When I upgraded mine from 4.1 to 4.3.1 then 4.3.3, I saw my battery life drop by about 60%. I used to get two days of use out of my iPhone 4 (mostly text, a few short calls, and minimal browsing). Suddenly I dropped about 10% an hour minimum with a few texts and no surfing.
On 4.3.3 if I surf the net on car rides (I travel quite a bit for work) and text, my phone dies in about 6-8 hours tops. I always made the entire day before on long trips, with 20-30% to spare. Thankfully I jailbroke my iPhone and saved my SHSH blobs. I just went back to 4.2.1 and have noticed a huge increase in battery life on the first day. After 6 hours of heavy texting I'm only down to 78%, whereas yesterday at this time with similar usage I was in the 30's. I even had Apple do a factory restore and reinstall the software to make sure it was a clean install.
Apple's reluctance to let you downgrade your OS when you see it doesn't work well with your phone will be a deciding factor on whether I purchase another iPhone in the future. Then again, On my Android phone (I bounce back and forth between phones) and my Blackberries previously, I always try out different OSes. If one didn't play nicely with my phone, I simply tried another.