I've done the same with the HTC Desire. I had an iPhone 3GS which I replaced with an iPhone 4 but decided to move to the Desire because of Apple's attitude towards the antenna issue and because I thought Android would be better because there is less control over what Apps can and cannot do.
However I've now realised I've made a mistake. I echo the comments about the browser - it isn't as intuitive and clicking links isn't as accurate as the iPhone (although I do like the feature which reformats text as you zoom in).
The apps and general feel of the Desire is not as polished as the iPhone and the games are no where near as good.
The interface can be sluggish. The whole thing just locks up when it is downloading and installing apps (not a huge problem in itself but the iPhone doesn't do this).
As others have said, it does feel a bit more like a computer and doesn't all hang together quite as well as an iPhone.
I think everyone should try both devices before they get into arguments about which is better because I think it probably is a case of horses for courses but I'm definitely going back to the iPhone 4!