The name doesn't annoy me too much. I think it was a bigger deal when they changed it to a new number (iPhone 4) and then you had 4G networks rolling out. I talk to a lot of casual users looking at phones who just assume it's a 4G phone because of the name.
But they seem to have a pattern of changing the name whenever there's a major body redesign. The iPhone 4 body was well-received, but it has some technical problems, and the 4s fixes those without really changing the form factor. I think that they didn't want to call it an "iPhone 5" and get flamed for really not offering any dramatic improvements.
Changing the form factor isn't a dramatic improvement. Dramatic improvements are a combination of improved screens, better processors, more memory, better battery, better camera, etc.
The 4S was "dramatic" improvement in my opinion as it had a lot of improvements.. I'd have no problem if they had called it iphone 5. Just about everything but the form factor changed for the better. Here's a few:
1. 2x processor of previous and 7x graphics. A big deal and it improves every other feature speeding things up.
2. Much better camera, including more MP, face detection, 5 element lens, lets more light in.
3. 1080p video including stabilization.
4. Better mic.
5. More storage.
6. Siri
7. HSPA+ and gsm/cdma dual mode
8. Bluetooth 4.0
9. Wireless mirroring
10. Better antenna
I hear the "it looks like the iphone 4" arguments but that's just looks. Personally, i don't want something different each year meaning all the accessories are incompatible. Bored with how the phone looks? I don't get it when most cover it up with a case.