Obvious questions out of the way...is the lens clean? Can you see any damage to it (scuffing, etc)?
I had this problem with the first iPhone 4 i got a couple of years ago (with no damage to the lens itself)...i just swapped it out for a new one so i never got a full answer on it, but given the complexity of the camera in it (and the tiny parts that are in it), it's no surprise some will be faulty.