Like finn5975, I adjust mine based on the current needs, but I do have Auto-Brightness turned on.
That said, I find that sometimes with the Griffin Survivor case on, I have to crank the brightness higher than I would have to if it weren't for the additional glare/haziness its screen protector causes. Even with Zagg GLASS installed though, I still don't trust the phone without that extra layer...