if something below is flat wrong just say so. i'm no expert.
handbrake gets my vote for layout and ease of use but they did remove some features i liked so it's not as versatile.
But I also use the following for certain jobs when Handbrake won't cut it with great results:
MediaCoder
Xilisoft HD video Converter 6
Xvid4PSP is rarely mentioned but will convert damn near anything. I almost never use it but never delete it. But it was one of the only things i knew of that could convert a 1080i ts file to something my xbox 360 could play.
as for handbrake killing the quality i haven't seen that. I think you may have to manually adjust the bitrate of a file to make sure it's not shrinking the quality too much to degrade it. Plus it helps to have a great source. my suggestion is find a small clip, like 15. Or cut one down to 15 seconds and do some trial and error encodes. at only 15 seconds it won't take too long to encode each time. I don't have an iphone 4 yet or appletv so i can't give you specific bitrates. But i have some nice looking Bluray disc rips at 3000kbps. they look great on 50 inches. Now the problem is according to this
Apple - Apple TV - View all technical specifications for Apple TV. i think appletv only plays up to 2500kbps. "MPEG-4 video, up to 2.5 Mbps". Correct me if i'm wrong. i'm no expert. is that the proper conversion? regardless the quote is from apple so i trust even if my conversion is wrong they are correct in what it can play. But so for me i'd start right just below 2500 and work my way back till i start to get unacceptable. i'm to tired to look up iphone stuff and since i'm stuck on sprint for a few more months it will only depress me.
Some other tools i used alot when i was converting files. Note these are Windows tools. I've no idea what works for Macs:
Yamb muxing and demuxing mp4 files. may work with avis too but i don't remember.
LameXP for converting audio
Nandub trims avis (make sure you set audio and video streams to direct copy)
XenonMKV MKV to Mp4 converter, works w/o reencoding
mediainfo used to check what a file consists of, example, bitrate, how many tracks, subtitles, works great if you can't figure out if a file is has a wrong extension or you forgot to ad an audio track or something.
MediaInfo
Avidemux 2.5 I think i use this to cut trim mkv's w/o reencoding. but honestly i don't remember for sure.
mp3gain for changing the volume of mp3. only use really for music not really audio i'd put in video.
PS i have no clue about DRM issues on these. I have no media that has DRM so i've never had to deal with that..
Oh and as a side note i now convert everything only to MP4 h246 for compatibility reasons. I have some avis, and 1080p mkvs, but personally i convert to mp4 just for consistency and lots of stuff plays. h246 codecs.
Edit: i also came across this
The Complete Guide to iPod, Apple TV and iPhone Video Formats | iLounge Article