I have both a iPhone 3G 16GB and a 2nd Gen Ipod Touch 32GB. I find that the battery life on the iPod is longer than the iPhone, because of the radios running all of the time.
The iPod is also thinner and has a metal back plate. The cut out on the back is for the WiFi antenna, so it can get signal through the back.
Comparing the 2nd Gen iPod Touch and the 3G is a no-brainer, as they are the same basic hardware (processor, memory, etc.). The iPod does have a integrated speaker, so playing audio without a headset is possible. The max memory for the iPod is 32 GB, but the new version rumored to be coming out in September will probably drop with 64GB max. It should also be running the same hardware as the 3GS, with the compass and even a rumored camera.
This, once again, leaves only the phone parts as the discernible difference in the builds, other than the possible thickness of the devices, also.
I find that I use my iPod as much now than I ever did, after getting a 3G iPhone, as I can now load the iPod up with media until full and save the memory on my iPhone for things like documents, emails, contacts, pictures and ringtones.
I honestly don't mind carrying two devices, either, as they, together equal a similar size to the old AT&T 8925 Tilt that I used to carry. Try carrying that and a 5th Gen iPod Video at the same time.
Sacrificing the iPod battery and still having a charge on the iPhone to make calls, messaging, browsing, etc. is why I do it.