I'm constantly using my iPad 2 during live performances.
I'm a 2nd drummer in a jamband, so I have lots of freedom to freak out and make weird noises and whatnot. Until recently I've mostly been using Pianist Pro to drive various VSTs in Ableton Live with MIDI over WiFi, but lately I've been using instruments directly on the iPad, especially since the new Moog app came out -- this thing is very possibly the coolest app yet.
Thumbjam is also a perennial favorite, with a new version coming out very soon. Jesse has been churning out betas very quickly, and the functionality in the new version will be otherworldly. For live performance, or even just sketching ideas, it's an incredible tool and a giant leap forward in terms of interfaces for musical instruments. I can't begin to explain how much I love Thumbjam, it's given me the ability to finally get the string concertos that've been floating around my head for decades now out into real form -- then I can just record the MIDI on my Macbook and print out the score!
Imagine if Mozart or Beethoven had had these tools...
[edit] Oh, and I've also invented an electric kazoo, and this interface would be the ultimate interface for recording it on the go, as well as applying effects to it for live performance, plus recording everything on the go, plus a thousand other uses I can't even take the time to write out...
I bought the iPad as a musical instrument. In fact, I sold my EWI 4000S to buy it, and haven't regretted it for a second. This thing is the ultimate musician's tool.