Yes, you can restore without updating...but before you do this, you must make sure that you have the firmware you wish to restore to downloaded on your computer...here's a quick clip from the Apple forums that shows the way to force the update to whatever firmware you choose...
The key language on that page is: You will be prompted with one or more restore options that may prompt iTunes to automatically download the latest iPod Software. The additional options are only offered if the older software is already stored on the computer; if not, the sole choice is to get the latest/greatest.
Fortunately, you can fool iTunes...if you can get your hands on the older software. On a Mac (for instance), the software is stored at ~/Library/iTunes/iPhone/iPod/iPad Software Updates/ (that''s where iTunes puts it after dowloading). Once you get a copy of the version you want, just stick it in there; then disconnect from internet; fire up iTunes, connect iPhone/iPad/iPod, and click the Restore button; iTunes has no choice but to offer the older" software.