I am not sure about actually "Canceling" it? You can always go into "Settings" in GV and just not have it forward the number or notify you in any way. It will just sit there. It's free so it should not hurt anything.
You know I always hated getting text notifications from GV. The text-to-speech was that that good IMO. What I do is have the message emailed to me. The email would have a link to my GV account ("Play Message"). that I could press and it would log me into GV. That way I can actually listen to the message from my iPhone.