Lonnie Lazar at Voxilla discusses a security flaw in Google Voice. If you call Voice from your own phone you could have access to your account. Someone else could spoof you calling phone number and access your account. They would have access to your messages and the ability to place calls from your account. According to the story Google has partially fixed this.

http://voxilla.com/2009/03/24/despite-fixes-google-voice-security-flaws-remain-1627

http://www.google.com/support/forum/p/voice/thread?tid=2a2b7ad2e7441921&hl=en