Install and then run Bitcoin-Qt. Then quit and uninstall.
RESULT: C:\Program Files\Bitcoin\locale folder is left behind. RESULT: %APPDATA%\Bitcoin\addr.dat, .lock, db.log, debug.log, etc left behind
I'd expect everything except wallet.dat to be removed (and it would be nice if there was an uninstall checkbox, NOT checked by default, to remove the wallet, too).