scp with spaces in a remote directory path and a wildcard

It's not as simple as you might think:

scp -r username@hostname:"'/some/annoying/directory path/'*" /destination/

There are other forms of this command on the web, but this is the one that worked for me on FC12, connecting to an SME 7.4 server.