We use ssh a lot, especially for access to our source code repository (currently CVS). We use a public key system with ssh, and most of our server accounts don’t have a password at all; they can be accessed only with a key.
For security, we ask people to protect their key with a passphrase. However, it is very inconvenient to have to type in the passphrase on every single access to the repository. The solution to this is to use an ssh agent, but finding out the best way to do this isn’t all that clear from the documentation.