I noticed that some servers running CentOS 5.4 exhibit noticeable delays during authentication. At some point, this started to really bother me. The issue was resolved quite simply by enabling connection debugging:
ssh –v user@host
During the connection process, the connection would hang at the following point:
Unspecified GSS failure. Minor code may provide more information
This indicates that the system attempts to use GSSAPI authentication, which is not configured, before falling back to password authentication. To fix this, you need to comment out the lines containing the word GSSAPIAuthentication in sshd_config on the server side or ssh_config on the client side.