Understanding the SSH Key Authentication Process

When connecting to a server using SSH keys ssh user@hostname, the process follows these steps:

1. Connection Initiation: The client initiates an SSH connection to the server
2. Server Identification: The server sends its public host key to prove its identity
3. Authentication Method Negotiation: The client and server agree on public key authentication
4. Key-Based Authentication: The actual authentication using the SSH keys
5. Session Establishment: Upon successful authentication, the secure session begins

Let's break down each step in more detail.

1. Connection Initiation

When you run a command like ssh user@server, your SSH client begins by establishing a TCP connection to the server's SSH port (typically port 22).

2. Server Identification

• Once connected, the server sends its public host key to identify itself.
• Your client checks this key against its known_hosts file to verify the server's identity and prevent man-in-the-middle attacks.
• If this is the first connection, you'll see:

  The authenticity of host 'hostname (IP)' can't be established.
  RSA key fingerprint is SHA256:xxxx.
  Are you sure you want to continue connecting (yes/no)?

• Upon accepting, the host key is stored in known_hosts for future verification

3. Authentication Method Negotiation

The client and server negotiate which authentication methods to use. When public key authentication is available, SSH typically tries it before falling back to password authentication.

4. Key-Based Authentication

This is where SSH keys come into play:

1. The client sends a message to the server indicating it wants to authenticate using a specific public key
2. The server checks if this public key is in the user's ~/.ssh/authorized_keys file
3. If found, the server generates a random challenge message and encrypts it with the public key
4. Only the holder of the corresponding private key can decrypt this challenge
5. The client decrypts the challenge using its private key and sends a response
6. The server verifies the response, proving the client possesses the private key

This process verifies the client's identity without ever transmitting the private key over the network.

5. Session Establishment

Once authenticated, a secure channel is established for the rest of the SSH session, allowing secure command execution, terminal access, or other SSH features.

Troubleshooting SSH Key Authentication

If you're having issues with SSH key authentication:

1. Check file permissions: Private keys should be 600 (chmod 600 ~/.ssh/id_rsa)
2. Verify the public key is in the server's authorized_keys file
3. Check the server's SSH configuration to ensure public key authentication is enabled
4. Run SSH in verbose mode (ssh -v user@server) to see detailed connection information
5. Confirm the key is loaded in your SSH agent with ssh-add -l

By understanding how SSH uses keys for authentication, you can better secure your server connections and troubleshoot any issues that arise in your SSH authentication workflow.

Conclusion

SSH key authentication is a powerful and secure method for accessing remote servers. By following best practices and understanding the underlying process, you can ensure a smooth and secure connection every time.