Trusting a Key: authorized_keys, ssh-agent, and Closing the Password Door

Key-based SSH login is deceptively simple: once configured, it just works. But "configured" hides three separate mechanics worth understanding — how the server actually decides to trust your key, how to avoid typing your passphrase on every connection, and how to shut the door on password authentication entirely once keys are working.

How the Server Decides Who Gets In

Every user account on a Linux server has a hidden file in their home directory: ~/.ssh/authorized_keys. This file is a list of public keys — one per line — that the SSH daemon will accept as proof of identity for that user. If your public key appears in this file on a server, your matching private key can log in as that user. If it does not, you cannot — no matter how strong your key is.

That is the entire mechanism. Two files, in two places, mathematically linked: the private key on your machine, and the matching public key listed inside authorized_keys on the server. Everything else — helper tools, configuration flags, agents — is convenience wrapped around this simple fact.

Placing a Key by Hand

Automated helpers can install a public key on a server in a single command, but the underlying operation is nothing more than appending a line to a text file and setting the correct permissions. Seeing it done by hand once makes the whole mechanism concrete — and gives you a fallback for the systems where those helpers are not installed.

The whole thing fits in one pipeline from your local machine:

Reading left to right: cat ~/.ssh/id_ed25519.pub prints your public key locally, and the pipe sends it through an SSH connection to the server. On the remote side, the shell creates ~/.ssh if needed with the correct permissions, appends the incoming public key to authorized_keys, and tightens the file's permissions to 600.

If you prefer a step-by-step approach, log in with your password and run the commands directly on the server:

The required permissions on the server side mirror what is needed on your own machine: ~/.ssh must be 700 and authorized_keys must be 600. If any of these are too permissive, the SSH daemon's StrictModes check will refuse to read the file and silently fall back to password authentication, as if the key were not there.

Confirming a Key Login Actually Worked

Before hardening anything, confirm that authentication really succeeded via your key — and not via a password fallback because the permissions on authorized_keys were wrong. The verbose flag -v makes the SSH client narrate the handshake. Buried in the output is a single line that tells you exactly which method got you in:

The phrase Authentication succeeded (publickey) is your confirmation. If instead you see Authentication succeeded (password), the key is not being used and hardening the server would lock you out. Fix the setup first.

The ssh-agent: Type the Passphrase Once

A passphrase-protected key is secure, but typing the passphrase on every single connection quickly becomes tedious. The ssh-agent exists for exactly this problem.

The agent is a small background program that holds your decrypted private keys in memory for the duration of your session. You unlock the key once by typing the passphrase; from that point on, the agent silently answers signing requests on your behalf. The decrypted key never touches disk again and disappears the moment the agent exits.

If your desktop environment does not start an agent automatically, you can start one manually and load a key into it with two short commands:

eval "$(ssh-agent -s)" starts the agent and sets the environment variables that let ssh find it. ssh-add loads a specific key, prompting for the passphrase once. The -l flag lists everything the agent currently holds.

In practice you will rarely run these commands by hand. Most modern Linux desktop environments and macOS start an agent automatically when you log in. On macOS, the agent can additionally store passphrases in the system Keychain so they survive a reboot. The manual commands above are what you reach for in a bare terminal session, on a server, or inside a container where no agent is already running.

Closing the Door: Disabling Password Authentication

With key-based login confirmed working, the final hardening step is to turn off password logins entirely. This is what actually neutralises the brute-force bots — even if an attacker somehow guessed a correct password, the server will simply refuse to consider passwords as a valid authentication method.

The setting lives in /etc/ssh/sshd_config. Open it on the server and find (or add) this line:

Save the file and reload the SSH daemon. The exact service name varies by distribution:

On many cloud-provider images — Ubuntu on AWS, DigitalOcean, or Hetzner — the directory /etc/ssh/sshd_config.d/ may contain additional files (such as 50-cloud-init.conf) that are loaded after the main file and can override your change. If passwords still work after you set PasswordAuthentication no, this is almost certainly why. Confirm the daemon's effective setting with:

If that returns yes, look inside /etc/ssh/sshd_config.d/ for a file overriding the setting and reconcile it.

If step 3 breaks something — a typo in the config, an unexpected override file — you still have a working, authenticated shell on the server from step 2 and can revert. Close it only after the third terminal in step 4 has successfully logged in.

For extra hardening, administrators often pair PasswordAuthentication no with PermitRootLogin no in the same file, which forces attackers to guess a non-root username as well — but the password setting is the one that closes the brute-force door.

Managing Multiple Keys

As you accumulate servers and services, you may not want a single key responsible for everything. Using a different key per device — one for the laptop, one for the desktop — limits the blast radius if a private key is ever exposed. If your laptop is stolen, you can revoke just that one key from every server (by removing its line from each authorized_keys file) without disrupting access from your other machines.

When you have several keys, the -i flag tells ssh which private key to use:

The same -i flag works with scp, sftp, and other SSH-based tools. One subtlety: by default ssh will also offer your other keys to the server before settling on the one you specified, and after enough rejected offers the server may disconnect with a Too many authentication failures error. Pairing -i with -o IdentitiesOnly=yes tells ssh to offer only the specified key and nothing else.

Typing -i on every command becomes a chore once you maintain more than a handful of keys. The cleaner long-term solution is the SSH client configuration file at ~/.ssh/config, where you can map a friendly alias to a hostname, username, port, and identity file — so that a simple ssh work-server does the right thing automatically. That configuration file is a topic in its own right.