SSH is the single most attacked service on a typical Linux server. If yours is exposed to the internet, something is already trying to brute force it. Most of that attack surface can be removed in about twenty minutes. Here is the order I do it in.
Key Takeaways
- The order matters. Key-based authentication alone removes most of the risk; everything after it is narrowing what remains.
- Test every change in a second terminal before closing the session you are working in.
- A non-standard port is noise reduction, not security. Treat it as a way to make your logs readable.
- Hardening is a point-in-time action. Configuration drifts, and without monitoring you find out at the worst moment.
sshd -tbefore every restart. It catches the config error that would otherwise lock you out.
Table of Contents
1. Key-based authentication only
The highest-value change. Generate a key, copy it up, then in /etc/ssh/sshd_config:
PasswordAuthentication no
PubkeyAuthentication yes
Test in a second terminal before closing your session. Locking yourself out of a remote server is a bad afternoon.
This one change ends brute force as a category of risk, because there is no password to guess. Attempts still arrive and still fill your logs, but they cannot succeed. Everything below is narrowing what remains after this.
One caveat worth knowing: PasswordAuthentication no does not disable keyboard-interactive authentication on every distribution. If you want to be certain, set KbdInteractiveAuthentication no as well, then verify with sshd -T | grep -iE "passwordauth|kbdinteractive", which prints the effective running configuration rather than what the file appears to say. If you would rather read the whole effective configuration at once, paste sshd -T output into the free SSH config checker, which reviews it in your browser and uploads nothing.
2. Disable root login
PermitRootLogin no
Log in as a normal user and escalate. This also gives you an audit trail of who did what, which "everyone shares root" does not.
That audit trail is the underrated part. When something goes wrong at 2am, sudo entries attributed to a named account are the difference between reconstructing what happened and guessing.
3. Limit who can log in
AllowUsers deploy admin
An explicit allowlist is stronger than trying to enumerate everyone you want to block.
AllowGroups is often the better form on a server with more than a couple of people, since adding someone becomes a group membership change rather than an sshd config edit and restart.
4. Consider a non-standard port
Moving off 22 does not stop a targeted attacker, but it removes most of the automated background noise, which makes your logs readable. Treat it as noise reduction, not security.
The honest cost: it complicates every runbook, monitoring check, and deploy script that assumes 22, and it will confuse a colleague at some point. Worth it on a server you spend time reading logs for, not worth it as a reflex.
5. Disable unused features
X11Forwarding no
AllowAgentForwarding no
PermitEmptyPasswords no
Agent forwarding deserves a note. If you forward your agent to a server and that server is compromised, whoever controls it can use your key to reach everywhere your key works, for as long as your session is open. Use ProxyJump instead when you need to hop through a bastion.
6. Shorten the login grace period
LoginGraceTime 30
MaxAuthTries 3
MaxAuthTries counts attempts within a single connection, which is not the same as attempts overall. An attacker can reconnect indefinitely. This limits the cheap version; per-source limiting is the next item.
7. Rate-limit at the firewall
Block IPs that open many connections in a short window before sshd ever sees them.
With nftables or iptables this is a connection-rate rule on the SSH port. The benefit over waiting for fail2ban is that it applies before any authentication work happens, so it costs almost nothing under load. Allowlist your own ranges first, as with anything that blocks automatically, the failure modes are covered in how automated IP blocking works and how it goes wrong.
8. Install fail2ban
Automatic banning for whatever gets past the above. If you are deciding between fail2ban and a shared-intelligence alternative, I compared them in fail2ban vs CrowdSec.
9. Restart and verify
sudo sshd -t && sudo systemctl restart sshd
The -t catches config errors before you apply them.
Then verify what is actually in effect, which is not always what the file says once Include directives and drop-ins are involved:
sudo sshd -T | grep -iE "permitrootlogin|passwordauth|allowusers|maxauthtries"
ls -la /etc/ssh/sshd_config.d/
A drop-in file in sshd_config.d/ can silently override the main config, and on several distributions one ships by default. This is a common reason a change appears to have no effect.
10. Watch what happens next
Hardening is a point-in-time action. Config drifts, packages update, someone re-enables password auth for a contractor and forgets. Without monitoring, you find out at the worst moment.
Worth checking regularly: has sshd_config changed, are there new authorized_keys entries you did not add - this is exactly what file integrity monitoring is for - and are failed login patterns shifting to a new source.
An added authorized_keys entry is the one that matters most. It survives password rotation entirely, which means an attacker who plants a key keeps access through exactly the response most people perform after an incident. The wider set of artefacts worth watching is in detecting privilege escalation on a Linux server.
What this does not cover
Ten changes remove most of the SSH attack surface. They do nothing about the cases where SSH was never the way in.
An attacker who lands through a web application does not need your SSH configuration to be weak, they already have code execution, and the first thing they typically leave behind is a webshell in a web-writable directory. A stolen key or a leaked deploy credential authenticates correctly and looks like a normal login. And nothing here tells you whether anything is currently running on the box that should not be; if you suspect that, start with how to tell if your Linux server has been compromised.
Hardening reduces what can happen. Monitoring tells you what did.
SecAI audits SSH configuration and monitors your servers continuously and flags exactly these drifts, alongside the file integrity and behavioural monitoring that catches what hardening alone misses. Where this sits among the other controls worth running is set out in the guide to Linux server security software.
Frequently Asked Questions
Does changing the SSH port actually improve security? Not against anyone targeting you specifically. It removes most untargeted scanning noise, which makes real events visible in your logs. That is worth something, but it is not a control.
Should I disable SSH entirely and use my provider's console? For servers that genuinely need no interactive access, that is a defensible choice. For most, SSH with keys and no password authentication is both practical and strong.
Is fail2ban still needed if password authentication is off? It is less critical, since attempts cannot succeed. It still reduces log volume and resource use from constant probing, which has its own value.
How do I safely rotate SSH keys? Add the new key, verify you can authenticate with it from a separate session, then remove the old one. Never remove first. Keep a console fallback available through your provider.
What if I lock myself out? Most providers offer a web console or recovery mode that bypasses SSH. Confirm you know how to reach yours before you start changing configuration, not after.