Hardening SSH on a new VPS — keys, fail2ban, disabling root login

The default sshd_config on most cloud images is "open enough to be usable." Anything internet-facing draws constant brute- force traffic — your VPS auth.log will have thousands of failed root login attempts per day if you leave password auth on. This article walks the four moves that take SSH from "default" to "actually locked down" in about ten minutes.

1. Set up SSH key authentication

If you're still logging in with a password, this is step zero. SSH keys are stronger than any password and don't appear in any breach dump. From your local machine:

# Generate a key (skip if you already have one):
ssh-keygen -t ed25519 -C "$(whoami)@$(hostname)"

# Copy it to the VPS (one last password prompt):
ssh-copy-id root@<your-IP>

# Test:
ssh root@<your-IP>
# Should log in without asking for a password.

Ed25519 keys are smaller and faster than RSA, with equivalent security. Skip RSA unless you need compatibility with very old software.

2. Disable password authentication

Only after step 1 works. If you disable passwords without a working key, your only way back in is the browser console. Verify key login twice first.

nano /etc/ssh/sshd_config

# Find and set:
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
UsePAM yes

# Test the config:
sshd -t

# If silent (no errors), reload:
systemctl reload ssh    # Debian / Ubuntu
systemctl reload sshd   # AlmaLinux / RHEL

Now password brute-forcers can't even try — they hit the unsupported-method rejection and move on.

3. Disable direct root login

The convention is: log in as a normal user, then sudo to root for admin tasks. This adds a layer (attacker needs your unprivileged account credentials and your sudo password) and creates an audit trail in /var/log/auth.log.

# Create a non-root user:
adduser yourname

# Add to sudo group:
usermod -aG sudo yourname     # Debian / Ubuntu
usermod -aG wheel yourname    # AlmaLinux / RHEL

# Copy your SSH key to the new user:
mkdir -p /home/yourname/.ssh
cp /root/.ssh/authorized_keys /home/yourname/.ssh/
chown -R yourname:yourname /home/yourname/.ssh
chmod 700 /home/yourname/.ssh
chmod 600 /home/yourname/.ssh/authorized_keys

Test logging in as the new user from another terminal (don't close your root session yet):

ssh yourname@<your-IP>
sudo whoami    # should print "root"

Once that works, disable root SSH:

nano /etc/ssh/sshd_config
# Set:
PermitRootLogin no

sshd -t && systemctl reload ssh

Root SSH is now off; you log in as yourname and sudo for elevated commands.

4. Install fail2ban

Even with password auth off, the log noise from brute-forcers is annoying. fail2ban watches the auth log and temp-bans IPs that hit thresholds.

Debian / Ubuntu:

apt install -y fail2ban

AlmaLinux:

dnf install -y epel-release
dnf install -y fail2ban

The default config (/etc/fail2ban/jail.conf) is sensible — SSH jail enabled, 5 attempts in 10 minutes triggers a 10-minute ban. Override only what you want to change in /etc/fail2ban/jail.local:

# /etc/fail2ban/jail.local
[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5

# Don't ban your own home/office IP if you have a static one:
ignoreip = 127.0.0.1/8 ::1 203.0.113.0/24

[sshd]
enabled = true

Start it:

systemctl enable --now fail2ban
fail2ban-client status sshd

The status output will show banned IPs as they accumulate.

Optional moves worth considering

• Change SSH port: doesn't add security, but reduces log noise by ~90%. Pick something unprivileged (e.g., 2222) and update both /etc/ssh/sshd_config and your firewall.
• Limit user logins: in sshd_config, add AllowUsers yourname to whitelist just your account. Any other user is rejected at the SSH layer.
• Two-factor with Google Authenticator: the libpam-google-authenticator package adds TOTP as a second factor for SSH. Overkill for most personal servers; useful for production.
• Rate-limit at the firewall: UFW's ufw limit ssh or nftables / iptables --limit rules add a network-layer rate limit in front of fail2ban. Belt and suspenders.

What this does NOT protect against

• Your laptop being compromised. If an attacker has your SSH private key and key passphrase, they log in normally. Keep the private key off shared machines.
• Vulnerabilities in services you expose. SSH hardening doesn't help if you have an outdated web app with a remote code execution bug.
• Insider abuse. If you give another person an account, they have it. Trust accordingly.

The mandatory escape-hatch reminder

Before you tighten SSH further, confirm your fallback:

• The browser console works regardless of SSH state.
• You know the root password (set it if you haven't — the console needs it).
• Rescue mode is your nuclear option if both SSH and console somehow fail.

With key auth on, password auth off, root SSH disabled, and fail2ban running, your VPS is past the point where any opportunistic brute-force will succeed. The remaining attack surface is your applications and your laptop — both more useful to focus on than further SSH knob-twiddling.