SSH is the primary remote access protocol for Linux servers, and it is the single most targeted service by automated attackers. A freshly deployed server with SSH on port 22 will receive brute-force login attempts within minutes. While SSH is cryptographically strong by default, its security depends entirely on how it is configured. Default settings are rarely sufficient for production.

This guide goes beyond basic SSH hardening into advanced topics like SSH certificate authorities, jump hosts, connection multiplexing, and two-factor authentication -- the practices used by organizations that take security seriously.

Key-Based Authentication

Password authentication should be disabled on every production server. SSH keys are both more secure and more convenient:

Generating Strong Keys

# Ed25519 (recommended - modern, fast, secure)
ssh-keygen -t ed25519 -C "[email protected]"

# RSA (fallback for older systems - use 4096 bits minimum)
ssh-keygen -t rsa -b 4096 -C "[email protected]"

# With a specific filename
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_production -C "admin@production"
Tip: Always use a strong passphrase on your SSH private key. This protects you if the key file is stolen. Use ssh-agent so you only enter the passphrase once per session, not on every connection.

Deploying Keys

# Copy your public key to the server
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server

# Or manually
cat ~/.ssh/id_ed25519.pub | ssh user@server \
  'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'

# Verify permissions (critical - SSH refuses keys with wrong permissions)
# On the server:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R $USER:$USER ~/.ssh

Disabling Password Authentication

Edit /etc/ssh/sshd_config:

PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM yes
PermitRootLogin no
AuthenticationMethods publickey
Warning: Before disabling password authentication, verify that key-based login works by opening a second SSH session using the key. If your key is misconfigured and you disable password auth, you will be locked out.

SSH Daemon Hardening

A comprehensive /etc/ssh/sshd_config for production servers:

# Protocol
Protocol 2

# Listen on a non-standard port (reduces automated scanning noise)
Port 2222

# Authentication
PermitRootLogin no
MaxAuthTries 3
MaxSessions 3
LoginGraceTime 20
PubkeyAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no

# Restrict users/groups
AllowUsers deployer admin
# Or: AllowGroups ssh-users

# Cryptography - use only strong algorithms
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key

Ciphers [email protected],[email protected],[email protected]
MACs [email protected],[email protected]
KexAlgorithms [email protected],curve25519-sha256,[email protected]
HostKeyAlgorithms ssh-ed25519,rsa-sha2-512,rsa-sha2-256

# Disable unnecessary features
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no
PermitTunnel no
GatewayPorts no
PermitUserEnvironment no
PrintMotd no

# Timeouts
ClientAliveInterval 300
ClientAliveCountMax 2

# Logging
LogLevel VERBOSE

# Banner
Banner /etc/ssh/banner
# Validate config before restarting
sshd -t

# Restart
systemctl restart sshd

# Regenerate host keys (remove weak ones)
rm /etc/ssh/ssh_host_dsa_key*
rm /etc/ssh/ssh_host_ecdsa_key*
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ""
ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""

SSH Certificates

SSH certificates solve the trust-on-first-use (TOFU) problem and eliminate the need to distribute authorized_keys files to every server. Instead, a Certificate Authority (CA) signs user keys, and servers trust any key signed by that CA:

Setting Up a Certificate Authority

# Generate the CA key pair (keep the private key extremely secure)
ssh-keygen -t ed25519 -f /etc/ssh/ca_key -C "SSH Certificate Authority"

# Sign a user's public key
ssh-keygen -s /etc/ssh/ca_key \
  -I "[email protected]" \
  -n deployer,admin \
  -V +52w \
  ~/.ssh/id_ed25519.pub

# This creates ~/.ssh/id_ed25519-cert.pub
# -I = Key identity (shows up in logs)
# -n = Principals (allowed usernames)
# -V = Validity period (+52w = one year)

Configuring Servers to Trust the CA

# Copy the CA public key to the server
scp /etc/ssh/ca_key.pub server:/etc/ssh/ca_key.pub

# Add to sshd_config
TrustedUserCAKeys /etc/ssh/ca_key.pub

# Optionally, sign the server's host key too
# (eliminates "unknown host" warnings for clients)
ssh-keygen -s /etc/ssh/ca_key \
  -I "server.example.com" \
  -h \
  -n server.example.com,192.168.1.100 \
  -V +52w \
  /etc/ssh/ssh_host_ed25519_key.pub

# Server sshd_config:
HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub

# Client ~/.ssh/known_hosts (trust all hosts signed by this CA):
@cert-authority *.example.com ssh-ed25519 AAAA...ca_public_key...

Fail2ban for SSH

# Install
apt install -y fail2ban    # Debian/Ubuntu
pacman -S fail2ban         # Arch

# Create /etc/fail2ban/jail.local
[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 3
backend = systemd

[sshd]
enabled = true
port = 2222
filter = sshd
logpath = %(sshd_log)s
maxretry = 3
bantime = 86400

[sshd-aggressive]
enabled = true
port = 2222
filter = sshd[mode=aggressive]
logpath = %(sshd_log)s
maxretry = 1
bantime = 604800
findtime = 86400

# Start
systemctl enable fail2ban
systemctl start fail2ban

# Check banned IPs
fail2ban-client status sshd

# Manually unban an IP
fail2ban-client set sshd unbanip 192.168.1.50

Jump Hosts and ProxyJump

A jump host (bastion host) acts as a gateway to your internal network. Internal servers are not directly accessible from the internet; you SSH through the jump host:

# Direct syntax
ssh -J jumphost.example.com internal-server

# Multiple jumps
ssh -J jump1.example.com,jump2.example.com internal-server

# In ~/.ssh/config (the recommended approach)
Host jump
    HostName jumphost.example.com
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_jump

Host internal-*
    ProxyJump jump
    User deployer
    IdentityFile ~/.ssh/id_ed25519

Host internal-web
    HostName 10.0.1.10

Host internal-db
    HostName 10.0.1.20

Host internal-docker
    HostName 10.0.1.30
# Now you can simply:
ssh internal-web
ssh internal-db
ssh internal-docker

# SCP through jump host
scp -o ProxyJump=jump localfile.txt internal-web:/tmp/

# Port forwarding through jump host
ssh -L 8080:internal-web:80 jump

Connection Multiplexing

SSH multiplexing reuses a single TCP connection for multiple SSH sessions, dramatically reducing connection time:

# In ~/.ssh/config
Host *
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h-%p
    ControlPersist 600
# Create the socket directory
mkdir -p ~/.ssh/sockets
chmod 700 ~/.ssh/sockets

# First connection establishes the master
ssh server          # Takes normal time

# Subsequent connections are nearly instant
ssh server          # Reuses existing connection
scp file server:    # Reuses existing connection

# Check multiplexed connections
ssh -O check server

# Close a multiplexed connection
ssh -O exit server

Two-Factor Authentication with PAM

Add a second factor to SSH authentication using Google Authenticator or any TOTP application:

# Install
apt install -y libpam-google-authenticator   # Debian/Ubuntu
pacman -S libpam-google-authenticator        # Arch

# Run setup as the user who will log in
google-authenticator
# Answer yes to time-based tokens
# Scan the QR code with your authenticator app
# Save the emergency codes

Configure PAM in /etc/pam.d/sshd:

# Add at the top of the file:
auth required pam_google_authenticator.so nullok

# The 'nullok' option allows users who haven't set up 2FA to still log in
# Remove 'nullok' after all users have configured their tokens

Update /etc/ssh/sshd_config:

ChallengeResponseAuthentication yes
UsePAM yes

# Require both key AND TOTP
AuthenticationMethods publickey,keyboard-interactive

# Or key OR password+TOTP
# AuthenticationMethods publickey keyboard-interactive
systemctl restart sshd

SSH Config Tips and Tricks

# ~/.ssh/config

# Keep connections alive across network changes
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    TCPKeepAlive yes

# Automatically add keys to agent
Host *
    AddKeysToAgent yes
    IdentitiesOnly yes

# Per-host configuration
Host production-*
    User deployer
    IdentityFile ~/.ssh/id_ed25519_prod
    ForwardAgent no

Host staging-*
    User developer
    IdentityFile ~/.ssh/id_ed25519_staging

# Git over SSH through a proxy
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_github

# Alias for quick access
Host db
    HostName database.internal.example.com
    User postgres
    LocalForward 5432 localhost:5432
    ProxyJump jump

Agent Forwarding Security

# Agent forwarding is convenient but dangerous
# It allows any root user on the remote host to use your keys

# Instead of ForwardAgent yes, use ProxyJump:
Host internal
    ProxyJump jump

# If you must use agent forwarding, restrict it:
Host trusted-server
    ForwardAgent yes

Host *
    ForwardAgent no    # Disabled by default

SSH Security Audit Checklist

Check Command Expected
Root login disabled sshd -T | grep permitrootlogin no
Password auth disabled sshd -T | grep passwordauthentication no
Strong ciphers only sshd -T | grep ciphers No CBC, 3DES, or RC4
Fail2ban active fail2ban-client status sshd Jail running
No weak host keys ls /etc/ssh/ssh_host_*_key Only ed25519 and rsa
Key permissions correct stat -c %a ~/.ssh/authorized_keys 600
LogLevel set sshd -T | grep loglevel VERBOSE

For organizations managing many servers, usulnet's built-in SSH terminal provides secure, audited access to Docker hosts without requiring individual SSH configurations on each administrator's workstation, while maintaining the security controls described in this guide.

Final thought: SSH security is not a one-time setup. Rotate keys annually, audit access logs regularly, and review your authorized_keys files for stale entries. A key belonging to a former employee is an open door into your infrastructure.