If you’re running a Linux system, you need SSH access. It’s just a fact for any administrator. More over, you need quick, secure access with a minimum of security prompts. It’s not hard to get that set up and use the latest elliptical-curve security to boot!

We are going to set up our SSH server to only allow connections from non-root accounts using public-key authentication instead of passwords. This means only computers that have your private key can connect and that is much much much safer than passwords since there’s a limit on the length and complexity of a password you can reasonably expect to remember.


Since we will be preventing root-access via SSH, you might want to make sure you have sudo installed and set up on the system so non-root users can perform root actions! If you need help doing this, check out > this section < of my article, Installing and setting up sudo on Debian/Ubuntu and understanding sudoers


Root user

Virtually all aspects of setting up and securing your SSH server are easier if you are logged in as the root user. For the rest of this article, I’m assuming you’re using the root user account unless otherwise noted. If you’re using a non-root user, please just sudo -s before all your other commands so that you don’t have to remember to type ‘sudo’ all the time.

Configure everything within an SSH session

Sometimes it’s easier to set up SSH from within an SSH session. This is especially true on Hyper-V hosts dealing with Linux guests since an SSH connection is really the only practical way to copy keyfiles, etc. from the guest to the host. Plus, a bilateral clipboard is not a luxury we have via the default Hyper-V RDP interface. If you want to work at the console, then you can skip this section. If you want to do the rest of the set up within an SSH section, expand the section below to make that possible:

By default, SSH does not allow the root user to connect using just a password. Let’s go ahead and change that temporarily so we can connect via SSH and get stuff set up without any additional hassle. Open your sshd_config using your favourite editor, I like nano:

(o)# backup configuration file
cp /etc/ssh/sshd_config /etc/ssh/sshd_config.original
(o)# open configuration file in nano
nano /etc/ssh/sshd_config

In your configuration file, you’ll see the following line:

#PermitRootLogin prohibit-password

The comment tells you it’s a default option. We will add an option immediately after that to override the default for now. Change the file so it looks like this:

#PermitRootLogin prohibit-password
PermitRootLogin yes

This allows the root user to login and use just his/her password. You can now connect via SSH and carry out the following steps. Don’t worry, we’ll close this security hole soon.

Create your keys

We will need to create two sets of keys. First, host keys so the server can positively identify itself to clients and we can skip ‘thumbprint’ authentication prompts or worrying about man-in-the-middle attacks. Second, user keys so we don’t need to remember super-long passwords and only machines where your key is available will be able to connect as you. For this first part, we will only generate host keys. We’ll get to the user keys later after we’ve confirmed everything else is working.

Host keys

When a connection is first set up between the host and the client, the host (server) can sign this communication using a private key (host key). If the client has a corresponding copy of the public key, it can decrypt this communication and thus, verify that it is talking to the right host. This eliminates the need to verify thumbprints and avoids that super annoying prompt that no one reads anyways.

You’ll want to generate these as the root user or using sudo as your non-root user. I’m assuming you are using your root user. You can generate a bunch of types of host keys, but ED25519 and RSA are the most common. Wherever possible, you should use ED25519 since it’s newer and faster to generate and verify. We will generate both for compatibility.

(o)# change to SSH directory, easier to keep keys here
cd /etc/ssh
(o)# delete default keys for security
rm -f ssh_host_*
(o)# generate ed25519 keypair
ssh-keygen -t ed25519 -N '' -C "comment to identify your key" -f ./filename1.key
(o)# generate RSA 4096-bit keypair
ssh-keygen -t rsa -b 4096 -N '' -C "comment to identify your key" -f ./filename2.key

Here’s a practical example of the commands on lines 3 & 4 that correspond to the configuration we’ll be using in the next section:

ssh-keygen -t ed25519 -N '' -C "ED25519 host key for linux1.internal.mytechiethoughts.com" -f ed25519.key
ssh-keygen -t rsa -b 4096 -N '' -C "RSA4096 host key for linux1.internal.mytechiethoughts.com" -f RSA4096.key

This creates an ED25519 keypair with filenames ed25519.key and ed25519.key.pub and an RSA 4096-bit keypair with filenames RSA4096.key and RSA4096.key.pub. Check out the file permissions, you’ll see that the keys are restricted to root access only — keep it this way!

As a side note, pay special attention to the fact that we use -N ” (that’s two single quotes with nothing inside them, you can also use two double-quotes like “”). This tells ssh-keygen not to use a password. This is critical since there is no one available to enter a password!

As a final step, you need to get at least one of your .pub files copied over to whatever client system you want to use to connect. I leave that up to you since there are many ways of doing this. SCP is probably easiest. WinSCP is a great program for Windows users or if you’re already connected via SSH, then your SSH client should have a file-transfer utility. You can use WinSCP with PuTTY or Bitvise has a built-in client. SCP is built-in on most Linux systems.

Configuring sshd

Before doing anything else, you obviously need to get your SSH server set up correctly. Assuming you’re using the standard SSH package, that’s all handled via one configuration file. I encourage you to take a look at the official documentation about the sshd_config file but we’ll go over my typical setup which should be quite sufficient for most use cases. Let’s start by taking a look at the entire file:

### OpenSSH Configuration

### Connection settings
Port 222
#ListenAddress your.private.ipv4.address
#ListenAddress [your:private:ipv6:address:use:square:brackets]
TCPKeepAlive no
ClientAliveInterval 60
ClientAliveCountMax 30
Protocol 2

### Authentication settings
#HostKey /etc/ssh/ed25519.key
#HostKey /etc/ssh/RSA4096.key
#PasswordAuthentication no
PasswordAuthentication yes
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM yes
AuthorizedKeysFile .ssh/authorized_keys

### Login settings
LoginGraceTime 1M
MaxAuthTries 3
MaxSessions 5
#PermitRootLogin no
PermitRootLogin yes

### Program settings
#Banner /etc/ssh/banner
X11Forwarding yes
PrintMotd no
PrintLastLog yes
AcceptEnv LANG LC_*
Subsystem sftp /usr/lib/openssh/sftp-server

Connection settings

In this section, we handle the basic connection settings like ports and addresses, good place to start, right?

The port on which the sshd daemon listens for connections. Default is 22. I like using the non-standard port 222. You may choose any otherwise unused port you like.
The address on which the sshd daemon listens for connections. You can specify multiple addresses, each with their own ListenAddress directive on their own line. IPv6 addresses must be enclosed in [square brackets]. This is useful on systems with multiple IPs and/or adapters. For example, your system may have an address/NIC used for a webserver and another for LAN access — you’d want to limit SSH connections to the LAN address only. The default is which means all available addresses. Leave these lines commented or delete them on DHCP systems since your address changes all the time! If you have a static IP address, uncomment and fill-in these lines to increase the security of your system.
Send packets to keep connection active. This can be spoofed or blocked by remote firewalls so we turn it off and use ClientAlive options instead.
Number of seconds the server waits before sending a NOOP packet to the client to keep the connection alive.
Number of NOOP packets sent to the client before terminating the connection for being idle. In my example setup, absent any input, the server will send a packet every 60 seconds to the client and will do so a maximum of 30 times before terminating the connection. In other words, a total of 1800 seconds (30 minutes) of idle time may elapse before the server terminates a connection.
Only use SSH-2 protocol. The SSH-1 protocol is old and not secure. SSH-2 also allows multiple shell sessions over a single connection.

Authentication settings

This section specifies what types of authentication are supported and expected by the server.

Location of the private key the host will use to sign communication with connected clients. If the client has a corresponding public key to decrypt this communication, you won’t get any messages about ‘confirming identity’ or be asked to verify ‘server thumbprints’. You can specify multiple host keys to be used, each on their own line. In my example, I use an ED25519 and an RSA4096 key for compatibility. If you’re following along in order, you’ve already generated these keys and they are already named and located correctly. In that case, you’ll want to uncomment these lines to activate them.
Determines whether or not passwords are allowed as an authentication mechanism. Keys are far more secure, so eventually we will turn this option off. For now, we need to set this to ‘yes’ because we have not generated any user keys. Default: Yes
This should always be set to no. Otherwise, you are allowing connections with a null password. Default: no
This is a method of authentication where one or more questions are posed to the user and, if they answer correctly, they can connect. We are not allowing this method in our setup. Default: no
Allows SSH to use the pre-installed Pluggable Authentication Modules to run authentication checks… complex topic, leave it set to yes. Default: yes
This tells SSH where to look for a list of allowed keys for each user when using public-key authentication. This will make sense when we configure user keys, for now leave as-is which is the default setting anyways.

Login settings

Controls how login occurs and user-related restrictions.

The amount of time the server waits for the user to complete the login sequence. If you have super long passwords, you might want to increase this so the user has time to type.
Amount of times the user can attempt to login before the connection is dropped.
The maximum amount of sessions a user is allowed to run over one connection.
This has 3 settings: yes/no/prohibit-password. We have this set to yes for now so we can connect using our root account until we get things set up. This is for convenience. In practice, this should ideally be set to no or at least set to prohibit-password. The latter means that you need a keypair to connect. Default: prohibit-password.

Program settings

These are all the other settings for which I couldn’t think of a better category. I’m only going over two of them here since the rest are pretty standard to most SSH installs.

Text file to display to clients upon successful connection. This can be terms and conditions, advisories, legal disclosures or just a hello message. If you don’t want a banner, leave this commented or delete the line. Otherwise, uncomment it and specify a plain text file with the message you want displayed. Default: none
Level of logging. Keep this verbose if you want to use programs like Fail2Ban to secure your SSH connections. This is almost always set to verbose unless there is a reason to change it. Default: verbose.

Activate new setup

You can grab a copy of my latest sshd_config or you can manually type or copy/paste the above configuration to your system. If you want to download my sshd_config directly to your target system and you don’t have a browser/gui:

cd /etc/ssh
mv sshd_config sshd_config.backup
wget https://git.asifbacchus.app/asif/DebianConfigs/raw/branch/master/config/etc/ssh/sshd_config

If you’re following my Setting up a Debian base-system article and have downloaded the DebianConfigs helper-archive, you already have a copy of this file. Either way, open sshd_config using your favourite text editor and make your changes:

cd /etc/ssh
nano /etc/ssh/sshd_config

If you’re using my sample configuration, make sure the host key lines are correct and those files exist — change the filenames as needed. Remember to uncomment the applicable host key lines to activate them. After saving your changes and exiting your editor, test your sshd configuration:

sshd -t

If you get a blank response, you’re all good to go. If not, you’ll get an error telling you what’s wrong. Assuming all is good, restart the ssh service to activate the new configuration (this will NOT terminate an existing session if you’re doing this all over SSH, don’t worry):

systemctl restart ssh.service && systemctl status ssh.service

You should see the ssh service has restarted successfully and is now listening (on port 222 instead of 22 if you followed my configuration).

Initial testing

Make sure you’ve copied at least one of the generated .pub host key files to your client system and installed it within your client software as appropriate. Make sure you’ve also updated your client connection profile to use port 222 instead of the default 22, assuming you made that change in your sshd_config. If so, go ahead and exit any existing connections. Now reconnect — you should NOT be prompted to verify the server since the host keys are accepted and doing that for you. You should also now be connecting on port 222. You should have no problem logging in as root with your password. If that’s all good, let’s get things nice and secure with user keys.

Generate user keys

For this part, you’ll want to login with your regular user account instead of your root account. Alternatively, if you’re already logged in as root, you can switch to your user account (replace ‘username’ with your account):

(o)# using substitute-user (note spaces around hyphen)
su - username
(o)# using sudo
sudo -u username -s

We’ll be using the ssh-keygen command again but a little differently. Again, you can generate either RSA or ED25519 keys. I suggest using ED25519 whenever possible and only using RSA when you need to connect from older clients. Let’s create a private directory for our keys and related files and then generate our keys — remember, you only need to generate one set, I’m just showing the command for both so you know how to do it (replace things like the password and comment with values appropriate for your environment):

(o)# switch to your home directory
cd ~
(o)# create a hidden 'ssh' directory
mkdir .ssh
(o)# grant access only to yourself
chmod 700 .ssh
(o)# switch to our new 'ssh' directory
cd .ssh
(o)# generate ed25519 keypair
ssh-keygen -t ed25519 -N 'C0nn3cT_' -C "[email protected] SSH key" -f ./asif_ed25519_ssh_linux1.key
(o)# generate RSA 4096-bit keypair
ssh-keygen -t rsa -b 4096 -N 'C0nn3cT_' -C "[email protected] SSH key" -f ./asif_rsa_ssh_linux1.key

As a side note, if you don’t want to enter a password each time you use your keys to connect, simply do what we did when generating host keys earlier: -N ” (two single-quotes with nothing between them). Keep in mind, this is less secure since anyone who has your key can use it without needing to know a password.

Now, remember in our sshd_config file there was a line that looked like this?

AuthorizedKeysFile .ssh/authorized_keys

That line is saying that for whomever is connecting, the list of valid public keys for that user is contained in a file called authorized_keys located in .ssh/ within their home folder. So, it seems that we need to add our keys to that file and we should secure that file so only we can add keys to it, right? (use your public key filenames instead of mine!)

(o)# switch to our 'ssh' directory
cd ~/.ssh
(o)# create 'authorized_keys' file
touch authorized_keys
(o)# grant access to our account only
chmod 600 authorized_keys
(o)# append our ed25519 public key to 'authorized_keys' file
cat asif_ed25519_ssh_linux1.key.pub >> authorized_keys
(o)# append our RSA public key to 'authorized_keys' file
cat asif_rsa_ssh_linux1.key.pub >> authorized_keys
cat authorized_keys

The last command prints out your authorized_keys file so you can see your public key(s) listed for verification.

Testing: Round 2

Go ahead an copy your private key(s) to whatever client system you will be connecting from. Once that’s done and they’re installed in your client software, you no longer need a password to connect. Go ahead an try it out! Assuming it works, we can lock down our SSH configuration.

Final security settings

We’ve set things up so that you are not prompted to verify the server because our communication is encrypted and verified using host keys. In the same way, we don’t need a password to prove it’s actually us connecting, our user-keypair takes care of that. So let’s do away with password authentication altogether (you’ll still need your private key password though, but that can be shorter and easy to remember) and prevent the root user from connecting since that’s a huge security risk.

The good news is, we already have our directives in the sshd_config file, just commented out. Prevent root logins by deleting line 29 and uncommenting line 28 so your file looks like this:

MaxSessions 5
PermitRootLogin no

### Program settings

Now prevent password usage (allow keypairs only) by deleting line 18 and uncommenting line 17 so it looks like this:

HostKey /etc/ssh/RSA4096.key.pub
PasswordAuthentication no
PermitEmptyPasswords no

Go ahead and save the file, exit your editor and test the file to make sure nothing is screwed up:

sshd -t

Assuming no errors are reported, restart the SSH service to activate your new configuration.

systemctl restart ssh.service && systemctl status ssh.service

Final thoughts

Go ahead and try logging in as root or using a password — it won’t work! There you go, secure SSH access with the latest elliptical key security and no root access outside of sudo from a verified regular-user account. Safe but still very functional. Thanks for reading my techie-thoughts on this issue. Have any comments? Suggestions? Want to add your tips? Things you want me to cover in a future article? Comment below!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Close Menu