Remote server access using Signed OpenSSH Certificates. This is Scalable and secure access with SSH. When you have Virtual Private Cloud (VPC) at any service provider like AWS, GCP, Azure, or similar, and you don't want to depend on fixed/shared ssh private key-based authentication then this option is great. Here the User uses his own ssh private ssh key pair for ssh login to remote nodes. If you just want to harden SSH for VPS instance with Public IP (DigitalOcean, Linode, or similar) then please refer to Part-1. Or if you want to use single shared private key-based access then please refer to Part-2.
This setup needs 3 kinds of nodes.
1. First is CA-Server, which will host the signing key and related services.
2. Second is Server, which is the target node, where we will do ssh to.
3. Third is Client, which is the source node, wherefrom we will do ssh.
On, CA-Server create 2 signing CA keys. One for the authenticity of the Server (server where we are ssh-ing to) and another for the authenticity of the User (user using which we are ssh-ing from). Further, these will be used to set up both Server and Client.
# On CA-Server become root
$ sudo su -
$ mkdir ssh-keys
$ cd ssh-keys
$ rm -rf server_ca* user_ca*# Create host CA key-pair
$ ssh-keygen -t rsa -N '' -C SERVER_CA -b 4096 -f server_ca# Create user CA key-pair
$ ssh-keygen -t rsa -N '' -C USER_CA -b 4096 -f user_ca
Now on CA-Server, we will be generating a signed key for the Server. It will generate “/etc/ssh/ssh_host_rsa_key-cert.pub” file. This ideally needs to be created on each Server(s) in the real world. But for understanding, here I’m showing for one, mainly the process. For other servers to use the same signed key, you need to copy “/etc/ssh/ssh_host_rsa_key*” to all Server(s). Further, you need to append “HostCertificate /etc/ssh/ssh_host_rsa_key-cert.pub” at bottom “/etc/ssh/sshd_config” on Server(s) and restart ssh service.
# On CA-Server generate host signed certificate
$ ssh-keygen -s server_ca -I host_auth_server -h -n *.sidh.local -V +52w /etc/ssh/ssh_host_rsa_key.pub
$ scp /etc/ssh/ssh_host_rsa_key* root@Server:/etc/ssh/# Set sshd for host signed certificate on Server
$ ssh root@Server
$ echo "HostCertificate /etc/ssh/ssh_host_rsa_key-cert.pub" >> /etc/ssh/sshd_config
$ systemctl restart sshd
With this, the setting up a signed host certificate is done. Let us understand the command options :
-s server_ca # This is the host CA key used for signing
-I host_auth_server # This is just an identifier for host
-h # This just indicates we are generating host-key
-n *.sidh.local # The host principals which are allowed
-V +52w # Is to set validity period - can be anything
/etc/ssh/ssh_host_rsa_key.pub # SSH Server's ssh private keyYou can validate Principals and Expiry etc. for the signed host certificate using ssh-keygen command as below :
$ ssh-keygen -L -f /etc/ssh/ssh_host_rsa_key-cert.pub/etc/ssh/ssh_host_rsa_key-cert.pub:
Type: ssh-rsa-cert-v01@openssh.com host certificate
Public key: RSA-CERT SHA256:+SWaQzDIq5a4mKM3tjKWy2QSHTtwjlNtzOTLZDiKVlg
Signing CA: RSA SHA256:c/yCSt8wwDRI52E7XQMdibgA/2Ar3Zsps1OBGYxKA60 (using ssh-rsa)
Key ID: "host_auth_server"
Serial: 0
Valid: from 2020-12-15T16:10:00 to 2021-12-14T16:11:49
Principals:
*.sidh.local
Critical Options: (none)
Extensions: (none)
Now on CA-Server, we will be generating signed key for User from Client. for this we first need to copy User’s ssh public key (~/.ssh/id_rsa.pub) to CA-Server. It will generate “id_rsa-cert.pub” and signing key we will be using is “user_ca”. Then, copy the generated signed key back to User’s ~/.ssh/ on Client system. Now just ssh to Server as usual until the signed key “id_rsa-cert.pub” is valid.
# Copy User's id_rsa.pub from Client to CA-Server
$ scp ~/id_rsa.pub root@CA-Server:~/ssh-keys/# On CA-Server
$ ssh root@CA-Server
$ cd ssh-keys
$ ssh-keygen -s users_ca -I user_auth_client -n ec2-user,ubuntu,hadoop -V +5w -z 0 id_rsa.pub# Copy "users_ca.pub" key to Server and set sshd
$ scp users_ca.pub root@Server:/etc/ssh/
$ echo "TrustedUserCAKeys /etc/ssh/users_ca.pub" >> /etc/ssh/sshd_config
$ systemctl restart sshd# Copy the User's signed public key back to Client
$ scp id_rsa-cert.pub User@Client:~/.ssh/# From Client ssh login to Server as usual
$ ssh ec2-user@Server
With this, the setting up User public key signed certificate is done. Now, let us understand the command options :
-s users_ca # This is User CA key used for signing
-I user_auth_client # This is just an identifier for User/Client
-n ec2-user,ubuntu # list of allowed user pricipals
-V +5w # Is to set validity period - can be anything
-z 0 # Indicates serial number
id_rsa.pub # User's ssh public keyYou can validate Principals and Expiry etc. for the User public key signed certificate using ssh-keygen command as below :
$ ssh-keygen -L -f ~/.ssh/id_rsa-cert.pubid_rsa-cert.pub:
Type: ssh-rsa-cert-v01@openssh.com user certificate
Public key: RSA-CERT SHA256:VlFWgZpVT20kBKwyiGMb1ZCOZdHlA9s6hpGq0ctUd+M
Signing CA: RSA SHA256:9K7kssYrtDxxqDYYrx9Uy4Mn4ZMHcQCVhNWUfobkp+M (using ssh-rsa)
Key ID: "user_auth_client"
Serial: 0
Valid: from 2020-12-16T08:13:00 to 2021-12-15T08:14:38
Principals:
ec2-user
ubuntu
Critical Options: (none)
Extensions:
permit-X11-forwarding
permit-agent-forwarding
permit-port-forwarding
permit-pty
Now, for each Server(s), the creation of “signed host certificate” and for each User’s the creation of “signed public certificate” manually every time like above is a tedious task. Of-course we need automation around it. There are some free and paid options out there. Look at below-mentioned reference links :
Further, as best practice, a Custom AMI image (Instance OS) can be created with all these Server-side SSH settings and pre-copied all SSH keys (/etc/ssh/ssh_host_rsa_key*) in the image, etc. by Hassicorp’s Packer easily. The same way an sdk/tool can be created for signing the User’s public key easily. Or if you want to use single shared private key-based access then please refer to Part-2.
That’s all.
