TLS & SSL & mTLS In A Nutshell — For Developers
Introduction
As developers, we often come across HTTP/S and instantly think — shit, not certs again. This guide will hopefully keep that gap in knowledge each year to a minimum which subtle reminders and explanations to what each component of the certification process is, including creating your own certificate authority and signing your own certificate signing requests with your newly created CA.
I apologise if this article is a little confusing as you read through, but certificates are kind of hard to explain in some sequential orderly fashion. So if you do read a section and it doesn’t make sense, just keep reading and come back to it later.
Note: I am not an expert in TLS, but hope to provide at least a little awareness of what is going on and by no means expect everything in this article to be 100% correct. Although I think the majority is? :D But anyways, if anything is wrong in this article feel free to leave a comments and/or note for me and I’ll update it and give credit :P
This article is slightly outdated and SAN is recommended without CN, but hey ho, it’s roughly still the same: https://mta.openssl.org/pipermail/openssl-users/2017-August/006283.html
What is RSA and what has TLS/SSL got to do with it?
RSA (Acronym for Rivest–Shamir–Adleman) is a public key-oriented asymmetric cryptographic encryption system. In a nutshell, given a public key, we can encrypt data, whether that be binary, ASCII, or whatever. And given a private key, we can decrypt the publically encrypted data.
The underlying implementation of RSA is often irrelevant to developers as there are many tools to generate keys for us. But if interested, it isn’t too bad to use RSA yourself for a custom application — For example, I’m terrible with Math but used it to encrypt my operation codes for a RuneScape private server such that the payload wasn’t identifiable without the decrypted operation code.
RSA is used in the TLS (Acronym for Transport Layer Security)/SSL(Acronym for Secure Socket[s] Layer) protocol.
Technically, and usually, other PKI (Public-Key-Infrastructure) can be used, but for example's sake, we’ll talk only of RSA.
So what is SSL exactly…?
We’ll be referring to TLS instead of SSL as TLS superseded SSL way back in ~2015.
TLS (1.3) is the current modern standard for RSA within a HTTP protocol as of 13th April 2021.
In a generic TCP scenario, RSA doesn’t explicitly really need a protocol as we can encrypt/decrypt whatever part of our custom binary packet structure we very well choose.
In HTTP, the protocol is stateless and can therefore apply consistency across the use of RSA, now in the form of TLS. It isn’t entirely as simple as this, but the other details matter little to us as Developers and we should often just use the latest TLS we can.
So, to clarify, TLS is an PKI (Whether that be RSA or another) oriented protocol in an HTTP environment in addition to providing identification metainformation, in the form of certificates.
Pre-requisites to understanding how TLS works:
There are a couple of things we need to understand before we can understand TLS.
These include:
- Client Certificates
- Server Certificates / Application Certificates
- Certificate Authorities
- Root Certificate Authorities
- Certificate Signing Requests
- TLS Sessions (and the fact they’re asymmetric)
We’ll go through each step by step and this should ultimately explain how TLS works.
What is a Certificate?
A certificate is a PKI Public key and identifiable meta-information about the owner in a standardised file format.
The specification used to create the meta information can come in a range of formats, but you’ve probably seen x509.
Common Certificate File Formats:
- .CRT — Is not password protected, only stores the certificate
- .PEM — Is not password protected, can store the certificate AND a private key, but it doesn’t have to store the private key
- .P12/PFX — Is password protected, should store the certificate AND a private key
- JKS — Java’s homebrew of PKCS#12 (P12/PFX)
We’ll cover private key usage soon and why we have formats that can store both.
What is a Certificate Authority?
A Certificate Authority is simply a certificate that has been generated to act as a CA purposefully or is an unsigned certificate generated using some tool, such as OpenSSL command-line tool. To create a CA using OpenSSL we can run the following commands:
# Create CA Key
openssl genrsa -aes256 -out ./myRootCAKey.key 4096# Create CA Cert
openssl req -new -config ./myRootCACertMetaInformation.cnf -x509 -sha256 -days 730 -key ./myRootCAKey.key -out ./myRootCACert.crt
The .cnf:
This is the meta information a certificate contains, it identifies the owner and other bits of information utilised in the certificate creation/usage, here’s an example .cnf:
# the fully qualified server (or service) name
FQDN = my.domain.com# the name of your organization
# (see also https://www.switch.ch/pki/participants/)
ORGNAME = My Company ROCKS ltd.# subjectAltName entries: to add DNS aliases to the CSR, delete
# the '#' character in the ALTNAMES line, and change the subsequent
# 'DNS:' entries accordingly. Please note: all DNS names must
# resolve to the same IP address as the FQDN.
ALTNAMES = DNS:my.domain.com # , DNS:bar.example.org , DNS:www.foo.example.org# --- no modifications required below ---
[ req ]
default_bits = 2048 # Read below
default_md = sha256 # We can override these in the command as you see above, or alternatively let it default here
prompt = no
encrypt_key = no
distinguished_name = dn # A DN is basically an identifier string of key/values for who the certificate belongs to, in this case we're using a variable to place it below so things look a bit prettier :)
req_extensions = req_ext # Extensions are[ dn ]
C = Country Code (USA, UK, whatever)
ST = Some State
O = $ORGNAME
OU = Organisational Unit
CN = $FQDN[ req_ext ]
subjectAltName = $ALTNAMES
We’ll go over how these work in their own section after understanding how TLS works.
The .key:
This is the private key we used to generate our public key, which is ultimately placed within the certificate we will be outputting. Note the public key is placed in the crt file, NOT the PRIVATE key and the PRIVATE key should be kept SAFE and NEVER distributed.
The .crt:
A combination of the CNF and public key encompassed into a single file, known as a certificate.
What is the purpose of a Certificate Authority?
A Certificate Authorities primary purpose is to be a single place of trust.
The uses of a CA are as follows:
- To sign Certificates for us developers to use within our applications
- To verify certificates that have been signed and substantiate that they are authentic
How do I get a Server Certificate?
As we’re not VeriSign, and we’ve previously just created our own CA, we’ll do something known as “self-signing”. Basically, we’ll use our own CA to sign server certificates.
Consider the following OpenSSL example:
Creating a CSR
# Create "Server Application" Key
openssl genrsa -out myServerRSAKey.key 4096# Create CSR (Certificate Signing Request)
openssl req -new -config ./myServer.cnf -key myServerRSAKey.key -sha256 -out myServer.csr
We create another key that will be associated with our Server Certificate when it is eventually created.
We then create a CSR using a .cnf [so that the CA can identify us using this meta information], specific to the domain we want the certificate to be registered for (i.e., mywebsite.com and my organisation details, etc.).
Notice the CSR is generic, we haven’t mentioned any CA yet whatsoever, that is because we would send our CSR to the CA to sign and return us a certificate.
Accepting a CSR
A CA will use the CSR to create our TLS certificate, but, it does not need your private key.
We need to keep our private key secret. The certificate created with a particular CSR will only work with the private key that was generated with it.
To accept a CSR with our self-signing CA, we can use the following command:
# Create "Server Application" Cert [signed with our CA]
openssl x509 -req -days 365 -sha256
-in myServer.csr
-CA ./myRootCACert.crt
-CAkey ./myRootCAKey.key
-set_serial 1
-out ./myServer.crtIt accepts our CSR, uses the CA’s certificate and key, identifies our certificate is certificate 1 (via set_serial) and outputs it to the same directory under myServer.crt.
Now we have a public certificate signed by the CA and the private key associated with it. With all of these steps in place, we can look into how TLS works and where we would install these certificates in our development environment.
Pre-requisite final
Now we have the pre-requisite knowledge of what is required to implement TLS, namely:
For self-signed TLS:
- A self-created/appointeed CA
- A signed Server Certificate
For a verified CA global internet TLS:
- A signed Server Certificate from a trusted CA (GlobalSign, VeriSign, DigiSign etc.)
We can look over how TLS works, and then we can look at how we would install these certificates in Spring Boot, NodeJS for the server-side and Mac for the client CA side.
How does TLS work then?
The TLS procedure happens in roughly 8 steps, depending on the version of TLS you’re using, the procedure varies slightly between TLS 1.2 and 1.3 and PKI methods used for example. But a roundabout example for RSA in TLS 1.2 would be…
Handshake Phase:
- The client makes a request to the server
- The server responds that it is indeed TLS
- The client sends a “ClientHello” message
- The server responds with a “ServerHello” message
- The server sends its certificate to the client
- The server sends a “ServerHelloDone” message
- The client verifies the certificate sent from the server with CA’s installed on the client’s machine/client application, it does this in a number of ways:
— Checking the digital signature
— Verifying the Certificate chain
— Checking for issues with the cert, i.e., is it expired? Is the domain name not the same as who sent it? etc.
If the client is happy with the certificate, the TLS handshake proceeds - The TLS Session is now able to commence as the authorisation stage is complete, the client is aware that the server is who it says it is and is prepared to enter asymmetric session encryption. Now a key point to recognise is the client expects the server to have its private key, if it doesn’t the next step will fail and the connection will terminate
TLS Asymmetric Session Phase for the Client:
- To begin the encrypted TLS session, the client generates something called a “client random” also known as the “pre-master secret”, but I prefer “client random”
- The client encrypts the “client random” using the public RSA key contained within the certificate sent by the server and sends it to the server
- The client sends a separate message known as the “Change Cipher Spec”, which lets the server known the client will only communicate via encrypted messages from now on
- Finally, the client sends a “Finished” message to indicate the handshake has completed from the client-side. The “Finished” message is the first encrypted message and is crucial, as it contains MAC (A 12 byte long, unless specified otherwise in the Change Cipher Spec message) data to ensure the handshake has not been compromised or tampered with. See here for information on the “Finished” message.
TLS Asymmetric Session Phase for the Server:
- The server receives the “Client Random” and “Change Cipher Spec”, it decrypts the “Client Random” using the private key initially created when the server made its CSR.
- The server then sends its own “Change Cipher Spec” to indicate it will only communicate over an encrypted channel now
- The server finally sends its own “Finished” message using the asymmetric key (which is the “Client Random” basically) for encryption
And that completes the TLS handshake and TLS asymmetric key sharing, the next step is they perform ordinary HTTP but encrypt the request and responses with the “Client Random”.
What to take away from understanding the TLS Handshake process
We’ll probably, very unlikely at least, remember the entire process in a year's time. So what I would suggest remembering, is that the following is required:
- A server certificate
- A server key (associated with the CSR) to decrypt the client's response and initiate TLS
- The client needs the CA certificate installed to verify the initial sending of the server certificate
I like this image below, it’s simple and direct:
It skips a fair few of the steps and shows us from a developer-esque perspective, we simply need to remember that we need a CA installed on our client, which signed the server's certificate. And the server simply has to install the certificate and private key from the initial CSR. That’s it!
Finally, depending on your operating system, installing the CA certificate is different. In Windows, it’s the Certificate Manager, in Mac it’s KeyChain Access. Whatever OS you are on, there’s plenty of guides on how you install your CA’s .crt file.
As for the server, in NodeJS it’s just referencing the file via an input stream to the key and server certificate. This should be fairly common across most technologies like Python, Java Servlets etc.
Some framework provide built-in TLS reading, such as Spring Boot in which it’s just a configuration YAML / Prop property which you point relatively to your certificate and key. Again, whatever the case, there’s plenty of guides out there.
KeyStore VS TrustStore
When learning TLS, I distinctly remember being confused by KeyStore and TrustStores and what exactly was meant by the two.
KeyStore
Let’s start with KeyStore, a KeyStore, such as PEM, P12, PFX, whatever, contains both the certificate AND private key. So in a NodeJS TLS example, instead of having two file input streams to pick up our certificates, we can have one which leads to our KeyStore and loads it into our application.
Don’t worry, the technology should be aware that it shouldn’t send the key contents and will only send the certificate on request.
TrustStore
Now TrustStores, they’re literally identical to KeyStores but there are two kinds:
- A P12/PFX/JKS/PEM whatever, which stores the CA’s certificate AND private key. The use case here would be possibly uploading it to your ACME, but that’s absolutely out of the scope of this guide.
- The same as above, but it DOES NOT store the private key and instead stores multiple CA certificates, hence “trust-store’. This is done literally via taking N CA certificates and pasting them into a single .P12/PFX, whatever file. This is common practice in mutual TLS, which we’ll discuss below.
So in a way, we can think of all the pre-installed CA certificates on the client’s host operating system as a TrustStore in and of itself, as after all, it contains multiple CA certificates.
Mutual TLS (mTLS)
In an ordinary TLS scenario, the server sends its certificate to the client, but the client never verifies itself. Mutual TLS is simply that the client verifies itself too.
Why would we use mTLS? Well most probably for very secure clients we manually distribute certificates too, or for internal service-2-service communication (Microservices) whereby we want to be super-duper secure.
To implement it, it would look like so:
On the client’s host machine certificate store (KeyChain Access/Windows Cert Manager/Whatever), it will have a certificate created from a CSR to a CA present on the servers TrustStore.
You can either:
- A) Create a single CA which generates both the server and client certificates, installs both the certificates on their respective machines/applications, and additionally install the CA on them both too. There’s a little fine tuning usually in server applications to tell it that it needs to perform mTLS but it’s most likely just a flag.
- B) Create 2 CA’s, install 1 on the client, and 1 on the server. Generate a single certificate from each CA and give it to the opposite party. For example:
CA 1 installed on the client.
CA 1 signed certificate installed on the server
CA 2 installed on the server
CA 2 signed certificate installed on the client
Server has mTLS flag turned on (or whatever config is required to enable mTLS)
And that’s it. Nothing overly complicated, just repeating the same process on the opposite party.
Final
I hoped this help understand TLS and mTLS a little bit more, or a lot more, who knows. If it did, leave a clap, and good luck in keeping your services secure!