TLS handshake, certificates and public-private key pairs

TLS/SSL uses certificates of the X.509 v3 certificate standard for the authentication of the communication partner. The structure of such certificates is expressed in ASN.1 (Abstract Syntax Notation One) and therefore is very flexible. For a basic understanding of the authentication during the TLS handshake, only a few certificate components are considered here: the issuer name, the subject name, the subject public key and the certificate signature. Besides these, certificates have a defined begin and end date of their validity, a serial and a version number, among other components. The issuer and subject names are distinguished names that commonly contain fields like country, organization, organizational unit, distinguished name qualifier, state or province name, common name and a serial number. Less often used fields in a distinguished name are a locality, title, surname, given name, initials, pseudonyms and a generation qualifier. In some cases, an e-mail address is included in the common name field.

The issuer name is the distinguished name of the certificate issuer. Usually, the issuer of a certificate is a certificate authority (CA), an entity that is trusted by all communication partners. When issuing (or "signing") a certificate, the CA creates the certificate signature using its own private key and thereby confirms the validity of the certificate. In particular, this means that the subject public key of the certificate is bound to the subject name. The validity of the certificate signature in turn can be verified by using the public key of the CA. The CA's public key is contained in the CA's own certificate, the so called CA certificate.

The subject name is the distinguished name of the certificate owner, i.e. the person or entity that receives the issued certificate from the CA. Additional verification checks may be done by a CA to verify that the certificate owner really corresponds to the subject name. The subject public key is the public key for which the certificate owner posses the matching private key.

When a client authenticates a server, the client has already received the server's certificate. In this certificate, the client finds the issuer name, where the issuer normally is a CA. Using the issuer name, the client checks whether it has its own copy of this issuer's CA certificate. The client (hopefully) finds a copy of this CA certificate and uses the CA's public key in this CA certificate to verify the certificate signature in the server's certificate. As the client itself has its own copy of the CA certificate, the client trusts that this really is not a manipulated copy of the original CA certificate. And the client trusts the CA, i.e. that the CA issued and signed the server's certificate for this server (and not somebody else). With that, the client rests assured, that the server's certificate received from the server really is from this desired server (and not from an imposter). The client now has successfully authenticated the server and proceeds with the TLS handshake.

At this point, you may note that anyone could have previously received the server's certificate (or picked it up by eavesdropping) and use it to pose as the server. However, continuing with the TLS handshake, the client uses the public key in the server's certificate to encrypt the randomly created symmetric key for the connection. An imposter could receive this encrypted symmetric key, but would not be able to decrypt it, because only the real server posses the private key needed for the decryption. Therefore, a wannabe imposter, who only somehow obtained the server's certificate, cannot successfully establish a secure connection with the client.

As we have just seen, the server's certificate received by the client serves two purposes: For one, the client uses it to authenticate the server. But as the certificate also contains the server's public key, the client can extract it from the certificate and use it right away to encrypt the randomly created symmetric key before sending it to the server.

Certificates and the corresponding keys play an important role in the TLS handshake. They need to be readily available for use during the TLS handshake. But at the same time they should be sufficiently protected. The server wants to keep its private key out of sight from everybody else. The server and the client also don't want anybody to manipulate the certificates. Especially the client wants to be sure that the CA certificates used to authenticate servers are authentic copies of the CA's original certificates rather than fake certificates planted by somebody else to falsely authenticate some fake server certificate. Therefore, both, the server as well as the client, use keystores to keep their key and certificates safe and accessible at the same time.