TLS in Kubernetes
In this tutorial, we will look at securing your Kubernetes cluster with TLS Certificates. In the previous tutorial, We saw what public and private keys are how a server uses public and private keys to secure connectivity.
There are 3 types of certificates, sever certificates configured on the servers, root certificate configured on the CA servers and then client certificates configured on the clients.
You’re going to see a lot of certificate files in this tutorial and it could be very confusing. So use this technique to know which one is which.
Simple technique
Usually certificates with Public keys are named CRT or pem extension. So that’s server.crt, server.pem for server certificates or client.crt or client.pem for client certificates.
Private keys are usually with extension .key, or with a –key in the filename. For example server.key or server-key.pem.
So just remember private keys have the word key in them usually either as an extension or in the name of the certificate and one that doesn’t have the word key in them is usually a public key or certificate. That’s how I remember it.
We will now see how these concepts relate to a Kubernetes cluster. The Kubernetes cluster consist of a set of master and worker nodes.
Of course all communication between these nodes need to be secure and must be encrypted all interactions between all services and their clients need to be secure.
For example an administrator interacting with the Kubernetes cluster through the kubectl utility or via accessing the Kubernetes API directly must establish secure TLS connection.
Communication between all the components within the Kubernetes cluster also need to be secured.
So the two primary requirements are to have all the various services within the cluster to use server certificates and all clients to use client certificates to verify they are who they are.
Let’s look at the different components within the Kubernetes cluster and identify the various servers and clients and who talks to who.
Server Certificates for Servers
Let’s start with a Kube API server. As we know already the API server exposes an HTTP a service that other components as well as external users use to manage the Kubernetes cluster.
So it is a server and it requires certificates to secure all communication with its clients.
So we generate a certificate and key Pair. We call it apiserver.crt and apisever.key. We will try to stick to this naming convention going forward. Anything with a .CRT extension is the certificate and .key extension is the private key.
Also remember the certificate names could be different in different Kubernetes setups depending on who and how the cluster was set up. So these names may be different in yours.
In this tutorial we will try to use names that help us easily identify the certificate files.
Another server in the cluster is the ETCD server. The ETCD server stores all information about the cluster so it requires a pair of certificate and key for itself. We will call it at etcdserver.CRT and etcdserver.key.
The other server component in the cluster is on the worker nodes. There are the kubelet services but also expose an HTTPs API in point that the Kube API server talks to to interact with the worker nodes. Again that requires a certificate in keypair. We call it kubelet.crt and kublet.key.
Client Certificates for Clients
Lets now look at the client components who are the clients who access the services.
The clients who access the Kube API server are us the administrators, through Kubectl or REST API. The admin user requires a certificate and key pair to authenticate the Kube API server. We will call it admin.crt and admin.key.
The scheduler talks to the Kube API server to look for pods that require scheduling and then get the API server to schedule the pods on the right worker nodes.
The scheduler is a client that accesses the Kube API server. As far as the Kube API server is concerned, the scheduler is just another client like the admin user. So the scheduler needs to validate its identity using a client TLS certificate. So it needs its own pair of certificate and keys. We will call it scheduler.crt and scheduler.key.
The Kube Controller manager is another client that accesses the Kube API server. So it also requires a certificate for authentication to the Kube API server. So we create a certificate pair for it.
The last client component is the Kube proxy. The Kube proxy requires a client certificate to authenticate to the Kube API server. And so it requires its own pair of certificate and keys. We will call them kube-proxy.crt and kube-proxy.key.
The servers communicate amongst them as well. For example, the Kube API server communicates with the ETCD server. In fact of all the components the Kube API server is the only server that talks to the ETCD server.
So as far as the ETCD server is concerned the Kube API server is a client. So it needs to authenticate.
The Kube API server can use the same keys that it used earlier for serving its own API service. The apiserver.crt and the apiserver.key files Or you can generate a new pair of certificates specifically or the Kube API server to authenticate to the ETCD server.
The Kube API Server also talks to the kubelet server on each of the individual nodes. That’s how it monitors the worker nodes for this. Again it can use the original certificates or generate new ones specifically for this purpose. So that’s too many certificates.
There are a set of client certificates mostly used by clients to connect to the Kube API Server. And there are a set of server side certificates used by the Kube API Server, ETCD server and kubelet to authenticate their clients.
Generate certificates
We will now see how to generate these certificates. As we know already, we need a certificate authority to sign all of these certificates.
Kubernetes requires you to have one certificate authority for your cluster. In fact you can have more than one one for all the components in the cluster and another one specifically for ETCD. In that case the ETCD servers certificates and the ETCD servers client certificates, which in this case is the api-server client certificate will be all signed by the ETCD server CA.
For now we will stick to just one CA for our cluster. The CA as we know has its own pair of certificate and key. We will call it ca.crt and ca.key.