Taints and Tolerations

In this tutorial we well discuss about Taints and Tolerations, — another feature for the advanced Pod scheduling in Kubernetes.

The taints and toleration are all about the relationship between the pod and node. Taints and Toleration will tell what pods can be placed on what nodes.

The concept of Taints and Tolerations are can be a bit confusing for Kubernetes beginners. Taints and tolerations work together to make sure that pods are not scheduled onto inappropriate nodes.

Traints and tolerations are mainly used to set restrictions on what PODs can be scheduled on a node.

Let us start with the simple cluster with 3 worker nodes. The nodes are named 1, 2 and 3. We also have a set of PODs that are to be deployed on these nodes. Let’s call them as A, B, C and D.

When the PODs are created Kubernetes scheduler tries to place these PODs on the available worker node. As of now, there are no restrictions or limitations and asserts the scheduler places the PODs across all of the nodes to balance them out equally.

Now let’s assume that we have dedicated resources at Node 1 for particular use case or application. So here we would like only those PODs that belongs to this application to be placed on Node 1.

First we prevent all PODs from being placed on the node by placing a taint on the node. Let’s call it blue. By default PODs have no toleration which means unless specified otherwise none of the PODs can tolerate any taint.

So in this case none of the PODs can be placed on Node 1 as none of them can tolerate the taint blue. This solves half of our requirement.

No unwanted PODs are going to be placed on this Node. The other half is to enable certain PODs to be placed on this node. For this we must specify which PODs are tolerant to this particular taint.

In our case we would like to allow only POD D to be placed on this node. So we add a toleration to POD. POD D is now tolerant to blue. So when the scheduler tries to place this POD on Node 1 it goes through. Node 1 can now only accept PODs that can tolerate the taint blue.

So with all the taints and toleration in place. This is how the PODs would be scheduled.

The scheduler tries to place POD A on Node 1 due to the taint it is thrown off and it goes to Node 2, the scheduler then tries to place POD B on Node 1 but again due to the taint it is thrown off and it is placed on Node 3 which happens to be the next free node. The scheduler then tries to place POD C on Node 1 but again due to the taint it is thrown off and ends up on Node 2. Finally the scheduler tries to place POD D on Node 1, since the POD is tolerant to the Node 1.

Note

Taints are set on nodes and toleration are set on PODs. So how do we do this.

How to set a Taint on the node

Syntax

kubectl taint nodes [NODE_NAME] [KEY]=[VALUE]:[TAINT-EFFECT]

Taint effects define how nodes with a taint react to Pods that don’t tolerate it. There are 3 types of taint effects,

NoSchedule: Instructs Kubernetes scheduler not to schedule any new pods to the node unless the pod tolerates the taint.
PreferNoSchedule: The PreferNoSchedule taints prevents scheduling Pods on a node, BUT, if no suitable untainted node can be found then it WILL schedule the Pod on that node.
NoExecute: Which means that new pods will not be scheduled on the node and the existing PODs on the node if any will be evicted if they don’t tolerate the taint.

These PODs may have been scheduled on node before the taint was applied to the node.

kubectl taint nodes node1 app=blue:NoSchedule

Now create tolarations on PODs

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod

spec:
  containers:
  - name: nginx
    image: nginx
  tolerations:
  - key: "app"
    operator: "Equal"
    value: "blue"
    effect: "NoSchedule"

To remove the taint, you have to use the [KEY] and [EFFECT] ending with [-].

Syntax

kubectl taint nodes <node-name> [KEY]:[TAINT-EFFECT]-

E.g

kubectl taint nodes node1 app=blue:NoSchedule-

Taints and Tolerations