Workloads on specific nodes

SPIKE: Specific workloads on Specific Nodes

JIRA: link

Resources:

• [1] Advanced pod scheduling https://docs.openshift.com/container-platform/4.4/nodes/scheduling/nodes-scheduler-about.html
• [2] Pod placing using node affinity rules https://docs.openshift.com/container-platform/4.4/nodes/scheduling/nodes-scheduler-node-affinity.html
• [3] Pod placing using node taints, and pod tolerations https://docs.openshift.com/container-platform/4.4/nodes/scheduling/nodes-scheduler-taints-tolerations.html
• [4] Pod affinity/anti-affinity https://docs.openshift.com/container-platform/4.4/nodes/scheduling/nodes-scheduler-pod-affinity.html#nodes-scheduler-pod-affinity
• [5] Cluster/Project wide Node Affinity https://docs.openshift.com/container-platform/4.4/nodes/scheduling/nodes-scheduler-node-selectors.html#nodes-scheduler-node-selectors-project_nodes-scheduler-node-selectors
• [6] Cluster https://github.com/storageos/cluster-operator

I’m thinking we should go with a mixture of two modes, taint/toleration and node affinity in order to target a specific class of compute node with privileged network access and ensure only specific workloads may be scheduled there, and so access them.

Creating a POC:

Steps:

oc new-project targeting-specific-nodes
oc get nodes
    NAME                    STATUS   ROLES  AGE VERSION
    kempty-n10.ci.centos.org   Ready    worker   7d5h   v1.17.1
    kempty-n11.ci.centos.org   Ready    worker   7d5h   v1.17.1
    kempty-n12.ci.centos.org   Ready    worker   7d5h   v1.17.1
    kempty-n6.ci.centos.org Ready   master   7d5h   v1.17.1
    kempty-n7.ci.centos.org Ready   master   7d5h   v1.17.1
    kempty-n8.ci.centos.org Ready   master   7d5h   v1.17.1
    kempty-n9.ci.centos.org Ready   worker   7d5h   v1.17.1

oc adm taint nodes kempty-n9.ci.centos.org examplekey=examplevalue:NoSchedule
oc label node kempty-n9.ci.centos.org examplekey=examplevalue
oc get node kempty-n9.ci.centos.org -o yaml
    …
      labels:
               ...
        examplekey: examplevalue
               ...
   ...
    spec:
      taints:
      - effect: NoSchedule
        key: examplekey
        value: examplevalue
    …

oc apply -f targetspecificnodes.yaml
oc get pods --all-namespaces -o wide | grep kempty-n9
    ...
    targeting-specific-nodes                                example-57d74cf6b4-mlppz                                        1/1     Running     0       13s     10.130.2.15 kempty-n9.ci.centos.org <none>          <none>

    targeting-specific-nodes                                example-57d74cf6b4-sgnzh                                        1/1     Running     0       13s     10.130.2.17 kempty-n9.ci.centos.org <none>          <none>
    targeting-specific-nodes                                example-57d74cf6b4-swllc                                        1/1     Running     0       13s     10.130.2.16 kempty-n9.ci.centos.org <none>          <none>
    ...