Introduction to Kubernetes Scheduling

Scheduling in Kubernetes determines where Pods are placed within the cluster. By default, the Kubernetes scheduler assigns Pods to nodes based on resource availability and fairness. However, advanced scheduling techniques allow for more precise control over how and where Pods run.

Why is Advanced Scheduling Important?

1. Resource Optimization: Efficiently use available resources across nodes.
2. Workload Isolation: Ensure certain workloads run on specific nodes.
3. High Availability: Spread workloads across failure domains.
4. Compliance: Adhere to policies requiring certain applications to run on specific nodes.

Key Concepts in Advanced Scheduling

1. Node Affinity: Schedule Pods based on node labels.
2. Taints and Tolerations: Prevent Pods from being scheduled on unsuitable nodes unless explicitly allowed.
3. Pod Affinity and Anti-Affinity: Control Pod co-location or separation.
4. Custom Schedulers: Define custom scheduling logic for specific use cases.

Step-by-Step Implementation

Step 1: Using Node Affinity

Node affinity allows Pods to be scheduled on nodes with specific labels.

Example: Configuring Node Affinity

1. Label a Node:

kubectl label nodes <node-name> environment=production

2. Define Pod Specification with Node Affinity:

apiVersion: v1
kind: Pod
metadata:
  name: node-affinity-example
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: environment
            operator: In
            values:
            - production
  containers:
  - name: nginx
    image: nginx

3. Deploy the Pod:

kubectl apply -f node-affinity-pod.yaml

4. Verify the Pod Placement:

kubectl get pods -o wide

Step 2: Taints and Tolerations

Taints allow you to repel Pods from nodes, and tolerations allow specific Pods to override those taints.

Example: Adding a Taint

1. Taint a Node:

kubectl taint nodes <node-name> key=value:NoSchedule

2. Define a Pod with a Matching Toleration:

apiVersion: v1
kind: Pod
metadata:
  name: toleration-example
spec:
  tolerations:
  - key: "key"
    operator: "Equal"
    value: "value"
    effect: "NoSchedule"
  containers:
  - name: nginx
    image: nginx

3. Deploy the Pod:

kubectl apply -f toleration-pod.yaml

4. Verify the Pod Placement:

kubectl get pods -o wide

Step 3: Pod Affinity and Anti-Affinity

Pod affinity ensures Pods are co-located with other Pods, while anti-affinity ensures separation.

Example: Configuring Pod Affinity

1. Define Pod Affinity:

apiVersion: v1
kind: Pod
metadata:
  name: pod-affinity-example
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app: frontend
        topologyKey: "kubernetes.io/hostname"
  containers:
  - name: nginx
    image: nginx

2. Deploy the Pod:

kubectl apply -f pod-affinity.yaml

3. Verify Placement:

kubectl get pods -o wide

Example: Configuring Pod Anti-Affinity

1. Define Pod Anti-Affinity:

apiVersion: v1
kind: Pod
metadata:
  name: pod-anti-affinity-example
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app: backend
        topologyKey: "kubernetes.io/hostname"
  containers:
  - name: nginx
    image: nginx

2. Deploy and Verify: Repeat the steps above to confirm the Pods are separated.

Step 4: Custom Schedulers

1. Deploy a Custom Scheduler:
   • Create a custom scheduler configuration.
   • Deploy it as a Deployment with a unique name.
2. Specify Scheduler for a Pod:

apiVersion: v1
kind: Pod
metadata:
  name: custom-scheduler-pod
spec:
  schedulerName: custom-scheduler
  containers:
  - name: nginx
    image: nginx

Best Practices for Scheduling in Production

1. Use Labels and Selectors:
   • Clearly label nodes and use selectors for workload placement.
2. Avoid Hard Constraints:
   • Use preferredDuringSchedulingIgnoredDuringExecution unless strict placement is required.
3. Monitor Scheduling Events:

kubectl get events --field-selector reason=FailedScheduling

4. Test Policies:

Deploy test workloads to verify scheduling behavior before production rollout.

Production Example: Advanced Scheduling

Scenario

You manage a production e-commerce platform with the following requirements:

1. Frontend Pods must run on nodes labeled zone=frontend-zone.
2. Backend Pods must avoid nodes running Frontend Pods.
3. Critical services must tolerate specific node taints.

Implementation

1. Label Nodes:

kubectl label nodes <frontend-node> zone=frontend-zone

2. Deploy Frontend Pods with Node Affinity:

• Use the nodeAffinity example.

3. Deploy Backend Pods with Anti-Affinity:

• Use the podAntiAffinity example.

4. Taint Database Nodes:

kubectl taint nodes <db-node> role=database:NoSchedule

5. Deploy Database Pods with Tolerations:

• Use the tolerations example.

6. Monitor and Verify Placement:

kubectl get pods -o wide

Conclusion

In this chapter, you learned:

1. How to implement advanced scheduling using Node Affinity, Taints, and Pod Affinity/Anti-Affinity.
2. How to deploy custom schedulers.
3. Best practices for applying these concepts in production.