Kubernetes Advanced Guide 🚀

Pod Management with kubectl 🛠️

Replace a Pod

To delete and recreate a pod using a configuration file:

kubectl replace --force -f nginx.yaml

Get Pods Information

Watch the status of all pods:

k get pods --watch/-o wide

Get information about all nodes:

k get nodes

Get information about pods in the kube-system namespace:

k get pods -n kube-system

Scheduling Pods in Specific Nodes 🗂️

Schedule a Pod on a Specific Node

Use the node name extension to schedule a pod. If the pods are not scheduling, ensure that the scheduler is available to auto-schedule the pod:

k get pods -n kube-system

Use the command above to get information about kube-system pods or the scheduler.

Labels & Selectors 🔍

Filtration with Labels & Selectors

Get pods with a specific label:

kubectl get pods --selector app=app1

Filter pods using multiple labels:

kubectl get pods --selector app=app1,env=prod,type=front-end

Filtering with Metadata Labels

In the below example, metadata labels are used only to filter the replica-set/deployments created. However, if you put the labels in the spec.template section, you will be able to filter the pods of those labels in your cluster. The selector section connects the replica set to the container in the spec section.

Annotations 📝

Annotations are used to store information that might be usable in some cases, such as phone numbers, email IDs, build numbers, etc.

Taints and Tolerations 🚫

Taints (Node Level)

There are three types of taints:

• NoSchedule: Do not schedule new pods to the node, but existing pods will remain.
• PreferNoSchedule: Prefer not to schedule pods, but it's not guaranteed.
• NoExecute: Do not schedule new pods to this tainted node, and existing pods will be evicted if they do not tolerate the taint.

To apply a taint:

kubectl taint nodes "node-name" app=blue:NoSchedule

Tolerations (Pod Level)

Values of tolerations must be in quotes:

tolerations:
- key: "app"
  operator: "Equal"
  value: "blue"
  effect: "NoSchedule"

Notes on Taints and Tolerations

Taints and tolerations do not guarantee that a pod will only schedule to the tainted node. They protect against unwanted pods scheduling to that node. For guaranteed scheduling, use NodeAffinity.

Removing a Taint

To remove a taint from a node:

kubectl taint nodes "node-name" app=blue:NoSchedule-

Node Selector & Label Nodes

Label nodes according to their capacity. For example, label a node as Large:

kubectl label nodes node56 Size=Large

Set the pod definition to use nodeSelector to schedule pods on the labeled node.

Node Affinity

Node affinity rules:

• requiredDuringScheduling: Only schedule the pod if the match expression is met.
• preferDuringScheduling: Prefer scheduling the pod according to the match expression.
• ignoreDuringExecution: Do nothing if node labels change in the future.
• RequiredDuringExecution: Evict or terminate pods if node labels change in the future.

Combining Taints and Node Affinity

To ensure that a pod schedules only to the desired node, use a combination of taints and node affinity.

Pod Resources 📊

YAML for Pod Resources & Limits

Pods cannot exceed their CPU limit as defined in the YAML file. However, they can exceed their memory limit, resulting in termination with an OOM (Out of Memory) error.

Namespace Level Resource Quota

Limit the resources for all pods in a namespace.

Editing Pods and Deployments ✏️

Editing a Pod

You cannot edit certain specifications of an existing pod. Editable fields include:

• spec.containers[*].image
• spec.initContainers[*].image
• spec.activeDeadlineSeconds
• spec.tolerations

To edit other properties, extract the pod definition, modify it, and recreate the pod.

Editing Deployments

With deployments, you can easily edit any field/property of the pod template. The deployment will automatically delete and create a new pod with the updated changes:

kubectl edit deployment my-deployment

DaemonSets 🌐

DaemonSets ensure that a pod runs on each node in the cluster. They are useful for monitoring and logging solutions, such as deploying fluentd in all pods for log collection.

Get information about DaemonSets:

kubectl get daemonsets/ds
k describe daemonset/ds abc123

Static Pods 🛡️

Static pods run standalone on any node in the cluster. Place the pod YAML file in /etc/kubernetes/manifests, and the kubelet will detect and manage the pod. Static pods are managed by the kubelet of the node and not the API server.

To find static pods, look for pods with node names at the end or check the ownership section in the pod's YAML.

Multiple Schedulers 🔄

Learn more about Kubernetes scheduling:

• Scheduling Code Hierarchy Overview (opens in a new tab)
• Advanced Scheduling in Kubernetes (opens in a new tab)
• How Does the Kubernetes Scheduler Work? (opens in a new tab)

Check scheduler events and logs:

kubectl get events -o wide
kubectl logs my-custom-scheduler -n=kubesystem