Health Checks: Readiness, Liveness, and Startup Probes

Kubernetes probes help the platform decide when a workload is started, when it is ready for traffic, and when a container should be restarted.

Kubernetes Field Notes · Operations Level · Estimated reading time: 26 minutes

What this lesson covers

Kubernetes can restart containers, remove unhealthy Pods from Service traffic, and delay traffic until an application is actually ready. It does this through probes.

By the end of this lesson, you should understand:

• What liveness probes are
• What readiness probes are
• What startup probes are
• Why readiness is different from liveness
• How probes affect Services and traffic routing
• How to create HTTP-based probes
• How to intentionally break a probe
• How to troubleshoot probe failures
• How probes fit into production workload design

The short version

Kubernetes probes are health checks performed by the kubelet on the node running the Pod.

startupProbe:
  Has the application finished starting?

readinessProbe:
  Is the application ready to receive traffic?

livenessProbe:
  Is the application healthy enough to keep running?

These checks help Kubernetes make safer decisions about restarts and traffic routing.

Readiness vs liveness

Readiness and liveness are often confused, but they answer different questions.

Readiness probe

A readiness probe tells Kubernetes whether the Pod should receive traffic from a Service.

If readiness fails, the Pod can keep running, but Kubernetes removes it from Service endpoints until it becomes ready again.

Liveness probe

A liveness probe tells Kubernetes whether the container should be restarted.

If liveness fails repeatedly, Kubernetes restarts the container.

What startup probes solve

Some applications take longer to start than others. If a liveness probe starts too early, Kubernetes may restart a slow-starting application before it has a fair chance to finish initializing.

A startup probe gives the application a separate startup window. While the startup probe is running, liveness and readiness checks are delayed.

Startup probes are useful for:

• Legacy applications with long boot times
• Applications that warm caches
• Applications that run startup migrations
• Containers that need time to initialize dependencies

Common probe types

Kubernetes supports several probe methods:

• HTTP GET: Kubernetes requests an HTTP endpoint.
• TCP socket: Kubernetes checks whether a port is accepting connections.
• Exec: Kubernetes runs a command inside the container.
• gRPC: Kubernetes can check a gRPC health endpoint when configured.

This lab uses HTTP GET probes because they are easy to understand and common for web applications.

Before you start the lab

This lab assumes Docker Desktop is running, your kind cluster exists, kubectl is installed, and your current context is kind-stealth-k8s-lab.

kubectl config current-context
kubectl get nodes

If your cluster no longer exists, recreate it:

kind create cluster --name stealth-k8s-lab

Step 1: Create a Namespace for the lab

kubectl create namespace probes-lab

Step 2: Create a Deployment with readiness and liveness probes

Create a file named:

probes-deployment-lab.yaml

Paste this into the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: probes-web-lab
  namespace: probes-lab
spec:
  replicas: 2
  selector:
    matchLabels:
      app: probes-web-lab
  template:
    metadata:
      labels:
        app: probes-web-lab
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 3
          periodSeconds: 5
          failureThreshold: 3
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 10
          periodSeconds: 10
          failureThreshold: 3
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "256Mi"

Apply it:

kubectl apply -f probes-deployment-lab.yaml

Check the Deployment and Pods:

kubectl get deployments -n probes-lab
kubectl get pods -n probes-lab

Step 3: Describe a Pod and inspect the probes

Get a Pod name:

kubectl get pods -n probes-lab

Describe one Pod:

kubectl describe pod <pod-name> -n probes-lab

Look for the Liveness and Readiness lines. You should see the HTTP path, port, initial delay, timeout, period, success threshold, and failure threshold.

Step 4: Create a Service

Create a Service for the Deployment:

kubectl expose deployment probes-web-lab --port=80 --target-port=80 --type=ClusterIP -n probes-lab

Check endpoints:

kubectl get endpoints probes-web-lab -n probes-lab

Ready Pods should appear as endpoints behind the Service.

Step 5: Test the Service

Port-forward to the Service:

kubectl port-forward -n probes-lab service/probes-web-lab 8080:80

Open:

http://localhost:8080

You should see the nginx welcome page. Press Ctrl+C when done.

Step 6: Break readiness on purpose

Create a file named:

broken-readiness-lab.yaml

Paste this into the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: broken-readiness-lab
  namespace: probes-lab
spec:
  replicas: 1
  selector:
    matchLabels:
      app: broken-readiness-lab
  template:
    metadata:
      labels:
        app: broken-readiness-lab
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
        readinessProbe:
          httpGet:
            path: /does-not-exist
            port: 80
          initialDelaySeconds: 3
          periodSeconds: 5
          failureThreshold: 2

Apply it:

kubectl apply -f broken-readiness-lab.yaml

Check the Pod:

kubectl get pods -n probes-lab

The Pod may be Running but not Ready.

Step 7: Troubleshoot the readiness failure

Describe the broken Pod:

kubectl describe pod <broken-pod-name> -n probes-lab

Look at the events. You should see readiness probe failures because the probe checks a path that returns an error.

This demonstrates that readiness affects traffic routing without necessarily restarting the container.

Step 8: Check endpoints for the broken workload

Create a Service for the broken Deployment:

kubectl expose deployment broken-readiness-lab --port=80 --target-port=80 --type=ClusterIP -n probes-lab

Check endpoints:

kubectl get endpoints broken-readiness-lab -n probes-lab

Because the Pod is not Ready, the Service should not have ready endpoints for it.

Step 9: Break liveness on purpose

Create a file named:

broken-liveness-lab.yaml

Paste this into the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: broken-liveness-lab
  namespace: probes-lab
spec:
  replicas: 1
  selector:
    matchLabels:
      app: broken-liveness-lab
  template:
    metadata:
      labels:
        app: broken-liveness-lab
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
        livenessProbe:
          httpGet:
            path: /does-not-exist
            port: 80
          initialDelaySeconds: 5
          periodSeconds: 5
          failureThreshold: 2

Apply it:

kubectl apply -f broken-liveness-lab.yaml

Watch the Pod:

kubectl get pods -n probes-lab -w

Press Ctrl+C after you see restarts increase.

Step 10: Troubleshoot the liveness failure

Describe the Pod:

kubectl describe pod <broken-liveness-pod-name> -n probes-lab

Look for liveness probe failure events and restart count. Kubernetes restarts the container because the liveness probe keeps failing.

Step 11: Add a startup probe example

Create a file named:

startup-probe-lab.yaml

Paste this into the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: startup-probe-lab
  namespace: probes-lab
spec:
  replicas: 1
  selector:
    matchLabels:
      app: startup-probe-lab
  template:
    metadata:
      labels:
        app: startup-probe-lab
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
        startupProbe:
          httpGet:
            path: /
            port: 80
          failureThreshold: 30
          periodSeconds: 2
        readinessProbe:
          httpGet:
            path: /
            port: 80
          periodSeconds: 5
        livenessProbe:
          httpGet:
            path: /
            port: 80
          periodSeconds: 10

Apply it:

kubectl apply -f startup-probe-lab.yaml

Describe the Pod and look for the startup, readiness, and liveness probe sections:

kubectl get pods -n probes-lab
kubectl describe pod <startup-pod-name> -n probes-lab

Step 12: Clean up

Delete the lab Namespace and everything inside it:

kubectl delete namespace probes-lab

Confirm cleanup:

kubectl get namespaces
kubectl get pods -A

Common probe troubleshooting commands

kubectl get pods -n <namespace>
kubectl describe pod <pod-name> -n <namespace>
kubectl logs <pod-name> -n <namespace>
kubectl get endpoints <service-name> -n <namespace>
kubectl get events -n <namespace> --sort-by=.metadata.creationTimestamp
kubectl rollout status deployment/<deployment-name> -n <namespace>

Common probe mistakes

Using liveness for dependency failures

If a database or downstream API is temporarily unavailable, restarting the app may make the outage worse. Use readiness to stop traffic when dependencies are unavailable.

Setting delays too aggressively

Probes that start too soon can cause unnecessary failures during startup.

Checking the wrong endpoint

A probe endpoint should reflect the health question being asked. A shallow “server is up” endpoint is different from a deep dependency check.

Making probes expensive

Health endpoints should be cheap. If every probe performs heavy database queries, the health check itself can become load.

Where this fits in real clusters

Probes are part of workload reliability. In production, teams usually define dedicated endpoints such as:

• /startupz: startup complete
• /readyz: ready to receive traffic
• /healthz: process is healthy enough to keep running

The exact names do not matter as much as the behavior. Each endpoint should answer a clear operational question.

What you just learned

• Created Deployments with readiness and liveness probes
• Verified that ready Pods appear behind Services
• Broke readiness and saw that traffic routing is affected
• Broke liveness and saw container restarts
• Created a startup probe example
• Used describe output and events to troubleshoot probe failures

Check your understanding

1. What does a readiness probe control?
2. What happens when a liveness probe repeatedly fails?
3. What problem does a startup probe solve?
4. Can a Pod be Running but not Ready?
5. Why should liveness not be used for every dependency failure?
6. What command shows probe failure events?
7. What command shows whether a Service has ready endpoints?
8. What is one common HTTP path used for health checks?
9. Why should health endpoints be cheap?
10. Which probe should usually fail when an app should temporarily stop receiving traffic?
11. Which probe should usually fail when restarting the container is the right action?
12. What is the main difference between readiness and liveness?