Skip to main content
Back to Library
mastering ckad certified kubernetes application developer

NetworkPolicy and Debugging Solutions

7 min read Chapter 39 of 87
Summary

Step-by-step solutions for Exercises 3 and 4: creating...

Step-by-step solutions for Exercises 3 and 4: creating a NetworkPolicy that allows only frontend Pods to reach backend Pods on port 80, and diagnosing and fixing a Service with a wrong label selector.

NetworkPolicy and Debugging Solutions

Solution: Exercise 3 — NetworkPolicy: Frontend to Backend Isolation

This exercise requires creating a namespace with backend and frontend Pods, applying a NetworkPolicy that allows only frontend-to-backend traffic on port 80, and verifying that an unauthorized Pod is blocked.

CNI Note: NetworkPolicies require a CNI plugin that supports them. If you are using Kind’s default kindnet CNI, NetworkPolicies may not be enforced. For full functionality, install Calico:

kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

Wait for Calico Pods to become ready before proceeding.

Step 1: Create the Namespace

kubectl create namespace netpol-lab

Step 2: Deploy the Backend Pod

cat > backend.yaml << 'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: backend
  namespace: netpol-lab
  labels:
    tier: backend
spec:
  containers:
    - name: nginx
      image: nginx:1.25
      ports:
        - containerPort: 80
EOF
kubectl apply -f backend.yaml

Verify the backend is running:

kubectl get pod -n netpol-lab backend

Expected output:

NAME      READY   STATUS    RESTARTS   AGE
backend   1/1     Running   0          5s

Step 3: Deploy the Frontend Pod

cat > frontend.yaml << 'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: frontend
  namespace: netpol-lab
  labels:
    tier: frontend
spec:
  containers:
    - name: busybox
      image: busybox:1.36
      command: ["sleep", "3600"]
EOF
kubectl apply -f frontend.yaml

Step 4: Verify Connectivity Before the Policy

Before applying any NetworkPolicy, confirm that the frontend can reach the backend:

kubectl exec -n netpol-lab frontend -- wget -qO- --timeout=3 http://backend.netpol-lab:80

Expected output: the default NGINX welcome page HTML. This confirms baseline connectivity.

Get the backend Pod’s IP for direct testing:

BACKEND_IP=$(kubectl get pod -n netpol-lab backend -o jsonpath='{.status.podIP}')
echo "Backend IP: $BACKEND_IP"

Step 5: Apply the NetworkPolicy

cat > netpol.yaml << 'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-allow-frontend
  namespace: netpol-lab
spec:
  podSelector:
    matchLabels:
      tier: backend
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              tier: frontend
      ports:
        - protocol: TCP
          port: 80
EOF
kubectl apply -f netpol.yaml

This policy:

  • Targets: Pods with tier: backend in namespace netpol-lab
  • Allows ingress from: Pods with tier: frontend (same namespace) on TCP port 80
  • Denies: All other ingress traffic to backend Pods

The policyTypes: [Ingress] combined with a single ingress rule means any traffic not matching the rule is denied. Egress from backend Pods is not restricted by this policy.

Step 6: Verify Frontend Access

kubectl exec -n netpol-lab frontend -- wget -qO- --timeout=3 http://$BACKEND_IP:80

Expected output: the NGINX welcome page HTML. The frontend Pod has label tier: frontend, which matches the ingress rule’s podSelector.

Step 7: Deploy and Test the Intruder Pod

kubectl run intruder -n netpol-lab --image=busybox:1.36 --restart=Never -- sleep 3600

This Pod has no tier: frontend label. Test its access to the backend:

kubectl exec -n netpol-lab intruder -- wget -qO- --timeout=3 http://$BACKEND_IP:80

Expected output:

wget: download timed out
command terminated with exit code 1

The connection times out because the intruder Pod does not match the NetworkPolicy’s ingress rule. The backend Pod’s ingress is restricted to only Pods with tier: frontend.

Step 8: Verify the Policy Configuration

kubectl describe networkpolicy -n netpol-lab backend-allow-frontend

Expected output includes:

PodSelector:     tier=backend
Allowing ingress traffic:
  To Port: 80/TCP
  From:
    PodSelector: tier=frontend
Not affecting egress traffic

Troubleshooting

SymptomCauseFix
Intruder can still reach backendCNI does not enforce NetworkPoliciesInstall Calico or Cilium and restart the test
Frontend also blockedPolicy podSelector mismatchVerify frontend Pod has label tier=frontend: kubectl get pod frontend -n netpol-lab --show-labels
Timeout on DNS resolutionDefault deny also blocking DNSEnsure no deny-all egress policy exists, or add a DNS egress allow rule
Policy exists but has no effectNamespace mismatchVerify the policy is in netpol-lab: kubectl get netpol -n netpol-lab

Cleanup

kubectl delete namespace netpol-lab

Deleting the namespace removes all resources within it — Pods, Services, and NetworkPolicies.

Solution: Exercise 4 — Debug a Broken Service

This exercise requires creating a Deployment and a Service with an intentionally wrong selector, diagnosing the problem, and fixing it.

Step 1: Create the Deployment

kubectl create deployment debug-app --image=nginx:1.25 --replicas=2

Verify the Pods are running and check their labels:

kubectl get pods -l app=debug-app --show-labels

Expected output:

NAME                         READY   STATUS    RESTARTS   AGE   LABELS
debug-app-6d9f8b7c5-abc12   1/1     Running   0          5s    app=debug-app,pod-template-hash=6d9f8b7c5
debug-app-6d9f8b7c5-def34   1/1     Running   0          5s    app=debug-app,pod-template-hash=6d9f8b7c5

Note the label: app=debug-app.

Step 2: Create the Broken Service

cat > broken-svc.yaml << 'EOF'
apiVersion: v1
kind: Service
metadata:
  name: debug-svc
spec:
  type: ClusterIP
  selector:
    app: debug-wrong
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
EOF
kubectl apply -f broken-svc.yaml

The selector app: debug-wrong does not match the Pods’ label app: debug-app.

Step 3: Observe the Symptom

Try to reach the Service:

kubectl run curltest --image=busybox:1.36 --rm -it --restart=Never -- wget -qO- --timeout=3 http://debug-svc

Expected output:

wget: download timed out
command terminated with exit code 1

The Service exists but routes to nothing.

Step 4: Diagnose the Problem

Check Endpoints — this is always the first diagnostic step:

kubectl get endpoints debug-svc

Expected output:

NAME        ENDPOINTS   AGE
debug-svc   <none>      15s

<none> means no Pods match the Service’s selector. This is the root cause indicator.

Compare the Service selector with Pod labels:

kubectl describe svc debug-svc | grep Selector

Output:

Selector:          app=debug-wrong
kubectl get pods --show-labels | grep debug-app

Output:

debug-app-6d9f8b7c5-abc12   1/1     Running   0          30s   app=debug-app,...
debug-app-6d9f8b7c5-def34   1/1     Running   0          30s   app=debug-app,...

The mismatch is clear: Service selector says app=debug-wrong, Pods have app=debug-app.

Step 5: Fix the Service

Edit the Service to correct the selector:

kubectl patch svc debug-svc -p '{"spec":{"selector":{"app":"debug-app"}}}'

Alternatively, edit the YAML and re-apply:

cat > fixed-svc.yaml << 'EOF'
apiVersion: v1
kind: Service
metadata:
  name: debug-svc
spec:
  type: ClusterIP
  selector:
    app: debug-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
EOF
kubectl apply -f fixed-svc.yaml

Step 6: Verify the Fix

Check Endpoints immediately:

kubectl get endpoints debug-svc

Expected output:

NAME        ENDPOINTS                       AGE
debug-svc   10.244.1.3:80,10.244.2.5:80    45s

Endpoints are now populated with the Pod IPs. Test connectivity:

kubectl run curltest --image=busybox:1.36 --rm -it --restart=Never -- wget -qO- --timeout=3 http://debug-svc

Expected output: the default NGINX welcome page HTML, confirming the Service now routes traffic to the backend Pods.

The Diagnostic Pattern

This exercise demonstrates a systematic approach to Service debugging that applies to the exam:

  1. Check Endpoints: kubectl get ep <service-name> — if empty, the selector is wrong.
  2. Compare selectors: kubectl describe svc vs. kubectl get pods --show-labels.
  3. Fix the mismatch: Patch or re-apply the Service with the correct selector.
  4. Verify: Endpoints populate, connectivity restored.

Memorize this four-step sequence. On the CKAD exam, broken Services with wrong selectors are a recurring pattern. The entire diagnosis takes under 30 seconds once you know where to look.

Troubleshooting

SymptomCauseFix
Endpoints populated but still getting timeoutWrong targetPortVerify the container listens on the port specified in targetPort
kubectl patch returns errorJSON syntax issueEnsure proper JSON: '{"spec":{"selector":{"app":"debug-app"}}}'
Only one endpoint despite 2 replicasOne Pod not runningCheck kubectl get pods -l app=debug-app for non-Ready Pods
DNS does not resolve debug-svcService in different namespaceSpecify namespace: debug-svc.default or deploy test Pod in same namespace

Cleanup

kubectl delete deployment debug-app
kubectl delete svc debug-svc