Solutions: Tasks 1–10

Each solution below follows a consistent structure: the fastest approach (imperative commands when available), the complete declarative YAML, verification commands with expected output, and a list of common mistakes that cost points on this task.

---

Solution 1 — Namespace with ResourceQuota

Time target: 4 minutes

Step 1: Create the namespace

kubectl create namespace quota-lab

Step 2: Create the ResourceQuota

The imperative approach does not cover all ResourceQuota fields. Use a YAML manifest:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: pod-memory-quota
  namespace: quota-lab
spec:
  hard:
    pods: "2"
    requests.memory: 1Gi
    limits.memory: 1Gi

Apply it:

kubectl apply -f quota.yaml

Step 3: Verify the quota

kubectl describe resourcequota pod-memory-quota -n quota-lab

Expected output:

Name:            pod-memory-quota
Namespace:       quota-lab
Resource         Used  Hard
--------         ----  ----
limits.memory    0     1Gi
pods             0     2
requests.memory  0     1Gi

Step 4: Test the quota enforcement

Create two Pods that fit within the quota:

kubectl run test-pod-1 --image=nginx:1.25 -n quota-lab \
  --overrides='{"spec":{"containers":[{"name":"test-pod-1","image":"nginx:1.25","resources":{"requests":{"memory":"256Mi"},"limits":{"memory":"256Mi"}}}]}}'

kubectl run test-pod-2 --image=nginx:1.25 -n quota-lab \
  --overrides='{"spec":{"containers":[{"name":"test-pod-2","image":"nginx:1.25","resources":{"requests":{"memory":"256Mi"},"limits":{"memory":"256Mi"}}}]}}'

Attempt a third Pod:

kubectl run test-pod-3 --image=nginx:1.25 -n quota-lab \
  --overrides='{"spec":{"containers":[{"name":"test-pod-3","image":"nginx:1.25","resources":{"requests":{"memory":"256Mi"},"limits":{"memory":"256Mi"}}}]}}'

The third Pod is rejected with: Error from server (Forbidden): pods "test-pod-3" is forbidden: exceeded quota: pod-memory-quota, requested: pods=1, used: pods=2, limited: pods=2.

Common Pitfalls

• Forgetting resource requests/limits on test Pods. When a ResourceQuota specifies requests.memory or limits.memory, every Pod in the namespace must declare those fields. A Pod without memory requests is rejected — not because of the quota limit, but because the quota requires the field to be set.
• String vs integer for the pods field. The pods value must be a string ("2") in YAML, though Kubernetes accepts integers in most cases. Use quotes to be safe.
• Wrong namespace. Applying the quota to default instead of quota-lab is a silent error — the quota works, but the test Pods go to the wrong namespace.

---

Solution 2 — Pod with Strict SecurityContext

Time target: 5 minutes

Step 1: Create the namespace

kubectl create namespace secure-app

Step 2: Create the Pod

This task requires a declarative manifest because the security context fields are too complex for kubectl run overrides:

apiVersion: v1
kind: Pod
metadata:
  name: locked-pod
  namespace: secure-app
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 3000
  containers:
  - name: nginx
    image: nginx:1.25
    securityContext:
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false
      capabilities:
        drop:
          - ALL
    volumeMounts:
    - name: cache-vol
      mountPath: /var/cache/nginx
    - name: run-vol
      mountPath: /var/run
  volumes:
  - name: cache-vol
    emptyDir: {}
  - name: run-vol
    emptyDir: {}

Apply:

kubectl apply -f locked-pod.yaml

Step 3: Verify the Pod is running

kubectl get pod locked-pod -n secure-app

Expected: STATUS is Running.

kubectl exec locked-pod -n secure-app -- id

Expected output: uid=1000 gid=3000 groups=3000

kubectl exec locked-pod -n secure-app -- touch /test-file 2>&1

Expected: touch: /test-file: Read-only file system — confirming readOnlyRootFilesystem is active.

Common Pitfalls

• Missing emptyDir volumes. Without writable mounts for /var/cache/nginx and /var/run, nginx fails to start because it cannot write to its cache directory or PID file. The Pod enters CrashLoopBackOff.
• Placing runAsNonRoot at the container level instead of Pod level. The task specifies Pod-level runAsNonRoot and container-level readOnlyRootFilesystem. Mixing up the levels is a common error.
• Forgetting allowPrivilegeEscalation: false. This field defaults to true when not specified. The task requires it explicitly set to false.
• Using drop: ["all"] instead of drop: ["ALL"]. The capability name is case-sensitive and must be uppercase ALL.

---

Solution 3 — Fix Broken Deployment

Time target: 4 minutes

Step 1: Create the namespace and apply the broken manifest

kubectl create namespace debug-deploy

Apply the broken Deployment from the task.

Step 2: Diagnose the errors

kubectl get pods -n debug-deploy

All Pods show ImagePullBackOff or ErrImagePull. The image name ngnix:1.25 is misspelled — it should be nginx:1.25.

Step 3: Fix Error 1 — Image name

kubectl set image deployment/broken-web web=nginx:1.25 -n debug-deploy

Step 4: Fix Error 2 — Readiness probe

After fixing the image, Pods start but remain NotReady because the readiness probe targets port 8080 path /healthz. Nginx listens on port 80 and serves its default page at /.

Edit the Deployment:

kubectl edit deployment broken-web -n debug-deploy

Change the readiness probe:

readinessProbe:
  httpGet:
    path: /
    port: 80
  initialDelaySeconds: 5
  periodSeconds: 5

Alternatively, patch it:

kubectl patch deployment broken-web -n debug-deploy --type='json' -p='[
  {"op": "replace", "path": "/spec/template/spec/containers/0/readinessProbe/httpGet/port", "value": 80},
  {"op": "replace", "path": "/spec/template/spec/containers/0/readinessProbe/httpGet/path", "value": "/"}
]'

Step 5: Verify

kubectl get pods -n debug-deploy

Expected: 2 Pods in Running state, READY column shows 1/1.

kubectl rollout status deployment/broken-web -n debug-deploy

Expected: deployment "broken-web" successfully rolled out.

Common Pitfalls

• Fixing only one error. The task has two distinct problems. Candidates who fix the image but skip the probe end up with running-but-not-ready Pods, which still fail the task.
• Using kubectl edit for the image fix. While functional, kubectl set image is faster and less error-prone. Save kubectl edit for fields that lack imperative equivalents.
• Changing the readiness probe to a TCP probe. The task says to fix the HTTP probe, not replace it with a different type. Changing to tcpSocket works functionally but does not match the requirement.

---

Solution 4 — Canary Deployment

Time target: 5 minutes

Step 1: Create the namespace

kubectl create namespace canary-ns

Step 2: Create the stable Deployment

kubectl create deployment web-stable --image=nginx:1.24 --replicas=4 -n canary-ns --dry-run=client -o yaml > web-stable.yaml

Edit web-stable.yaml to ensure the Pod template labels include both app: web and track: stable:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-stable
  namespace: canary-ns
spec:
  replicas: 4
  selector:
    matchLabels:
      app: web
      track: stable
  template:
    metadata:
      labels:
        app: web
        track: stable
    spec:
      containers:
      - name: nginx
        image: nginx:1.24
        ports:
        - containerPort: 80

kubectl apply -f web-stable.yaml

Step 3: Create the Service

The Service selector must match only app: web — not track. This is what allows both stable and canary Pods to receive traffic:

kubectl expose deployment web-stable --name=web-svc --port=80 --target-port=80 -n canary-ns --dry-run=client -o yaml > web-svc.yaml

Edit web-svc.yaml to remove track: stable from the selector (keep only app: web):

apiVersion: v1
kind: Service
metadata:
  name: web-svc
  namespace: canary-ns
spec:
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80

kubectl apply -f web-svc.yaml

Step 4: Create the canary Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-canary
  namespace: canary-ns
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
      track: canary
  template:
    metadata:
      labels:
        app: web
        track: canary
    spec:
      containers:
      - name: nginx
        image: nginx:1.25
        ports:
        - containerPort: 80

kubectl apply -f web-canary.yaml

Step 5: Verify

kubectl get endpoints web-svc -n canary-ns

The endpoint list must contain 5 IP addresses (4 from web-stable + 1 from web-canary).

kubectl get pods -n canary-ns --show-labels

Four Pods show track=stable, one shows track=canary, and all five show app=web.

Common Pitfalls

• Including track in the Service selector. If the Service selector is app: web, track: stable, the canary Pods are excluded. The canary pattern requires the Service to select on a common label shared by both Deployments.
• Using kubectl expose without modifying the selector. kubectl expose deployment web-stable copies the Deployment’s labels into the Service selector, including track: stable. You must manually edit the selector to remove track.
• Mismatched labels between the Deployment selector and Pod template. The Deployment’s selector.matchLabels must be a subset of the Pod template’s labels.

---

Solution 5 — CronJob and Job Extraction

Time target: 4 minutes

Step 1: Create the namespace

kubectl create namespace batch-ns

Step 2: Create the CronJob

Imperative approach:

kubectl create cronjob log-timestamp \
  --image=busybox:1.36 \
  --schedule="*/5 * * * *" \
  --restart=Never \
  -n batch-ns \
  -- sh -c "echo Timestamp: \$(date) && sleep 5"

Then patch for history limits — or use a declarative manifest from the start:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: log-timestamp
  namespace: batch-ns
spec:
  schedule: "*/5 * * * *"
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: logger
            image: busybox:1.36
            command: ["sh", "-c", "echo Timestamp: $(date) && sleep 5"]
          restartPolicy: Never

kubectl apply -f cronjob.yaml

Step 3: Create a Job from the CronJob

kubectl create job log-now --from=cronjob/log-timestamp -n batch-ns

This creates a one-off Job using the CronJob’s template without waiting for the next scheduled trigger.

Step 4: Verify the Job

kubectl wait --for=condition=complete job/log-now -n batch-ns --timeout=60s
kubectl logs job/log-now -n batch-ns

Expected output: Timestamp: <current date/time>

Step 5: Confirm the CronJob still exists

kubectl get cronjob log-timestamp -n batch-ns

The CronJob must show its schedule and remain active.

Common Pitfalls

• Forgetting restartPolicy: Never. CronJob templates require either Never or OnFailure. If you omit it, the default is Always, which is invalid for Jobs and produces an API error.
• Using \$(date) vs $(date). In a YAML manifest, $(date) inside a command string is interpreted by the container’s shell at runtime, which is correct. In a kubectl imperative command, you must escape the $ to prevent your local shell from expanding it before the command reaches Kubernetes.
• Not specifying history limits. While the CronJob works without explicit history limits, the task requires successfulJobsHistoryLimit: 3 and failedJobsHistoryLimit: 1. Missing them loses partial credit.

---

Solution 6 — Pod with Secret (env) and ConfigMap (volume)

Time target: 5 minutes

Step 1: Create the namespace

kubectl create namespace config-ns

Step 2: Create the Secret

kubectl create secret generic db-credentials \
  --from-literal=DB_USER=admin \
  --from-literal=DB_PASS=exam-secret-2026 \
  -n config-ns

Step 3: Create the ConfigMap

kubectl create configmap app-config \
  --from-literal=app.properties="log.level=INFO
cache.ttl=300
feature.flag=true" \
  -n config-ns

Alternatively, create a file first and use --from-file:

cat <<EOF > app.properties
log.level=INFO
cache.ttl=300
feature.flag=true
EOF

kubectl create configmap app-config --from-file=app.properties -n config-ns

The --from-file approach is cleaner because it preserves the key name as the filename (app.properties).

Step 4: Create the Pod

apiVersion: v1
kind: Pod
metadata:
  name: config-consumer
  namespace: config-ns
spec:
  containers:
  - name: app
    image: busybox:1.36
    command: ["sh", "-c", "echo DB_USER=$DB_USER DB_PASS=$DB_PASS && cat /config/app.properties && sleep 3600"]
    env:
    - name: DB_USER
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: DB_USER
    - name: DB_PASS
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: DB_PASS
    volumeMounts:
    - name: config-volume
      mountPath: /config
  volumes:
  - name: config-volume
    configMap:
      name: app-config

kubectl apply -f config-consumer.yaml

Step 5: Verify

kubectl exec config-consumer -n config-ns -- env | grep DB_

Expected:

DB_USER=admin
DB_PASS=exam-secret-2026

kubectl exec config-consumer -n config-ns -- cat /config/app.properties

Expected:

log.level=INFO
cache.ttl=300
feature.flag=true

Common Pitfalls

• Using envFrom with secretRef instead of individual secretKeyRef. The task specifies injecting each key individually with secretKeyRef. Using envFrom works functionally but does not match the requirement.
• ConfigMap key mismatch. If you create the ConfigMap with --from-literal=config=..., the mounted file is named config, not app.properties. The cat /config/app.properties verification command fails.
• Forgetting the volume mount. Defining the volume in spec.volumes but not mounting it in spec.containers[].volumeMounts means the ConfigMap data is never accessible inside the container.

---

Solution 7 — NetworkPolicy: Deny All Then Allow by Namespace

Time target: 5 minutes

Step 1: Create namespaces and label

kubectl create namespace netpol-ns
kubectl create namespace trusted-ns
kubectl label namespace trusted-ns purpose=trusted

Step 2: Create the target Pod

kubectl run web-server --image=nginx:1.25 -n netpol-ns --labels=app=web-server

Step 3: Create the deny-all ingress NetworkPolicy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
  namespace: netpol-ns
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress: []

kubectl apply -f deny-all.yaml

The empty ingress: [] list means no ingress traffic is allowed to any Pod in netpol-ns.

Step 4: Create the allow-from-trusted NetworkPolicy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-from-trusted
  namespace: netpol-ns
spec:
  podSelector:
    matchLabels:
      app: web-server
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          purpose: trusted
    ports:
    - protocol: TCP
      port: 80

kubectl apply -f allow-trusted.yaml

Step 5: Verify (conceptual)

The deny-all policy blocks all ingress. The allow-from-trusted policy adds an exception for Pods in namespaces labeled purpose: trusted to reach web-server on port 80. Both policies coexist — NetworkPolicies are additive. If any policy allows traffic, it passes.

kubectl describe networkpolicy -n netpol-ns

Two policies must appear: deny-all-ingress and allow-from-trusted.

Common Pitfalls

• Omitting policyTypes: ["Ingress"] on the deny-all policy. Without policyTypes, the policy type is inferred from the presence of ingress or egress rules. An empty ingress: [] with no policyTypes field creates ambiguity. Always specify policyTypes explicitly.
• Using podSelector instead of namespaceSelector in the allow rule. The task requires allowing traffic from a specific namespace, not from specific Pods. Using podSelector alone (without namespaceSelector) matches Pods in the same namespace as the policy.
• Placing the allow policy in trusted-ns instead of netpol-ns. NetworkPolicies apply to the namespace they are created in. The allow policy must be in netpol-ns because it governs ingress to Pods in that namespace.
• Combining namespaceSelector and podSelector in the same from entry without understanding the AND/OR semantics. A single from entry with both selectors means AND (both must match). Separate from entries mean OR. The task requires only namespaceSelector.

---

Solution 8 — StatefulSet with Headless Service and volumeClaimTemplates

Time target: 6 minutes

Step 1: Create the namespace

kubectl create namespace stateful-ns

Step 2: Create the headless Service

apiVersion: v1
kind: Service
metadata:
  name: db-headless
  namespace: stateful-ns
spec:
  clusterIP: None
  selector:
    app: db
  ports:
  - port: 5432
    targetPort: 5432

kubectl apply -f db-headless.yaml

Step 3: Create the StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: db
  namespace: stateful-ns
spec:
  serviceName: db-headless
  replicas: 3
  selector:
    matchLabels:
      app: db
  template:
    metadata:
      labels:
        app: db
    spec:
      containers:
      - name: postgres
        image: postgres:16
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_PASSWORD
          value: exam-pass
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 1Gi

kubectl apply -f db-statefulset.yaml

Step 4: Wait for all Pods

kubectl rollout status statefulset/db -n stateful-ns --timeout=120s

Or watch directly:

kubectl get pods -n stateful-ns -w

Pods appear in order: db-0 first, then db-1, then db-2. Each Pod must be Running before the next one starts.

Step 5: Verify DNS

kubectl exec db-0 -n stateful-ns -- nslookup db-headless.stateful-ns.svc.cluster.local

This resolves to the IP addresses of all three Pods. Individual Pods are addressable as db-0.db-headless.stateful-ns.svc.cluster.local.

Step 6: Verify PVCs

kubectl get pvc -n stateful-ns

Expected:

NAME        STATUS   VOLUME   CAPACITY   ACCESS MODES
data-db-0   Bound    ...      1Gi        RWO
data-db-1   Bound    ...      1Gi        RWO
data-db-2   Bound    ...      1Gi        RWO

Common Pitfalls

• Forgetting to create the headless Service before the StatefulSet. The StatefulSet’s serviceName references the headless Service. While the StatefulSet still creates Pods without the Service, DNS resolution does not work until the Service exists.
• Setting clusterIP to a value other than None. A headless Service requires clusterIP: None. Any other value creates a regular ClusterIP Service, breaking the StatefulSet DNS pattern.
• Missing POSTGRES_PASSWORD environment variable. The postgres:16 image requires this variable. Without it, the container exits with an error message about missing authentication configuration.
• Mismatched volumeClaimTemplates name and volumeMounts name. The name field in volumeClaimTemplates[].metadata.name must exactly match the volumeMounts[].name in the container spec.

---

Solution 9 — Debug: Pod Returns 503

Time target: 4 minutes

Step 1: Create namespace and apply the broken manifest

kubectl create namespace debug-svc

Apply the Pod and Service from the task.

Step 2: Diagnose the problem

Check the Pod status:

kubectl get pod api-server -n debug-svc

The Pod is Running and Ready. The issue is with the Service, not the Pod.

Check the Service endpoints:

kubectl get endpoints api-svc -n debug-svc

If the Pod is Ready, the endpoint should show the Pod IP — but with the wrong targetPort. The http-echo container listens on port 5678, but the Service targetPort is 8080.

Step 3: Fix the Service targetPort

kubectl patch service api-svc -n debug-svc --type='json' -p='[{"op":"replace","path":"/spec/ports/0/targetPort","value":5678}]'

Or edit directly:

kubectl edit service api-svc -n debug-svc

Change targetPort: 8080 to targetPort: 5678.

Step 4: Verify the fix

kubectl get endpoints api-svc -n debug-svc

The endpoint must show the Pod IP with port 5678.

kubectl run curl-test --image=curlimages/curl --rm -it --restart=Never -n debug-svc -- curl http://api-svc.debug-svc.svc.cluster.local

Expected output: healthy

Common Pitfalls

• Assuming the readiness probe is the problem. The http-echo image returns 200 on all paths, so the /ready path in the readiness probe actually works. The Pod becomes Ready. The real issue is the Service’s targetPort mismatch.
• Changing the Service port instead of targetPort. The port is the Service’s external port (what clients connect to). The targetPort is what the Service forwards traffic to on the Pod. The container listens on 5678, so targetPort must be 5678.
• Forgetting to verify with an actual curl test. Checking endpoints alone confirms connectivity plumbing, but a curl test confirms end-to-end data flow.

---

Solution 10 — Helm: Install, Upgrade, Rollback

Time target: 5 minutes

Step 1: Create the namespace

kubectl create namespace helm-ns

Step 2: Add the Bitnami repository

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

If the repo is already added, the add command prints a warning. That is fine — helm repo update ensures you have the latest index.

Step 3: Install the release

helm install my-nginx bitnami/nginx -n helm-ns --set replicaCount=1

Step 4: Verify installation

helm list -n helm-ns

Expected: one entry for my-nginx with status deployed and revision 1.

kubectl get deployment -n helm-ns

The nginx Deployment should show 1/1 ready replicas.

Step 5: Upgrade to 3 replicas

helm upgrade my-nginx bitnami/nginx -n helm-ns --set replicaCount=3

Step 6: Verify upgrade

kubectl get deployment -n helm-ns

Expected: 3/3 ready replicas.

helm history my-nginx -n helm-ns

Expected: two revisions.

REVISION  STATUS      DESCRIPTION
1         superseded  Install complete
2         deployed    Upgrade complete

Step 7: Rollback to revision 1

helm rollback my-nginx 1 -n helm-ns

Step 8: Verify rollback

kubectl get deployment -n helm-ns

Expected: 1/1 ready replicas (back to revision 1 settings).

helm history my-nginx -n helm-ns

Expected: three revisions.

REVISION  STATUS      DESCRIPTION
1         superseded  Install complete
2         superseded  Upgrade complete
3         deployed    Rollback to 1

Common Pitfalls

• Forgetting -n helm-ns. Without the namespace flag, Helm installs the release in the default namespace. Every Helm command must include -n helm-ns.
• Using helm upgrade without --set or -f. If you run helm upgrade my-nginx bitnami/nginx -n helm-ns without specifying values, the chart’s default values override your previous --set overrides. Always re-specify values you want to keep, or use --reuse-values (though --reuse-values has limitations with chart version changes).
• Confusing revision numbers. A rollback creates a new revision, not a revert. After rollback, the history shows 3 entries, and the current state matches revision 1 but carries revision number 3. The helm history output confirms this.
• Not running helm repo update before install. Without an updated index, Helm may fail to find the chart or use a stale version. Always run helm repo update after adding a repository.