# Kubernetes for AI, in reading order

> Kubernetes for AI workloads in reading order: the 1.36 and 1.37 upgrades, GPUs with DRA and gang scheduling, Gateway API inference routing and agents.

URL: https://www.matthewswong.com/en/ai-devops/kubernetes-for-ai/

- Author: Matthews Wong (https://www.matthewswong.com/en/about/)
- Reviewed on: 17 October 2026

> **Key takeaway:** This path starts with what the 1.36 and 1.37 releases change for platform teams, then covers GPU allocation with DRA, gang scheduling and the KAI and Volcano schedulers, then the move from Ingress to Gateway API and its inference extension, and ends with agents on the cluster: the Agent Sandbox resource, kagent, and read-only MCP access.

The articles on this site about running AI workloads on Kubernetes, arranged as one path: upgrade the cluster, give it GPUs it can schedule, route model traffic through Gateway API, then let agents work inside it safely.

## Read in this order

Each stage builds on the one before. If you already know a stage, skip ahead; the line under each title says what that article adds.

### Stages 1: Upgrade first: what 1.36 and 1.37 change

The release changes that land on every cluster before any AI feature does.

1. [Kubernetes 1.37 Release: What Breaks When Upgrading From 1.36](https://www.matthewswong.com/en/blog/kubernetes-1-37-garhwal-whats-new/): The 1.37 overview for platform teams: defaults that flip on, deprecation dates and a preflight script.
2. [Kubernetes 1.36 Security Changes: userns, kubelet, gitRepo](https://www.matthewswong.com/en/blog/kubernetes-1-36-user-namespaces-kubelet-authz-gitrepo-removed/): The 1.36 security changes: user namespaces GA, fine-grained kubelet authorization and gitRepo removed, with an audit script.
3. [Kubernetes Manifest-Based Admission Control Beta in 1.37](https://www.matthewswong.com/en/blog/kubernetes-manifest-based-admission-control-beta/): Admission webhooks and CEL policies loaded from disk, so they still apply when etcd is down.
4. [containerd runAsNonRoot Bypass: CVE-2026-46680 Fix and Audit](https://www.matthewswong.com/en/blog/containerd-cve-2026-46680-runasnonroot-bypass/): A crafted image could defeat runAsNonRoot; the fixed containerd versions, the Pod mitigation and a cluster audit.
5. [KYAML in Kubernetes 1.37: Stable YAML Subset, kubectl -o kyaml](https://www.matthewswong.com/en/blog/kyaml-kubernetes-stable-yaml-format/): KYAML is stable: what the strict YAML subset enforces and how to use it in CI.
6. [kube-proxy ipvs Deprecated: Migrate to nftables by v1.40](https://www.matthewswong.com/en/blog/kube-proxy-ipvs-deprecation-nftables-migration/): kube-proxy's ipvs mode is deprecated; a sourced nftables migration plan with kernel checks.

### Stages 2: GPUs: DRA, gang scheduling and AI conformance

How a cluster hands out accelerators, starts multi-pod jobs together, and proves it is ready for AI work.

7. [Kubernetes DRA GPU Fallback: Prioritized List in 1.36](https://www.matthewswong.com/en/blog/kubernetes-1-36-dra-prioritized-list-gpu-fallback/): Ask for one GPU type and fall back to another with the DRA prioritized list, plus MIG partitions and device taints.
8. [Kubernetes 1.37 DRA Extended Resources GA: Run example.com/gpu Without a Device Plugin](https://www.matthewswong.com/en/blog/kubernetes-1-37-dra-extended-resources-ga/): Leave the device plugin without rewriting manifests: map one DeviceClass and migrate nodes one by one.
9. [Kubernetes Gang Scheduling Beta in 1.37: What Works, What Not](https://www.matthewswong.com/en/blog/kubernetes-1-37-gang-scheduling-beta-ai-training/): Native gang scheduling for training jobs: the Workload and PodGroup APIs, and what still needs Kueue or Volcano.
10. [Volcano Scheduler GPU Gang Preemption: v1.15 DRA Queue Quota](https://www.matthewswong.com/en/blog/volcano-1-15-gang-preemption-dra-queue-quota/): Volcano's gang preemption, DRA queue quota and scheduling gates, set against the native gang scheduling.
11. [NVIDIA Run:ai Open Source: KAI Scheduler, GPU Fractioning](https://www.matthewswong.com/en/blog/nvidia-runai-open-source-kai-scheduler-gpu-fractioning/): The open-sourced Run:ai parts: KAI Scheduler, GPU fractioning, the CUDA catch and the licence terms.
12. [Kubernetes HPA Scale to Zero for GPU Queue Workers (1.37)](https://www.matthewswong.com/en/blog/kubernetes-hpa-scale-to-zero-gpu-queue-workers/): Scale GPU queue workers to zero with the native HPA, and the trap to avoid.
13. [Kubernetes AI Conformance: What the KARs Demand of a Platform](https://www.matthewswong.com/en/blog/kubernetes-ai-conformance-program-explained/): Check a platform against the AI conformance requirements, and what the badge does not prove.

### Stages 3: Traffic: Gateway API and inference routing

Off Ingress and onto Gateway API, then the extension that routes requests to model servers.

14. [Kubernetes Gateway API vs Ingress: Why It Replaces Annotations](https://www.matthewswong.com/en/blog/kubernetes-gateway-api-vs-ingress/): The groundwork: how Gateway API's role-oriented resources replace Ingress.
15. [ingress-nginx Retirement: Contour or Gateway API Migration](https://www.matthewswong.com/en/blog/ingress-nginx-retirement-migration-paths/): With ingress-nginx retired, compare a Contour lift-and-shift with a move to Gateway API.
16. [Ingress2Gateway 1.0: Migrate ingress-nginx to Gateway API](https://www.matthewswong.com/en/blog/ingress2gateway-1-0-migration-walkthrough/): The hands-on conversion with Ingress2Gateway: read its warnings and fix what it cannot translate.
17. [Gateway API Inference Extension: EPP Moves to llm-d, What Now](https://www.matthewswong.com/en/blog/gateway-api-inference-extension-epp-moves-to-llm-d/): Gateway API for model traffic: what the Inference Extension keeps upstream now the Endpoint Picker lives in llm-d.
18. [llm-d on Kubernetes: Distributed Inference and v0.10 Hardening](https://www.matthewswong.com/en/blog/llm-d-kubernetes-native-distributed-inference/): The serving stack built on that extension, and what its hardening release changes for rollouts.

### Stages 4: Agents on the cluster

Running agents as Kubernetes workloads, and giving them cluster access that cannot break it.

19. [Kubernetes Agent Sandbox CRD: Isolated Stateful AI Agents](https://www.matthewswong.com/en/blog/kubernetes-agent-sandbox-crd-ai-agents/): Isolated, stateful agents as a Kubernetes resource: Sandbox, templates, claims and warm pools.
20. [kagent Kubernetes Tutorial: AI Agents on a Laptop Cluster](https://www.matthewswong.com/en/blog/kagent-ai-agents-kubernetes-tutorial/): Run AI agents on a laptop kind cluster with kagent and wire MCP tools into them.
21. [Kubernetes MCP Server: read_only, Toolsets, Denied Resources](https://www.matthewswong.com/en/blog/kubernetes-mcp-server-go-read-only-toolsets/): Give an agent cluster access that cannot uninstall anything: a view ServiceAccount, read-only mode and denied resources.
22. [kubectl-ai Guide: Confirmation Gate, Limits and MCP Server Risk](https://www.matthewswong.com/en/blog/kubectl-ai-google-human-in-loop-guide/): How kubectl-ai's confirmation gate classifies commands, and why its MCP HTTP server needs OAuth.
23. [AI DevOps Assistants for Kubernetes Production Triage](https://www.matthewswong.com/en/blog/ai-devops-assistant-production-triage/): What an AI assistant really shortens in Kubernetes triage, and the calls it should not make for you.
24. [KubeCon North America 2026 Guide: AI Track, Co-located Days](https://www.matthewswong.com/en/blog/kubecon-na-2026-salt-lake-city-guide/): To go further in person: a platform engineer's guide to KubeCon North America and its AI Inference and Agentic track.

## More from the AI & DevOps guide

- [MCP & Agent Security](https://www.matthewswong.com/en/ai-devops/mcp-agent-security/): Prompt injection, the 2026 MCP incidents, access controls, sandboxes and vetting agent skills. (25 articles)
- [LLM Inference Serving](https://www.matthewswong.com/en/ai-devops/llm-inference-serving/): vLLM and SGLang, disaggregated serving, llm-d and KServe, autoscaling and cost per token. (25 articles)
- [AI SRE & Agentic Ops](https://www.matthewswong.com/en/ai-devops/ai-sre-and-agentic-ops/): Incident-investigation agents, MCP servers for ops tools, agents in CI, and the IaC beneath. (25 articles)
- [Supply Chain & Regulation](https://www.matthewswong.com/en/ai-devops/software-supply-chain-and-regulation/): npm and PyPI worms, model signing and AIBOM, secrets in agent commits, the CRA and EU AI Act. (23 articles)

---

- HTML version: https://www.matthewswong.com/en/ai-devops/kubernetes-for-ai/
- Bahasa Indonesia: https://www.matthewswong.com/id/ai-devops/kubernetes-for-ai/
- Site index for agents: https://www.matthewswong.com/llms.txt
- Sitemap: https://www.matthewswong.com/sitemap.xml
- API description (OpenAPI): https://www.matthewswong.com/openapi.json