Kubernetes 1.37 Release: What Breaks When Upgrading From 1.36

I did not find the risky parts of Kubernetes 1.37 by scrolling to the Stable list. I found them by asking one question of every line in the release post: does this change behaviour on a cluster where nobody touched a flag? Most of the 67 enhancements answer no. A handful answer yes, and those are the ones that wake someone up after the control plane rolls.

This is a platform-team read of the 1.37 release, not a feature tour. I have not run 1.37 in production, so every claim below traces to the release announcement, the July sneak peek, or one of three follow-up posts on kubernetes.io that I read in full on 17 October 2026. Where the release post is vague or contradicts itself, I say so. Separate deep dives cover DRA, gang scheduling, HPA scale to zero, KYAML, manifest-based admission and the ipvs migration; this one is the map that tells you which of them you need.

67 enhancements, and why the count is the least useful number

The announcement, published on 26 August 2026 under the theme name Garhwal, counts 67 enhancements: 16 graduating to Stable, 23 to Beta, 27 entering Alpha and 1 deprecation or removal. That is a normal-sized release. The number tells you nothing about upgrade risk, because it counts KEPs, not behaviour changes, and a KEP that adds an opt-in Alpha field costs you nothing until you opt in.

One small thing to know when you quote the headline. The Stable list in the same post, under the heading that says 16, contains 17 bullets. Pod Certificates and ClusterTrustBundles are two bullets and, per the post, two KEPs (4317 and 3257), which may explain the gap, but the post does not say so. I would cite the 16 as the project's figure and not do arithmetic on it.

Instead of counting, sort. I split the release into three piles: changes that are on by default and touch behaviour you already depend on, changes that are opt-in and can wait for a staging cluster, and deprecation clocks that started ticking. The rest of the post follows that order.

The nine defaults that change on a cluster you did not touch

This table is the post in miniature. Every row is something the release post states is on, enforced or removed in 1.37 without an operator opting in. Stage and default come from the release announcement; the last column is the failure mode as the project itself describes it.

Notice the shape. Three of the nine are control-plane scale work that you will never see when it works. Two are refusals you can detect before the upgrade, on nodes and in static Pod manifests. One, SELinux, is a genuine behaviour change for a subset of workloads. The last three are Beta features whose defaults are built to stay quiet until a workload meets them. That asymmetry is why I treat the audit as a short, ordered job and not a project.

The control plane gets quietly harder to knock over

The most important Stable item for large clusters is the one with the least glamour. The watch cache initialisation work is finished: the remaining WatchCacheInitializationPostStartHook gate graduates and is locked on, having defaulted to enabled since 1.36. Cache warm-up no longer creates a spike of requests against etcd, and kube-apiserver now delegates bounded requests and rejects others with HTTP 429, instead of letting expensive list and watch calls exhaust etcd or API Priority and Fairness capacity. The post says plainly that custom controllers and operators should respect Retry-After and use exponential backoff.

Two Beta items attack the same startup cost from different sides. EtcdRangeStream adds a server-streaming RangeStream RPC that pins one MVCC revision and returns chunks, so the API server no longer assembles a whole list in memory. It needs etcd 3.7 or newer; against older etcd the API server sees Unimplemented and falls back to unary Range. The kubernetes.io write-up adds that the same payload used to sit in memory on both etcd and the API server at once. ConcurrentWatchObjectDecode, in Beta and off since 1.31, is now on and decodes watch events on a pool of 10 workers while a collector preserves event order.

The numbers are the project's, not mine: over 150k pods, concurrent decode alone cuts cache initialisation by about 40 percent, and about 55 percent combined with RangeStream. I have not reproduced them, and I would not promise them to a stakeholder. The trade-off the post names is real and specific, though: with the gate on, up to 10 conversions can run concurrently against a conversion webhook during cache initialisation. If a webhook caps its own concurrency below 10, expect it to be the new bottleneck.

# Does your etcd meet the RangeStream floor? The release post says etcd 3.7+.
# Read the VERSION column; older etcd just means a silent fallback, not a failure.
etcdctl endpoint status --cluster -w table

# The post says etcdctl get --stream ships alongside the feature. Confirm the
# flag exists in your build before scripting around it.
etcdctl get --help | grep -i stream

# Which CRDs depend on a conversion webhook? These are the ones that now see
# up to 10 concurrent conversions while the watch cache warms.
kubectl get crd -o custom-columns=NAME:.metadata.name,CONVERSION:.spec.conversion.strategy | grep -i webhook

# Are any clients already being told to slow down? Needs access to /metrics.
kubectl get --raw /metrics | grep 'apiserver_request_total' | grep 'code="429"' | head

SELinux is the one that actually changes workload behaviour

If your nodes do not run SELinux, skip this section: the April post by Jan Safranek and Swathi Rao says the kubelet skips the whole SELinux logic when it is unavailable or disabled. If they do, SELinuxMount and SELinuxChangePolicy are Stable and on in 1.37. Volumes then mount with the label as a mount option, in constant time, instead of the runtime relabelling every file, but only when the CSI driver opts in with seLinuxMount true on its CSIDriver object (the in-tree fc and iscsi plugins also support it) and the Pod sets at least seLinuxOptions.level.

The cost is one mount, one label. Two cases worked under recursive relabelling and do not now: Pods with different labels sharing a volume through different subPaths, and a privileged and an unprivileged Pod sharing a volume. In both, one Pod sits in ContainerCreating until the other terminates. The project calls the first case niche and says the second is rare but has been observed in applications. The release post adds that the behaviour is not locked until 1.38, so you can still disable it cluster-wide for one more release.

# 1. Find the conflicts BEFORE upgrading (v1.36 tooling, from the April post).
#    Add the controller to kube-controller-manager:
--controllers=*,selinux-warning-controller

#    Then read the conflict metric. It names the Pods; the kubelet metric
#    volume_manager_selinux_volume_context_mismatch_warnings_total only counts them.
selinux_warning_controller_selinux_volume_conflict

# 2. Opt a conflicting workload out, per Pod. Recursive keeps the old behaviour.
apiVersion: v1
kind: Pod
metadata:
  name: legacy-shared-volume
spec:
  securityContext:
    seLinuxChangePolicy: Recursive   # Opt-out: fine for a legacy app, a poor default
    seLinuxOptions:
      level: "s0:c123,c456"
  containers:
    - name: app
      image: registry.example.com/app:1.0

# 3. After the upgrade, the kubelet counts Pods that could not start:
volume_manager_selinux_volume_context_mismatch_errors_total

The April post also suggests applying the opt-out in bulk with a MutatingAdmissionPolicy, a mutating webhook, or Kyverno or Gatekeeper, instead of editing every Deployment. One caution from that post: the conflict metric exposes namespace names, so it assumes only cluster administrators can read kube-controller-manager metrics.

Four deprecation clocks to put in the calendar

None of these break on upgrade day, but each now has a date or a trigger. The release post and the sneak peek agree on all of them. The ones with dates say expected, not committed, so treat them as planning inputs.

# kube-dns or CoreDNS? On kubeadm clusters both keep the k8s-app=kube-dns label,
# so the label proves nothing. Read the image.
kubectl -n kube-system get deploy -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.template.spec.containers[*].image}{"\n"}{end}'

# ipvs mode, exactly as the release post gives it:
kubectl -n kube-system get configmap kube-proxy -o jsonpath='{.data.config\.conf}' | grep 'mode:'

# cgroup v1 or v2, run on each node. cgroup2fs means v2; tmpfs means v1.
stat -fc %T /sys/fs/cgroup

# Temporary kubelet override, which the post calls a short-term fix only.
# Memory QoS and in-place resize of memory-backed volumes need cgroup v2.
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
failCgroupV1: false

# Static Pods that reference Secrets or ConfigMaps (kubeadm default path).
# My grep is wider than the post's wording, so expect some false positives.
grep -lE 'secretRef|configMapRef|secretKeyRef|configMapKeyRef' /etc/kubernetes/manifests/*.yaml

The ipvs row is the one most likely to be a surprise, because ipvs was introduced in v1.8 to fix iptables bottlenecks and many clusters picked it years ago. The post gives the reason: the kernel ipvs API cannot fully implement Kubernetes Services, so ipvs mode keeps using iptables underneath. A clean migration target is nftables mode, which also gains a netlink-based rule path and an opt-in localhost NodePort proxy in this release.

Opt-in features worth a staging cluster, and the ones to skip

Everything here is off by default or Alpha, so none of it can hurt you until you decide. The question is which are worth your staging cluster's time. I would rank them by how directly they replace something you already operate.

First, HPA scale to zero, now Beta and on by default: set minReplicas to 0 on a workload driven by an object or external metric, such as a queue, and the HPA records a ScaledToZero condition so it can tell its own scale-down from a Deployment someone zeroed by hand. Second, the StorageVersionMigration API is Stable and on: a declarative object now replaces the kubectl replace loops and the out-of-tree migrator, and the kubernetes.io post says it also rewrites data after an encryption at rest change. Third, the Node lifecycle conditions DrainInProgress, Drained, MaintenancePlanned, MaintenanceInProgress and GracefulNodeShutdownInProgress, in Alpha, are worth reading if you maintain a node-drain controller. Skip pod-level checkpoint and restore unless your runtime already implements the new CheckpointPod and RestorePod RPCs; the post says it must.

# Scale a queue consumer to zero. External metrics only: the post is explicit that
# CPU and memory cannot drive scale to zero because they need running Pods.
# The metric name below is a placeholder from my own example, not from the post.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-worker
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-worker
  minReplicas: 0
  maxReplicas: 20
  metrics:
    - type: External
      external:
        metric:
          name: queue_messages_ready
        target:
          type: AverageValue
          averageValue: "30"

---
# Rewrite stored objects after a CRD storage version change. From the kubernetes.io
# storage version migration post.
apiVersion: storagemigration.k8s.io/v1
kind: StorageVersionMigration
metadata:
  name: crontabs-migration
spec:
  resource:
    group: example.com
    resource: crontabs

---
# Kubelet-side gates from the release post, all off by default. Staging only.
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
featureGates:
  PodAndContainerStatsFromCRI: true
  PodLevelResourceManagers: true

The release also graduates DRA pieces to Stable, namely extended resources, device taints and tolerations, the ResourceClaim status field and a standard numaNode attribute, and moves gang scheduling and workload-aware preemption to Beta. Those deserve their own reading and are covered in separate posts rather than repeated here.

Where 1.37 can still hurt you

The honest trade-off in this release is that the safest-looking changes are the ones that move load around. Streaming reads and concurrent decoding make the API server calmer, and shift pressure to etcd 3.7 and to your webhooks. Neither shows up in a feature checklist, and I would expect the first sign to be a conversion webhook running hot during an API server restart, not an error message.

Three smaller traps. Scale to zero looks like a cost feature and fails quietly when pointed at a CPU metric, because the project does not support it. Memory QoS is on by default, but its defaults were chosen to avoid surprise throttling, so the behaviour you get depends on the kubelet settings memoryReservationPolicy and memoryThrottlingFactor, and those are worth reading before you assume nothing changed. And the SELinux change leaves a conflicting Pod waiting in ContainerCreating, so alert on that state, not only on crash loops.

What to do on Monday

This is the order I would run it in, cheapest and most informative first. None of the steps change the cluster; they all read.

1. Inventory nodes: kubelet version, container runtime, and cgroup version. A single cgroup v1 node blocks the kubelet unless you set failCgroupV1 to false, so find it before the rollout does.
2. Check etcd. If you are below 3.7, plan the etcd upgrade as its own change; RangeStream will fall back safely, so this is a gain you are choosing, not a blocker.
3. If any node runs SELinux in enforcing mode, enable selinux-warning-controller on 1.36 and read the conflict metric for a full week of traffic, not an afternoon.
4. List your CRD conversion webhooks and check the concurrency each can take. Raise it, or accept that cache warm-up will now use up to 10 concurrent conversions.
5. Read the ipvs ConfigMap and the DNS image. If either answer is the old one, file the migration against the v1.40 date, not against the v1.43 removal.

#!/usr/bin/env bash
# Read-only 1.37 preflight. Run from a machine with kubectl and etcdctl.
set -euo pipefail

echo '== node versions and runtimes =='
kubectl get nodes -o custom-columns=NAME:.metadata.name,KUBELET:.status.nodeInfo.kubeletVersion,RUNTIME:.status.nodeInfo.containerRuntimeVersion

echo '== kube-proxy mode =='
kubectl -n kube-system get configmap kube-proxy -o jsonpath='{.data.config\.conf}' | grep 'mode:' || echo 'no mode line found'

echo '== CRDs with conversion webhooks =='
kubectl get crd -o custom-columns=NAME:.metadata.name,CONVERSION:.spec.conversion.strategy | grep -i webhook || echo none

echo '== etcd versions =='
etcdctl endpoint status --cluster -w table

echo '== 429s already served =='
kubectl get --raw /metrics | grep 'apiserver_request_total' | grep 'code="429"' | head -5 || true

echo '== static pod manifests touching Secrets or ConfigMaps =='
grep -lE 'secretRef|configMapRef|secretKeyRef|configMapKeyRef' /etc/kubernetes/manifests/*.yaml 2>/dev/null || echo none

Run the cgroup check on each node separately, since kubectl cannot see it: stat -fc %T /sys/fs/cgroup. Then read the full 1.37 changelog for anything specific to your distribution before you schedule the rollout.

The rule I take from 1.37 is to sort a release by defaults, not by stage. Sixteen Stable features did not matter much to me, while a handful of already-on Beta gates and one SELinux change decide whether the upgrade is boring. Audit what is on without being asked, put the deprecation dates in the calendar, and leave the Alpha list for a staging cluster that has the time.