NVIDIA Run:ai Open Source: KAI Scheduler, GPU Fractioning

NVIDIA's announcement on 13 October 2026 says the full Run:ai platform is now open source, and that sentence hides four separate things with different licences, different dependencies and very different amounts of novelty. I read it twice before I could say which of them I would actually try on a cluster.

I have not installed any of it. Everything below comes from NVIDIA's technical blog post, the dsx-ai-factory/run-ai repository, KAI Scheduler's own README and docs pages, and NVIDIA's March KubeCon post, all read on 17 October 2026. Where a page was silent I say so instead of filling the gap, and the one YAML file I composed myself is labelled as such.

The four parts, and what each one needs

NVIDIA describes the release as four modular components that can be adopted independently. The useful question for a platform team is not whether Run:ai is open but which of these four solves a problem you currently pay for or work around.

So the scheduler and the operator are the two parts a single team can try this week, Resource Management is for the platform owner who has to divide GPUs between departments, and Workload Map is a library for people building tooling. An NGC Helm chart for the integrated stack also exists, as a Community Product build.

What is actually new in the October release

KAI Scheduler is not the news. NVIDIA's March KubeCon post says it was onboarded as a CNCF Sandbox project on 24 March 2026, and the DEV Community analysis I read notes it was already Apache 2.0 before October. That analysis is secondary and AI-assisted, so treat the exact history as its claim, but the primary sources agree on the direction: the scheduler was open first and the rest followed.

What the October release adds is the platform around it. The dsx-ai-factory/run-ai repository is a monorepo whose README lists a control plane, cluster services (controllers and admission webhooks deployed into each managed cluster), a runai-installer CLI, a runai-mcp-server that exposes Run:ai operations as tools for AI assistants, generated clients, and scale and load-testing tools. Images and Helm charts are published to NVIDIA NGC under nvcr.io/nvidia/runai.

NVIDIA also lists eight DSX ecosystem partners integrating the platform: Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud and vCluster. The same post covers two adjacent projects, Model Streamer (wired into vLLM through the flag --load-format runai_streamer) and Grove, a Kubernetes API for multi-component inference. The post carries partner-published loading benchmarks for Model Streamer; I am not repeating them because I could not verify any of them.

Fractional GPUs: the CUDA 13.4 dependency and the pod spec

The most consequential line in the announcement is that Run:ai's proprietary fractional-GPU implementation has been retired in favour of native fractions in CUDA 13.4, and that the GPU Fractioning Operator is the Kubernetes integration for that driver feature. If you wanted fractions, the question is no longer a licence but whether your drivers are new enough. KAI Scheduler's gpu-sharing page shows what a fractional pod looks like.

# From KAI Scheduler's docs/gpu-sharing/README.md: a pod that asks for half a GPU.
# Two things are mandatory on every fractional pod: schedulerName and the queue label.
apiVersion: v1
kind: Pod
metadata:
  name: half-gpu
  labels:
    kai.scheduler/queue: default-queue
  annotations:
    gpu-fraction: "0.5"          # must be greater than 0 and less than 1
spec:
  schedulerName: kai-scheduler   # without this the default scheduler never sees the annotation
  containers:
    - name: gpu-workload
      image: nvidia/cuda:13.0.2-base-ubi8
      command: ["nvidia-smi"]
      args: ["--query-gpu=name,memory.total", "--format=csv,noheader"]

# The same page also documents these alternatives (metadata only):
#   gpu-memory: "4096"                                         # memory in MiB
#   nvidia.com/container.gpu-workload.gpu-memory.request: 4096Mi   # NvFractions
#   nvidia.com/container.gpu-workload.gpu-memory.limit: 8192Mi
#   gpu-fraction-num-devices: "2"                              # spread over two devices

Those annotations are not one mechanism. The page documents several modes, and which annotations apply depends on the mode your cluster runs.

The docs do not say in so many words that NvFractions is the Kubernetes face of the CUDA 13.4 fractions, so I am inferring that from the naming and from NVIDIA's announcement. Check that with the operator's install documentation before you plan around it.

Queues and quotas: the part that replaces a spreadsheet

KAI Scheduler models sharing with a Queue custom resource under the scheduling.run.ai/v2 API. A queue declares a guaranteed quota, a hard limit and an overQuotaWeight for dividing spare capacity, and queues nest through parentQueue.

# Field names and units are from KAI Scheduler's docs/queues/README.md.
# Units: CPU in millicores, memory in MB, GPU in whole units.
# Only leaf queues (no children) can run jobs.
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: research-team
spec:
  displayName: "Research Team"
  resources:
    gpu:
      quota: 1       # guaranteed
      limit: 2       # hard cap
---
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: ml-team
spec:
  displayName: "ML Team"
  parentQueue: "research-team"   # makes research-team a non-leaf, so it can no longer run jobs
  priority: 200                  # higher goes first when spare capacity is shared out
  resources:
    gpu:
      quota: 1
      overQuotaWeight: 1         # share of the spare GPUs among queues at the same priority

Two details from the docs matter more than the syntax. Only leaf queues can run jobs, so turning a team queue into a parent silently retires it as a place to submit work. And a quota of -1 means unlimited while 0 or unset means no guarantee, so a copy-pasted burst queue can be far more generous than you intended. The queues page does not say how a pod names its queue; the gpu-sharing page does, through the kai.scheduler/queue label.

Gang scheduling: all of a job or none of it

By default KAI schedules pods individually. To get all-or-nothing placement you add the kai.scheduler/batch-min-member annotation to a Job or JobSet, and no pod starts until that minimum can be placed together. For JobSets the scheduler builds one PodGroup with a SubGroup per replicatedJob and per replica, and for PyTorchJob it needs the Kubeflow training operator v1.

# My composition from the annotation semantics in docs/batch/README.md.
# The docs name the example files but I did not read their text, and I have not run this.
# Six pods are requested; none starts unless at least two can be placed together.
apiVersion: batch/v1
kind: Job
metadata:
  name: train-gang
  annotations:
    kai.scheduler/batch-min-member: "2"
spec:
  parallelism: 6
  completions: 6
  template:
    metadata:
      labels:
        kai.scheduler/queue: ml-team
    spec:
      schedulerName: kai-scheduler
      restartPolicy: Never
      containers:
        - name: worker
          image: nvidia/cuda:13.0.2-base-ubi8
          command: ["nvidia-smi", "-L"]
          resources:
            limits:
              nvidia.com/gpu: 1

The docs also let you create a PodGroup yourself and have pods join it through pod-group-name, with kai.scheduler/skip-podgrouper set to true so the podgrouper leaves membership alone. That is the escape hatch for controllers KAI does not understand, and also the case Workload Map is aimed at.

The licence fine print, layer by layer

Open source is true of the code and only partly true of the ways to get it. Four layers apply, and they say different things.

The practical rule is to decide which layer you are consuming before anyone says the word free. Building from source and running the Helm chart from NGC are two different legal relationships with NVIDIA.

Where it can hurt: four trade-offs I would name up front

None of these is a reason to skip the release. Each is a place where a pilot that looked fine in a demo can stall.

• The driver floor. Fractions depend on CUDA 13.4 and a compatible driver, and NVIDIA says the proprietary implementation is retired. A cluster pinned to older drivers for stability has a real upgrade to schedule first.
• Strict pod rules. A fractional pod cannot also request a whole nvidia.com/gpu, only one container may request a fraction, and users cannot set the reserved devices annotation because the KAI Binder writes it.
• Unclear memory isolation. The docs name a mode called NonMemoryEnforced and say NvFractions needs CDI and NRI for runtime memory enforcement. I read the first name as meaning no memory enforcement, which I did not verify, so test what a greedy neighbour can do before you share a GPU between teams.
• Maturity and support. KAI is a CNCF Sandbox project, the NGC build has no enterprise support, and the DEV Community analysis says no public adoption numbers exist yet. A vendor announcement is not a production track record.

A fifth consequence follows from the pod spec: every pod must set schedulerName to kai-scheduler, so a team that forgets it silently gets the default scheduler and none of the queue or fraction behaviour. Treat that as a policy to enforce with admission rules, not a habit to hope for.

What to do on Monday

I would run a narrow pilot of the scheduler alone, on one non-production GPU pool, and leave the full platform for later.

1. Record the driver and CUDA version on every GPU node and compare them with the CUDA 13.4 requirement for fractions.
2. Install KAI Scheduler from its README Helm command, pinned to a release, into its own namespace.
3. Create a leaf Queue with a small GPU quota and a limit, then run the half-gpu pod in a separate namespace, never in kai-scheduler.
4. Run a Job with the batch-min-member annotation larger than the free GPUs and confirm that it waits instead of starting partially.
5. Ask legal about the NGC terms and ask yourselves whether you need the enterprise support the Community Product lacks.

# 1. Which driver and CUDA version do the GPU nodes report? Fractions need CUDA 13.4
#    and a compatible driver, so read this before reading anything else.
nvidia-smi --query-gpu=name,driver_version --format=csv,noheader
nvidia-smi | grep "CUDA Version"

# 2. Install only the scheduler, from the KAI README. Replace VERSION with a release
#    from https://github.com/kai-scheduler/KAI-scheduler/releases
helm upgrade -i kai-scheduler \
  oci://ghcr.io/kai-scheduler/kai-scheduler/kai-scheduler \
  -n kai-scheduler --create-namespace --version VERSION

# 3. Never submit workloads into the kai-scheduler namespace. Use your own.
kubectl create namespace gpu-trial

# 4. Apply a leaf Queue (see the queue section), then the half-gpu pod with
#    metadata.namespace set to gpu-trial, and confirm which scheduler placed it.
kubectl -n gpu-trial get pod half-gpu -o jsonpath='{.spec.schedulerName}{"\n"}'
kubectl -n gpu-trial get events --field-selector involvedObject.name=half-gpu

The install command comes from the KAI README and the checks are standard kubectl. The Queue and pod YAML above are the docs' examples, so adjust names to your own namespaces.

Read the release as four decisions, not one: the scheduler is mature enough to trial now, the operator is gated by CUDA 13.4, the platform layer is new code with no public adoption record, and the licence you accept depends on whether you build from source or pull from NGC. Pick the layer first and the rest follows.