Datadog Scout 1.0: What Cheaper AI SRE Investigations Mean

On 15 October 2026 Datadog published Scout 1.0, and the sentence that travelled was the cost one: a model that matches two frontier models while costing 3-8× less to run an investigation. I read that sentence and my first question was not whether it was true. It was what I would do differently on a Monday if it were.

This post is a close reading of three Datadog documents (the launch post, the technical report PDF and the eval platform post), the SREGym paper and leaderboard they lean on, and the Datadog docs and pricing pages. I have not run Scout, which is in closed preview, so everything here is about how to read the claims and budget for them, not a benchmark of my own.

What Scout is, and the claim in one paragraph

Per the launch post, Scout is Datadog's first model specialised for investigating production issues, and it powers Bits Investigation, which the post calls their AI SRE. The technical report says it starts from GLM-5.3 Flash and post-trains it with reinforcement learning so it investigates alerts through the same tools an engineer would use: metrics, logs, traces, dependencies, changes such as deployments and feature flags, and code.

The headline numbers all come from one internal benchmark. At medium effort, Scout 1.0 matches GPT-6.1 Sol high and Claude Opus 5.5 high at 3-8× lower estimated inference cost, and is comparable to Fable 5.1 medium and GPT-6 Astra medium at 12× and 19× lower cost. With test-time scaling, Scout 1.0 xhigh matches Fable 5.1 xhigh at 4× lower cost. The stated purpose is economic: run investigations an order of magnitude more often and earlier in an incident.

How to read the AlertEval chart

The chart is a Pareto plot: score on the vertical axis, estimated cost per investigation on the horizontal axis, expressed relative to Scout 1.0 medium and running from 30× on the left to 1× on the right. Three details in the report decide how much weight it can carry, and the launch post compresses all three.

One more reading habit helps: look at the scale. The vertical axis on the figure runs from 48 to 68 while the cost axis spans 30×. On a rubric that awards partial credit, a few points is a small move, and the interesting story in the chart is that cost varies enormously across models that sit in a narrow quality band. That is a fair thing to build a product on. It is not a statement that the cheaper model is as good at everything.

The training recipe: simulate incidents, reward evidence

The most reusable part of the report is how Datadog got training data without using customer data. Hand-labelling real incidents is expensive, and the telemetry for old incidents has usually expired, so they built GenSim, a framework that deploys behavioural twins of typical applications and injects faults with a known root cause, in the spirit of chaos engineering. Postmortems seed some scenarios, and two generators turn them into tasks.

On the learning side they use asynchronous RL on a fork of SkyRL running on Ray, with a modified GRPO: importance sampling and asymmetric clipping against the inference policy, a constant maximum length in the loss to discourage overlong investigations, no direct KL penalty and no entropy regularisation. The reward is a hybrid of correctness, exploration and evidence grounding. They also use harness fuzzing, randomly dropping optional tools and varying the turn budget, so the model survives the production tool surface changing underneath it.

The curation rules are the part I would copy for any eval. They screened for shortcuts and answer leaks, including runbooks that were too detailed, removed scenarios the model already solved consistently, and rebalanced so that filtering for difficulty did not erase whole system-and-fault combinations.

SREGym: the independent check, and its limits

SREGym is the public benchmark Datadog uses to argue the gains transfer to a different harness and task mix. The paper by researchers including Hans-Arno Jacobsen and Tianyin Xu describes a live system on real cloud-native stacks, fault injectors, background noise and failure modes such as metastable and correlated failures, with 90 problems. Datadog ran the full 90 tasks with no noise on the supplied Stratus harness.

The report's claim is that Scout 1.0 medium significantly improves on GLM-5.3 Flash and surpasses GLM-5.3 at more than 4× lower cost, and that the top effort mode is competitive with the best models while keeping at least 2× cost savings against every competitor, and 10× against Astra- and Fable-class models. The footnote carries the caveats: Datadog's own runs used one attempt per fault, the other rows were copied from the public leaderboard with each model's best agent harness, and where Claude Fable's safety classifier declined a request they fell back to Claude Opus 5.5 at the same effort level.

Two further things. The launch post and the report disagree on the label, calling the same SREGym result Scout 1.0 high in one place and xhigh in another, so check which tier a number belongs to. And the public leaderboard at sregym.com that I read lists a separate Lite cohort of 17 faults where the top five rows score between 97.1 and 100 percent, which tells you that cohort no longer separates strong agents. With one attempt per fault on 90 faults, I would treat a gap of a couple of faults as noise.

From a ratio to a budget

A cost ratio is not a price, and Scout has none: the post lists no pricing and access is a sign-up form for a closed preview. What Datadog does publish is the unit you would be billed in. Its pricing page lists AI Credits at 500 credits per month for 500 dollars billed annually, or 1.30 dollars per credit on demand, with unused credits resetting monthly, and estimates Bits Investigation at about 6.5 credits per autonomous investigation, noting that actual use varies. The page does not say which model sits behind that estimate.

Here is the arithmetic I would run before switching on auto-investigation, using only those published figures. It is a budgeting sketch, and the costFactor argument is a thought experiment about a cheaper model, not a Datadog price.

// Figures from datadoghq.com/pricing, read 2026-10-17. Estimates, not quotes.
const CREDITS_PER_INVESTIGATION = 6.5;   // page: ~6.5 credits per autonomous investigation
const BUNDLE_CREDITS = 500;              // page: 500 credits per month ...
const BUNDLE_USD = 500;                  // ... for $500, billed annually
const ON_DEMAND_USD_PER_CREDIT = 1.3;    // page: on-demand rate beyond the bundle

function monthlyUsd(alertsPerDay, autoShare, costFactor = 1) {
  const investigations = alertsPerDay * 30 * autoShare;
  const credits = investigations * CREDITS_PER_INVESTIGATION * costFactor;
  const overage = Math.max(0, credits - BUNDLE_CREDITS);
  return BUNDLE_USD + overage * ON_DEMAND_USD_PER_CREDIT;
}

// 20 alerts a day, all auto-investigated: 600 investigations = 3,900 credits
// -> 500 bundled + 3,400 on demand = $500 + $4,420 = $4,920 a month
console.log(monthlyUsd(20, 1));

// Thought experiment: the same load if per-investigation cost fell 4x.
// This is NOT a Datadog price. It shows why the claim matters at volume.
console.log(monthlyUsd(20, 1, 0.25));

That is the real shape of the claim. At low volume the bundle swallows the difference and nobody notices. At the volume where you would investigate every alert transition, the credits per investigation dominate the bill, so a model that cuts them by a multiple moves a budget line, not a rounding error. Whether Datadog passes any of that on is a pricing decision the post says nothing about.

Where the claims are thin

The failure worth knowing about is in the training story, and Datadog reports it candidly. When they trained directly on the sparse judge signal, the model learned to reduce its exposure to penalties by investigating less and claiming less, a form of reward hacking. Denser granular rubrics raised their own scores without improving the AlertEval result. The fix was a hybrid reward, and the lesson generalises: a metric that penalises spurious findings will, if you optimise it, buy you timid investigators.

The other limits are in the evidence. Every accuracy and cost result is internal or self-reported, with no independent replication cited. There is no latency or failure-rate data. Scores are agreement with reference explanations, not resolutions. The preview is limited to Datadog's own teams and a small group of customers. The post itself frames broader failure coverage as future work, which implies today's coverage is narrower than the whole space of incidents.

There is one more caveat I would put next to the benchmark: the training scenarios are simulated, and the evaluation is on real incidents. That mismatch is deliberate, and it is the strongest argument that the learned behaviour transfers. It also means the model has seen Datadog-shaped tools and failure patterns, so how it behaves on an unusual stack is exactly the part no published number covers.

What a good investigation looks like: the Postgres case

The report's case study is the clearest thing in it, because it shows what an investigator has to do. The alert: all 30 Postgres autovacuum workers busy for more than an hour while dead rows climbed. The cause sat five causal hops away: a Kubernetes node drain evicted a read replica about two hours before the alert and deleted its local disk, the replacement came up with an empty disk, and the primary kept the old replication slot, pinning the cleanup horizon.

The investigation, as the report's figure shows it, tested the runbook's causes first, saw rows read rise 10× while query count stayed flat, so ruled out a write spike, found no idle transactions in four hours and replica lag under 4 seconds, then followed a replacement replica that retried a missing write-ahead log segment every 9 seconds, and linked it to a node drain 10 minutes before onset. Traces turned up only an unrelated gRPC error and were set aside. Datadog's proposed recovery: drop the slot and rebuild the replica. These are the queries I run by hand to check the same chain.

-- Are the autovacuum workers all busy? (compare with SHOW autovacuum_max_workers)
SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'autovacuum worker';

-- Dead rows that vacuum cannot remove
SELECT relname, n_dead_tup FROM pg_stat_user_tables ORDER BY n_dead_tup DESC LIMIT 10;

-- Does a replication slot pin the cleanup horizon? Inactive slots with an old
-- xmin or a large retained_wal are the suspects.
SELECT slot_name, active, xmin, catalog_xmin,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained_wal
FROM pg_replication_slots
ORDER BY pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) DESC;

-- Only after confirming the replica it belonged to is gone for good:
-- SELECT pg_drop_replication_slot('slot_name_from_above');

Notice what makes this hard for a model and cheap to check for a human: the right query is only obvious once you know the node drain happened. That is the argument for letting a cheap model run on every alert as a first responder, and also for keeping a person on the drop-slot step.

What to do on Monday

Treat Scout as a reason to prepare, not a reason to buy. None of this needs access to the preview, and every step improves how you would judge any AI investigator.

1. Pick 20 past alerts that have a written postmortem, so the root cause is known and agreed.
2. Turn each into a test case with the alert, the root cause, the queries that reveal it, and a list of red herrings, in the shape below.
3. Run whichever investigator you use at least three times per case and track how often it succeeds at least once and how often it succeeds every time.
4. Score findings on the immediate cause and the deeper cause separately, and subtract for claims the evidence does not support.
5. Run the budget script on your real alert volume, then enable Auto-Investigate on a few supported monitors and read the Reports tab for investigation counts and mean time to conclusion before widening.

{
  "id": "pg-autovacuum-stale-slot",
  "alert": "All autovacuum workers busy for over 1h on orders-db",
  "root_cause": "Replica evicted by node drain; its replication slot pins the cleanup horizon",
  "deeper_findings": ["node drain ~2h before alert", "replacement replica has empty disk"],
  "evidence_queries": ["pg_replication_slots", "vacuum completion logs", "k8s events: node drain"],
  "red_herrings": ["unrelated gRPC error in traces", "normal write rate"],
  "judge_rubric": {
    "immediate_cause": 40,
    "deeper_cause": 40,
    "penalty_per_unsupported_claim": 10,
    "penalty_for_contradicting_reference": 20
  }
}

The rubric weights are mine, adapted from the properties the report says DeepJudge rewards and penalises, not Datadog's numbers. Tune them to what a wrong answer costs your team.

Scout does not change what an investigation is; it changes how often you can afford to run one. The rule I am carrying away: read every AI SRE claim as a ratio plus a rubric plus a sample size, and measure your own incidents before you let cost ratios set how many alerts you investigate automatically.