DeepSeek V4.1 Flash on vLLM vs SGLang: What the Numbers Mean

Two numbers landed in my feed within a month of each other: 5x and 35 to 873. Both are about the same model, DeepSeek-V4.1-Flash, and both look like the answer to the question of which engine is faster. They are not. Before I could say anything useful I had to find out what each number was measured against.

This is a reading, not a benchmark: I did not run either engine. I read the vLLM post of 7 October 2026, the SGLang kernel optimisation post, the Hugging Face model card and the earlier vLLM AgentX post, and I quote only figures I saw on those pages. Where the pages disagree with each other or with the headline, I say so.

What makes this model different to serve

The model card describes a causal encoder-decoder with 40 layers: a 20-layer encoder followed by a 20-layer decoder, 8B activated parameters per token in prefill and 16B in decode, a 196B-parameter Engram memory, and a global KV cache of 890 bytes per token, about a quarter of DeepSeek-V4-Flash. The weights are MIT licensed and the context window is up to one million tokens. SGLang's post derives the 890 from FP4 entries of 288 bytes of main KV plus 68 bytes of indexer K, pooled across layers.

Two consequences drive everything that follows. A long prompt mostly runs through the first 20 layers, because the decoder takes its global KV from the encoder output, while generated tokens still run all 40. And the sliding-window KV is a separate, uncompressed FP8 cache covering the last 128 positions in each of 40 layers, which prefix caching has to deal with. Both engines spent their three weeks on those two facts.

Why the headline numbers cannot be compared

The two posts answer different questions. Putting their set-ups side by side shows that the only thing they share is the model name. Note also that the scouting date I started from, 10 September, is the LMSYS day-0 post that SGLang's page links to, not the optimisation write-up itself, which carries no date of its own on the page.

So 873 is not a faster number than 5x; it is a different axis. A batch-size-1 decode figure with speculative decoding says how quickly one stream can talk to itself. A throughput figure under an interactivity floor says how many sessions a GPU can carry. AgentX itself, per the earlier vLLM post, has a median input of 142K tokens, a median output of 444 tokens and a prefix-cache hit rate above 96 per cent, a workload that is almost all prefill and cache reuse. Decode speed barely appears in it.

vLLM: skip the half of the model you do not need

The biggest single idea in the vLLM post is SWA bounded replay, on by default behind a --[no-]swa-bounded-replay flag. On a prefix-cache hit of length H, vLLM caches only global KV and reruns tokens H minus 128 up to H to rebuild the sliding-window KV, clipping the window at the replay start. On the decoder side, layer 20 runs on every token to produce global KV, while layers 21 to 39 run only on each request's last 128 tokens. For long prompts that skips nearly half the model.

# vLLM: bounded replay is ON by default (per the vLLM post).
# Serve command from the model card; no other launch flags are published.
vllm serve "deepseek-ai/DeepSeek-V4.1-Flash"

# Wrong: comparing outputs across runs with replay on and expecting identical text.
# Right: turn the approximation off for regression and replay tests.
vllm serve "deepseek-ai/DeepSeek-V4.1-Flash" --no-swa-bounded-replay

# What the replay does, as described in the post (H = prefix-cache hit length):
#   encoder: cache global KV only, rerun tokens [H - 128, H) to rebuild SWA KV
#   decoder: layer 20 on every token; layers 21-39 only on the last 128 tokens
#   CUDA graphs: layers 0-20 on the full batch, 21-39 on the trimmed batch (PIECEWISE)

The trimmed layers are small, so run eagerly they are bound by kernel launch cost rather than compute. vLLM captured layers 0 to 20 as a CUDA graph on the full batch and layers 21 to 39 separately on the trimmed batch, using a breakable PIECEWISE graph. With graphs, decoder replay cut prefill compute time by 30 to 40 per cent. Without them, short prompts got slower than the baseline, by up to 12 per cent at 1K tokens on DEP2.

The TTFT claims also need care. The summary says about 30 per cent lower TTFT from bounded replay, while the figure caption says nearly 70 per cent at around 100K throughput from three optimisations combined. The measurements are a median of three runs on GB200 with prefix caching off. I would quote neither without saying which one.

vLLM: the kernel list, and the gap between kernel and end-to-end

Most of the remaining vLLM gain comes from integrating DeepSeek's fused kernels. The table lists what each one fuses and the effect the post reports, and the unit matters: some figures are per kernel, some per layer, some end to end.

Read the sparse MQA row twice. A 14 to 23x layer speedup at 512K context sounds like a revolution, and it turns into a 3 to 6 per cent decode gain end to end, because the layer is a small share of a decode step. The honest summary of this table is that no single kernel produced 5x. The 5x is a chart in the post, with the underlying data not given in the text, and it compounds a smaller KV footprint, trimmed layers, graphs and many small fusions.

The configuration split is the part that generalises. For low latency, vLLM used TP4 with FlashInfer attention; it tried MegaAttention there, but the gain was small and FlashInfer was faster. For high throughput it used DEP2, data-parallel attention with experts split across GPUs, because DP avoids replicating the shared KV latent that TP would duplicate, and session affinity keeps prefix-cache locality across turns. NVFP4 KV nearly halves per-request KV against FP8, which raises concurrency per GPU.

SGLang: a ladder from 35 to 873 tokens per second

SGLang's post is a cumulative ladder on four GB300s with attention at TP4, and it is more useful than the headline because every rung has a number. Read it rung by rung and the title stops meaning what it seems to mean.

Step 2 is the lesson. Some dense FP8 weights had fallen back to a slower path because their block and scale layout did not match the fast kernel, and fixing the layout took the build from 35.2 to 117.8, about 3.3 times (my arithmetic from their table). The authors say checking which kernel a GEMM actually dispatches to was worth more than tile tuning. The rest of plain decode, steps 3 to 10, added another 1.7 times by fusing small operations and overlapping the mHC statistics on a second stream.

The jump from 203.3 to 558.2 at step 11 is speculative decoding switching on, not a kernel. DSpark uses a block size of 5 with three draft blocks, and the post's own benchmark table gives 223.50 tokens per second with it off and 853.49 with it on, both EP4. The last rung, 873.63, comes from moving MoE from EP4 to TP4 with the expert intermediate dimension padded from 576 to 640 per GPU, a same-round gain of 2.22 per cent. Most of the 25x in the title is therefore speculation with a simulated accept length.

Configs you can actually run

Only SGLang published a full launch command. The vLLM post gives one flag and describes TP4 and DEP2 in prose, so the vLLM block below uses the model card's own serve command and marks everything else as a gap rather than inventing flags. The SGLang block is copied from the post's reproduction section, with the environment variables it lists.

# vLLM: not published as commands. The post names the shapes only:
#   low latency     TP4 + FlashInfer attention
#   high throughput DEP2 (DP attention, experts split across GPUs) + session affinity
# Check your installed version's docs for the exact flags before copying anyone's.

# SGLang: reproduction block from the post (4x GB300, attention TP4).
export MODEL_PATH=/path/to/DeepSeek-V4.1-Flash
export SGLANG_RAGGED_VERIFY_MODE=static
export SGLANG_SIMULATE_ACC_LEN=5.5          # simulated accept length, NOT measured
export SGLANG_SIMULATE_ACC_METHOD=match-expected

CUDA_VISIBLE_DEVICES=0,1,2,3 PYTHONPATH="$PWD/python" MAX_JOBS=16 python -m sglang.launch_server   --model-path "$MODEL_PATH"   --served-model-name deepseek-ai/DeepSeek-V4.1-Flash   --tp 4 --ep-size 1 --trust-remote-code   --moe-a2a-backend none --moe-runner-backend flashinfer_mxfp4   --mem-fraction-static 0.80 --max-total-tokens 33554432   --chunked-prefill-size 4096   --cuda-graph-bs-decode 1 2 4 8 16 32 64   --max-running-requests 128   --speculative-algorithm DSPARK --speculative-dspark-block-size 5   --skip-server-warmup --reasoning-parser deepseek-v41   --random-seed 42 --decode-log-interval 10   --host 127.0.0.1 --port 30021
# EP4 comparison in the post: change --ep-size 1 to --ep-size 4.

Notice what the SGLang command pins: the random seed, 33,554,432 max total tokens, chunked prefill of 4,096, CUDA graphs for decode batch sizes 1 to 64, and a simulated accept length through environment variables. Remove the simulation and the 873 figure is no longer the one you will see. The post also states that none of its results use the new kernels DeepSeek released with V4.1, so the two posts are not even optimising the same kernel set.

Where the numbers break

Four things in the sources should stop you from copying a conclusion. First, the vLLM page disagrees with itself: the TL;DR says 5.3x under a 150 TPS constraint while the body and figure say about 5x, and the constraint is never restated in the body. Second, the 1.9x and 5x figures exist only in a chart, so I cannot check them against text. Third, SGLang's final results table lists one fewer accept-length value than it has columns, so the final accept lengths are ambiguous. Fourth, SGLang throughput excludes full prefill and is measured from first to last streamed event.

The operational risks are separate. Bounded replay makes outputs non-reproducible against an exact baseline. The Engram layers rely on transparent huge pages for up to 10 times faster lookup kernels in prefill, and vLLM added a fallback for systems that lack enough huge pages, which means a host with the wrong memory configuration quietly takes the slow path. SGLang's DSpark numbers assume an accept length that real traffic may not reach: the ceiling is 6, and your prompts decide the real value.

My own mistake while reading is a useful warning. I first took 35 to 873 as one engine getting 25 times faster at serving. It is a single stream, with speculation, on simulated acceptance, against a first build that had a layout bug. Every one of those qualifiers was on the page; none was in the title.

What to do on Monday

You do not need GB300s to use what these posts teach. Do the following, in order, on whatever you serve.

1. Write down your own workload shape before any tuning: median input, median output, prefix-cache hit rate and turns per session. AgentX publishes those four for its traces; yours decide whether decode or prefill and cache reuse matter.
2. Confirm which kernel your biggest GEMMs dispatch to. SGLang's 3.3x came from a layout mismatch, not from tuning.
3. Put the KV footprint per token in your capacity sheet and recompute concurrency per GPU. A smaller KV cache buys concurrency before it buys speed.
4. Report two configs, not one: a low-latency one and a high-throughput one, as vLLM did with TP4 and DEP2.
5. Run any bounded or approximate optimisation against your own evaluation set with the flag on and off before turning it on in production.

# bench-card.yaml: fill this in before you quote any serving number.
engine:            vllm | sglang   # exact version or commit
model:             deepseek-ai/DeepSeek-V4.1-Flash
hardware:          4x GB300        # chip, count, interconnect
parallelism:       TP4 | DEP2 | EP4
kv_dtype:          fp8 | nvfp4
speculation:       off | dspark    # if on: simulated or measured accept length?
approximations:    swa-bounded-replay on | off
workload:
  input_tokens:    4096            # SGLang: random ids, seed 42, no chat template
  output_tokens:   1024            # temperature 0, ignore_eos true
  concurrency:     1               # or the session count for an agentic trace
  prefix_cache:    cleared before each run
runs:              6               # one warm-up discarded; report the median
metric:            decode tokens/s | TPGS at P90 interactivity above 50 tok/s
excludes:          full prefill?   # SGLang measures first to last streamed event
quality_checked:   yes | no        # no means the speed number is unverified

The card above is my template for recording a comparison. It copies the fields SGLang pinned, because a result without them cannot be reproduced and cannot be compared with the next post that appears.

A serving speedup is a claim about a workload, a baseline and a metric, and only then about an engine. vLLM and SGLang both got DeepSeek-V4.1-Flash fast in three weeks by the same boring route: make the KV cache small, skip the layers you can, fuse the small kernels, and check what actually dispatches. Copy the route, not the number.