Blog

AI / Machine Learning

Deeper machine learning articles — model selection, embeddings, and the core ML concepts behind applied AI systems.

18 articles

AI/MLOctober 202612 min read

Datadog Scout 1.0: What Cheaper AI SRE Investigations Mean

Datadog says its RL-trained Scout model investigates alerts at a fraction of frontier-model cost. A close reading of AlertEval, SREGym, the credit pricing and the limits.

AI/MLOctober 202611 min read

LMCache Multi-Process Mode: Shared KV Cache for MoE Serving

LMCache multi-process mode replaces per-rank KV offload with one shared host-memory pool for vLLM DP+EP serving. A close reading of the benchmark, config and limits.

AI/MLOctober 202612 min read

vLLM Semantic Router v0.4: Mixture-of-Models Routing Guide

How vLLM Semantic Router v0.4 routes one model name across a pool with signals, decisions, algorithms and plugins, and when it beats a static model choice.

AI/MLOctober 202612 min read

KServe LLMInferenceService Guide: llm-d, WVA and vLLM

A close reading of KServe v0.17 and v0.18 LLMInferenceService: llm-d routing, WVA autoscaling, LeaderWorkerSet, and where it beats raw vLLM Deployments.

AI/MLOctober 202612 min read

OCI Model Artifacts: Faster Kubernetes Model Cold Start

A close reading of the 2026 KServe cold-start study: image volumes, modelcar and object storage compared at 2, 14 and 140 GB, plus what the first cold pull and integrity checks cost.

AI/MLOctober 202612 min read

NVIDIA Dynamo on Kubernetes: Rust EPP and v1beta1 Upgrade

What Dynamo 1.5 changes for operators: the Rust Endpoint Picker, v1beta1 CRD storage, TLS and mTLS for transports, KVBM deprecation, and a safe blue/green EPP migration.

AI/MLOctober 202612 min read

DeepSeek V4.1 Flash on vLLM vs SGLang: What the Numbers Mean

vLLM reports 5x agentic throughput and SGLang 35 to 873 tokens per second for DeepSeek-V4.1-Flash. A close reading of both posts: what each measured, what is comparable, and what to copy.

AI/MLOctober 202612 min read

SGLang Unified Radix Cache: Prefix Caching for Hybrid Models

How SGLang's Unified Radix Cache and HiCache tiers (GPU, host, Mooncake) make prefix caching work for hybrid attention and Mamba models, with real numbers.

AI/MLOctober 202612 min read

vllm-metal on Apple Silicon: a Local vLLM Server for Mac

What vllm-metal brings to a Mac: continuous batching, a paged KV cache and an OpenAI-compatible server on MLX. Install, memory budget, MTP, and honest limits.

AI/MLOctober 202612 min read

vLLM 0.31 Release Notes: Preload, Model Runner V2 and the Flags That Break

A close reading of the vLLM v0.29, v0.30 and v0.31 release pages and the preload docs: the weight-cache daemon that makes restarts fast, where it silently falls back to disk, and the removed flags to grep for first.

AI/MLOctober 202612 min read

vLLM Bi-Directional KV Transfer for Multi-Turn Agents

Multi-turn agents resend their whole history, and a disaggregated prefill node recomputes what the decode node already holds. A close reading of vLLM's bi-directional KV transfer: the six steps, the TTL and threshold knobs, the numbers and what breaks.

AI/MLOctober 202612 min read

vLLM Disaggregated Prefill Decode: A NixlConnector Guide

How to split vLLM into prefill and decode instances with NixlConnector, what vLLM's own L40S goodput numbers show, and the transfer checks that stop you benchmarking a broken setup.

AI/MLOctober 202612 min read

vLLM NixlPushConnector: Push vs Pull KV Transfer and TTFT

vLLM's push-based KV transfer registers decode blocks early and lets prefill write them, cutting TTFT roughly 3x in its own test. A close reading of the mechanism, the numbers, the failure modes and when pull still wins.

AI/MLJuly 202610 min read

Hybrid Search and Reranking: Better RAG Retrieval

Boost RAG accuracy with hybrid search and reranking, fusing BM25 keyword search with vector search, then reranking with cross-encoders before generation.

AI/MLJuly 20269 min read

Choosing an Embedding Model: A Practical Benchmark

A practical framework for picking an embedding model for RAG: dimension versus cost versus recall, why MTEB scores can mislead you, and a script to benchmark candidates on your own data.

AI/MLMay 202610 min read

Semantic Search with pgvector: Skip the Dedicated Vector Database

You probably do not need Pinecone. How to build production semantic search with pgvector inside the PostgreSQL you already run, indexes and hybrid search included.

AI/MLApril 20269 min read

RAG vs Fine-Tuning: An Honest Decision Framework

RAG and fine-tuning solve different problems, and most teams pick wrong. A practical framework for deciding which one your LLM feature actually needs.

AI/MLMarch 202612 min read

Building Production RAG LLM Applications: Architecture, Chunking, and Evaluation

A practical guide to architecting production-grade Retrieval-Augmented Generation systems using LangChain, pgvector, and RAGAS evaluation metrics. Covers chunking strategies, vector store setup, retrieval optimization, and how to measure answer quality in real deployments.

Back to all posts