Skip to main content

Blog

Thoughts & Stories

Sharing my experiences, learnings, and adventures in tech and beyond.

Browse by Topic

DevOps & Cloud Infrastructure (193)AI & LLMs (143)Backend & APIs (102)Web Development (90)ERP Systems (77)Security (65)Software Development (58)Career Reflections (29)Cloud (29)AI / Machine Learning (18)Product & SaaS (12)University (1)Hackathons (1)
Gateway API Inference Extension: EPP Moves to llm-d, What Now
DevOpsOctober 202611 min read

Gateway API Inference Extension: EPP Moves to llm-d, What Now

A migration note on the Endpoint Picker and Body Based Router moving to llm-d: what stays in the SIG repo, the Helm key mapping, fail-open behaviour and a Monday checklist.

Read More
llm-d on Kubernetes: Distributed Inference and v0.10 Hardening
DevOpsOctober 202612 min read

llm-d on Kubernetes: Distributed Inference and v0.10 Hardening

llm-d layers an LLM-aware router on vLLM and the Gateway API Inference Extension. I read the v0.10 notes, values files and shutdown config to see what hardening means.

Read More
NVIDIA Dynamo on Kubernetes: Rust EPP and v1beta1 Upgrade
AI/MLOctober 202612 min read

NVIDIA Dynamo on Kubernetes: Rust EPP and v1beta1 Upgrade

What Dynamo 1.5 changes for operators: the Rust Endpoint Picker, v1beta1 CRD storage, TLS and mTLS for transports, KVBM deprecation, and a safe blue/green EPP migration.

Read More
DeepSeek V4.1 Flash on vLLM vs SGLang: What the Numbers Mean
AI/MLOctober 202612 min read

DeepSeek V4.1 Flash on vLLM vs SGLang: What the Numbers Mean

vLLM reports 5x agentic throughput and SGLang 35 to 873 tokens per second for DeepSeek-V4.1-Flash. A close reading of both posts: what each measured, what is comparable, and what to copy.

Read More
SGLang Unified Radix Cache: Prefix Caching for Hybrid Models
AI/MLOctober 202612 min read

SGLang Unified Radix Cache: Prefix Caching for Hybrid Models

How SGLang's Unified Radix Cache and HiCache tiers (GPU, host, Mooncake) make prefix caching work for hybrid attention and Mamba models, with real numbers.

Read More
vllm-metal on Apple Silicon: a Local vLLM Server for Mac
AI/MLOctober 202612 min read

vllm-metal on Apple Silicon: a Local vLLM Server for Mac

What vllm-metal brings to a Mac: continuous batching, a paged KV cache and an OpenAI-compatible server on MLX. Install, memory budget, MTP, and honest limits.

Read More

Showing 121–126 of 818 posts

Explore More

Projects

See the projects I've built

Experience

My professional career

Certifications

Professional certifications