LLM and GenAI system design for Staff and Principal engineering interviews: worked designs with cost math, practical coding problems, and reference numbers. By Sunil Pandey.https://llmsystemdesign.com/https://llmsystemdesign.com/designs/enterprise-rag-sub-second/https://llmsystemdesign.com/designs/enterprise-rag-sub-second/A multi-tenant retrieval-augmented assistant for service desk agents that must answer inside a live workflow. Latency budget, cost math, access control, freshness, and how to know it's working.Fri, 16 Oct 2026 00:00:00 GMThttps://llmsystemdesign.com/designs/llm-evaluation-platform/https://llmsystemdesign.com/designs/llm-evaluation-platform/A shared platform that lets product teams measure and protect the quality of their LLM features, offline in CI and online in production. Versioned datasets, calibrated LLM judges, statistics that don't lie, and the cost of judging.Fri, 16 Oct 2026 00:00:00 GMThttps://llmsystemdesign.com/designs/multi-agent-service-desk/https://llmsystemdesign.com/designs/multi-agent-service-desk/An agentic system that triages support tickets, takes real actions to fix them, and writes up the resolution. Where to use agents and where not to, action safety, durable execution, and evaluating something that changes the world.Fri, 16 Oct 2026 00:00:00 GMThttps://llmsystemdesign.com/problems/attention-and-kv-cache/https://llmsystemdesign.com/problems/attention-and-kv-cache/Multi-head attention with grouped-query attention and a KV cache in NumPy, plus the test that proves cached decoding matches the full forward pass. Increasingly asked of ML and AI engineers.Fri, 16 Oct 2026 00:00:00 GMThttps://llmsystemdesign.com/problems/lru-ttl-cache/https://llmsystemdesign.com/problems/lru-ttl-cache/The classic cache question with a time dimension added, the way AI platform teams ask it. Design choices, a tested Python solution, and the follow-ups that decide the round.Fri, 16 Oct 2026 00:00:00 GMThttps://llmsystemdesign.com/problems/token-bucket-rate-limiter/https://llmsystemdesign.com/problems/token-bucket-rate-limiter/Per-key rate limiting with weighted costs, so it can limit tokens per minute as well as requests. A thread-safe Python solution with tests, and the distributed follow-up.Fri, 16 Oct 2026 00:00:00 GMThttps://llmsystemdesign.com/notes/why-this-site/https://llmsystemdesign.com/notes/why-this-site/Most LLM interview material is either a list of questions or a diagram without numbers. This site tries to be neither.Fri, 16 Oct 2026 00:00:00 GMT