System designs

Worked LLM system designs

Each design follows the same seven steps, so you can compare them and practise the method instead of memorising answers. Read one, then close the tab and try it yourself in 45 minutes.

• 01Design enterprise RAG with a sub-second latency budget

  A multi-tenant retrieval-augmented assistant for service desk agents that must answer inside a live workflow. Latency budget, cost math, access control, freshness, and how to know it's working.

  RAGlatencymulti-tenantevaluation

• 02Design an AI agent that resolves IT tickets end to end

  An agentic system that triages support tickets, takes real actions to fix them, and writes up the resolution. Where to use agents and where not to, action safety, durable execution, and evaluating something that changes the world.

  agentstool usesafetyorchestration

• 03Design an evaluation platform for 50 LLM features

  A shared platform that lets product teams measure and protect the quality of their LLM features, offline in CI and online in production. Versioned datasets, calibrated LLM judges, statistics that don't lie, and the cost of judging.

  evaluationLLM-as-judgeplatformobservability

In progress: an LLM gateway for 50 internal teams, a chat assistant serving at scale, and a coding agent with tools and MCP.