LLM System Design
Note
Why I started LLM System Design
Most LLM interview material is either a list of questions or a diagram without numbers. This site tries to be neither.
When I look at what’s online for LLM system design interviews, I mostly find two things.
The first is long lists: “50 RAG interview questions”, each with a paragraph-long answer. They’re useful for vocabulary, and useless for a 45-minute design round, where nobody asks you to define a vector database. They ask you to design something, and then they push.
The second is architecture diagrams with boxes labelled “LLM”, “Vector DB” and “Guardrails”, and not a single number anywhere. A diagram like that can’t be wrong, which is exactly the problem. At Staff level the interviewer wants to see you commit: this many tokens, this latency budget, this cost, and this is the part that will break first.
So this site does a small number of designs properly. Each one states its assumptions, does the arithmetic in the open, argues the trade-offs, and ends with the weakest parts of its own design. If you disagree with an assumption, you can change it and redo the math. That’s the skill.
I’ll add designs and problems steadily, and I’ll correct mistakes with a dated note rather than silently. If there’s a design you’d like to see worked through, tell me on LinkedIn or X.