Problem solving

Practical coding for AI engineering loops

AI labs and platform teams have moved away from puzzle questions toward practical implementation: build a cache, a rate limiter, attention from scratch, then extend it as the interviewer adds requirements. Every solution here is Python with tests, because the tests are graded too.

• Implement an LRU cache with per-entry TTL

  The classic cache question with a time dimension added, the way AI platform teams ask it. Design choices, a tested Python solution, and the follow-ups that decide the round.

  Warm-up · Common in AI platform and backend loops

• Build a token bucket rate limiter for an LLM gateway

  Per-key rate limiting with weighted costs, so it can limit tokens per minute as well as requests. A thread-safe Python solution with tests, and the distributed follow-up.

  Medium · Common in AI platform and infrastructure loops

• Implement causal attention and a KV cache from scratch

  Multi-head attention with grouped-query attention and a KV cache in NumPy, plus the test that proves cached decoding matches the full forward pass. Increasingly asked of ML and AI engineers.

  Hard · Reported in ML and AI engineer loops at frontier labs