Chapter 5 · Techniques

Chapter 5 exam

The serving half — caching, parallelism, disaggregation. Eight questions. Clear it to move on to the modalities.

Builds on

Dynamic disaggregation with NVIDIA Dynamo

Chapter progress

0 / 19 lessons completed

Up next in this chapter: Vision language models →

The second half of the chapter was about serving the model once the tokens are cheap: reusing KV across shared prefixes, tiering and routing the cache, bounding the O(n²) cost of long context, sharding the matmul and the experts across GPUs, respecting the cross-node bandwidth cliff, and finally splitting prefill from decode — statically, then dynamically. This is the capstone: prove the systems reasoning holds together.

Eight questions across caching, parallelism, and disaggregation. Score 80% or better to finish Chapter 5 and move on to the modalities. Every option explains itself, and you can retake it any time.

01 · Chapter exam

Serving at scale — caching, parallelism, disaggregation.

Prefix caching and RadixAttention, the KV storage hierarchy, cache-aware routing, the long-context cost curves, tensor vs expert parallelism and their collectives, the cross-node bandwidth cliff, and when to split prefill from decode — statically and dynamically.

Check your understanding0 / 8

Question 1

Two requests share a long system prompt. How does prefix caching turn that into a win, and what does RadixAttention add over a flat prefix hash?

Question 2

Where should the KV cache live?

Question 3

You deployed prefix caching across many replicas but the hit rate is near zero. What's the most likely cause?

Question 4

As context length n grows, which cost curves does the lesson say you're fighting, and how do sliding-window attention + attention sinks help?

Question 5

Tensor parallelism shards each matmul across GPUs. What does it buy, what does it cost per transformer block, and what constrains the TP size?

Question 6

Expert parallelism spreads a MoE's experts across GPUs. What dominates its wall-clock, and why does fabric choice matter so much?

Question 7

Crossing a node boundary, the lesson says bandwidth drops roughly 18–75×. What's the resulting placement rule for TP and PP?

Question 8

Disaggregated serving splits prefill and decode onto separate pools. Why, what joins them, and when does dynamic disaggregation (à la Dynamo) earn its keep?