You've built the serving foundation: containerizing a model the right way, pinning the fragile dependency chain, deploying from a NIM instead of rolling your own, turning static batching into continuous batching, understanding what makes a cold start slow, routing load with an awareness of queue depth, scaling to zero responsibly, and decomposing services so each component scales to its own bottleneck. Prove it stuck before the infrastructure and ops chapters.
Eight questions. Score 80% or better to unlock the GPU procurement, geo-routing, reliability, and observability lessons. Every option explains itself, and you can retake any time.
01 · Checkpoint
Containers, batching, routing — the serving mechanics.
When building a serving container with Docker, what principle should govern the order of your COPY/ADD layers?
Question 2
The inference dependency chain — driver, CUDA runtime, PyTorch, vLLM, kernels — has a specific property that makes upgrading it tricky. What is it?
Question 3
What does an NVIDIA NIM container include that distinguishes it from a plain vLLM Docker image?
Question 4
A static-batching engine holds all sequences in a batch until the longest one finishes. What does continuous batching change?
Question 5
Cold-start time for a 70B model is dominated by two phases. Which are they?
Question 6
A load balancer routes inference requests across a pool of vLLM workers. Which algorithm is best suited to minimize queue depth and latency when request lengths vary significantly?
Question 7
Scale-to-zero reduces cost during idle periods but introduces a trade-off. What is it?
Question 8
A serving system has three components: a tokenizer service, a vLLM decode engine, and a reranker. The tokenizer is CPU-bound, the engine is GPU-bound, and the reranker is CPU-bound. Why should you scale them independently rather than as a single monolith?