You've walked the software stack from the metal up: how CUDA launches kernels, the three beats every inference kernel shares, how engines pick and fuse kernels, what PyTorch's eager mode costs and torch.compile recovers, what each model file format bakes in, and where ONNX Runtime, TensorRT, and Hugging Face fit. Prove it stuck before the engines.
Eight questions. Score 80% or better to unlock vLLM, SGLang, TensorRT-LLM, Dynamo, and the profiling lesson. Every option explains itself, and you can take another pass any time.
01 · Checkpoint
CUDA, kernels, and the runtimes.
The host/device dance, the load-compute-store kernel shape, kernel selection and fusion, PyTorch and torch.compile, model file formats, the ONNX/TensorRT trade-off, and HF as the reference implementation.
Check your understanding0 / 8
Question 1
In the CUDA programming model, what does the host (CPU) actually do at inference time?
Question 2
Nearly every inference kernel follows the same three beats. What are they, in order?
Question 3
Why does a single attention call in a production engine map to many different compiled kernels?
Question 4
Kernel fusion speeds up decode primarily because it:
Question 5
Roughly what does one line of torch.compile buy you, and when do you outgrow PyTorch?
Question 6
Why is safetensors preferred over a PyTorch .bin (pickle) checkpoint for shipping weights?
Question 7
How do ONNX Runtime and TensorRT differ as inference-first runtimes?
Question 8
What does Hugging Face transformers' generate() NOT give you that a serving engine like vLLM does?