Understanding Inference for Systems Engineers

A stage-by-stage walk through what actually happens when a large language model generates text, told through one real engine built for one model on one GPU: tokenization, the forward pass layer by layer, fp8 weights, prefill, CUDA graphs and megakernels, sessions, speculative decoding, grammar-constrained decoding, and correctness.