INFERENCE NOTEBOOK
Decode faster, verify exactly
Interactive concept lab
Let a small model guess. Let the big model check.
Speculative decoding separates token proposal from token verification. Move the controls and watch accepted runs, verifier work, and effective speed change together.
The invariant
The verifier still decides the final next-token sequence. The draft model only proposes a short candidate block.
Draft and verify timeline
Each row is one decoding round. Green tokens were accepted by the verifier; coral tokens were rejected and replaced.
1.00x
accepted draft tokenverifier replacementverification pass
Primary sources
Accelerating Large Language Model Decoding with Speculative Sampling
SpecInfer: Predicting Text Generation with Speculative Inference
Build the mental model
The chart is a compact simulator, not a benchmark. Real throughput also depends on batching, kernels, memory bandwidth, and whether draft and target models share infrastructure.