sampling

Choose between local candidates, radix selection and sorting from k and the required output. A tested FP32 reference makes ties, signed zero, NaNs and threshold selection explicit.

Greedy decoding chooses the highest-scoring token, but identical prompt text does not guarantee identical inputs or numerical execution. Check the effective request and deployment, investigate the first differing token, and define the reproducibility guarantee precisely.

Test the complete serving bundle with contract fixtures, representative tasks and load-aware outcomes. Preserve paired results, calibrate the scorer and distinguish a relative improvement from meeting an absolute product requirement.

Trace a stalled stream without unbounded telemetry, understand what sampling loses, and reconcile GPU allocation with a stated cost policy.

Implement stable temperature, top-k and top-p sampling with explicit tie rules, masks, invalid inputs and probability tests.