Tune in the order the bottleneck moves, and log a prediction for every step

Before each change, write which resource limits the system, what the change should do and by roughly how much. Then rerun the same workload. A measured log went from 789 to 6,568 tokens per second, and its wrong predictions taught more than its right ones.

26 MIN · PREMIUM