09Your new model deployment produces fluent answers that score badly on evaluations. Where do you look?▼mediumNewBasetenTogether AIModal4 repliesunlockedFluent and wrong is a plumbing failure, not a model failure, because a model whose weights are broken produces obvious nonsense. Five places the plumbing goes wrong, the reproduction that isolates each, and the one that is invisible until an evaluation runs.Open full answer →
32Your agent product's tool-call success rate fell from 94 to 61 percent after a deploy. Debug it.▼mediumNewBasetenTogether AIModal4 replies◆ premiumTool calling passes through four layers and a break in any one produces the same symptom, so the debugging is a bisection rather than an investigation. Which layer the size of the drop points at, the deploy diff that usually contains the answer, and why no latency metric moved.Open full answer →
