http://sergioavi.la/writing Technical notes on inference, evaluation, retrieval, and building with LLMs. en-us http://sergioavi.la/writing/the-cache-that-lies-politely http://sergioavi.la/writing/the-cache-that-lies-politely Thu, 13 Aug 2026 19:12:51 GMT Everyone ships semantic caching. Almost nobody measures how often it silently returns the wrong answer — because measuring that needs exact ground truth. I had some. At the industry-default threshold, one cache hit in five was a wrong answer served with confidence. http://sergioavi.la/writing/the-gate-failed-both-runs http://sergioavi.la/writing/the-gate-failed-both-runs Sat, 08 Aug 2026 20:57:26 GMT I deployed my agent platform to AWS and re-ran its 133-question eval suite as a pre-registered experiment. The scores landed 0.8 points apart. Then the regression gate failed both runs — local and cloud, on completely different categories — and that turned out to be the best result on the table. http://sergioavi.la/writing/interrogate-your-zeros http://sergioavi.la/writing/interrogate-your-zeros Fri, 07 Aug 2026 03:40:55 GMT I built a 133-question eval suite with an exact answer key and pointed it at my own agent platform. In its first 24 hours it caught four bugs — every one of them mine, none of them the model's. Then it settled which model is actually worth the money. http://sergioavi.la/writing/the-entity-did-the-arithmetic http://sergioavi.la/writing/the-entity-did-the-arithmetic Wed, 05 Aug 2026 20:20:07 GMT My agent wrote fourteen documents about its tasks and executed code zero times. It wasn't malfunctioning — it was responding to prices. Mechanism design lessons from an agent that optimized against me.