Blog · RSS

Built, then measured

Engineering write-ups on building with AI, every number from a run committed beside it.

1. November 11, 2026EvalsAsk the book

   What a golden set found in my book's chatbot

   Thirty-two questions, three runs and under two dollars. The book's index had never built, code questions never got their code, and my scorer was older than my rules.

2. October 30, 2026Tuning the demo

   The typewriter effect, with a fact-checker in the way

   The demo checks every answer against its sources, so I taught it to type one checked sentence at a time. The eval set caught a version losing 8 of 26 answers.

3. October 26, 2026Running the shop

   Teaching my site to run the shop

   Every five minutes my site works out what is due and does each thing it can, as me. It tells me only what needs my hands, one line with one button.

4. October 25, 2026Finding clients

   Teaching my site to find its own prospects

   One button on my admin page searches the web for companies that may need my services, checks each find in code, and learns from earlier finds.

5. October 21, 2026Tuning the demo

   Giving the answer the rest of the point

   Adding each passage's neighbours answered all 5 two-passage questions, up from 4, at 60% more a question, over my 50% limit. I switched it on anyway.

6. October 21, 2026Tuning the demo

   “The passages don't list his job history.”

   My demo chatbot said my résumé listed no jobs. How I fixed it test-first, then tuned the pipeline with a 25-question eval on live models, for $1.96.