Choosing What It Reads

This chapter builds the second model job: a reranker that reads the question and the pool together and says which three laws answer it.

Retrieval, with everything Chapters 4 through 6 added, ranks by how a law is worded. A manual law written by a person as a plain sentence beside a question outranks the code’s own text for a plain question, whether or not it answers it, because the manual law was written to match questions and the code was not. The deployed pipeline measured this when it first ingested the states’ codes: of eleven rules that the first states’ codes held and the manuals did not, retrieval reached two.

The eight laws retrieval returns are the eight that look most like the question. What the model needs is the three that answer it, and telling the difference means reading the question and each law together and judging. That is a reranker: a stage that takes a pool of candidates and the question, and orders the candidates by whether they answer it rather than by whether they resemble it. A candidate is one law in the pool.

There are two kinds. A cross-encoder is a small model fitted for exactly this: it takes the question and one candidate, and produces a score. It is fast, it runs on the same service as the embedding model, and it scores by likeness, which is its limit. The deployed pipeline’s cross-encoder gave Nevada’s side-window tint question the windshield strip, Pennsylvania’s early licence question the automatic licence at 18, and California’s adult blood-alcohol question three under-21 limits: each a candidate very like the question and not an answer to it.

The other kind is a language model asked to choose. It reads every candidate at once, it can tell an adult’s limit from an under-21 limit because it reads the question’s words, and it answers with a list of numbers. Yours uses that kind, and the deployed pipeline asks it first and keeps the cross-encoder behind it.