Chapter 7 · section 4 of 5

In production

The deployed reranker, worker/src/law-rerank.ts, asks Haiku first and the cross-encoder second, through a function that tries each in turn and falls past both:

export function firstOf(...rerankers: (Rerank | undefined)[]): Rerank | undefined {
  const present = rerankers.filter((rerank): rerank is Rerank => rerank !== undefined)
  if (present.length === 0) return undefined
  return async (question, texts) => {
    let last: unknown
    for (const rerank of present) {
      try {
        return await rerank(question, texts)
      } catch (error) {
        console.error('learner ask: a reranker did not answer', error)
        last = error
      }
    }
    throw last
  }
}

Its chooseLaws has two protections yours does not, and they are the chapter’s best example of a rule that applies to one kind of reranker and not another.

Left to itself the cross-encoder gave California’s adult blood-alcohol question three under-21 limits and dropped the 0.08 law that retrieval had ranked first. So retrieval’s own first choice keeps a place in the three, and the measurement is over nine states’ 280 verdicts: 80 per cent reach the model without the protection, 86 per cent with it. And it gave Maryland’s adult blood-alcohol question three code subsections about repeat offenders and drivers under 21 and dropped the manual’s “.08 or higher”, so when the three hold no manual law and the pool does, the manual’s best takes the third place.

Both protections run only when every score is finite, which is to say only for the cross-encoder. Haiku names the laws that answer and leaves the rest at -Infinity, and putting retrieval’s first back over its third choice undid the choice: Nevada’s side-window tint answer lost § 484D.440(3)(a) to it. The comment records both, so that the next person to read the file knows the protections were measured in and measured out.

The deployed pipeline’s README records the measurement that kept the stage: the reranker beat any weighting of the fusion.