Chapter 5 · section 4 of 7

Two rankings into one

The pipeline now has two lists for every question: the sixteen best by words and up to eight by meaning. The scores are not comparable. A BM25 score is a sum that grows with the number of matching words and has no ceiling; a cosine is a number between zero and one. Adding them, or weighting one by a constant, would be a guess, and the deployed pipeline makes none: neither ranking’s raw scores are compared with the other’s, because they are not on the same scale.

Reciprocal rank fusion merges ranked lists by position rather than by score: a law in position r of a list contributes 1 / (k + r) to its total, summed across every list it appears in, and the lists are merged in order of total. The constant k is usually 60, and it is there to keep a first place in one list from swamping a tenth place in all the others. A law near the top of either list makes the shortlist; a law high on both leads it.

Two rankings over the same laws, by words and by meaning, fused by position rather than by score. A law that is first in one list and absent from the other is on the shortlist; a law in the top three of both leads it.

Listing: ask/fuse.ts

import type { Law } from "./corpus.js";
import { RETRIEVED_LAWS, bm25 } from "./retrieve.js";

// Two rankings over the same laws, merged. Neither ranking's raw scores are
// compared with the other's, because they are not on the same scale; what is
// compared is where each law stands in each list.

/** The usual constant for reciprocal-rank fusion: it keeps one list's first place from swamping the other's. */
export const RRF_K = 60;

/** The question again in the law's words (Chapter 6), and what the index found for it. */
export interface Search {
  search: string;
  near: string[];
}

/** One list from several, by reciprocal rank: a law near the top of any list makes the shortlist, and one high on all of them leads it. */
export function fuse(rankings: string[][], k = RRF_K): string[] {
  const fused = new Map<string, number>();
  for (const ids of rankings) {
    ids.forEach((id, rank) => fused.set(id, (fused.get(id) ?? 0) + 1 / (k + rank + 1)));
  }
  return [...fused.entries()].sort((a, b) => b[1] - a[1]).map(([id]) => id);
}

/**
 * The laws most likely to answer `question`, best first: BM25 over the
 * question, the ids the index found nearest it, and each rephrased search's
 * own two rankings, fused. With no `near` and no `searches`, this is BM25
 * alone, byte for byte.
 */
export function retrieve(
  laws: Law[],
  question: string,
  limit = RETRIEVED_LAWS,
  near: string[] = [],
  searches: Search[] = [],
): Law[] {
  const byId = new Map(laws.map((law) => [law.id, law]));
  // Only ids the file holds: a vector left behind for a law since removed is
  // dropped here, never shown.
  const meant = near.filter((id) => byId.has(id));
  if (meant.length === 0 && searches.length === 0) return bm25(laws, question, limit);
  const rankings = [bm25(laws, question, limit * 2).map((law) => law.id), meant];
  for (const { search, near: nearer } of searches) {
    rankings.push(bm25(laws, search, limit * 2).map((law) => law.id));
    rankings.push(nearer.filter((id) => byId.has(id)));
  }
  return fuse(rankings)
    .slice(0, limit)
    .map((id) => byId.get(id) as Law);
}

retrieve is the function the pipeline calls from here on, and bm25 is a detail inside it. Two things about it hold for every stage after this one.

With nothing extra, it is what it was. No ids from the index and no rephrased searches, and retrieve returns bm25 alone, byte for byte. The stage you built this chapter is additive: switching it off gives you Chapter 4’s pipeline, not a broken one. Chapter 10 makes that a rule.

The searches argument is Chapter 6’s. Each one is a restatement of the question with its own two rankings, and retrieve fuses all of them. It is in the signature now so that the function is shown once, complete.