Chapter 9 · section 4 of 8
A model may choose, and code decides
The second half of that listing is the one place in the corpus where a model does work, and it is the pattern for every such place.
The agency’s pages, from Chapter 8, hold the answers no section of a code holds: how soon a failed test can be retaken, what a ticket means. There are about six pages a state and fifty-one states, and reading 306 pages by hand to pick the sentences that state a rule is a week’s work. So a model is asked to copy out at most five passages from each page that state the family’s rule, character for character.
A model asked to copy can paraphrase. It can also fix a spelling, join two sentences that are apart on the page, or write “10 days” where the page says 15. So every passage it proposes is kept only if it is a substring of the page as fetched, after the one normalisation pageText applies: tags dropped, white space collapsed, curly quotes made straight. Nothing else is forgiven. Not a digit, not a letter, not case. And the match is on whole words: “5 days before the re-test” is in a page that says “15 days before the re-test” as letters, and onThePage rejects it.
A passage that fails is dropped and named in the script’s output, with the reason. The deployed script’s comment says what the alternative would have been: a paraphrase stored under the agency’s name is a wait or a deadline nobody published.
The curly-quote rule is itself a measurement. Maryland’s “Hold a learner’s permit for at least 9 months” was dropped for its apostrophe alone, because the page typesets a curly one and the model straightened it. The normalisation forgives that and nothing more.