Joel Stevick

Chapter 1 · section 7 of 9

The system prompt

A prompt is the text you give the model to act on. In casual use people mean the question they typed; in the API it is the whole input: every message in the array plus the field this part of the chapter is about.

The system prompt is a separate top-level system field carrying instructions about how to behave, as opposed to what to answer. It is not a message and has no role; it sits outside the array. Two things make it different from putting the same words in the first user message:

  • It comes first, always, no matter how long the conversation grows.
  • It reads as operator instruction rather than as something a participant in the conversation said, which makes it harder for later text to talk the model out of it.

In Chapter 2 the rules the model writes under go in system, and the laws it writes from go in the user message. The split is not cosmetic. The laws are text from outside: a statute, a court’s schedule, a page from an agency’s website. Putting them in the same channel as the rules would mean that a passage of fetched text sits where instructions sit.

Rule: Instructions to the model go in system. Material for the model to work on goes in messages. When you find yourself concatenating the two into one string, stop: you are removing the only structural distinction the API gives you between what to do and what to do it to.