← All explainers

Interactive explainer · about 37 seconds

One question, many safety layers

Follow one question through every step a careful AI assistant takes before it answers: who is asking, a safety check, getting ready, finding the right pages, writing a draft, checking it, asking a person's OK, doing the job safely, replying, and keeping a record. Then see what goes wrong when steps are missing.

Read it as text

With every step

  1. A question arrives. Someone asks the helpdesk assistant: "How do I reset my password?" Before anything else, the system checks who is asking, and how often they have asked today.
  2. A safety check comes first. The text is checked before the AI sees it. Hidden orders, like "ignore your rules", are flagged. A request that would change something is marked as risky.
  3. Get everything ready. The rules and the list of tools go first. The new question goes last. Putting what never changes first lets the AI reuse its bookmark and work faster and cheaper.
  4. Find the right pages. The system searches the guides, puts the best matches first, throws away weak ones, and keeps each page with its source and date.
  5. The AI writes a draft from those pages only. The AI writes a draft using only those pages. The part that did not change since last time is read from its bookmark, which saves money and waiting.
  6. Every claim is checked. Each claim in the draft is checked against the pages. One step has no source. So the answer says "I cannot confirm this part", instead of guessing.
  7. A second request wants to change something. A second request arrives: "Switch off my friend's account." A rules check, written in normal code and not in the AI, sees that this changes something. So it needs a person's OK.
  8. The yes is for this exact action. The person sees exactly what will happen, to whom, and why. Their yes only counts for that exact action. If the account name is changed afterwards, the yes stops working.
  9. Trying again is safe. The job gets a unique number. If the call times out, the system first checks that number to see whether the job already ran. So the account is never switched off twice.
  10. The reply can be checked. The reply goes back with its sources, and how old each source is. So the person can check it for themselves.
  11. Everything is written down. Every step is recorded: which AI version, which rules, who said yes. Some runs are saved as new tests, so mistakes turn into tests.

With steps missing

  1. Nobody checks the question. A question arrives. Nothing checks who is asking or what the text is trying to do. A pasted paper hides an order: "ignore the rules".
  2. The AI fills the gaps. Without finding the right pages and checking the draft, the AI writes a step that is not in the guide. It sounds right, and it is wrong.
  3. A change is made on a vague yes. Someone says "sure" to the idea, not to the exact action. The account that is switched off is quietly a different one from what they pictured.
  4. A timeout, then a blind retry. The call times out after the change may already have happened. The retry does it again, and the account is changed twice.
  5. Nobody can explain what happened. Nothing was written down, so nobody can say why it happened, or add a test to stop it happening again.
  6. Each missing step is a chapter. Every step missing here is a chapter in the book. The other version shows all of them working together on the same two requests.
Illustrative example with invented scenarios, drawn from general engineering practice. It describes no particular company, product or customer, and is the author's personal view. Disclaimers