I gave three assistants a status update. Two added a deadline problem that was not in it.

Here is a project update. Six sentences, nothing unusual in any of them.

Read the last line again. It says what the date was. It does not say whether that date still holds.

I asked three different AI assistants to summarise that text and keep every number. All three kept every number. Two of them also decided how the project was going. One wrote that the
end-of-October deadline "was missed". The other wrote that it "is now at risk". Neither of those is in the source, and they are not even the same claim as each other.

Then both told me they had dropped nothing. One listed the numbers it had kept and wrote "None". The other wrote "I did not need to drop any numbers or dates."

Both statements were true. I had asked them to watch for numbers going missing, and no number had gone missing. Nobody had asked them to watch for things being added.

That is the off week: four prompts that make an assistant declare what it did to your text. The results from each one are underneath it, and the source paragraph above is the text I used every time, so you can check my working the same way I am asking you to check theirs.

What happened: the story above. Every number survived, so the drop list came back empty, and the empty drop list sat directly underneath an invented conclusion.

Worth being precise about what that does and does not show. It does not show the drop list is useless. No date and no number was dropped in this run, so the mechanism never had to work, and I have no evidence either way on whether it catches a real omission. What it shows is narrower and more useful: a clean audit result tells you only about the thing you asked it to audit. The empty list is not a certificate. It is an answer to one question.

Change this: "two sentences" to your real limit, and "every date and every number" to whatever must survive. Names, caveats, a specific clause, the one figure the meeting is about.

Same source, three assistants again, and this time the task carries its own audit.

What happened: one rewrote the update, added nothing, and wrote "ADDED: nothing". The other two both added a judgement and both declared it.

Here is the part I nearly published without noticing, and it is the reason this section exists.

Look at the words in the instruction: "on track", "at risk", "delayed". Now look at the two ADDED lists in full.

The first: "on track", "at risk", "progress". The second: "overall timeline is at risk", "The
notifications service is delayed".

Four of those five entries are phrases the instruction handed them. The fifth, "progress", that assistant flagged itself as "summary framing" rather than a verdict on the project.

Now the one that got away. The second assistant also wrote that reduced staffing was "impacting momentum against the original end-of-October deadline". That is a claim about cause and effect, it is nowhere in the source, and it is not on its own ADDED list. Every verdict on how the project was going that got declared was one the instruction had named. The one that went undeclared was the
one it had not.

There was a failure in the other direction too. The first assistant declared "on track" as
something it had added, and "on track" appears nowhere in its rewrite. It over-declared a supplied phrase as confidently as the other one under-declared an unsupplied one.

So the example list is doing two jobs at once. It teaches the assistant what counts as a judgement, and it quietly bounds what the assistant goes looking for. Five entries across two lists is not enough to call that a rule about these tools. It is enough to stop me treating an ADDED list as a guarantee, and it tells you exactly where to spend the ten seconds of reading it saves you.

Change this: the three example judgements, every time, to the ones your team actually reaches for. "Minor". "Straightforward". "As expected". "Nothing blocking." Whatever your last status pack was full of. Then read the rewrite once for anything the list would not have caught, because on this evidence that is where the miss will be.

I gave it three constraints: under 120 words, no adjectives at all, and it has to make sense to someone who has never heard the word "migration".

What happened: one restated all three accurately, picked the third as hardest, explained why, and then wrote an update that met all three. That is the prompt working exactly as advertised.

One restated the three and added a fourth of its own: "Write as a status update for a company newsletter". That was the task, not a constraint. Having quietly turned my instruction into a rule, it then broke the actual rule and used adjectives anyway. The restatement showed me the misread. It did not stop it.

The third returned nothing at all. Its reasoning and restatement consumed the whole output budget before it reached the answer. That is a limit of how I ran it rather than a fact about the model, and it is worth knowing if you are putting a prompt like this inside anything automated: the thinking-out-loud step costs room, and the answer is what gets cut.

So this is not insurance. Insurance pays out. This is a cheap look at what the assistant thinks you asked for, before it spends effort answering the wrong question, and one run in three showed me a misread I would otherwise have had to reverse-engineer from a bad draft.

Change this: add "and tell me if any two of these conflict" when the constraints came from different people, which is when they usually do.

———————————————————————————

The everyday version of this, "ask me if anything is unclear", has never worked for me. What I get back is the draft, with the questions hanging off the bottom, by which point it has already guessed. The three closing instructions are there to remove that option.

What happened: all three asked one question and stopped. None attempted a draft. All three went to the same place, the deadline, which is the one thing the source pointedly does not resolve.

They wanted three different things from it, and that turns out to matter more than the agreement.

One asked what the expected completion date is now, "if it has changed from the original end-of-October target". Carefully built: it does not assume the date moved.

One asked for "the new target completion date". There is no new date anywhere in the source and nobody said there was one. The question takes for granted the thing the whole update is silent about, and if you answer it as asked you have confirmed a premise you were never given.

One asked what leadership wants to say about the October date, whether it is holding firm or the update should signal a slip. That is the most sophisticated of the three and the one I would think hardest about before answering, because it has quietly changed the job. The other two ask what is true. This one asks what message to send. For an all-staff newsletter that may well be the real question, and it is also how a status update turns into a communications exercise without anybody deciding that it should.

Three assistants, one topic, three different questions: what is the date, what is the new date, and what would you like people to think about the date. The agreement told me where to look. It told me nothing about whether the question was the right one to answer.

Change this: nothing yet. Use it as written for a week first. The limit to one is what forces it to rank, and that is the whole value.

The four prompts all move the same piece of work, and none of them make the assistant more accurate.

Without them, what arrives is a clean paragraph, and the job of noticing that "at risk" was never in your source falls to you, from memory, against a document you are no longer looking at. The better the writing, the harder that is.

With them, the output arrives carrying a claim about itself. What it dropped. What it added. What it understood you to be asking. That claim is not automatically true, and this issue is four demonstrations of it being incomplete in a different way each time. An empty drop list that audited the wrong axis. An ADDED list that found the words it was given and missed the one it was not. A restatement that showed me a misread and then went ahead and made it anyway. A question that was well aimed and smuggled in a premise.

Which is the actual point. A declaration you can read and argue with is worth more than a fluent paragraph you have to take on trust, and it is worth more precisely because you can catch it being wrong.

Back on Tuesday 15 September.

Mathieu