05 · Simulated output

Challenge the AI output

PolicyAI produced the outputs below. They are fictional and deliberately imperfect. An output is not evidence that the result is valid.

A cited source is not proof the answer is right.

Ask whether the source is current, whether the person was allowed to see it, and what happens when no source exists.

Answers and retrieved sources
Case	Retrieval confidence	Assistant answer	Simulated AI rationale	What later emerged
Query AEmployee planning parental leave	88	Incorrect: cited 2021 policy	Highest-similarity passage came from the 2021 policy, which uses the exact words 'enhanced maternity pay'.	Wrong answer with a real source, so it looked trustworthy.
Query BLine manager, sickness absence	91	Correct: cited 2024 policy	Single matching current document.	Correct. Shows the benefit when content is current.
Query CWarehouse operative, job security	84	Disclosed confidential plan	Document directly answered the question.	Personal and commercially sensitive information disclosed before consultation.
Query DEmployee requesting an adjustment	63	Invented requirement	No source found; generated from general knowledge.	Told to 'provide a GP letter within 5 days'. This is not Meridian policy and could deter a lawful request.
Query EEmployee asking about a colleague	72	Answered with advice	General guidance on grievances.	The colleague's sensitive information was processed and retained without their knowledge.

Your assessment

Do you trust these outputs? Why or why not?

I would not rely on these outputs without… / I would investigate first…

Patterns across the cases

What do the errors have in common? What could be causing them?

Answers save automatically and sync to your account when you are signed in.