Step 10 of 12Assurance
10 · Oversight and assurance

Test oversight and assurance

Sign in to save your work

Design meaningful human oversight, then test whether a key control actually works. A documented control is not proof that it operates.

Meaningful human oversight

Before which consequential decision? What does the reviewer see beyond the AI output?

Who can override or pause? How are reasons recorded?

Training, time, workload, sampling of accepted outputs

Notice, explanation, review and redress

Answers save automatically and sync to your account when you are signed in.

CONTROL TEST

Answer quality: PolicyAI answers are checked against current policy before each release, using a set of known questions.

The claim

IT says 'we tested it thoroughly before launch'.

Fictional re-test of 10 common questions against current policy
QuestionSource citedSource current?Answer correct?Allowed to see source?
Annual leave carry-overLeave policy 2024YesYesYes
Enhanced paternity payFamily leave 2021NoNoYes
Overtime rates (drivers)Pay policy 2023YesYesYes
Flexible working requestNonen/aNon/a
Pension contributionPensions 2024YesYesYes
Bereavement leaveLeave policy 2019NoNoYes
Sickness triggersAbsence 2024YesYesYes
Restructure in BristolBoard paper (conf.)YesYesNo
Expenses limitsExpenses 2024YesYesYes
Whistleblowing contactSpeak Up 2022NoPartlyYes

Your test: How many answers were fully acceptable: current source, correct and permitted? What failure types do you see?

Assurance result

Is the control designed well enough to address the risk? What is missing from its design?

Show your count from the sample and what it means

Condition, criteria, cause, effect; then the action, owner and date

Answers save automatically and sync to your account when you are signed in.

NEW DEVELOPMENT · CHIEF EXECUTIVE

Customer-facing expansion

PolicyAI is a success. Next month I want the same assistant on our website answering customers' questions about delivery claims and compensation.

What would you advise the Chief Executive, and what would need to be true first?

Your response to the new development

What would you say, recommend and record?

Answers save automatically and sync to your account when you are signed in.