AI GRCAPractitioner Programme
Step 3 of 12Evidence
03 · Facts, claims and gaps

Examine the evidence pack

Sign in to save your work

Read every document. Separate what is confirmed from what is claimed, and note what is missing.

Business request

Request for an independent review

From: Chair, AI Governance Committee

Complaints about blocked payments have doubled since launch. Fraud Operations says the model is working; Customer Operations says it is hurting customers. The Committee wants an independent recommendation before the supplier's next model update.

  • Decide whether automated blocking may continue unchanged
  • Identify the controls and evidence required
  • Report to the Committee in two weeks
Performance data

Twelve-week operating figures

From: Fraud Operations MI

All figures are fictional. Every flagged transaction has since been confirmed as fraudulent or genuine.

MeasureResult
Card transactions scored100,000
Flagged for verification or block4,000
Flagged and confirmed fraudulent1,000
Flagged and confirmed genuine3,000
Complaints about incorrect blocks120
Urgent customer cases (essential payments)25
  • The data does not show how much fraud the model missed
  • It does not separate verifications from blocks
  • It is not broken down by customer group, age or vulnerability
Supplier claim

Sentinel Analytics product sheet (extract)

Sentinel FraudScore is 95% accurate and reduces fraud losses by up to 40%. Validated on 50 million transactions across European markets.
  • No validation report has been supplied to Northbridge
  • 'Accuracy' is not defined
  • Performance on Northbridge's own customers has not been measured
Control inventory

Documented controls

From: Fraud Operations procedures

ControlDocumented ownerEvidence held
Customer verification by app or SMSFraud Operations ManagerSystem configuration
Analyst review of blocks on requestFraud Operations ManagerCase notes (free text)
Urgent-case escalation within 2 hoursCustomer Operations LeadProcedure document
Monthly false-positive monitoringFraud Analytics LeadDashboard, no minutes
Quarterly model-change approvalNot assignedNone
Complaint extracts

Three customer complaints

  • 'I was abroad at 11pm and couldn't pay for the hotel. I waited six hours for someone to unblock my card.'
  • 'My payment for my mother's prescription was blocked. I'm hard of hearing and couldn't do the phone check.'
  • 'I got a message asking me to confirm the payment twenty minutes after it had already been declined.'
Meeting note

Fraud Operations stand-up (extract)

Analysts are clearing about 300 blocks a day. We usually release them if the customer sounds genuine. Nobody has time to look at the ones who don't call.
  • No record of how many customers never call
  • Analysts' release decisions are not sampled or checked

Facts, claims and gaps

Keep these three apart. A decision-maker needs to know which is which.

What does the evidence pack establish?

What is asserted by the supplier or the business but not yet evidenced?

Show any calculation. Say what the figures do and do not tell you.

Name the specific documents, data or tests you would ask for

Answers save automatically and sync to your account when you are signed in.

BackSave & continue