Module 2: Responsible AI, Trustworthy AI & Harm

Module 1 asked what HireAI does. Module 2 asks what happens to people when the system is wrong, unequal, unclear or poorly controlled.

By the end of this module, you should be able to:

• explain responsible and trustworthy AI in practical terms;
• identify individual, group, organisational and societal harm;
• trace where bias can enter an AI lifecycle;
• distinguish fairness from overall accuracy;
• assess severity and likelihood;
• evaluate transparency, privacy, safety, security and human oversight; and
• make a defensible launch recommendation.

The continuing HireAI case uses fictional applicant profiles. The aim is not to guess whether a person should be hired. It is to test how system limitations could create disadvantage and what evidence would be needed to control that risk.

Responsible AI means designing, using and governing AI so that benefits are pursued while foreseeable harm is prevented or reduced. It turns principles into ownership, controls, evidence and action.

Trustworthy AI means there is reasonable evidence that the system is fit for its purpose and is valid, reliable, safe, secure, fair, explainable, privacy-aware, accountable and monitored.

Confidence is not trust. A polished answer, score or dashboard may still be wrong. Trust depends on evidence such as test results, approval records, control logs, incident data and monitoring outcomes.

Responsible AI is not a public statement that the organisation uses “ethical AI”. It requires named owners, defined boundaries, meaningful review, routes for challenge and the ability to pause the system.

A useful practitioner question is: What evidence would justify allowing this system to affect a person?

AI harm is a negative consequence caused or contributed to by an AI system or the way it is used.

Harm can be direct or indirect, immediate or delayed, visible or difficult to detect. The person operating the system is not always the person affected by it.

A strong harm statement follows the path from a system condition to an output, decision and real consequence. For example: inaccessible CV formatting may be parsed poorly, leading to a lower score, rejection and reduced opportunity for a disabled applicant.

Bias is a systematic tendency that creates distorted or unequal outcomes. It can enter before, during or after model development.

• Purpose and design: the wrong objective or proxy is selected.
• Data: historical records exclude groups or reproduce past decisions.
• Labels: human judgements are inconsistent or discriminatory.
• Model and testing: averages hide weak performance for a subgroup.
• Deployment: users rely on the output in a context for which it was not tested.
• Human behaviour: automation bias causes reviewers to accept the score without challenge.
• Monitoring: complaints and unequal outcomes are not measured.

Bias does not require malicious intent. A neutral-looking variable may operate as a proxy for age, disability, ethnicity or socioeconomic status. Practitioners therefore examine both the technical system and the surrounding process.

Accuracy measures whether outputs are correct according to a defined test. Fairness asks how benefits, errors and burdens are distributed.

A system can be 90% accurate overall and still perform poorly for career changers, disabled applicants or people with overseas qualifications. The average can hide unequal false-negative rates—the rate at which suitable applicants are incorrectly screened out.

Fairness requires context. Different fairness measures can conflict, so the organisation must explain which outcomes matter, which groups were examined and why the chosen threshold is acceptable.

Ask:

• Accurate for whom?
• Which error causes the greater harm?
• Were relevant groups represented in testing?
• Do thresholds create unequal outcomes?
• Can applicants understand and challenge the result?
• Are overrides and appeals monitored by group?

Fairness is a governance decision supported by evidence, not a single universal score.

A simple harm assessment considers severity and likelihood.

The rating should include reasoning. Do not select “High” simply because AI is involved.

Treatment options are:

1. Avoid — do not use AI for the decision or remove the harmful feature.
2. Reduce — add testing, human review, accessibility, explanation and monitoring.
3. Share — allocate obligations contractually while retaining accountability.
4. Accept — document why remaining risk is within authority and tolerance.
5. Monitor — use indicators, complaints and outcome data to detect change.

Controls must connect to evidence. A review policy is not proof that reviews happen effectively.

Transparency tells people that AI is being used, for what purpose and how it affects the process. Explainability provides a meaningful reason for an output or decision. They are related but different.

“The AI ranked you low” is not a useful explanation. A better explanation identifies the relevant criteria, the information used, the role of human review and the route to challenge the decision.

Privacy begins when personal data enters the system. Recruitment data may reveal contact details, work history, education, location and sensitive information directly or by inference.

Practitioners ask:

• Is every data item necessary for the purpose?
• Were applicants told how their information would be used?
• Is there a lawful basis and an appropriate retention period?
• Who, including suppliers, can access the data?
• Could the model expose or infer sensitive information?
• Can people exercise their rights and request human review?

More data can increase capability, but it also increases exposure and governance responsibility.

Safety means preventing unacceptable harm. Reliability means performing consistently enough for the intended task. Security protects the model, data, connected tools and records from unauthorised access or manipulation.

Controls should match impact. A film recommendation and a recruitment recommendation do not justify the same testing threshold.

Meaningful human oversight requires more than placing a person after the model. The reviewer needs:

• enough information to understand the recommendation;
• training on limitations and possible bias;
• time to review the evidence;
• authority to override, escalate or pause;
• freedom from targets that reward rubber-stamping; and
• a record of the decision and reasons.

Automation bias occurs when people give excessive weight to an automated output. Override rates, review quality and rejected-candidate samples can help show whether oversight works in practice.

A stakeholder has an interest in the system. An affected person experiences its consequences. HireAI stakeholders include applicants, recruiters, hiring managers, HR leaders, legal and privacy teams, the supplier and senior decision-makers.

The fictional evidence pack focuses on four applicant contexts:

• Aisha has relevant overseas qualifications.
• Daniel has a career gap caused by caring responsibilities.
• Maya is disabled and uses an accessible CV format.
• Jordan entered the profession through a bootcamp and portfolio.

These characteristics do not prove that HireAI will discriminate. They identify plausible tests. The reviewer should trace how parsing, labels, proxies, thresholds or reviewer behaviour could affect the recommendation.

The assessment should name the affected group, describe the harm pathway and distinguish evidence from assumption.

Possible HireAI controls include subgroup performance testing, accessible application routes, structured human review, explanations, appeals, override monitoring, rejected-candidate sampling, incident reporting and a pause mechanism.

Each control needs an owner and evidence. Examples include test reports, accessibility results, reviewer training records, override logs, appeal outcomes, monitoring dashboards and signed approval conditions.

A launch recommendation may be:

• Proceed — evidence and controls support the proposed boundary.
• Proceed with conditions — specified actions and monitoring are mandatory.
• Pause — material evidence is missing or controls are not ready.
• Do not proceed — the use is unsuitable or the harm cannot be reduced sufficiently.

The decision should balance benefit, affected-person impact, control effectiveness and uncertainty. Vendor confidence cannot replace the deployer’s evidence and judgement.

Key points

• Responsible AI converts principles into ownership, controls and evidence.
• Trust depends on demonstrated fitness for purpose, not confident outputs.
• Harm can affect individuals, groups, organisations and society.
• Bias can enter at every lifecycle stage.
• Overall accuracy can hide unequal errors.
• Severity and likelihood ratings require contextual reasoning.
• Transparency, explanation, privacy and meaningful oversight protect affected people.
• Every control needs an owner and evidence.

Before continuing

You should be able to select one HireAI applicant context, trace a credible harm pathway, assess its seriousness, propose evidence-backed controls and decide whether launch should proceed. Your homework records that judgement in the HireAI AI Harm & Stakeholder Map.