For Individuals
- Learning & Certification
- Specialist Tracks & Tools
Consulting & Enterprise
- Programmes & Solutions
- Diagnostics & Evidence
Insights
- Analysis & Research
  - Executive Reports In-depth whitepapers for decision-makers
  - Academic Research Peer-reviewed publications and frameworks
- Editorial & Dispatches
  - The C-Suite AI Memo Weekly executive strategy newsletter
  - News & Commentary Press, updates, and market notes
Events
About
Contact
LOGIN

Skip to content

AI Bias Testing: Measuring Fairness With Numbers You Can Defend

What Bias Means Statistically, Legally and Socially

Quiz: What Bias Means Statistically, Legally and Socially

Where Bias Enters: Data, Labels, Features and Deployment

Quiz: Where Bias Enters: Data, Labels, Features and Deployment

Protected Characteristics and Proxy Variables

Quiz: Protected Characteristics and Proxy Variables

Collecting Demographic Data Lawfully for Testing

Quiz: Collecting Demographic Data Lawfully for Testing

Group Fairness Metrics: Parity, Equal Opportunity and Equalised Odds

Quiz: Group Fairness Metrics: Parity, Equal Opportunity and Equalised Odds

Impact Ratios, the Four Fifths Rule and Legal Thresholds

Quiz: Impact Ratios, the Four Fifths Rule and Legal Thresholds

Individual and Counterfactual Fairness Approaches

Quiz: Individual and Counterfactual Fairness Approaches

Why Fairness Definitions Conflict Mathematically

Quiz: Why Fairness Definitions Conflict Mathematically

Sample Size, Confidence and Testing Small Groups Honestly

Quiz: Sample Size, Confidence and Testing Small Groups Honestly

1 of 2

Previous Lesson

Quiz: Sample Size, Confidence and Testing Small Groups Honestly

AI Bias Testing: Measuring Fairness With Numbers You Can Defend
Sample Size, Confidence and Testing Small Groups Honestly
Quiz: Sample Size, Confidence and Testing Small Groups Honestly

Time limit: 0

Quiz Summary

0 of 5 Questions completed

Questions:

Information

You have already completed the quiz before. Hence you can not start it again.

Quiz is loading…

You must sign in or sign up to start the quiz.

You must first complete the following:

Results

Quiz complete. Results are being recorded.

Results

0 of 5 Questions answered correctly

Your time:

Time has elapsed

You have reached 0 of 0 point(s), (0)

Earned Point(s): 0 of 0, (0)
0 Essay(s) Pending (Possible Point(s): 0)

Categories

Not categorized 0%

Review
Answered
Correct
Incorrect

Question 1 of 5

1. Question

According to ISO 25012:2011 clause 7.2.3, what is a key requirement for bias testing in AI systems?

Testing must include at least 100 observations per group
Correct Incorrect Correct answer

Testing must provide sufficient statistical power to identify relevant differences
Correct Incorrect Correct answer

Testing must use exact statistical tests for all subpopulations
Correct Incorrect Correct answer

Testing must always involve clinical decision support systems
Correct Incorrect Correct answer

Correct / 1 Points

Incorrect / 1 Points
Question 2 of 5

2. Question

What is one reason why small samples cannot reliably detect bias even when it exists?

Small samples are too expensive to collect
Correct Incorrect Correct answer

Small samples cannot provide confidence intervals
Correct Incorrect Correct answer

Small samples cannot detect differences smaller than 5%
Correct Incorrect Correct answer

Small samples lack sufficient statistical power
Correct Incorrect Correct answer

Correct / 1 Points

Incorrect / 1 Points
Question 3 of 5

3. Question

Which of the following is a recommended practice when reporting results from AI bias testing?

Report only point estimates without any measure of uncertainty
Correct Incorrect Correct answer

Always include the actual sample sizes used in the analysis
Correct Incorrect Correct answer

Avoid using confidence intervals for clinical applications
Correct Incorrect Correct answer

Use approximations instead of exact tests for small samples
Correct Incorrect Correct answer

Correct / 1 Points

Incorrect / 1 Points
Question 4 of 5

4. Question

According to the lesson, what is a benefit of using confidence intervals in AI bias testing?

They allow practitioners to ignore statistical variation
Correct Incorrect Correct answer

They help determine whether observed differences are meaningful or due to chance
Correct Incorrect Correct answer

They reduce the need for large sample sizes
Correct Incorrect Correct answer

They guarantee that no false positives will occur
Correct Incorrect Correct answer

Correct / 1 Points

Incorrect / 1 Points
Question 5 of 5

5. Question

What approach is suggested for testing smaller demographic groups where traditional tests may not apply?

Use larger samples to approximate normal distributions
Correct Incorrect Correct answer

Apply chi-square tests regardless of sample size
Correct Incorrect Correct answer

Use exact statistical tests rather than approximations
Correct Incorrect Correct answer

Report results without any confidence intervals
Correct Incorrect Correct answer

Correct / 1 Points

Incorrect / 1 Points

Login

Accessing this course requires a login. Please enter your credentials below!

Username or Email Address

Password

Remember Me

Continue with Facebook

Continue with Google

Lost Your Password?

Register

Don't have an account? Register one!

Register an Account

Keep building practical AI skills

Short, practical courses from Tesseract Academy, founded by Dr Stelios Kampakis, CStat.

Explore Individual Learning

© 2026 Tesseract Academy. All Rights Reserved.