Skip to main content

Detecting algorithmic bias

Instruction and application
Complete

Identifying Disparities​

So far, we have seen that bias in our ML models can undermine our organisation's goals and erode public trust.

To effectively detect bias, we turn to statistical measures, model cards, and specialised toolkits that allow us to quantify disparities and ensure our models remain fair and ethical.

Hand pointing illustration

Bias detection techniques​

So how do we detect bias in our models? There are a variety of techniques, including statistical measures, model cards and third-party tools. Let’s take a look at each category and see how they benefit us in the ML model lifecycle.

Statistical measures​

Statistical measures quantify disparities in outcomes across different demographic groups (race, gender, age). The goal is to identify significant deviations that indicate potential bias.

Demographic parity​

This metric assesses whether the proportion of individuals receiving a positive outcome is the same across all demographic groups.

Mathematically, for groups A and B, demographic parity holds if: P(Ŷ=1 | A) = P(Ŷ=1 | B)

Where Ŷ is the predicted outcome (e.g., loan approval).

Demographic Parity illustration

Equal opportunity​

This metric focuses on the true positive rate (TPR) being equal across different groups. For a hiring model, it means qualified candidates are identified at the same rate regardless of gender.

Equal opportunity holds if: P(Ŷ=1 | Y=1, A) = P(Ŷ=1 | Y=1, B)

Where Y is the true outcome (e.g., qualified candidate).

Equal Opportunity illustration

Disparate impact​

Quantifies bias by looking at the ratio of positive outcomes. A common threshold is the "80% rule," which suggests adverse impact if the rate for the disadvantaged group is less than 80% of the rate for the advantaged group.

Disparate impact ratio: P(Ŷ=1 | group A) / P(Ŷ=1 | group B)

Disparate Impact illustration

Benefits of Statistical Measures​

  • Quantifiable: Provides concrete numbers that can be tracked over time.
  • Early detection: Identify potential biases early in the development process.
  • Accountability: Promotes transparency and organisational responsibility.

Model cards and audits​

Bias is often multifaceted and not captured by simple metrics alone.

  • Model Cards: Structured documents providing essential information about a model's intended use, training data, evaluation metrics, and potential risks.
  • Model Audits: Systematic evaluations of a model's fairness, safety, and reliability, often involving in-depth distribution analysis and user testing.
Model Audit illustration

Specialised Toolkits​

Several open-source tools provide resources to detect and mitigate bias:

  • IBM AI Fairness 360 (AIF360): A comprehensive toolkit for pre-, in-, and post-processing bias mitigation.
  • Fairlearn: Microsoft’s tool focused on the trade-offs between fairness and accuracy.
  • Aequitas: Developed by the University of Chicago, specifically for bias audit and reporting in real-world applications.

Action item: Self-reflection​

Reflect on how you will apply these bias detection techniques to your own work.

Questions & Reflections
How could you incorporate statistical bias detection techniques into your current or future machine learning projects to improve fairness?

Type your reflection here...

Which of these fairness metrics best aligns with your organisation’s goals, and how might you advocate for its adoption?

Type your reflection here...