Balancing model complexity and bias

Instruction and application

Complete

We have seen in this unit that there is a tension between complexity and bias when building a machine learning model.

Bias occurs when a model is unable to find the true pattern in the data and as a result underfits. We counter bias by increasing the complexity, which in turn increases the risk of overfitting.

Therefore, making a deployable model is a balancing act between these two concepts.

Let’s look at some strategies we can use to help our models perform as effectively as possible.

Strategies for mitigating overfitting data while minimising bias

Often, when trying to build highly accurate models, we can inadvertently introduce or amplify biases present in the training data, or create models that are too complex and generalise poorly (overfitting).

The key is to find that sweet spot where the model performs well on unseen data without unfairly discriminating against certain groups.

Here are some strategies to navigate this balancing act:

Regularisation techniquesRegularisation adds a penalty term to the model's loss function, discouraging overly large weights. This simplifies the model and reduces its tendency to fit the noise in the training data.

Common techniques include L1 (Lasso) and L2 (Ridge) regularisation.

When we use it:

When we suspect our model is overfitting, evidenced by a large discrepancy between training and validation performance.

Regularisation is particularly useful with complex models that have a large number of parameters, such as deep neural networks or high-degree polynomial regressions.

Benefits:

Improves the model's generalisation ability, making it more robust to new, unseen data. Can also help in feature selection (L1 regularisation drives some feature weights to zero).

Risks:

If the regularisation strength is too high, it can lead to underfitting, where the model becomes too simple and fails to capture the underlying patterns in the data. It might also inadvertently penalise features that are important for reducing bias if those features happen to have slightly larger weights.

Cross-validationCross-validation is a technique where the training data is split into multiple subsets (folds). The model is trained on some folds and evaluated on the remaining fold. This process is repeated multiple times, with each fold serving as the validation set once.

The performance metrics are then averaged across all folds, providing a more reliable estimate of the model's generalisation ability. Techniques like stratified k-fold cross-validation ensure that the proportion of different classes (or sensitive groups) is maintained in each fold.

When to use it:

During model development and evaluation to get a more robust estimate of performance and to tune hyperparameters, including regularisation strength. Stratified cross-validation is crucial when dealing with imbalanced datasets or when fairness across different groups is a concern.

Benefits:

Provides a more reliable estimate of the model's performance on unseen data, reduces the risk of overfitting to a specific train-test split, and helps in selecting more robust models. Stratification helps ensure that fairness considerations are taken into account during evaluation.

Risks:

Can be computationally expensive, especially with large datasets and complex models. If the data distribution in the folds doesn't accurately reflect the real-world distribution, the performance estimates might still be biased.

Feature selection and engineeringThis involves carefully selecting and transforming the input features used to train the model. It includes removing irrelevant or redundant features that might introduce noise and potentially exacerbate biases. It also involves creating new features that might capture important information while being mindful of fairness implications.

For example, instead of using a direct indicator of a sensitive attribute, we might use a less direct proxy if it's truly informative and less discriminatory.

When to use it:

During the data preprocessing stage, before training the model. It's an iterative process that might need to be revisited as we gain more understanding of the data and the model's behavior.

Benefits:

Can lead to simpler, more interpretable models that are less prone to overfitting. Thoughtful feature selection can also help in mitigating bias by removing or transforming features that are highly correlated with sensitive attributes and contribute to unfair predictions.

Risks:

Removing seemingly irrelevant features might lead to loss of important information. Feature engineering requires domain expertise and careful consideration to avoid introducing new biases or masking existing ones. Over-reliance on proxy features might still lead to discriminatory outcomes if the proxy is strongly correlated with the sensitive attribute.

Ensemble methodsEnsemble methods combine the predictions of multiple individual models to make a final prediction. Techniques like bagging (e.g., Random Forests) and boosting (e.g., Gradient Boosting) can improve generalisation by reducing variance and bias. Fairness-aware ensemble methods are an emerging area where the individual models are trained or combined in a way that explicitly considers fairness constraints.When to use it:

To improve the overall performance and robustness of the model. Fairness-aware ensembles are used when we want to explicitly optimise for both accuracy and fairness.

Benefits:

Can lead to more accurate and stable models with better generalisation. Fairness-aware ensembles offer the potential to directly address bias issues while maintaining good predictive performance.

Risks:

Ensemble models can be more complex and harder to interpret. Fairness-aware ensemble methods are still an active area of research, and their implementation and theoretical guarantees might vary.

Early stoppingEarly stopping is a technique used during the training of iterative algorithms (like gradient descent for neural networks) where the training process is halted when the performance on a separate validation set starts to degrade. This prevents the model from continuing to learn the noise in the training data and thus reduces overfitting.

When to use it:

When training complex models with many parameters where overfitting is a significant risk, such as deep learning models.

Benefits:

Helps prevent overfitting and can lead to models that generalise better. It can also save computational time by stopping the training process early.

Risks:

Requires a well-representative validation set. If the validation set is too small or not representative, early stopping might occur prematurely or too late.

Case studies

Let’s look at some real examples of when an organisation had needed to balance bias and complexity in their ML models.

Case study: ProPublica

The Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) is a proprietary algorithm used in the United States to assess the risk of recidivism (likelihood of re-offending) for defendants. ProPublica's investigation revealed that while the algorithm showed similar overall accuracy in predicting recidivism for Black and white defendants, it exhibited significant disparities in its false positive and false negative rates. Specifically, COMPAS was more likely to incorrectly label Black defendants as high-risk and white defendants as low-risk.

Trade-off

The developers likely aimed for a complex model that could incorporate numerous factors to achieve high predictive accuracy. However, this complexity, trained on potentially biased historical criminal justice data, resulted in unfair outcomes for different racial groups. There was a trade-off between optimising for overall accuracy and ensuring fairness metrics like equalised odds (similar false positive and false negative rates across groups).

Fairness considerations

The case highlights the conflict between overall accuracy and group fairness. While the model's overall predictive power might have seemed acceptable, its disparate impact on different racial groups raised serious ethical and legal concerns.

Case study: Google’s word embeddings and gender bias

Research into word embeddings, a technique used in natural language processing to represent words as vectors, revealed that these embeddings often captured and amplified societal biases present in the training data (typically large text corpora). For example, the relationship "man is to computer programmer as woman is to homemaker" was found to be encoded in some popular word embedding models.

Trade-off

The goal of developing these complex word embeddings was to create representations that accurately capture semantic relationships between words, leading to high performance in various NLP tasks. However, the complexity of the models and the vast, often biased, training data resulted in the encoding of harmful gender stereotypes. Researchers then had to work on debiasing techniques, often involving reducing the dimensionality or adjusting the embeddings, which could potentially impact the model's performance on certain tasks (a trade-off between fairness and task-specific accuracy).

Fairness considerations

This case illustrates representational harm, where AI models perpetuate harmful stereotypes. Efforts to mitigate this bias often involve modifying the model or the data, potentially affecting its utility for certain applications.

Case study: Facial recognition technology

Multiple studies have demonstrated that facial recognition algorithms from various companies exhibit significantly lower accuracy and higher error rates when identifying individuals with darker skin tones and women compared to individuals with lighter skin tones and men.

Trade-off

Developers aimed to create highly accurate and complex models for facial recognition, often using large datasets. However, these datasets frequently lacked sufficient representation of diverse demographic groups. The pursuit of high overall accuracy led to models that were less accurate and thus less fair for certain populations.

Addressing this often requires retraining on more balanced datasets or developing bias mitigation techniques, which might involve increasing model complexity in specific ways or even sacrificing some overall accuracy to improve performance across all groups.

Fairness considerations

This highlights issues of accuracy parity across different demographic groups. The lower accuracy for certain groups can have severe consequences in applications like law enforcement and security.

Comparing models

An important part of the ML ecosystem is comparing the performance of several models to see which yields the most accurate predictions.

However, using accuracy as the sole measure can be misleading when fairness is a concern. We need fairness-adjusted performance metrics that provide a more holistic view.

Demographic parityThis metric checks if the selection rate (or positive outcome rate) is the same across different protected groups.

For example, in a loan application model, demographic parity would be achieved if the proportion of approved loans is roughly the same for all racial groups.

Metric:

The difference in acceptance rates between the most and least privileged group should be close to zero.

Example:

If 80% of white applicants are approved and 70% of Black applicants are approved, the demographic disparity is 10%.

Equal opportunityThis metric focuses on the true positive rate (TPR) or recall for different groups. It aims to ensure that the model has a similar ability to correctly identify positive instances across all groups.

For example, in a hiring model, equal opportunity would mean that the model has a similar rate of correctly identifying qualified candidates from all gender groups.

Metric:

The difference in true positive rates between the most and least privileged group should be close to zero.

Example:

If the TPR for male candidates is 85% and for female candidates is 75%, the equal opportunity difference is 10%.

Equalised oddsThis is a stricter criterion that requires both the true positive rate (TPR) and the false positive rate (FPR) to be similar across different groups.

In the context of a criminal risk assessment tool, this would mean that the model should have a similar rate of correctly identifying future offenders (TPR) and incorrectly identifying non-offenders as high-risk (FPR) across different racial groups.

Metric:

Differences in both TPR and FPR between the most and least privileged group should be close to zero.

Example:

Consider a credit risk assessment model with "age" as a protected attribute (privileged: under 30, unprivileged: over 50).

If the TPR ((correctly identifying individuals who will repay their loan) is 80% for those under 30 and 65% for those over 50, the TPR difference is 15%.

If the FPR (incorrectly identifying individuals who will default) is 10% for those under 30 and 25% for those over 50, the FPR difference is 15%.

For Equalised Odds to be met, both these differences should ideally be close to zero.

CalibrationA well-calibrated model has predicted probabilities that align with the actual outcomes. For example, if a model predicts a 70% chance of default for a group of loan applicants, then approximately 70% of that group should actually default.

Fairness in calibration implies that the model should be well-calibrated across different protected groups.

Metric:

While there isn't a single "difference" metric like with TPR and FPR, calibration is often evaluated by comparing predicted probabilities to actual outcomes within each group. Disparities in calibration across groups indicate unfairness.

One way to quantify this difference is by looking at the Expected Calibration Error (ECE) per group or the difference in ECE between groups.

Example:

Consider a model predicting the probability of hospital readmission within 30 days, with "race" as a protected attribute (Group A and Group B).

• Calibration for Group A: when the model predicts a 70% chance of readmission for a subset of Group A patients, approximately 70% of those patients should actually be readmitted.
• Calibration for Group B: if the model also predicts a 70% chance of readmission for a subset of Group B patients, but only 40% of them are actually readmitted, the model is poorly calibrated for Group B compared to Group A. The difference in the actual readmission rates for the same predicted probability highlights a calibration disparity across groups.Fairness-aware accuracySome composite metrics attempt to combine accuracy and fairness considerations into a single score. These metrics often involve penalising models that exhibit significant disparities across groups while rewarding those with high overall accuracy.

Metrics

Accuracy with demographic parity: aims to achieve high predictive accuracy while ensuring that the proportion of positive outcomes predicted by the model is similar across different protected groups.

One way to measure this is by considering a primary accuracy metric (like overall accuracy) alongside a demographic parity metric (like the difference in acceptance rates).

A fairness-aware evaluation would then consider a model "good" if it achieves a reasonable level of accuracy while keeping the demographic disparity within an acceptable threshold.

There isn't a single combined "Fairness-Aware Accuracy" score here, but rather a joint evaluation.

Accuracy with equalised odds: This seeks to balance predictive accuracy while also ensuring that the true positive rates (TPR) and false positive rates (FPR) are similar across different protected groups. This involves evaluating a primary accuracy metric in conjunction with the metrics for Equalised Odds (TPR difference and FPR difference).

A fairness-aware assessment would look for a model that performs well overall while also minimising the disparities in TPR and FPR across groups.

Example

• Accuracy with demographic parity: consider a loan application model with "ethnicity" as the protected attribute (Group A and Group B).Accuracy: The overall accuracy of the model in correctly classifying loan applicants as likely to repay or default is 85%.

• Demographic parity difference: If the loan approval rate (positive prediction) is 70% for Group A and 50% for Group B, the demographic parity difference is 20%.

• Accuracy with equalised odds: Consider a recidivism prediction model with "gender" as the protected attribute (Male and Female).Accuracy: The overall accuracy of the model in predicting whether an individual will re-offend is 78%.

• TPR difference: If the TPR (correctly identifying future offenders) is 60% for males and 45% for females, the TPR difference is 15%.

• FPR difference: If the FPR (incorrectly identifying non-offenders as high-risk) is 15% for males and 5% for females, the FPR difference is 10%. When comparing models, it is important to follow these steps:

• Define fairness: clearly articulate what fairness means in the specific context of the application. Different fairness definitions might be more appropriate depending on the domain and potential impact.

• Calculate relevant metrics: evaluate models not only on traditional accuracy metrics but also on the chosen fairness metrics.

• Visualise trade-offs: often, there's a trade-off between accuracy and fairness. Visualising these trade-offs (e.g., using ROC curves with fairness metrics overlaid) can help stakeholders make informed decisions.

• Consider the context: the acceptable level of bias might vary depending on the application. In high-stakes scenarios, even small biases can have significant consequences.

Key points

By employing these fairness-adjusted performance metrics, we can move beyond a purely accuracy-driven approach and build machine learning models that are not only effective but also more equitable.

Action item: Pause and think

Before moving on, try applying what you've learned by answering the questions below.