Managing bias

Instruction and application

Complete

Even the most advanced models can fail when they unknowingly reinforce unfair patterns. That’s why knowing how to spot and mitigate bias isn’t just a nice-to-have—it’s a must-have for anyone building or working with machine learning models.

Ready to become a guardian of fairness in AI?

Bias mitigation strategies

Now let’s turn our attention to some strategies we can use to mitigate bias if it arises in our models.

By proactively addressing bias at different stages of the machine learning pipeline, we can strive for more equitable and fair predictions.

We will consider strategies for before, during, and after the model build.

Pre-processing

Pre-processing techniques aim to modify the training data itself to reduce bias before it even reaches the model.

The advantage of addressing bias at this stage is that it provides a cleaner, more balanced foundation for the model to learn from, potentially leading to more inherently fair models regardless of the specific algorithm used.

Here are some techniques we can use:

Rebalancing datasetsThis involves adjusting the class distribution within the training data, particularly concerning the protected attribute (e.g., race, gender).

We can achieve this through oversampling the minority group (replicating their instances) or undersampling the majority group (removing some of their instances). The goal is to provide the model with a more representative view of all groups.

When to use it:

It is particularly useful when the dataset exhibits a significant class imbalance with respect to the protected attribute and the target variable.

For example, if a loan application dataset has far fewer approved applications for a particular demographic group, rebalancing can help the model learn more effectively from the underrepresented group.

Benefits:

• More equitable learning: by providing a more balanced representation of all groups, the model is less likely to be dominated by the patterns of the majority group and can learn more nuanced relationships for the minority group, leading to fairer predictions.

• Improved performance on minority groups: rebalancing can provide the model with sufficient examples of the minority class to learn meaningful decision boundaries, potentially improving the accuracy and recall for these previously underperforming groups.

• Reduces majority bias: prevents the model from simply predicting the majority class outcome, which can disproportionately disadvantage minority groups.Risks:

• Oversampling and overfitting: artificially increasing the minority class by duplicating existing samples can lead the model to memorise these specific instances. This can result in excellent performance on the training data but poor generalisation to new, unseen data, effectively overfitting to the artificial duplicates.

• Oversampling and computational burden: significantly increasing the dataset size through oversampling can dramatically increase the time and computational resources required for model training.

• Undersampling and information loss: discarding data points from the majority class, even if done randomly, can lead to the loss of potentially valuable information and the discarding of important patterns that could have improved the model's overall robustness and accuracy.

• Undersampling and introduction of selection bias: if the selection process for removing majority class instances is not truly random or doesn't account for underlying data distributions, it can inadvertently introduce a different form of bias, where the remaining majority samples are not representative of the original distribution.

Data anonymisationData anonymisation involves removing or obscuring sensitive personal information (e.g., name, address, specific demographic identifiers) from the dataset.

The goal is to prevent the model from directly learning and potentially discriminating based on these attributes. Various techniques exist, such as masking, generalisation, and perturbation.

When to use: When dealing with datasets containing personally identifiable information (PII) that could be a source of bias.

It's often a necessary step to comply with privacy regulations and ethical considerations, especially before sharing or using sensitive data for model training.

Benefits:

• Reduces direct discrimination: by removing explicit identifiers and sensitive attributes, the model is prevented from directly conditioning its predictions on these factors.

• Enhances privacy protection: anonymisation is crucial for complying with data privacy regulations and protecting the privacy of individuals in the dataset.

• Facilitates data sharing and collaboration: anonymised datasets can be shared more readily for research and development purposes without the same privacy concerns associated with raw data.Risks:

• Loss of information and reduced model utility: aggressively anonymising data can remove or significantly distort features that are actually important for the predictive task, leading to a substantial decrease in the model's accuracy and overall usefulness.

• Risk of re-identification: even after anonymisation, if enough correlated or quasi-identifying features remain in the dataset, sophisticated data mining techniques might be used to re-identify individuals, leading to privacy breaches.

• Failure to address proxy discrimination: anonymisation by removing direct sensitive attributes might not be sufficient if other features in the dataset act as strong proxies (highly correlated variables) for the removed attributes. The model could still indirectly learn to discriminate based on these proxies.Removing sensitive featuresThis straightforward approach involves directly removing protected attributes (e.g., race, gender, religion) from the training data. The rationale is that if the model doesn't have access to these features, it cannot directly discriminate based on them.When to use:

This can be considered when there is a strong belief that these features are the primary drivers of bias and are not essential for the predictive task itself.

However, it's important to be cautious, as removing these features might not eliminate bias if other remaining features are highly correlated with them.

Benefits:

• Eliminates direct dependence on sensitive attributes: by removing these features, the model is structurally unable to make decisions solely based on them.

• Simplified model development: can lead to simpler models as fewer features need to be processed and interpreted.Risks:

• Significant reduction in model performance: if the removed sensitive features contain information that is genuinely relevant to the prediction task (independent of bias), their removal can lead to a less accurate and less useful model.

• Strong potential for proxy discrimination: bias can still manifest if other remaining features in the dataset are highly correlated with the removed sensitive attributes. The model might learn to use these proxies to indirectly discriminate in a similar way as if the sensitive feature were still present. This is a major limitation of this approach.

• Loss of information for fairness auditing: removing sensitive features entirely can make it more difficult to audit the model for fairness after training, as the protected group information is no longer directly available in the data used for evaluation.

• Ethical implications of ignoring societal realities: in some contexts, understanding the relationship between sensitive attributes and outcomes (even if historically biased) might be necessary to identify and address systemic inequalities. Simply removing these features can mask important societal patterns.

In-processing

In-processing methods aim to directly integrate fairness considerations into the model training process itself.

The main benefit is that the model learns to make predictions while being explicitly constrained or guided to reduce bias, often leading to a more direct and potentially more effective way to achieve a better balance between accuracy and fairness.

Adversarial debiasingAdversarial debiasing involves training a primary prediction model alongside an auxiliary "adversary" model.

The adversary's objective is to predict the protected attribute from the learned representations of the primary model. The primary model is simultaneously trained not only to make accurate predictions of the target variable but also to generate internal representations that are uninformative about the protected attribute, thereby "fooling" the adversary.

This encourages the primary model to learn features that are useful for prediction but independent of the sensitive attribute.

When to use:

This technique is particularly useful when you suspect that even after pre-processing steps like removing sensitive features, the remaining features might still encode information about the protected attribute (proxy variables). Adversarial debiasing aims to actively remove this encoded information from the model's internal representations.

Benefits:

• Reduces reliance on protected attributes and their proxies: by explicitly training against the predictability of the sensitive attribute, the model learns to base its decisions on other, hopefully fairer, features.

• Potential for good accuracy-fairness trade-off: if implemented effectively, adversarial debiasing can lead to a model that is both accurate in its predictions and fair across different groups.

• Can be applied to various model architectures: the adversarial training framework can often be adapted to different types of machine learning models, including neural networks and other complex architectures.Risks:

• Complexity of implementation and hyperparameter tuning: training adversarial networks is a complex process that requires careful design of both the main model and the adversary, as well as meticulous tuning of various hyperparameters (e.g., learning rates, adversary strength).

• Potential for accuracy degradation: forcing the model to be invariant to the protected attribute might lead to a loss of predictive power if that attribute (or its correlates) genuinely contains some information relevant to the prediction task (even if it also carries bias).

• Sensitivity to adversary design: the effectiveness of the debiasing is highly dependent on the architecture and training of the adversary. A poorly designed or undertrained adversary might not effectively prevent the main model from learning biased representations.

• Reduced interpretability: the internal representations learned by a model trained with adversarial debiasing can be more abstract and harder to interpret, making it challenging to understand why the model makes certain decisions.Regularisation schemes that incorporate fairness constraintsThis approach involves modifying the standard loss function used during model training by adding penalty terms that explicitly quantify and penalise unfairness.

These fairness constraints are often based on specific fairness metrics, such as demographic parity (requiring equal proportions of positive outcomes across groups) or equalised odds (requiring equal true positive and false positive rates across groups).

By incorporating these penalties, the model is incentivised to learn parameters that not only minimise prediction errors but also satisfy the defined fairness criteria to a certain degree.

When to use:

It's useful when you want to directly embed fairness considerations into the model's learning objective, allowing for a more integrated approach compared to pre- or post-processing methods.

This is especially relevant when you aim for a specific type of fairness and want the model to learn representations that inherently satisfy those criteria.

Benefits:

• Direct optimisation for fairness: by including fairness metrics in the loss function, the model is directly trained to reduce bias according to the chosen definition of fairness.

• Potential for a better accuracy-fairness trade-off: compared to post-processing, in-processing regularisation can sometimes achieve a better balance between predictive accuracy and fairness because fairness is considered during the learning process itself, rather than as an afterthought.

• Learning fairer representations: the regularisation can encourage the model to learn data representations that are less correlated with the protected attribute while still being informative for the prediction task.Risks:

• Trade-off between accuracy and fairness: introducing fairness constraints as regularisation terms often involves a trade-off with the model's predictive accuracy. The strength of the regularisation needs to be carefully tuned to find an acceptable balance.

• Dependence on the chosen fairness metric: the effectiveness of this approach is highly dependent on the appropriateness and well-definition of the chosen fairness metric. Different metrics capture different aspects of fairness, and selecting the wrong one might not address the most critical fairness concerns in a given application.

• Complexity of implementation: defining and implementing custom loss functions that incorporate fairness constraints can be mathematically and technically complex, requiring a deep understanding of both the machine learning model and the chosen fairness metric.

• Potential for unintended consequences on other fairness metrics: Optimising for one specific fairness metric through regularisation might inadvertently negatively impact other important fairness considerations. It's crucial to evaluate the model across multiple fairness dimensions.Post-processing

Post-processing methods involve adjusting the output of a trained model after it has been trained, without altering the model's learned parameters.

The primary advantage is that these techniques can be applied to any pre-trained model, offering a way to improve fairness without the need for retraining, which can be computationally expensive or infeasible in some situations.

**Adjusting model outputs (e.g., threshold optimisation)**This involves modifying the final predictions of a trained model to satisfy specific fairness criteria. A common technique is to adjust the classification threshold for different demographic groups.

For example, to achieve demographic parity, you might use different probability thresholds for approving loans for different racial groups, even if the underlying model's probability estimates are the same. Other methods involve directly manipulating the predicted labels or probabilities based on group membership.

When to use:

This is particularly useful when you have a pre-trained model that exhibits unfairness and you want to mitigate this without retraining the model. It can also be applied when in-processing techniques were not implemented or did not achieve the desired level of fairness. Post-processing is often simpler to implement than in-processing methods.

Benefits:

• Applicable to any pre-trained model: this is a significant advantage as it allows for fairness interventions on existing models without the need for retraining, saving time and computational resources.

• Relatively simple to implement: compared to modifying training procedures or model architectures, adjusting outputs can often be done with simpler code and a better understanding of the model's final predictions.

• Flexibility in choosing fairness metrics: post-processing allows you to target specific fairness metrics (e.g., demographic parity, equalised odds) after the model is trained and adjust outputs accordingly.Risks:

• Suboptimal decision boundaries: adjusting outputs after the model is trained does not change the underlying decision boundary learned by the model. This might lead to suboptimal decisions for individuals, especially those whose predicted probabilities are close to the adjusted thresholds. The model's internal logic might still be biased, and the post-processing is just a superficial correction.

• Potential for reduced overall accuracy: forcing the model to meet certain fairness metrics through output adjustments can sometimes lead to a decrease in the overall predictive accuracy of the model, as the adjustments might override some of the model's correct predictions.

• Inability to address bias in learned representations: post-processing techniques operate on the model's predictions but do not change the potentially biased way the model has learned to represent the data. The model might still be relying on unfair signals internally, and the post-processing is merely masking the symptoms.

• Context-dependent thresholds: thresholds optimised for fairness on a specific validation set might not generalise well to new, unseen data or different operational contexts. Re-optimisation of thresholds might be necessary when the data distribution shifts.

• Fairness metric dependency: the adjustments made are specific to the chosen fairness metric. Optimising for one metric might inadvertently negatively impact other important fairness considerations.

Key points

Bias mitigation strategies help ensure that machine learning models make decisions that are fair and equitable, rather than reinforcing historical inequalities. These strategies can be applied at three key stages:

• Pre-Processing: Tackle bias before it enters the model by rebalancing datasets, anonymising sensitive data, or removing protected attributes.
• In-Processing: Embed fairness directly into model training using techniques like adversarial debiasing and fairness-aware regularisation.
• Post-Processing: Adjust model outputs after training to correct any remaining unfairness without needing to retrain the model. By applying these approaches thoughtfully, you’ll create models that not only perform well but also treat people fairly—making your work more responsible and impactful.

Pause and think

Before moving on, take a few minutes to consider how you can apply what you've learned back in your role. Consider the questions below:

• Before model development, what steps could you take to ensure your training data fairly represents all groups involved?
• When selecting a bias mitigation strategy, how might you balance fairness with maintaining acceptable model performance?
• Looking at a current or upcoming project, where in the ML pipeline could you introduce a simple fairness check or adjustment?