Model governance and reproducibility in practice

Learning

Complete

You’ve deployed your model, set up monitoring, and established feedback loops, but how do you ensure that your ML system remains trustworthy, explainable, and compliant six months down the line? Or a year from now? That’s where model governance comes in.

After deployment, your model needs more than just code — it needs supervision and strong governance. Even top-performing models can degrade as data, user behaviour, and systems evolve.

In this section, you’ll learn how to monitor ML models effectively, ensure accountability through governance practices, and detect issues before they impact business outcomes.

What is model governance?

Model governance refers to the policies, processes, and technical practices that support the safe, ethical, and accountable use of ML. It spans the entire ML life cycle, from data sourcing and experimentation to deployment, monitoring, and retirement.

Robust governance is not just about avoiding risk — it’s about building confidence with regulators, stakeholders, and users that your model can be trusted.

Traceability, reproducibility, and auditability in practice

A key part of model governance is ensuring traceability, reproducibility, and auditability so you can confidently answer questions like:

• Where did this data come from?
• How was this model trained?
• Can we reproduce the same results tomorrow or prove how a past prediction was made? These capabilities aren’t just technical — they’re essential for regulatory compliance, accountability, and stakeholder trust.

Key practices include:

Tracking data lineageMaintain clear records of data sources, transformations, and filtering used for training and testing.Experiment loggingUse tools like MLflow or Weights & Biases to track training details such as hyperparameters, evaluation metrics, dataset versions, and runtime environments.Model registriesUse centralised registries (e.g., MLflow Model Registry) to store model artifacts along with metadata — version tags, training data references, code commit hashes, and intended use cases..Audit trailsMaintain logs of who trained, deployed, and updated models, and when — creating a transparent history that supports internal reviews and regulatory audits.

<g></g><defs><clipPath><rect width="24" height="24" fill="white"></rect></clipPath></defs>## Example A retail company uses a model registry to manage all demand forecasting models.

• Model registration: Each model entry includes links to the original training dataset, preprocessing pipeline, code version, hyperparameters, and evaluation metrics.
• Audit scenario: Months later, the business identifies unexpected sales anomalies during a promotional period and needs to investigate the forecast behind inventory decisions.
• Traceability and reproducibility: The data science team uses the registry to locate the exact model version, trace the inputs and parameters used at the time, and fully reproduce the forecast using containerised environments and archived datasets.
• Outcome: The audit confirms the model behaved as expected, helping the team shift focus to external factors — demonstrating the value of strong governance practices.

Ethical AI, fairness, and bias detection in deployed models

Model governance extends beyond technical performance — it also ensures your ML systems behave ethically once deployed. Fairness and bias become especially important in live environments where real decisions are made, such as in credit approval, recruitment, or health care triage.

Key practices include:

• Bias detection: Monitor deployed models for performance discrepancies across different user groups. Use fairness metrics such as demographic parity or equal opportunity to detect bias after deployment — not just during training.

• Bias mitigation: Apply post-deployment strategies such as reweighting predictions, applying correction layers, or using model ensembles to reduce bias in live predictions.

• Transparency: Communicate limitations of deployed models clearly using updated documentation, such as model cards that reflect live behavior, known risks, and performance across demographics. <g></g><defs><clipPath><rect width="24" height="24" fill="white"></rect></clipPath></defs>## Example: Addressing bias in a deployed dynamic pricing model Context: A ride-hailing platform deploys an ML model to determine dynamic pricing based on demand, time, and location data.Issue discovered: After deployment, monitoring shows that ride prices fluctuate more frequently (and are often higher) in certain lower-income neighbourhoods, leading to customer complaints and reputational risk.Bias analysis:

• The team conducts a post-deployment fairness audit.

• They discover that location data, used as a proxy for demand, indirectly correlates with income level, creating unintended pricing bias. Mitigation steps:

• Adjust feature weighting to reduce reliance on sensitive location indicators.

• Retrain the model with fairness constraints (e.g., equal opportunity).

• Conduct A/B testing to verify that performance and fairness both improve. Governance outcome:

• The team updates a model card to document the issue, mitigation techniques, and ongoing monitoring plan.

• The internal governance team receives monthly fairness performance reports to ensure sustained compliance and ethical standards.

Regulatory compliance and documentation best practices

ML systems often operate in regulated environments where demonstrating how decisions are made is essential. This includes maintaining clear records of training data, code changes, evaluation results, and ensuring transparency and fairness in model behavior.

To support these requirements and promote responsible AI practices, use structured documentation formats, such as:

• Model cards: Describe the model’s intended use, training data, performance metrics, ethical considerations, and known limitations.
• Data sheets for datasets: Provide details about data sources, collection methods, preprocessing, and potential biases.

Action item: Deployed model governance essentials

Choose the most appropriate deployment strategy for each scenario. Review the explanation after each choice to reinforce your understanding.