Skip to main content

Risk factors in ML deployment

Learning
Complete

What could possibly go wrong with a model that passed all your tests? In production — plenty.

So you’ve learned how to deploy an ML model. But what happens next? In real-world production environments, deployment is only the beginning. This page focuses on what can go wrong — and why managing risk is a core skill for anyone working with live ML systems.

Microscope illustration

When models fail in production, it's rarely just a technical glitch. More often, it’s a complex mix of technical, operational, business, and ethical risks— and failing to account for them can lead to costly, even harmful, consequences.

Let’s break this down.

Categorising ML deployment risks​

As you go through this section, think like a detective. If something goes wrong in a deployed model, where might the root of the issue lie?

Select the headings below to learn about four major categories of risk to keep in mind during deployment.

Technical risksThese are the issues that most engineers expect — but they’re still often underestimated.

  • Model performance degradation: Over time, your model’s predictions become less accurate due todata drift (the data in production changes from what the model saw during training) ormodel drift (the underlying patterns the model learned no longer hold).

  • Data quality issues: Missing values, biased training data, or corrupted inputs can cause errors or flawed predictions.

  • Infrastructure failures: Resource limits, network timeouts, or incompatible updates to APIs or environments can crash your system.

  • Security vulnerabilities: Attackers can exploit exposed APIs, unpatched containers, or poorly secured endpoints.

  • Scalability bottlenecks: A model that works fine with 1,000 predictions a day might break with one million.

  • Integration complexities: Bugs or mismatches in how your model communicates with other systems (e.g., dashboards, customer apps) can result in the model misreading or losing the data.Example: A ride-sharing platform rolls out a new real-time pricing model. It works fine in testing, but once it’s live during a citywide event, the surge pricing calculation crashes the backend API due to a previously unseen load — causing booking delays and customer complaints.Operational risksOperational risks come from how you manage (or mismanage) the model once it's live.

  • Lack of monitoring: There is no system in place to detect whether predictions are off or if key metrics are declining.

  • Inadequate incident response: When something breaks, teams don’t know whom to contact or how to respond quickly.

  • Poor version control: Teams can’t tell which version of the model is running or how it was trained.

  • Insufficient documentation: When a new engineer joins the team, there’s no record of how the model works or what data it needs.

  • **Human error:**They create annual overrides, copy-paste mistakes, or incorrect assumptions during updates.

  • Maintenance overhead: Fragile systems require constant intervention, slowing down innovation.Example: A retail company’s demand forecasting model suddenly predicts zero demand for all products. The issue? An upstream system renamed a field from item_id to product_code, and no one updated the feature pipeline.Business risksThese risks are tied to the real-world impact of a model’s performance — or its failure.

  • Financial loss:** **A pricing or risk model fails and costs the company revenue.

  • **Reputational damage:**Customers lose trust due to inaccurate recommendations, delays, or offensive outputs.

  • Customer dissatisfaction: Users abandon your product because the experience becomes inconsistent or unhelpful.

  • Missed opportunities: A working model isn’t deployed in time due to excessive risk aversion or unclear ownership.

  • Regulatory noncompliance:** **A model uses personal data in a way that violates privacy laws or industry standards.Example: A loan approval model mistakenly denies hundreds of applicants due to a data entry error in a feature column. The media picks up the story, and the company faces public backlash and scrutiny from regulators.Ethical and societal risksThese risks aren’t just about the bottom line — they impact people and society in lasting ways.

  • Bias amplification: The model reflects or worsens historical biases in training data.

  • Fairness concerns: Certain user groups are treated unfairly (e.g., higher rejection rates, worse recommendations).

  • Lack of transparency: Stakeholders don’t understand how the model makes decisions and can’t challenge them.

  • Privacy violations: Sensitive information is mishandled, shared, or inferred inappropriately.

  • Unintended societal impact: A model influences behaviour in ways that people haven't anticipated, with harmful outcomes.Example: An online hiring platform uses a model trained on past hiring data. Over time, the model learns to favour applicants from certain backgrounds — excluding qualified candidates and reinforcing systemic bias.

Impact assessment: Severity and likelihood​

Not all risks deserve the same level of attention — and trying to fix everything at once is rarely realistic. That’s where impact assessment comes in. It's a structured way to decidewhich risks to tackle first based on two key factors:

Severity​

Ask yourself: If this risk occurs, how damaging would the consequences be?

  • Would it cause financial loss?
  • Could it lead to regulatory penalties?
  • Might it hurt users or damage your company’s reputation? Severity is about how bad the outcome could be. For example, a model that unintentionally denies loan approvals to a protected group might trigger serious legal and ethical consequences — making it ahigh-severity issue.

Likelihood​

Then ask: How likely is this risk going to happen, given our current safeguards?

  • Has this type of failure occurred before?
  • Are we operating in a complex, changing environment?
  • Are our testing, monitoring, and governance processes strong enough to catch issues early? Likelihood is about how probable it is that the risk will materialise. For example,model drift—where the relationship between inputs and outputs changes over time — is oftenhighly likely, especially in dynamic industries such as finance or retail.

Putting it together: The risk matrix​

To make these decisions visual and practical, many teams use a risk matrix. It mapsseverity againstlikelihood so you can prioritise at a glance.Low likelihoodHigh likelihoodHigh severityMonitor closely (e.g., security breach).Urgent: Address immediately (e.g., biased predictions).Medium severityMake it low priority (e.g., documentation gaps).Important: Add controls (e.g., model drift).Low severityConsider it an acceptable risk.Monitor occasionally (e.g., minor feature drift).

<g></g><defs><clipPath><rect width="24" height="24" fill="white"></rect></clipPath></defs>## Case spotlight: Balancing risks in a clinical ML deployment Imagine you're part of a data science team at a hospital network. You're preparing to deploy an ML model that analyses chest X-rays to support radiologists in identifying early signs of lung cancer. The goal is to improve diagnostic speed and accuracy, especially in high-volume clinics. But before going live, you need to assess the deployment risks and prioritise your mitigation plan.

Here’s how severity and likelihood can help guide your focus:

Risk 1: Security vulnerability

  • What: An internal API serves this model. If improperly secured, it could allow unauthorised access to patient images or manipulation of predictions.

  • Severity: High — Breach of protected health information could lead to HIPAA violations, lawsuits, and reputational harm.

  • Likelihood: Low — Network security protocols and access controls are already in place.

  • Action: Monitor and control closely — Conduct regular security reviews, ensure encrypted transmission, and apply role-based access control.Risk 2: Model drift

  • What: Over time, the X-ray inputs in production may change due to new imaging machines or shifts in patient demographics. The model might gradually become less accurate.

  • Severity: Medium — While radiologists still verify outputs, inaccurate suggestions could introduce diagnostic delays or missed findings.

  • Likelihood: High — Data distribution shifts are common in health care settings.

  • Action: Mitigate proactively — Set up automated monitoring for drift, and schedule periodic retraining with recent data.

Action item: Identifying deployment risks checkpoint​

You work for an e-commerce company that just launched an ML model to power personalised product recommendations on the home page. The model is integrated into the website via an internal API and retrained weekly using customer interaction data.

A few weeks postlaunch, users and stakeholders start raising concerns.