Skip to main content

Skills application

Instruction iconInstruction and application
Complete iconIn Progress

Case study: From launch to retirement of a recommendation model​

In this skills application, you will apply what you’ve learned about change management, logging and decommissioning to design a complete life cycle plan for an ML model in production.

Skills Application illustration

Context​

You are leading the MLOps effort for a new ‘Personalised Product Recommendation’ model deployed on an e-commerce platform. The initial version (V1) has just gone live. In six months, you expect to release an improved V2 that will fully replace V1. After V2 stabilises, V1 will eventually be decommissioned.

Your challenge is to plan this model’s life cycle with a focus on:

  • Managing change during the transition from V1 to V2.
  • Designing logging and monitoring to support operational stability.
  • Safely decommissioning V1 and archiving key artefacts for compliance and future analysis.

Success criteria​

To complete the skills application, you must:

  • Provide a clear change management plan for rolling out V2, including deployment strategies and rollback options.
  • Identify a structured logging and monitoring approach with at least five critical metrics and a method for comparing V1 vs V2.
  • Outline a safe and transparent decommissioning plan for V1, including specific archiving practices and compliance considerations.

Instructions and materials​

Follow the instructions below to complete this skills application. Use the form to submit your report. Completing this activity will unlock the solution example on the following page.##Change management plan for V2 deployment- List the key phases for replacing V1 with V2 (pre-deployment checks, rollout, rollback).

  • Suggest deployment strategies (e.g. A/B testing, canary rollout) that reduce risk.
  • Describe how you would apply versioning to code, data and models to ensure traceability.

**Logging and monitoring strategy for​

V1 and V2**

  • Identify at least five critical metrics or metadata to log (e.g. latency, error rates, feature drift, prediction confidence, user engagement).
  • Explain how these logs would help compare V1 and V2 performance and monitor for issues.
  • Recommend conceptual tools or dashboards to collect and visualise this data.

**

Decommissioning plan for V1**

  • Decide the appropriate time to retire V1, and justify your choice.
  • Outline the technical steps to safely decommission V1 while avoiding disruptions.
  • List which artefacts (e.g. model file, metadata, training data snapshot, deployment logs) should be archived, and explain why each is important for audits, compliance or historical learning.

Go deeper​

After completing the activity, consider these questions:

  • How would your strategy change if V2 underperforms compared to V1 during rollout?
  • If a regulator asked for evidence of how V1 made recommendations two years later, would your archive satisfy that requirement?
  • What risks could arise if V1 were decommissioned too quickly without proper communication to stakeholders?

Use the form below to record your answers​

Questions & reflections
Change management plan for V2 deployment
Key phases for replacing V1 with V2:
Deployment strategies:
Your reflection here...
Versioning:
Your reflection here...
Logging and monitoring strategy for V1 and V2
Logging metrics/metadata:
How to use logs to compare V1 and V2 performance and monitor for issues:
Your reflection here...
Tools/dashboards to collect and visualise data:
Your reflection here...
Decommissioning plan for V1
When to retire V1 (justify your choice):
Which artefacts to archive and why:
Your reflection here...