Skip to main content

Skills application

Instruction and application
In Progress

Skills application submit​

Building a data drift alert system​

In this skills application, you'll analyse input data distributions over time to detect data drift in a deployed ML system. Starting with a baseline dataset, you'll compare it against simulated production data, apply a statistical test (Kolmogorov–Smirnov), and build a basic monitoring loop that triggers alerts when drift is detected.

Learning this skill brings value to the workplace by enabling robust monitoring of ML systems, improving early detection of performance degradation, and supporting more stable, risk-aware deployments.

Activity resources​

To complete this activity, you’ll need a Jupyter Notebook andtwo synthetic datasets. These files simulate a real-world model monitoring scenario where data distributions shift between training and production environments.

  • **baseline_data.csv:**The reference dataset used to train the original model.
  • **production_data.csv:**Simulated production data containing a shifted distribution to illustrate data drift. Use the links below to access and download the required files before starting the activity:

Optional: Download a copy of the workshop slides​

Optional: Download a copy of the workshop slides​

Optional: Download a copy of the workshop slides​

Work on the challenges​

Follow the instructions in the Jupyter Notebook to load the datasets, visualise distributions, detect statistical drift, and simulate a monitoring loop with drift alerts.

Collaborate in the breakout room​

Discuss and share insights with fellow apprentices as you work on the activity challenges. While the activity is designed for individual completion, feel free to ask questions, collaborate, and compare approaches with your group.

Share your findings​

Submit your work showing your plots, drift detection code, and monitoring loop. Be sure to include any comments or explanations that clarify your approach.

Regroup and share​

Return to the main session after 25 minutes to discuss key takeaways and insights from the activity.

Action item: Activity share out​

  • How does the statistical test used in this activity support real-time monitoring and logging in a deployed ML system?
  • From a risk management perspective, how could automated drift detection like this help prevent major model failures?
  • If this monitoring system triggered an alert, what kind of rollback strategy would make sense in response?

Skills application​

Building a data drift alert system

In this skills application, you'll analyse input data distributions over time to detect data drift in a deployed ML system. Starting with a baseline dataset, you'll compare it against simulated production data, apply a statistical test (Kolmogorov-Smirnov), and build a basic monitoring loop that triggers alerts when drift is detected.

Learning this skill brings value to the workplace by enabling robust monitoring of ML systems, improving early detection of performance degradation, and supporting more stable, risk-aware deployments.

Activity instructions

  • Work on the challenges: Follow the instructions in the Jupyter Notebook to load the datasets, visualise distributions, detect statistical drift, and simulate a monitoring loop with drift alerts.
  • Collaborate in the breakout room: Discuss and share insights with fellow apprentices as you work on the activity challenges. While the activity is designed for individual completion, feel free to ask questions, collaborate, and compare approaches with your group.
  • Share your findings: Submit your work showing your plots, drift detection code, and monitoring loop. Be sure to include any comments or explanations that clarify your approach.
  • Regroup and share: Return to the main session after 25 minutes to discuss key takeaways and insights from the activity.