Skip to main content

Challenge instructions

Instruction and application
In Progress

To strengthen your deployed ML solution, you'll now focus on monitoring it in production, planning its lifecycle, and communicating its value and risks to stakeholders through a tailored team presentation.

Hackathon Resource Pack​

If you haven't already, or if you need to re-download it, download the Hackathon Resource Pack now. It contains:

  • Descriptions of the two scenarios.
  • Reference model artefacts for each scenario.
  • Deployment harness used in Workshop 1.
  • Monitoring plan template with drift-detection rubric.
  • The stakeholder presentation template you will use for the wrap up. AIMLF Hackathon 3 Resource Pack.zip

Once downloaded, open the files and keep them accessible for the rest of the hackathon.

### Instructions

Design the monitoring plan​

Complete the monitoring plan template for your deployed model, covering:

  • Operational health metrics (latency, error rate, throughput).
  • Prediction-quality metrics tied to the business objective.
  • Explainability spot-checks using SHAP or LIME on flagged predictions.

Focus on a plan you could actually operate — not an exhaustive theoretical list.

Drift detection and adaptation​

Define how you will detect model and data drift, and how you will respond. Cover:

  • Which input distributions and output distributions you will watch.
  • The thresholds or statistical tests that will fire an alert.
  • The retraining trigger — what evidence would justify retraining, and who signs off.

Identify:

  • Which drift signals are most likely to appear first.
  • Which are cheapest to remediate versus most business-critical.
  • Key trade-offs between alert sensitivity and alert fatigue.

Lifecycle and decommissioning plan​

Define the end-to-end lifecycle of your deployed model. Your plan should consider:

  • Retraining cadence and versioning strategy.
  • Security, privacy, and compliance considerations.
  • Ownership and on-call responsibility.
  • Criteria for decommissioning the model, and the runbook for doing so safely.

You may use LLMs to:

  • Critique your monitoring thresholds.
  • Challenge assumptions in your retraining triggers.
  • Help refine your decommissioning runbook.

Prepare the stakeholder presentation​

Build the required 5-minute audience-tailored team presentation, covering:

  • The deployed solution and its architecture.
  • The monitoring and drift-detection plan.
  • The lifecycle plan, including retraining and decommissioning.
  • Key risks, explainability findings, and recommendations.

Tailor language and depth for both a technical reviewer and a non-technical business stakeholder.

Practical risk and communication reflection​

Consider:

  • What happens if monitoring fails silently.
  • Where human oversight remains essential.
  • Which risks you would flag to executives versus operators.
  • What you would improve with more time, data, or budget.

Looking ahead to the wrap up workshop​

In the wrap-up workshop, you'll deliver your 5-minute team presentation and hand over your deployed solution, monitoring plan, and technical documentation as your end-of-programme artefacts.

In progress