Knowledge check

Instruction and application

In Progress

Your team is preparing to roll out V2 of a fraud detection model to replace V1. To minimise risk, you decide to test V2 with a small subset of live traffic before scaling up. What change management strategy does this represent?

A) A/B testingB) Canary deploymentC) Shadow deploymentD) Blue/green deployment

Which of the following is the most critical pre-deployment step before replacing V1 with V2 of a recommendation model?

A) Removing all V1 resources to reduce costsB) Archiving V2 training datasetsC) Retraining V1 with fresh dataD) Running performance validation on historical datasets and logging baseline metrics

You've deployed V2 of an NLP chatbot model. After rollout, users report delays in response time. Which logged metric would best help diagnose this issue?

A) Model accuracyB) Inference latency per requestC) User engagement scoresD) Feature importance values

Which of the following tools would be most useful for alerting stakeholders when a new model version causes unexpected error spikes?

A) A central log storage system with no alerting capabilityB) A code versioning repositoryC) A real-time monitoring dashboard with automated anomaly detectionD) A model registry

Your HR attrition prediction model is being retired. Which of the following steps should be done first in the decommissioning plan?

A) Archive the trained model and logs.B) Shut down the model endpoints and de-provision compute resources immediately.C) Communicate the decommissioning timeline to stakeholders.D) Delete all dependent pipelines.

When decommissioning an ML model, which of the following resources beyond the model file must typically be cleaned up or de-provisioned?

A) Archived model artefactsB) Temporary training datasets, APIs and compute clustersC) Compliance reportsD) Stakeholder communications

A bank is retiring an old credit risk model. Regulators may request to see the model's decision patterns years later. What should be archived?

A) The final trained model fileB) The model file, training dataset snapshot, inference logs and hyperparameter metadataC) Model accuracy reportsD) Feature importance plots

Why is archiving inference logs important when decommissioning an ML model?

A) To retrain the model on newer dataB) To reduce storage costsC) To simplify the deletion of dependenciesD) To ensure a trail of predictions for future audits and dispute resolution

You deployed a recommendation model and are preparing V2. Which combination of strategies best reduces the risk of disruption to customers?

A) Running canary deployment, logging key metrics and keeping rollback procedures readyB) Deploying blue/green testing with immediate full cutover to V2C) Running A/B testing on V2 against V1 without monitoring production KPIsD) Deploying shadow testing on V2 with synthetic traffic but no rollback plan

Which of the following best describes a holistic life cycle approach to retiring an ML model?

A) Gradually de-provisioning compute resources and archiving model weights to reduce costsB) Conducting performance monitoring up to shutdown and retaining compliance logs, with stakeholder updates planned after decommissioningC) Notifying stakeholders and archiving artefacts, but leaving infrastructure resources running to maintain continuityD) Coordinating change management, logging performance, communicating with stakeholders, de-provisioning resources and archiving artefacts

Evaluate your understanding of this unit by completing the

Knowledge check.

Action item: Knowledge check

In progress