Skip to main content

Async review

Instruction and application
In Progress

Recap core topics:​

  • Unit 2: Scalability and capacity management.
  • **Unit 3:**Model life cycle management and decommissioning.

Unit 2: Scalability and capacity management​

In Unit 2, you explored…

  • Scalability drivers: This section discusses how data volume, velocity, variety and quality affect ML system performance, efficiency and resource use.
  • Data handling strategies: These include batch vs streaming pipelines, distributed storage, partitioning, feature stores and schema enforcement for diverse data formats.
  • Capacity planning frameworks: These include defining operational requirements (throughput, latency, availability, scalability, cost-effectiveness), mapping them across ML pipelines and using workload characterisation, load/stress testing, benchmarking and profiling to prepare for growth.
  • Scaling strategies: These include horizontal, vertical and autoscaling approaches, plus load balancing to manage demand surges cost-effectively.
  • Resource estimation: This topic demonstrates calculating compute, memory, storage and bandwidth needs for inference and training, validated with load testing.
  • Supply chain risks: This section covers vendor lock-in, third-party API reliance, GPU shortages, software dependencies and governance issues, plus strategies such as redundancy, monitoring, contracts, multicloud set-ups and containerisation to mitigate them.

Unit 3: Model life cycle management and decommissioning​

In Unit 3, you explored…

  • Managing change in ML: How structured processes, versioning (code, data, models), automated testing and staged deployments (shadow, canary, A/B) ensure smooth, traceable and compliant updates.
  • Logging and monitoring: Why detailed logs (metadata, deployment records, inference data, audit trails) are critical for auditability, debugging, compliance and reproducibility, supported by tools such as MLflow, model registries, observability and alerting systems.
  • Decommissioning protocols: Safe retirement of ML models through dependency mapping, stakeholder communication, traffic diversion and resource de-provisioning while monitoring for disruptions.
  • Archiving for compliance and reproducibility: Preserving model artefacts, metadata, training data snapshots, logs and code to meet regulatory requirements (e.g. GDPR, HIPAA, EU AI Act) and support audits, dispute resolution and institutional learning.

Scalability essentials​

  • Throughput and** latency**define system performance under load.
  • **Scaling strategies:****Vertical scaling =**add more power to one machine.
  • **Horizontal scaling =**add more machines to share the load.
  • Autoscaling = automatic adjustment based on demand.
  • Data variety and formats (structured, unstructured, multimodal) affect scalability and resource requirements.
  • Data quality issues (missing values, schema changes, noisy inputs) increase compute costs and reduce efficiency.

Supply chain and governance risks​

  • Supply chain risks: Vendor lock-in, reliance on third-party APIs, GPU shortages.
  • **Change management:**Version control and model registries.
  • Staged deployments: Shadow, canary, A/B testing.
  • **Logging essentials:**Inference logs, metadata and audit trails for compliance and traceability.

Decommissioning protocols​

  • **Dependency mapping:**Identify systems and stakeholders relying on the model.
  • **Traffic diversion:**Safely shift users/services to newer models through staged rollouts.
  • **De-provisioning:**Retire compute and storage resources once dependencies are cleared.
  • **Archiving:**Retain models, datasets, logs and documentation for compliance and traceability.
  • **Communication:**Inform stakeholders of retirement timelines and mitigation plans to maintain business continuity.

Action item: Poll — from scaling to retiring​

It's time for a quick life cycle-focused poll. This will help you check your understanding of scalability, governance and model retirement practices in production ML systems. No pressure — just go with your best judgement!