Skills application
Instruction and application
In ProgressSearch at scale: Capacity planning and risk-proofing your ML system
In this skills application, you will apply what you’ve learned about capacity planning and supply chain risk mitigation to design a preliminary capacity plan for a new ML-powered feature.
Context
Your company is launching a new Smart Search feature for its e-commerce website. This feature uses an ML model to re-rank search results in real-time based on user click-through rates and purchase history. The model runs inferenceas users type, making low latency critical.Scenario details:
- Peak traffic: 1,000 search queries per second.
- Inference: Each query triggers one ML model prediction.
- Average inference time: 100 ms per prediction on current hardware.
- Traffic growth: Expected to double over the next 12 months.
- Deployment: Model deployed as a microservice on a cloud platform (e.g. AWS, Azure, GCP).
- Data sources: Internal website logs and a third-party product catalog API.
Success criteria
To complete the skills application, you must:
- Calculate initial resource requirements for inference at the current peak load.
- Recommend a scaling strategy and justify your choice.
- Adjust the plan for future growth and explain how you would validate it with load testing.
- Analyse three supply chain risks in detail, explaining their potential**impact (severity and likelihood)**on ML operations.
- For each risk, propose comprehensive, layered mitigation strategies, justifying their choice and considering any associated trade-offs (e.g. cost, complexity).
Instructions and materials
Follow the instructions below to complete this skills application. Use the form to submit your answers. Completing this activity will unlock the solution example on the following page.Calculate initial resource requirements- Determine the minimum number of concurrent inference requests the system must support at peak load.
- Estimate the total processing capacity (in inferences per second) needed.
- Suggest whether horizontal or vertical scaling is more suitable, and explain why. Plan for future capacity- Adjust your estimates to account for 2x traffic growth in the next 12 months.
- Describe how you would conduct load testing to confirm your plan before launch. Identify and mitigate supply chain risks- List at least three risks considering data, software and external dependencies.
- For each of the three risks, you must: Analyse the risk: Describe the risk in detail, outlining how it could manifest, its estimated likelihood, and its specific impact on the ML feature's performance, cost, security or reliability.
- Propose comprehensive mitigation: Design a layered mitigation strategy. Explain why each part of the strategy is effective and discuss any potential trade-offs (e.g. increased cost for improved reliability, reduced performance for higher security).
Go deeper
After completing the activity, consider these questions:
- Reflect on how your capacity plan and risk mitigation choices represent a strategic balance between operational needs (performance, availability), security and cost. Where did you make deliberate trade-offs, and how would you justify these to stakeholders?
- Considering the identified supply chain risks, how did these specific risks directly influence or constrain your design choices for scalability and the overall security posture of your ML/AI infrastructure? Provide concrete examples.
- What data-specific security challenges might arise from the use of internal website logs and a third-party product catalog API (considering their variety, quality and format), and what tailored mitigation strategies would you propose beyond general data validation?