Resource calculation and supply chain risk mitigation
Instruction and application
In ProgressWhen an ML system crawls during peak usage or stalls because a third-party API fails, the root cause is often not the algorithm — it’s poor planning. Accurate resource calculation and proactive supply chain risk mitigation can make the difference between a model that performs reliably and one that fails when it matters most.
In this section, you’ll learn a structured approach to estimating your model’s resource needs and building resilience against supply chain disruptions.
Calculating resource requirements for ML models
Before addressing supply chain risks, you need a clear view of the resources required to meet performance targets. This involves translating operational requirements, such as throughput, latency and availability, into specific compute, memory and bandwidth needs.
These calculations typically cover two areas: Inference, where the model serves predictions andtraining, which involves the model's build or updates. Select the tabs to learn more. ### Inference calculationsIn the inference stage, your goal is to ensure the system can handle real-time or batch predictions without exceeding latency or throughput limits. Accurate calculations here help prevent performance bottlenecks and maintain a smooth user experience even under peak load.
-
CPU/GPU: Factor in the number of operations per second (FLOPs), inference latency per request, desired throughput and batch size. For GPU-based inference, also consider tensor core utilisation and precision (FP32, FP16, INT8) to optimise cost and speed.
-
Memory: Account for model size, batch size and the size of input/output data, ensuring enough headroom to handle peak loads without paging to disk.
-
Network bandwidth: Plan for data transfer rates for inputs/outputs, model updates and any external API calls the inference service relies on. ### Training calculations In the training stage, the focus shifts to allocating enough compute, memory and storage to complete model training or retraining within acceptable timeframes. This ensures the ability to iterate quickly, test new architectures and update models efficiently.
-
CPU/GPU: Consider dataset size, model complexity (parameters), number of epochs, batch size and desired training completion time. For deep learning, factor in GPU memory constraints and any requirement for distributed training.
-
Memory: Include the dataset size, model parameters and intermediate activations stored during forward/backward passes.
-
Storage: Plan for raw datasets, processed features, model checkpoints and training logs, with redundancy for recovery in case of failure.
Example: Estimating resources for a computer vision pipeline
A retail company wants to deploy a vision model that detects product defects in real time on its manufacturing line and retrains weekly with fresh data. During peak production, the system processes 18,000 images per minute, with each inference taking 50 ms on a single CPU core.
-
Inference CPU calculation:Requests per second = 18,000 ÷ 60 = 300 TPS.
-
Each core processes 20 requests/sec (1 ÷ 0.05s).
-
Required cores = 300 ÷ 20 = 15 (rounded up to 18 for overhead and failover).
-
Training GPU calculation:Dataset size for retraining = 150 GB with 40M model parameters, trained for eight epochs at batch size 128.
-
Estimated total = ~160 GPU hours on a single NVIDIA V100.
-
Parallelising across 4 GPUs reduces training time to ~40 hours, enabling the weekly retraining schedule. By calculating these requirements up front, the company can provision enough compute for smooth real-time inference while maintaining an efficient retraining cycle, avoiding performance slowdowns and wasted resources.
Validate estimates with load testing
Once you’ve estimated your resource requirements, validate them with load testing to simulate peak conditions. This helps uncover hidden bottlenecks and ensures your model meets performance targets before deployment.
Provisioning resources: Cloud vs on-premises
Once you know your compute, memory, storage and bandwidth needs, the next decision is where to provision them.
- Cloud provisioning offers elasticity and pay-as-you-go pricing, making it ideal for variable workloads or unpredictable spikes. It allows rapid experimentation and scaling without the delays of hardware procurement, which can be critical in fast-moving ML projects.
- On-premises provisioning gives more control and consistent performance. It can be cost-effective for steady, high-volume workloads, although it requires higher upfront investment. It also avoids potential vendor lock-in and can better address strict data residency or compliance requirements.
- Hybrid approaches combine the predictability of on-premises with the burst capacity of the cloud, balancing cost and flexibility. This model lets teams keep sensitive workloads in-house while tapping into cloud resources for testing, seasonal peaks or special projects.
Mitigating supply chain risks in ML deployments
Provisioning resources is only part of the equation — ML systems also rely on external services, data providers and hardware. If any link fails, the pipeline can break. Supply chain risk in ML spans APIs, open-source libraries, cloud vendors and specialised hardware, making it a critical factor to plan for.
Data-related risks
When data stops flowing correctly, the entire ML pipeline can stall. Understanding where and how your data sources might fail is the first step in building resilience.
- Availability and reliability: External data sources, such as public APIs or third-party feeds, may change schema or throttle access or go offline entirely.
- Pipeline failures: Breaks in ETL or streaming jobs can halt training or inference, especially in real-time applications.
- Governance and compliance: Violation of regulatory obligations, such as GDPR and CCPA, can occur if data origins, consent or storage practices are not tracked and enforced.
For each risk, assess the following:
- Likelihood: How probable is it that this risk will occur?
- Severity of impact: What would the consequences be, such as data loss, downtime, financial impact or reputational damage?
- System-wide effects: How could this risk cascade or trigger failures in other parts of your ML pipeline?
Software and hardware risks
Beyond data, the tools and infrastructure you rely on also carry vulnerabilities that can limit scalability or even take systems offline.
- Framework dependencies: Relying heavily on a specific ML framework (e.g. TensorFlow, PyTorch) exposes you to API changes, licensing shifts or discontinued features.
- Hardware shortages: Supply chain disruptions or global demand spikes can affect GPU and specialised accelerator availability.
- Vulnerabilities in dependencies: Open-source libraries and pre-trained models can contain security flaws that compromise your system.
- Vendor lock-in: Exclusive reliance on a single cloud provider or proprietary tool can limit flexibility and raise migration costs.
Supply chain risks — mitigation strategies
Developing a truly comprehensive mitigation strategy goes beyond merely listing solutions. It involves a layered approach, careful justification of chosen methods and an understanding of the trade-offs in terms of cost, complexity and residual risk. Not every risk requires the most expensive mitigation — understanding the likelihood and impact (as discussed above) guides your strategic choices. The following strategies outline practical ways to strengthen your ML system against disruptions across data, software and infrastructure layers.
- Redundancy and diversification: Use multiple data sources, adopt multi-cloud strategies and implement failover mechanisms to reduce single points of failure. This often involves a trade-off with increased infrastructure cost and management complexity, which must be justified by the risk tolerance.
- Monitoring and alerting: Proactively watch external APIs, feeds and model dependencies for latency spikes, schema changes or downtime. Implement robust alerting that prioritises critical events based on their potential business impact.
- Data contracts and SLAs: Secure formal agreements for critical data pipelines to guarantee availability and performance. These should ideally include clear escalation paths and penalties for non-compliance, reflecting the financial and operational impact of a data supply chain disruption.
- Vendor due diligence: Assess providers’ stability, security posture and compliance track record and regularly review these assessments as part of a continuous risk management process.
- Robust data validation: Implement automated checks at ingestion to catch corruption or unexpected changes early. Consider multiple layers of validation (e.g. schema, range, integrity) and implement circuit breakers to prevent problematic data from propagating further into the pipeline.
- Security audits and vulnerability management: Regularly scan all dependencies and infrastructure for known weaknesses. This includes conducting regular penetration testing and red-teaming exercises to proactively identify vulnerabilities related to both software and data handling.Containerisation and orchestration: Use tools such as Docker and Kubernetes for environment consistency and portability. While enhancing flexibility, ensure container images undergo regular scanning for vulnerabilities and have minimal necessary components in their builds to reduce the attack surface.
- Standardisation: Stick to widely supported formats and protocols to reduce complexity and improve interoperability. This also simplifies the application of security policies and monitoring across different data flows.
Example scenario
A fintech company deploys a fraud detection model that relies on:
- A third-party transaction feed for near real-time purchase data.
- A specific GPU type available through a single cloud provider for both training and inference.
- An open-source anomaly detection library integrated into its pre-processing pipeline. One morning, the data provider changes the feed’s schema without notice, breaking the ETL pipeline. At the same time, a surge in demand for GPUs leads to limited availability in the cloud region the company uses, delaying model retraining. To make matters worse, a newly discovered vulnerability in the open-source library forces the team to halt updates until the release of a patch. Mitigation in action-Redundancy: Maintain a backup transaction feed from a secondary provider to switch over when the main one fails.-Multi-cloud strategy: Have GPU provisioning scripts ready for two different cloud vendors to avoid capacity shortages.
- Security monitoring: Use automated dependency scanning to flag vulnerabilities in open-source components, with a predefined protocol for rapid patching or temporary isolation.
Action item: Stress-testing your deployment plan
This activity will challenge you to connect resource estimation with real-world risk planning.
Scenario
You’re preparing to deploy an NLP summarisation model that must process 100,000 text inputs per hour at peak load. Each request takes approximately50 ms on a single CPU core, and the model’s size is500 MB. The deployment relies on apre-trained LLMaccessed via a third-party API and hosted on asingle cloud provider.Complete the form below as you work on the activity tasks.