Welcome to the workshop!
Workshop
In ProgressOptimising ML Inference Scaling for Performance and Cost
Today's icebreaker:
One giant or many small?
If you had to serve one million predictions per second, would you rather use one super-powerful computer or many smaller computers?
Share your response in the chat!
Today's agenda:
✨
Review:
Recap key concepts.
✨
Practice:
Simulating inference scaling trade-offs.
✨
Closing:
Key takeaways and next steps.
Today's learning objectives:
- Understand the fundamental trade-offs between latency, throughput, and resource allocation for ML inference workloads.
- Simulate the impact of different scaling strategies (vertical vs. horizontal) on the performance and theoretical cost of an ML inference service.
- Critically assess how various computational resources relate to operational requirements in ML/AI systems.
- Reinforce concepts of performance optimisation and cost-efficiency in designing scalable ML platform architectures.
Workshop slides
Preview or download a copy of the workshop slides:
PDF preview loads in your browser…
If it doesn’t appear, download it here.
In progress