Skip to main content

Skills application

Instruction and application
In Progress

Skills application submit​

Simulating inference scaling trade-offs​

In this skills application activity, you'll simulate how different infrastructure choices impact the performance and cost of an ML inference system. You'll test how latency and throughput respond to vertical and horizontal scaling strategies under varying request loads.

Learning this skill brings value to the workplace by building practical intuition around system design trade-offs. It supports better decision-making when selecting compute resources, helps manage cloud costs, and ensures that ML services remain responsive at scale.

Activity instructions and resources​

Select the link below to download and access the Jupyter Notebook for this activity. It includes the instructions needed to complete the activity.

Optional: Download a copy of the workshop slides​

Work on the challenges​

Follow the instructions in the Jupyter Notebook to work on the challenges to simulate inference performance and observe how system behavior changes under different scaling strategies.

Collaborate in the breakout room​

Discuss and share insights with fellow apprentices as you work on the activity challenges. While the activity is designed for individual completion, feel free to ask questions, collaborate, and compare approaches with your group.

Share your findings​

Submit your completed Jupyter Notebook and summarise your findings.

Regroup​

Return to the main session after 35 minutes to discuss key takeaways and insights from the exercise.

Action item: Activity share out​

  • Based on our simulations, what are the primary advantages and disadvantages of vertical scaling versus horizontal scaling for an ML inference service?
  • How does the "cost" aspect influence your architectural decisions for ML platforms, especially when considering performance targets?
  • Can you think of real-world cloud services or technologies that embody these vertical and horizontal scaling concepts?

Skills application​

Simulating inference scaling trade-offs

In this skills application activity, you'll simulate how different infrastructure choices impact the performance and cost of an ML inference system. You'll test how latency and throughput respond to vertical and horizontal scaling strategies under varying request loads.

Learning this skill brings value to the workplace by building practical intuition around system design trade-offs. It supports better decision-making when selecting compute resources, helps manage cloud costs, and ensures that ML services remain responsive at scale.

Activity instructions

  • Work on the challenges: Follow the instructions in the Jupyter Notebook to work on the challenges to simulate inference performance and observe how system behavior changes under different scaling strategies.
  • Collaborate in the breakout room: Discuss and share insights with fellow apprentices as you work on the activity challenges. While the activity is designed for individual completion, feel free to ask questions, collaborate, and compare approaches with your group.
  • Share your findings: Submit your completed Jupyter Notebook and summarise your findings.
  • Regroup: Return to the main session after 35 minutes to discuss key takeaways and insights from the exercise.