Skip to main content

Skills application

Skills application
In Progress

Skills application

In this skills application, you will apply your knowledge of performance metrics by selecting, justifying, and implementing the most appropriate evaluation approach for a real-world business scenario.

This will strengthen your ability to link model performance to business priorities and communicate trade-offs clearly to stakeholders.

Skills application illustration

Context​

You’re working with a subscription-based online education platform that offers tailored course recommendations to learners. The company is developing a machine learning model to predict whether a user is likely to complete a recommended course based on their profile, behaviour on the platform, and past engagement.

The business goal is to increase course completion rates while optimizing marketing spend on reminders and promotions. Marketing teams are concerned about:

  • ** Sending promotions to users who would complete the course anyway (false positives)** .
  • ** Missing users who would benefit from encouragement (false negatives)** . Your task is to evaluate the model’s performance and propose the most suitable metrics based on business impact, data characteristics, and stakeholder needs.** Success criteria**

To successfully complete the skills application, you must:

  • Select at least two performance metrics to evaluate the model.
  • Justify your choices by explaining their relevance to the business context and error trade-offs.
  • Communicate your reasoning in clear, stakeholder-friendly language.
  • Submit your responses using the form to unlock the solution example on the next page.

Instructions​

To complete this activity, review the scenario and respond to the following prompts. Use the form provided to record and submit your answers.

Once all fields are complete, the solution page will be unlocked.

Step 1: Understand the context​

In 1–2 sentences, summarize the business goal and the role of the ML model. What is the model predicting, and how will it support business outcomes?

Step 2: Define the error trade-off​

Briefly explain which type of error (false positive or false negative) is more costly in this context—and why.

Step 3: Select your metrics​

Choose at least two performance metrics (e.g., precision, recall, F1-score, accuracy, MAE, RMSE). For each one, explain:

  • What it measures.
  • Why it is appropriate for this use case.

Step 4: Communicate to stakeholders​

Write 2–3 sentences on how you would explain your metric choices to a non-technical stakeholder—like the marketing or leadership team.** Go deeper**

After completing the activity, consider these questions:

  • How would your metric selection change if the business goal shifted?
  • What assumptions did you make about user behaviour or business priorities?
  • How might these metrics inform future improvements to the model?
Questions & reflections
Describe the business goal and the role of the ML model. What is the model predicting, and how will it support business outcomes?
Your reflection here...
Briefly explain which type of error (false positive or false negative) is more costly in this context—and why.
Your reflection here...
Specify at least two performance metrics (e.g., precision, recall, F1-score, accuracy, MAE, RMSE). For each one, explain:

What it measures.

Why it is appropriate for this use case.

Your reflection here...

In 2–3 sentences, describe how you would explain your metric choices to a non-technical stakeholder—like the marketing or leadership team.
Your reflection here...