Skip to main content

Business alignment in metric selection

Instruction and application
Complete

Numbers alone don’t drive value​

** Numbers alone don’t drive value—they matter only when they speak the language of the business.**

You may know how to calculate precision or RMSE—but when should you care about one more than the other? Choosing the right evaluation criteria can significantly shift outcomes across different industries.

Business alignment illustration

Translating business requirements​

When stakeholders define success, they don’t ask for a "high F1-score" or "low MAE." They talk about business goals:

  • “We can’t afford to miss high-risk customers.”
  • “We want fewer false alarms in the system.”
  • “Late deliveries are driving up complaints.”

As a machine learning practitioner, your task is to ** translate those business concerns into measurable model goals.**

Tip: Questions to ask stakeholders​

  • What are the consequences of a false alarm?
  • What happens if we miss something important?
  • Is it worse to act unnecessarily or to miss acting at all?

Cost-sensitive evaluation: Understanding trade-offs​

Every model makes mistakes—but not all mistakes cost the business the same.

  • ** False positive (FP):** The model wrongly predicts something will happen (e.g., predicting fraud on a legitimate transaction).
  • ** False negative (FN):** The model fails to predict something that actually happens (e.g., failing to detect an actual fraud case).

Choosing metrics and thresholds that reflect the ** risk tolerance** of the business is known as** cost-sensitive evaluation** .

Examples of trade-offs​

** If false negatives are risky, prioritise recall.** In healthcare or equipment failure detection, you want to ** catch as many real positives as possible** , even if it means extra false alarms.

** If false positives are costly, focus on precision.** In marketing or fraud detection, too many false alarms waste money or erode trust. You want ** high precision** to ensure positive predictions are correct.

** If big errors hurt most, use RMSE.** In forecasting delivery times, ** large errors** disrupt operations more than small ones. RMSE penalises larger deviations, focusing you on reducing outliers.

Case study: Last-mile delivery​

An e-commerce company decided that ** false positives (late packages promised as on-time)** posed the greatest risk to customer trust.

  • ** The solution:** They prioritised** precision**—ensuring that when the model predicts "on-time," it is likely correct.
  • ** Result:** Higher customer trust and fewer complaints.

Action item: Model trade-off reflection​

You’re supporting a loan approval model for a micro-lending app. The team is deciding how to balance:

  • ** False positives** – approving loans for people who won’t repay.
  • ** False negatives** – rejecting people who would have repaid.

Reflect on which mistake is riskier and which metric you would prioritise.

Questions & reflections
1. Which mistake do you think is riskier in this case?

Type your reflection here...

2. Based on that, which metric would you prioritise—precision or recall?

Type your reflection here...

3. If neither precision nor recall fully captures the business risk, how could you design a custom metric?

Type your reflection here...