Skip to main content

Async review

Instruction and application
In Progress

Recap core topics:​

  • Unit 1: Performance Metric Selection and Implementation
  • Unit 2: Performance Optimisation and Model Refinement

Unit 1: Performance Metric Selection and Implementation​

In Unit 1, you explored…

  • Performance metrics selection

Compass vs. Ruler – How Classification (Accuracy, Precision, Recall, F1) steers categorical hits; and how Regression (MAE, MSE, RMSE, R²) gauges continuous misses.

  • Single-View Snapshot – How F1-score balances classification; and how R² reveals regression explainability.

  • Business alignment in metric selectionFalse-positive phobia – When too many false alarms erode trust, dial up precision.

  • False-negative fear – If missing a hit is disaster, crank recall to catch every signal.

  • Balanced battleground – When both misfires matter, wield the F1-score as your unified gauge.

Unit 2: Performance Optimisation and Model Refinement

In Unit 2, you explored…

  • Cross-validation techniques

Partition data into k folds, training on k−1 and testing on the hold-out fold to average out variance.

  • Preserve class proportions in each fold with Stratified Cross-Validation to avoid bias in imbalanced classification.

  • Aggregating results across folds delivers more robust, unbiased performance estimates than a single train/test split.

  • Use standard K-Fold for fairly uniform data; opt for stratification when class imbalance might skew your evaluation.

  • Error analysis methodsConfusion matrices decompose predictions into TP, FP, FN, TN to pinpoint exactly where your model errs.

  • ROC curves plot true-positive rate against false-positive rate across all thresholds, with AUC summarising overall discrimination.

  • Use confusion matrix insights to choose and tune metrics (Precision, Recall) based on specific error costs.

  • Leverage ROC/AUC threshold analysis to select the optimal operating point for your business priorities.

Knowledge check​

Let's see how much you can remember about async units 1 and 2.

Try taking the quiz below, remember it's not a test, it's to help you find out if there are any areas you should brush up on.