Foundations of ML models and system architectures

Instruction and application

Complete

Building on solid foundations

Like any construction project, the best place to start is with the foundations. Understanding what can and can’t be built in a given scenario is important for streamlining the building process and enabling you to better explain the outcomes to stakeholders.

In this section, we are going to explore different types of architecture and look at their scope, focus and common uses.

Model architecture

Model architecture is the inner workings of the model itself. What are the building blocks? Different types of models (neural networks, tree-based, linear) share an input-transformation-output blueprint.

1. Input and feature representation

Every model needs data. This component focuses on what the data contains and the transformations applied:

✨

Raw Data:

The text, numbers, images, or audio entering the model.

✨

Feature Engineering:

Transforming raw data into usable formats (e.g., vectorisation, PCA).

✨

Preprocessing:

Normalisation, scaling, and tokenisation to improve stability and performance.

2. Model parameters and structure

Learnable weights or coefficients that are adjusted during training to reach the optimal outcome.

✨

Linear Models:

Use intercepts and coefficients.

✨

Tree-based Models:

Use split thresholds.

✨

Neural Networks:

Use weights and biases across input, hidden, and output layers.

Why it matters

Being able to describe the internal parameters and structure helps stakeholders understand how a prediction is made, making the model more credible and trustworthy.

3. Loss (objective) function

How does a model know its predictions are accurate? Supervised models use a loss function to measure the difference between a prediction and the true value. The goal of training is to minimise this function, guiding how it updates the parameters.

Your team is developing a model to predict customer churn. Initial models show high accuracy on the training data but perform poorly on new, unseen data. Which of the following model architecture components and associated techniques would be MOST relevant to investigate and potentially adjust to address this issue?

A) Input and feature representation: Reviewing the raw data sources and ensuring all relevant features are included.B) Model parameters and structure: Increasing the number of layers in a neural network or the depth of a decision tree to capture more complex patterns.C) Regularisation and hyperparameters: Experimenting with L1 or L2 penalties, adjusting dropout rates, or implementing early stopping during training.D) Optimisation algorithm: Switching from stochastic gradient descent (SGD) to Adam to potentially improve convergence speed.

You are building a natural language processing model to classify customer reviews as positive or negative. You notice that the model performs well on longer reviews but struggles with very short, one-word reviews. Considering the model architecture components, which area would offer the MOST direct way to potentially improve the model's performance on these shorter inputs?

A) Loss (objective) function: Changing from binary cross-entropy to hinge loss to adjust the penalty for misclassified short reviews.B) Optimisation algorithm: Implementing a mini-batch gradient descent strategy to ensure the model sees a variety of short and long reviews during each training step.C) Inferences and prediction mechanisms: Adjusting the threshold for classifying a review as positive or negative based on the model's output probability.D) Input and feature representation: Revisiting the text vectorisation technique to ensure short reviews are adequately represented and potentially exploring techniques that are more sensitive to single-word context.

Your organisation is deploying a machine learning model as a web service to provide real-time product recommendations. The data science team needs to ensure the model is consistently receiving the latest product catalogue and user interaction data for accurate recommendations. Which aspect of the system architecture is MOST critical to establish and maintain for this purpose?

A) MLOps tools: Implementing tools for model versioning and rollback in case of deployment issues.B) Cloud platforms: Selecting a cloud provider with robust compute resources for hosting the web service.C) Containerisation: Packaging the model and its dependencies into a Docker container for consistent deployment.D) Data ingestion: Establishing reliable data pipelines to automatically pull the latest product catalogue and user interaction data from their respective sources into the system.

In progress