Skip to main content

Foundations of ML models and system architectures

Instruction and application
Complete

Building on solid foundations​

Like any construction project, the best place to start is with the foundations. Understanding what can and can’t be built in a given scenario is important for streamlining the building process and enabling you to better explain the outcomes to stakeholders.

In this section, we are going to explore different types of architecture and look at their scope, focus and common uses.

Foundations illustration

Model architecture​

Model architecture is the inner workings of the model itself. What are the building blocks? Different types of models (neural networks, tree-based, linear) share an input-transformation-output blueprint.

1. Input and feature representation​

Every model needs data. This component focuses on what the data contains and the transformations applied:

  • Raw Data: The text, numbers, images, or audio entering the model.
  • Feature Engineering: Transforming raw data into usable formats (e.g., vectorisation, PCA).
  • Preprocessing: Normalisation, scaling, and tokenisation to improve stability and performance.

2. Model parameters and structure​

Learnable weights or coefficients that are adjusted during training to reach the optimal outcome.

  • Linear Models: Use intercepts and coefficients.
  • Tree-based Models: Use split thresholds.
  • Neural Networks: Use weights and biases across input, hidden, and output layers.

Why it matters​

Being able to describe the internal parameters and structure helps stakeholders understand how a prediction is made, making the model more credible and trustworthy.

3. Loss (objective) function​

How does a model know its predictions are accurate? Supervised models use a loss function to measure the difference between a prediction and the true value. The goal of training is to minimise this function, guiding how it updates the parameters.