Core ML dimensions
Poll
CompleteNow that we've established a solid understanding of the foundations, history and relationship between ML and AI, let’s dive into a few of the core dimensions that serve as the backbone of ML and AI applications — learning paradigms and task types.
These dimensions represent different approaches to learning from data and solving complex problems, each with a unique set of techniques and use cases. Let's explore the key learning paradigms and task types that drive the most impactful AI and ML innovations.
Learning paradigms
A learning paradigm is the overarching approach or strategy used in training an ML model. It defines how a model learns from data and includes the assumptions and processes underlying the training mechanism. There are several key learning paradigms in ML, each suited to different types of problems and data structures and equipped with their unique challenges:
Supervised learning
Supervised learning is one of the most widely used ML methods. In this approach, the algorithm is trained on labelled data, where the input data is paired with the correct output. This allows the model to learn the relationship between the features (inputs) and labels (outputs) and make predictions on new, unseen data.
Common applications of supervised learning include regression tasks, where the model predicts continuous values (e.g. house prices), and classification tasks, where the goal is to categorise data into predefined classes (e.g. spam detection in emails or image recognition).
The algorithm's performance improves as it is exposed to more labelled examples, allowing it to make more accurate predictions.
Supervised learning in the real world:
- Spam detection: Supervised learning is used to classify emails as spam or not spam based on labelled training data.
- House price prediction: Real estate companies and analysts use supervised learning regression models to predict house prices based on features such as location, square footage, number of bedrooms and bathrooms, and age of the property.
| Challenge | Solution |
|---|---|
| Requires a large amount of labelled data, which can be expensive or time-consuming to gather. | Use semi-supervised or active learning, where a small amount of labelled data is supplemented by a larger amount of unlabelled data. |
Unsupervised learning
In contrast to supervised learning, unsupervised learning deals with unlabelled data, where the algorithm does not have predefined outcomes or target labels. Instead, it seeks to uncover hidden patterns or structures within the data on its own.
The most common techniques in unsupervised learning include clustering, where data points are grouped based on similarities (e.g. customer segmentation in marketing), and dimensionality reduction, which reduces the number of features in the data while preserving important information.
Unsupervised learning is especially useful for discovering insights when labelled data is scarce or unavailable.
Unsupervised learning in the real world:
- Customer segmentation: Businesses use unsupervised learning to group customers into distinct segments based on purchasing behaviour, demographics or other features.
- Anomaly detection: In fraud detection, unsupervised learning can identify unusual patterns or outliers in transaction data that may indicate fraudulent activity.
| Challenge | Solution |
|---|---|
| Identifying meaningful patterns without predefined labels can lead to ambiguous results. | Use domain knowledge to guide clustering or employ techniques like dimensionality reduction to simplify data. |
Semi-supervised learning
Semi-supervised learning strikes a balance between supervised and unsupervised learning by using both labelled and unlabelled data. Typically, only a small portion of the data is labelled due to the high cost or effort involved in manual labelling, while the majority remains unlabelled.
This approach allows the model to leverage a larger amount of unlabelled data alongside the limited labelled data to improve its learning accuracy. Semi-supervised learning is particularly effective in scenarios where labelled data is scarce but unlabelled data is abundant, such as in medical imaging or speech recognition tasks.
Semi-supervised learning in the real world:
- Medical imaging: Large datasets of medical images with limited labelled examples (e.g. only a few images labelled as containing tumors) can benefit from semi-supervised learning by using abundant unlabelled images to improve model performance.
- Speech recognition: A small set of labelled audio samples combined with vast amounts of unlabelled audio data can improve recognition accuracy.
| Challenge | Solution |
|---|---|
| The quality of the small labelled dataset significantly impacts the model's performance. | Ensure the labelled data is as accurate and representative as possible to guide the learning process effectively. |
Reinforcement learning
Reinforcement learning (RL) involves an agent that learns by interacting with an environment. The agent makes decisions based on the actions it takes, and the environment provides feedback in the form of rewards or penalties.
The goal of the agent is to optimise its actions over time to maximise cumulative rewards. Unlike supervised learning, RL does not require labelled data — it relies on trial and error for learning.
Reinforcement learning in the real world:
- Robotics: RL is used in training robots to perform complex tasks, such as navigating environments, assembling products or interacting with humans.
- Game-playing AI: RL algorithms have been used to train AI systems like AlphaGo and chess engines, where the agent learns optimal strategies by playing the game repeatedly.
- Self-driving cars: RL is applied to help autonomous vehicles learn to navigate traffic, make decisions in real time and improve safety.
| Challenge | Solution |
|---|---|
| Reinforcement learning models often require a significant amount of interaction with the environment, which can be resource-intensive. | Use simulation environments to accelerate training and reduce the costs of real-world trials. |
Paradigm benefits and drawbacks
Whether you're working with supervised, unsupervised, semi-supervised or reinforcement learning, each approach offers distinct benefits and potential drawbacks that can impact their suitability for different tasks. By examining these aspects, you'll be better equipped to select the most appropriate paradigm for your ML projects and navigate the trade-offs involved effectively.
| + Benefits | Paradigm | - Drawbacks |
|---|---|---|
| High predictive accuracy when trained on a well-curated dataset. | Supervised | Requires large amounts of high-quality labelled data. |
| Provides insights into how predictions are made, making them easier to understand and trust. | Models may struggle with unseen or imbalanced data, leading to overfitting. | |
| Works with raw, unlabelled data, reducing the time and cost of data preparation. | Unsupervised | Without labelled data, models may struggle to assign meaningful interpretations to clusters or patterns. |
| Good for identifying structures in data, such as customer segments, fraud detection or anomalies, without prior knowledge. | There’s no ground truth to compare results against, making performance assessment more challenging. | |
| Can adapt more easily to new and evolving datasets. | Insights generated may not always be intuitive or actionable without additional analysis. | |
| Reduces the cost and effort involved in labelling large datasets. | Semi-supervised | The quality of the model can still depend heavily on the small labelled dataset. |
| Can achieve high accuracy by utilising unlabelled data to complement labelled data. | If the unlabelled data is not representative or is noisy, it may negatively impact the learning process. | |
| Works well in scenarios where labelled data is scarce but unlabelled data is abundant. | More complex than purely supervised or unsupervised learning. | |
| Can handle complex decision-making tasks where actions lead to long-term outcomes (delayed rewards). | Reinforcement | Can require extensive computational resources for training, especially in large environments. |
| Highly effective in environments that require real-time feedback and dynamic decision-making. | The trial-and-error nature of learning can be inefficient and time-consuming. | |
| Useful in areas like robotics, game-playing and autonomous systems. | Difficult to implement in highly unpredictable or unstable environments. |
Task types and algorithms
With a clear understanding of the different ML paradigms, you're now ready to dive deeper into the most common ML task types and the algorithms that drive them. ML tasks and algorithms are the specific structures and processes through which data is analysed and predictions are made. Each of these tasks has unique characteristics, making them suitable for different types of problem-solving. Understanding when and how to use these task types is important for any aspiring ML engineer.
Each of the ML task types described below falls under the category of classical model types — a category that uses established algorithms (e.g. decision trees, linear regression) that often rely on manual feature engineering.
Key characteristics of classical models
- Require structured data with predefined features.
- Need manual feature engineering to extract relevant attributes from raw data.
- Based on well-understood statistical principles and have been widely used for decades.
- Generally less complex and computationally intensive compared to deep learning models.
- More interpretable and explainable than deep learning models.
Classification
Classification is one of the most widely used types of ML, where the task is to predict a categorical label for a given input. Essentially, the model learns to classify data points into predefined categories based on features provided during training.
Common classification algorithms:
- Logistic regression: Despite its name, logistic regression is used for binary classification tasks (e.g. spam vs non-spam). It estimates the probability of a class by applying a logistic function.
- Decision trees: A decision tree splits data into smaller subsets based on the values of features, and each leaf node represents a predicted class. It's easy to interpret and visualise.
- Random forests: An ensemble of decision trees, random forests aggregate the predictions of many decision trees to improve accuracy and reduce overfitting.
- Support Vector Machines (SVM): SVM works by finding a hyperplane that best separates classes in the feature space. It’s particularly effective for high-dimensional data.
- K-Nearest Neighbours (KNN): A simple but effective algorithm that classifies data points based on the majority class of their closest neighbours in the feature space.
When should I use classification?
Classification models are ideal when the output is a distinct category, such as predicting customer churn (yes/no), diagnosing diseases (positive/negative) or classifying images (dog/cat).
Regression
Regression is used when the output is a continuous variable, rather than a discrete category. Regression predicts numerical values based on input features and is commonly used for tasks where the goal is to predict trends, relationships or quantities.
Common regression algorithms:
- Linear regression: The simplest form of regression, where the relationship between input features and the output is modelled as a straight line. It’s used to predict continuous values based on linear relationships.
- Ridge and Lasso regression: These are variations of linear regression that include regularisation techniques (L2 and L1 penalties, respectively) to prevent overfitting and improve model generalisation.
- Decision trees for regression: Decision trees can also be used for regression tasks, predicting a continuous value by averaging the output values of data points that fall into each leaf node.
- Random forests for regression: Like in classification, random forests improve regression models by combining multiple decision trees to reduce variance and avoid overfitting.
- Support Vector Regression (SVR): A variant of SVM, SVR tries to find a hyperplane that best fits the data while minimising error within a certain threshold.
When should I use regression?
Regression models are appropriate when you are predicting continuous outcomes, such as forecasting sales revenue, predicting stock prices or estimating the age of an individual based on their features.
Clustering
Clustering is a type of unsupervised learning, where the goal is to group data points that are similar to each other into clusters. Unlike classification and regression, clustering doesn’t require labelled data and is often used to uncover hidden patterns in large datasets.
Common clustering algorithms:
- K-Means clustering: One of the most popular clustering algorithms, K-Means partitions data into K distinct clusters based on feature similarity. It’s efficient and works well for large datasets but can be sensitive to the initial choice of centroids.
- Hierarchical clustering: This algorithm creates a tree-like structure (dendrogram) to represent clusters. It’s useful for identifying hierarchical relationships between data points, such as grouping documents by topic.
- DBSCAN (Density-Based Spatial Clustering of Applications with Noise): DBSCAN groups together data points that are close to each other based on a distance metric, and it can handle clusters of arbitrary shapes and identify outliers.
- Gaussian Mixture Models (GMM): GMM assumes that data points are generated from a mixture of several Gaussian distributions and tries to estimate the parameters of these distributions to form clusters.
When should I use clustering?
Clustering is useful when you want to discover inherent groupings in data, such as segmenting customers into groups with similar interests, identifying patterns in social networks or grouping documents by topic.
Action item: Match the model to the problem
Read through the scenarios below and select the model you'd recommend.