Skip to main content

Defining statistical bias

Instruction and application
In Progress

Defining statistical bias

Ever tried to solve a complex problem with an oversimplified solution—only to find it misses the point entirely? That’s what happens when your machine learning model suffers from high bias.

In machine learning, bias is a type of systematic error—a consistent, repeatable mistake that occurs when a model’s assumptions are too simple to capture the underlying structure of the data. This typically results in ** underfitting** , where the model performs poorly even on training data because it fails to learn the true patterns.

Bias often arises when:

Defining statistical bias illustration

Why this matters​

High bias limits a model’s ability to learn from data and adapt to complexity. It doesn’t just affect performance metrics—it can lead to:

  • Inaccurate business forecasts, pricing, or customer targeting.
  • Missed signals in healthcare, safety, or fraud detection.
  • Unfair outcomes when critical data patterns are ignored—impacting individuals unequally based on gender, geography, or other factors. Understanding and detecting high bias is the first step to building models that learn effectively, generalise reliably, and support fair and informed decision-making.

Example: Bias and underfitting in action​

Let’s say you’re building a model to ** predict house prices** in a mid-sized city.

You collect a dataset with features like:

  • Square footage.

  • Number of bedrooms and bathrooms.

  • Neighborhood.

  • Distance to schools or transit.

  • Year built. To keep things simple, you choose a linear regression model and feed it only one feature: square footage. The model is:

  • Fast to train.

  • Easy to interpret.

  • Computationally efficient.** But it performs poorly. Why?**

Because it makes ** strong simplifying assumptions**: that price increases at a constant rate with size, and that no other features matter.** Here’s what happens:**

  • In a ** high-demand neighborhood** , a small house may sell for far more than a larger one in a less popular area—but the model underestimates that.
  • In the ** suburbs** , a large but outdated home far from amenities may be overpriced—because the model sees only square footage. This is ** underfitting caused by high bias** . The model is too simplistic to capture real-world patterns. Even with more data, it keeps making the same types of mistakes—a hallmark of** systematic error** .

Below is a chart showing ** underfitting due to high bias** in the housing price prediction scenario:

  • The ** grey dots** represent actual house prices, which follow a** nonlinear pattern** influenced by various real-world factors.
  • The ** red line** shows a** simple linear regression model** that only considers square footage, resulting in** systematic error**—underestimating in some areas, overestimating in others. This illustrates how ** high bias leads to underfitting** , failing to capture the complexity in the data.

Action item: Identifying bias and underfitting quiz​

Let's see if you can apply what you've learned by answering the questions below.