After building a regression model, one important question remains:

How good is the model?

Suppose two models predict house prices.

Model A:

Model B:

Clearly, Model A performs better.

However, computers need numerical measures to compare models objectively.

These measures are called Evaluation Metrics.

Evaluation Metrics help us:

• Measure prediction quality
• Compare different models
• Detect poor performance
• Select the best model
• Improve future predictions

In this article, we will explore the most important regression evaluation metrics used in Machine Learning and understand when to use each one.

Why Do We Need Evaluation Metrics?

Machine Learning models make predictions.

Predictions are rarely perfect.

Example:

Prediction Error:

$$100-90=10$$

A single prediction error is useful.

However, real datasets contain thousands of predictions.

We need a systematic way to summarize overall performance.

Evaluation metrics provide this summary.

Understanding Residuals

Residuals represent prediction errors.

Formula:

$Residual=y_{actual}-y_{predicted}$

Example:

Actual:

$$80$$

Predicted:

$$75$$

Residual:

$$5$$

Most regression metrics are built using residuals.

Desired Characteristics of a Good Metric

A good evaluation metric should:

• Reflect prediction quality
• Penalize mistakes appropriately
• Be easy to interpret
• Support model comparison

Mean Absolute Error (MAE)

MAE is one of the simplest regression metrics.

It calculates the average absolute prediction error.

Formula:

$MAE=\frac{1}{n}\sum |y_i-\hat y_i|$

Example Calculation

Dataset:

Errors:

MAE:

$$\frac{2+2+5}{3} = 3$$

Interpreting MAE

MAE = 3

means:

The model is off by approximately 3 units on average.

Advantages of MAE

• Easy to understand
• Same units as target variable
• Less sensitive to outliers

Disadvantages of MAE

• Does not penalize large errors strongly
• All errors are treated equally

Mean Squared Error (MSE)

MSE is one of the most commonly used regression metrics.

Instead of absolute errors, it uses squared errors.

Formula:

$MSE=\frac{1}{n}\sum(y_i-\hat y_i)^2$

Example Calculation

Errors:

$$2,2,5$$

Squared Errors:

$$4,4,25$$

MSE:

$$\frac{4+4+25}{3} = 11$$

Why Square Errors?

Squaring:

• Eliminates negative signs
• Penalizes large errors heavily

Example:

Large mistakes receive significantly larger penalties.

Advantages of MSE

• Strongly penalizes large errors
• Smooth mathematical properties
• Works well with optimization algorithms

Disadvantages of MSE

• Sensitive to outliers
• Harder to interpret due to squared units

Root Mean Squared Error (RMSE)

RMSE solves one major problem of MSE.

Formula:

$RMSE=\sqrt{MSE}$

Example

Suppose:

$$MSE=25$$

Then:

$$RMSE=5$$

Why RMSE is Popular

MAE:

Units = Original Target Units

RMSE:

Units = Original Target Units

MSE:

Units = Squared Units

RMSE is easier to interpret.

RMSE Interpretation

Example:

House Prices:

RMSE = ₹2 Lakhs

Interpretation:

Predictions are typically off by around ₹2 Lakhs.

Comparing MAE and RMSE

If large mistakes are costly:

RMSE is often preferred.

R² Score (Coefficient of Determination)

One of the most important regression metrics.

R² measures:

How much variance in the target variable is explained by the model.

Formula:

$R^2=1-\frac{SS_{res}}{SS_{tot}}$

Where:

• $SS_{res}$ = Residual Sum of Squares
• $SS_{tot}$ = Total Sum of Squares

Understanding R² Intuitively

Suppose:

House prices vary significantly.

If the model explains most of this variation:

R² becomes high.

If the model explains little:

R² becomes low.

R² Range

Typically:

$$0 \le R^2 \le 1$$

Interpretation:

Example

$$R^2=0.85$$

Interpretation:

The model explains 85% of the variability in the target variable.

Why R² is Popular

Advantages:

• Easy interpretation
• Scale-independent
• Useful for comparison

Limitation of R²

R² almost always increases when new features are added.

Even useless features can increase R² slightly.

This creates a problem.

Adjusted R²

Adjusted R² solves this issue.

Formula:

$Adjusted\ R^2=1-(1-R^2)\frac{n-1}{n-p-1}$

Where:

• $n$ = Number of observations
• $p$ = Number of features

Why Adjusted R² Matters

Adjusted R² penalizes unnecessary features.

Useful for:

• Multiple Linear Regression
• Feature selection

Example

Model A:

5 Features

Adjusted R² = 0.82

Model B:

20 Features

Adjusted R² = 0.78

Even though Model B has more features, Model A may actually be better.

Mean Absolute Percentage Error (MAPE)

MAPE expresses error as a percentage.

Formula:

$MAPE=\frac{100}{n}\sum\left|\frac{y_i-\hat y_i}{y_i}\right|$

Example

Actual:

100

Predicted:

90

Percentage Error:

$$10\%$$

Why Businesses Like MAPE

Easy interpretation.

Example:

MAPE = 5%

means:

Predictions are off by about 5% on average.

Limitation of MAPE

Problems occur when:

$$y=0$$

Division by zero becomes undefined.

Explained Variance Score

Measures how much variance is captured by the model.

Formula:

$$Explained\ Variance$$

focuses on variance rather than exact prediction error.

Values closer to:

$$1$$

are better.

Python Implementation

MAE

from sklearn.metrics import mean_absolute_error

mae = mean_absolute_error(
    y_true,
    y_pred
)

MSE

from sklearn.metrics import mean_squared_error

mse = mean_squared_error(
    y_true,
    y_pred
)

RMSE

import numpy as np

rmse = np.sqrt(
    mean_squared_error(
        y_true,
        y_pred
    )
)

R² Score

from sklearn.metrics import r2_score

r2 = r2_score(
    y_true,
    y_pred
)

Example Comparison

Suppose:

Model A:

Model B:

Model A is clearly superior.

When to Use MAE

Choose MAE when:

• Interpretability matters
• Outliers should not dominate
• Average error is important

Examples:

• Sales Forecasting
• Demand Prediction

When to Use RMSE

Choose RMSE when:

• Large errors are costly
• Outliers matter

Examples:

• Medical Predictions
• Financial Forecasting

When to Use R²

Choose R² when:

• Comparing regression models
• Understanding explanatory power

Real-World Example

House Price Prediction:

Actual:

₹50 Lakhs

Predicted:

₹48 Lakhs

Error:

₹2 Lakhs

After evaluating thousands of houses:

Interpretation:

The model performs quite well.

Common Mistakes

Using Only One Metric

No single metric tells the entire story.

Always evaluate multiple metrics.

Comparing Metrics Across Different Datasets

MAE and RMSE depend on scale.

Comparisons should be made carefully.

Ignoring Business Context

A 5% error may be acceptable in sales forecasting but unacceptable in medical applications.

Best Practices

• Report MAE and RMSE together
• Use R² for model comparison
• Use Adjusted R² for multiple regression
• Understand business requirements
• Evaluate on unseen test data

Regression Evaluation Workflow

A typical workflow is:

1. Train model
2. Generate predictions
3. Calculate residuals
4. Compute MAE
5. Compute RMSE
6. Compute R²
7. Compare models
8. Select the best-performing model

Summary of Regression Metrics

Why Evaluation Metrics Matter

Building a regression model is only half the task. The other half is determining whether the model is actually useful. Evaluation metrics provide objective ways to measure performance, compare models, and identify areas for improvement.

Understanding MAE, MSE, RMSE, R², Adjusted R², and MAPE is essential because these metrics are used in nearly every real-world regression project. They help transform model predictions into meaningful performance insights and guide the development of better Machine Learning solutions.

In the next article, we will explore Polynomial Regression, which extends Linear Regression to handle non-linear relationships that cannot be captured by a simple straight line.