MAE & RMSE

1. Definitions and properties #

For observations \(y_i\) and predictions \(\hat{y}_i\):

• MAE averages absolute errors. It is more robust to outliers and corresponds to the median in a Laplace error model.
• RMSE squares errors before averaging, then takes the square root. Large deviations are penalised heavily.
• Both retain the original units of the target; RMSE emphasises large mistakes, MAE emphasises typical error size.

2. Computing in Python #

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

import numpy as np
from sklearn.metrics import mean_absolute_error, mean_squared_error

y_true = np.array([10, 12, 9, 11, 10])
y_pred = np.array([9.5, 13.0, 8.0, 11.5, 9.0])

mae = mean_absolute_error(y_true, y_pred)
rmse = mean_squared_error(y_true, y_pred, squared=False)

print(f"MAE = {mae:.3f}")
print(f"RMSE = {rmse:.3f}")

Setting squared=False returns RMSE instead of MSE.

3. Effect of outliers #

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import numpy as np

rng = np.random.default_rng(42)
baseline = np.linspace(100, 110, 20)
prediction = baseline + rng.normal(0, 1.0, size=baseline.size)

mae_clean = np.mean(np.abs(baseline - prediction))
rmse_clean = np.sqrt(np.mean((baseline - prediction) ** 2))

prediction_with_outlier = prediction.copy()
prediction_with_outlier[0] += 15

mae_outlier = np.mean(np.abs(baseline - prediction_with_outlier))
rmse_outlier = np.sqrt(np.mean((baseline - prediction_with_outlier) ** 2))

print(f"MAE (clean, outlier) = {mae_clean:.2f}, {mae_outlier:.2f}")
print(f"RMSE (clean, outlier) = {rmse_clean:.2f}, {rmse_outlier:.2f}")

• MAE grows slowly when we introduce an outlier.
• RMSE increases sharply, highlighting large individual errors.

4. Choosing between MAE and RMSE #

• Few outliers and precision matters → choose RMSE to highlight subtle deviations.
• Many outliers or heavy-tailed noise → MAE (or median absolute deviation) stays stable.
• Costs scale quadratically → RMSE mirrors the business objective (e.g., energy loss, physical deviations).
• Need straightforward communication → MAE answers “on average we miss by ±X units”.

5. Related metrics #

• MAPE: percentage error; intuitive for business users but unstable near zero.
• RMSLE: RMSE on the log scale; punishes underestimation in growth/volume forecasts.
• Pinball loss: evaluates prediction intervals/quantiles for risk-sensitive forecasts.

Summary #

• MAE and RMSE complement each other; one is robust, the other sensitive to large errors.
• Report both to understand error distribution and pick the metric that matches your cost function.
• Combine with MAPE, RMSLE, or quantile losses to gain a richer view of model performance.