Pinball loss

Eval

Pinball loss

Created: Last updated: Read time: 2 min
まとめ
  • Pinball loss measures how far quantile predictions deviate above or below the desired quantile.
  • Compute the loss for a quantile regression model in Python and observe the asymmetric weighting.
  • Explore applications in demand forecasting and risk management, plus tips for setting quantile targets.

1. Definition #

For quantile \(\tau\):

$$ L_\tau(y, \hat{y}) = \begin{cases} \tau (y - \hat{y}) & \text{if } y \ge \hat{y} \ (1 - \tau)(\hat{y} - y) & \text{otherwise} \end{cases} $$

Predictions below the target quantile incur \(\tau\) times the shortfall; predictions above incur \(1-\tau\) times the surplus.

2. Computing in Python #

from sklearn.metrics import mean_pinball_loss

quantile = 0.9
loss = mean_pinball_loss(y_true, y_pred_quantile, alpha=quantile)
print(f"Pinball Loss (q={quantile}): {loss:.4f}")

Set alpha to the desired quantile. Pair this with a quantile regression model (e.g., GradientBoostingRegressor with loss="quantile").

3. Interpretation #

  • Lower is better: perfect quantile predictions yield zero loss.
  • \(\tau = 0.5\) reduces to MAE.
  • Asymmetry: gain control over the penalty for over- vs under-prediction by tweaking \(\tau\).

4. Practical applications #

  • Demand intervals: evaluate 90th-percentile forecasts to plan safety stock.
  • Risk management: Value at Risk (VaR) and Expected Shortfall rely on quantile evaluation.
  • Energy/load forecasting: train separate upper/lower quantile models and assess them with pinball loss.

5. Tips #

  • Training a separate model per quantile can be expensive; consider libraries that support multi-quantile training (LightGBM, CatBoost).
  • Extreme quantiles (close to 0 or 1) need many samples to remain stable.
  • Combine pinball loss with interval metrics such as PICP to evaluate both accuracy and coverage.

Summary #

  • Pinball loss is the core metric for quantile regression, enabling asymmetric error control.
  • mean_pinball_loss makes evaluation straightforward; choose \(\tau\) to match your risk appetite.
  • For interval forecasts, pair pinball loss with coverage metrics to obtain a complete view of performance.