Binning numerical features

<p><b>Binning</b> (discretisation) converts a continuous feature into ordered categories. It is useful when a model cannot handle real-valued inputs directly or when you wish to build features such as an “income decile”.</p>

Equal-width vs. equal-frequency #

Let (x_1, \dots, x_n) be a feature. A binning rule partitions the range into intervals (I_k) and replaces each (x_i) with the label of the interval that contains it.

• Equal-width binning divides the range into intervals of the same length.
• Equal-frequency (quantile) binning divides the sorted data so that each bin has approximately the same number of observations (pandas.qcut).

Equal-frequency bins are more robust to heavy tails, whereas equal-width bins preserve the notion of distance.

Visualising quantile bins #

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

rng = np.random.default_rng(18)
income = rng.lognormal(mean=10, sigma=0.55, size=1_000)
quantile_edges = np.quantile(income, np.linspace(0, 1, 6))

plt.hist(income, bins=40, color="steelblue", alpha=0.85)
for edge in quantile_edges[1:-1]:
    plt.axvline(edge, color="darkorange", linestyle="--", alpha=0.8)
plt.title("Quantile bin edges (equal-frequency, 5 bins)")
plt.xlabel("Income")
plt.ylabel("Count")
plt.show()

Comparing qcut and cut #

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

quantile_bins = pd.qcut(income, q=5)
width_bins = pd.cut(income, bins=5)

quantile_counts = pd.Series(quantile_bins).value_counts(sort=False)
width_counts = pd.Series(width_bins).value_counts(sort=False)
indices = np.arange(len(quantile_counts))

fig, ax = plt.subplots(figsize=(7, 4))
ax.bar(indices - 0.2, quantile_counts.values, width=0.4, color="seagreen", alpha=0.8, label="qcut (equal-frequency)")
ax.bar(indices + 0.2, width_counts.values, width=0.4, color="firebrick", alpha=0.6, label="cut (equal-width)")
ax.set_xticks(indices)
ax.set_xticklabels([f"Bin {i}" for i in indices])
ax.set_ylabel("Number of samples")
ax.legend()
ax.set_title("Distribution of samples per bin")
plt.tight_layout()
plt.show()

Equal-frequency binning produces almost identical counts for each bin, while equal-width binning assigns many observations to the dense centre and few to the extremes.

Practical tips #

• Clip extreme outliers before binning; even a single extreme value can stretch the range and break equal-width bins.
• Store the edges produced during training and reuse them later to ensure identical bin definitions.
• Tree-based models rarely need explicit binning. For linear models, however, binning can capture non-linear effects while keeping the feature space compact.