Gradient Boosting Explained: Why It Still Wins on Tabular Data

Deep learning dominates images, text, and audio. Tables are different. On rows-and-columns data, a well-tuned ensemble of shallow decision trees still beats most neural networks on accuracy, training time, and engineering overhead. This guide explains how gradient boosting works, why it keeps winning, and how to run XGBoost, LightGBM, and CatBoost with production-ready settings.

Quick Takeaways

• Gradient boosting builds trees sequentially. Each new tree fits the residual errors (negative gradients of the loss) of the ensemble so far.
• On datasets of roughly 1K to 1M rows with mixed numeric and categorical features, boosted trees match or beat deep models in most published benchmarks.
• LightGBM is typically fastest, CatBoost handles categoricals best out of the box, and XGBoost is the most battle-tested.
• Start with learning_rate = 0.03–0.1, max_depth = 4–8, and early stopping. Tune subsampling before anything exotic.

How Gradient Boosting Works

Boosting combines many weak learners into one strong predictor. Gradient boosting frames this as gradient descent in function space.

The core algorithm

1. Initialize the model with a constant: F₀(x) = argmin_γ Σ L(yᵢ, γ).
2. For each round m = 1…M:
   • Compute pseudo-residuals: rᵢₘ = −∂L(yᵢ, F(xᵢ)) / ∂F(xᵢ).
   • Fit a small regression tree hₘ(x) to those residuals.
   • Update: Fₘ(x) = Fₘ₋₁(x) + η · hₘ(x), where η is the learning rate.
3. Return F_M(x).

For squared error, the pseudo-residual is simply y − F(x). For log loss, it is y − p. The same machinery handles ranking, quantile, Poisson, and custom objectives, as long as the loss is differentiable.

Why trees, and why shallow ones

Weak learners are typically trees with depth 3–8. Shallow trees have high bias and low variance. Boosting then reduces the bias round by round. Deep trees overfit early and waste the sequential error-correction mechanism.

XGBoost’s second-order trick

XGBoost uses both the gradient g and the Hessian h of the loss. The optimal leaf weight is:

w* = −Σg / (Σh + λ)

The λ term is L2 regularization on leaf weights. A split’s gain also subtracts a complexity penalty γ, so weak splits get pruned automatically.

Why Boosted Trees Still Win on Tabular Data

Research such as Grinsztajn et al. (2022), Why do tree-based models still outperform deep learning on typical tabular data?, identified three structural reasons. Later benchmarks, including TabArena-style evaluations, have narrowed the gap on some datasets, but trees remain the default baseline to beat.

1. Uninformative features. Trees ignore irrelevant columns through split selection. MLPs are rotation-invariant and get diluted by noise features.
2. Non-smooth target functions. Tabular targets often contain sharp thresholds (age ≥ 65, balance > 0). Trees model step functions natively. Neural nets favor smooth functions.
3. Heterogeneous features. Columns with different units, scales, and distributions need no normalization for trees. Each feature is split independently.

Practical advantages add up:

• No feature scaling. Missing values are handled natively by learned default directions.
• Training on CPU in seconds to minutes.
• Built-in feature importance and SHAP support via TreeExplainer.
• Strong performance with limited data (under 100K rows), where neural networks overfit.

Where deep learning catches up

Pretrained tabular foundation models (such as TabPFN-style approaches) now compete on small datasets (under ~10K rows) with no tuning. Neural nets also win when data is very large, multimodal, or needs end-to-end embeddings. Treat them as a complement. Always benchmark a boosted-tree baseline first.

Library Comparison: XGBoost vs. LightGBM vs. CatBoost

Rule of thumb: pick CatBoost for categorical-heavy data and fast iteration, LightGBM for large datasets and tight latency budgets, XGBoost for maximum ecosystem compatibility.

Hands-On Implementation

Install

pip install xgboost lightgbm catboost scikit-learn shap

XGBoost with early stopping

import xgboost as xgb
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.metrics import root_mean_squared_error

X, y = fetch_california_housing(return_X_y=True, as_frame=True)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

model = xgb.XGBRegressor(
    n_estimators=2000,          # upper bound; early stopping picks the real number
    learning_rate=0.05,         # eta: smaller = more rounds, better generalization
    max_depth=6,
    subsample=0.8,              # row sampling per tree
    colsample_bytree=0.8,       # feature sampling per tree
    reg_lambda=1.0,             # L2 on leaf weights
    tree_method="hist",
    early_stopping_rounds=50,
    random_state=42,
)
model.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=100)

preds = model.predict(X_val)
print("RMSE:", root_mean_squared_error(y_val, preds))
print("Best iteration:", model.best_iteration)

LightGBM with categorical features

import lightgbm as lgb

train_set = lgb.Dataset(X_train, y_train, categorical_feature="auto")
val_set = lgb.Dataset(X_val, y_val, reference=train_set)

params = {
    "objective": "regression",
    "metric": "rmse",
    "learning_rate": 0.05,
    "num_leaves": 31,           # keep below 2^max_depth to limit overfitting
    "min_data_in_leaf": 40,     # raise on small or noisy data
    "feature_fraction": 0.8,
    "bagging_fraction": 0.8,
    "bagging_freq": 1,
    "lambda_l2": 1.0,
    "verbose": -1,
}

booster = lgb.train(
    params, train_set, num_boost_round=3000,
    valid_sets=[val_set],
    callbacks=[lgb.early_stopping(50), lgb.log_evaluation(100)],
)

CatBoost with raw categorical columns

from catboost import CatBoostClassifier

cat_cols = ["city", "device_type", "plan"]   # string columns, no encoding needed

clf = CatBoostClassifier(
    iterations=3000,
    learning_rate=0.05,
    depth=6,
    l2_leaf_reg=3,
    eval_metric="AUC",
    early_stopping_rounds=100,
    random_seed=42,
    verbose=200,
)
clf.fit(X_train, y_train, cat_features=cat_cols, eval_set=(X_val, y_val))

Explain predictions with SHAP

import shap

explainer = shap.TreeExplainer(model)
shap_values = explainer(X_val)
shap.plots.beeswarm(shap_values)   # global feature impact
shap.plots.waterfall(shap_values[0])  # single prediction

Hyperparameter Tuning Playbook

Tune in this order. Each step gives the largest gains per unit of effort.

Use Optuna with 50–200 trials and 5-fold cross-validation. Always fix random_state and stratify classification splits.

import optuna

def objective(trial):
    params = {
        "learning_rate": trial.suggest_float("learning_rate", 0.01, 0.2, log=True),
        "max_depth": trial.suggest_int("max_depth", 3, 9),
        "min_child_weight": trial.suggest_int("min_child_weight", 1, 50),
        "subsample": trial.suggest_float("subsample", 0.5, 1.0),
        "colsample_bytree": trial.suggest_float("colsample_bytree", 0.5, 1.0),
        "reg_lambda": trial.suggest_float("reg_lambda", 1e-2, 10, log=True),
    }
    m = xgb.XGBRegressor(n_estimators=2000, early_stopping_rounds=50,
                         tree_method="hist", **params)
    m.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=False)
    return root_mean_squared_error(y_val, m.predict(X_val))

study = optuna.create_study(direction="minimize")
study.optimize(objective, n_trials=100)
print(study.best_params)

Real-World Use Cases and Workflows

Fraud detection

Class imbalance is severe (often under 0.5% positives). Set scale_pos_weight to roughly negatives/positives, optimize PR-AUC instead of accuracy, and calibrate probabilities with isotonic regression before thresholding.

Credit risk scoring

Apply monotonic constraints (monotone_constraints in XGBoost and LightGBM) so that higher income never raises predicted default risk. Regulators and risk teams require this kind of explainability.

Demand forecasting

Convert time series to tabular form with lag features (t−1, t−7, t−28), rolling means, and calendar variables. Use time-based splits only. Random splits leak the future.

Churn prediction

Combine behavioral aggregates (30/60/90-day activity) with categorical plan data. CatBoost’s ordered target statistics reduce leakage on high-cardinality columns such as customer_segment.

A production workflow template

1. Build a baseline with default LightGBM (10 minutes).
2. Add feature engineering: aggregates, ratios, target-safe encodings.
3. Run Optuna with cross-validation.
4. Check SHAP for leakage (one dominant suspicious feature is a red flag).
5. Blend 2–3 boosters (XGBoost + LightGBM + CatBoost) for a typical 0.5–2% metric lift.
6. Export via model.save_model() or ONNX for serving.

Common Mistakes

• Tuning without early stopping. You will over- or under-train.
• Target leakage. Features computed using future or label-derived information inflate validation scores.
• Ignoring distribution shift. Re-validate on a recent holdout, not a random split.
• One-hot encoding high-cardinality columns. This explodes sparsity. Use native categorical support instead.
• Trusting default feature importance. Gain importance is biased toward high-cardinality features. Prefer SHAP or permutation importance.

FAQ

What is gradient boosting in simple terms?

Gradient boosting trains small decision trees one after another. Each tree corrects the mistakes of the combined previous trees by fitting the gradient of the loss function. The final prediction is the sum of all tree outputs, scaled by a learning rate.

Is XGBoost better than a neural network for tabular data?

Usually, yes, for datasets from about 1K to 1M rows. Boosted trees handle mixed feature types, missing values, and sharp thresholds without preprocessing. Neural networks can win on very large datasets, multimodal inputs, or small data when a pretrained tabular model is available.

XGBoost vs. LightGBM vs. CatBoost: which should I choose?

Choose LightGBM for speed on large data, CatBoost for categorical-heavy data with minimal tuning, and XGBoost for the broadest tooling and distributed support. Differences in accuracy are usually small, so benchmark all three with the same cross-validation folds.

How do I stop gradient boosting from overfitting?

Lower the learning rate, enable early stopping, limit max_depth or num_leaves, raise min_child_weight, and add subsampling (0.7–0.9) plus L2 regularization.