Causal Inference for Data Scientists: Beyond Correlation

Correlation predicts. Causation explains what happens when you intervene. A churn model can flag customers who will leave, but only a causal model tells you whether a discount will keep them. This guide covers the potential outcomes framework, DAGs, and four estimators, with working Python code using DoWhy, EconML, and scikit-learn.

Quick Takeaways

• Causal inference estimates the effect of an intervention, written as E[Y | do(T=1)] − E[Y | do(T=0)], not the observed association.
• Confounders (variables affecting both treatment and outcome) cause most naive estimates to fail. Control for them with adjustment, matching, weighting, or doubly robust estimators.
• Randomized experiments are the gold standard. When they are impossible, use observational methods such as propensity scores, DiD, and instrumental variables.
• Every observational estimate rests on untestable assumptions. Run refutation tests and sensitivity analysis before you ship a result.

Why Correlation Fails: Confounding and Simpson’s Paradox

Ice cream sales correlate with drowning deaths. Temperature drives both. The same structure appears in business data: customers who use a premium feature churn less, but engaged customers are both more likely to adopt the feature and less likely to churn.

Simpson’s paradox is the extreme case. A trend in aggregated data reverses once you stratify by a confounder. A model trained on the aggregate learns the wrong sign for the treatment effect.

Core Concepts

The Potential Outcomes Framework

Each unit i has two potential outcomes: Y_i(1) under treatment and Y_i(0) under control. You observe only one. This is the fundamental problem of causal inference.

Key estimands:

• ATE (Average Treatment Effect): E[Y(1) − Y(0)]
• ATT (Average Treatment Effect on the Treated): E[Y(1) − Y(0) | T=1]
• CATE (Conditional ATE): E[Y(1) − Y(0) | X=x], the basis of personalization and uplift modeling

Identification Assumptions

Causal Graphs (DAGs)

A directed acyclic graph encodes your assumptions. Three structures matter:

• Confounder (T ← Z → Y): adjust for Z.
• Mediator (T → M → Y): do not adjust if you want the total effect.
• Collider (T → C ← Y): do not adjust. Conditioning on C opens a spurious path.

Many analysts “control for everything.” That is a mistake. Adjusting for colliders and mediators introduces bias. Use the backdoor criterion to select the adjustment set.

Step 1: Simulate Data with a Known Ground Truth

Simulation lets you verify that an estimator recovers the truth. The true ATE here is 2.0.

import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
n = 20_000

# Confounders
age = rng.normal(40, 10, n)
income = rng.normal(60, 15, n)

# Treatment depends on confounders (selection bias)
logit = -0.05 * (age - 40) + 0.04 * (income - 60)
p_treat = 1 / (1 + np.exp(-logit))
treatment = rng.binomial(1, p_treat)

# Outcome depends on confounders AND treatment (true ATE = 2.0)
y = 0.3 * age + 0.1 * income + 2.0 * treatment + rng.normal(0, 3, n)

df = pd.DataFrame({"age": age, "income": income, "T": treatment, "Y": y})

# Naive estimate: biased by confounding
naive = df.loc[df["T"] == 1, "Y"].mean() - df.loc[df["T"] == 0, "Y"].mean()
print(f"Naive difference in means: {naive:.2f}")  # Not 2.0

The naive difference deviates from 2.0 because treated units are systematically different from controls.

Step 2: Four Estimators Compared

Regression Adjustment

import statsmodels.formula.api as smf

# Include confounders as covariates; the coefficient on T is the ATE estimate
ols = smf.ols("Y ~ T + age + income", data=df).fit(cov_type="HC3")
print(ols.params["T"], ols.conf_int().loc["T"].values)

Propensity Score Weighting (IPW)

The propensity score is e(x) = P(T=1 | X=x). Weighting by its inverse creates a pseudo-population where treatment is independent of X.

from sklearn.linear_model import LogisticRegression

X = df[["age", "income"]]
ps_model = LogisticRegression(max_iter=1000).fit(X, df["T"])
e = ps_model.predict_proba(X)[:, 1]

# Trim extreme scores to stabilize variance and enforce overlap
e = np.clip(e, 0.01, 0.99)

# Horvitz-Thompson style IPW estimator for ATE
w1 = df["T"] / e
w0 = (1 - df["T"]) / (1 - e)
ate_ipw = np.mean(w1 * df["Y"]) - np.mean(w0 * df["Y"])
print(f"IPW ATE: {ate_ipw:.2f}")

Doubly Robust Estimation (AIPW)

AIPW combines an outcome model and a propensity model. It stays consistent if either is correctly specified.

from sklearn.ensemble import GradientBoostingRegressor

# Outcome models fit separately per treatment arm
m1 = GradientBoostingRegressor().fit(X[df["T"] == 1], df.loc[df["T"] == 1, "Y"])
m0 = GradientBoostingRegressor().fit(X[df["T"] == 0], df.loc[df["T"] == 0, "Y"])
mu1, mu0 = m1.predict(X), m0.predict(X)

T, Y = df["T"].values, df["Y"].values
psi = (mu1 - mu0
       + T * (Y - mu1) / e
       - (1 - T) * (Y - mu0) / (1 - e))
print(f"AIPW ATE: {psi.mean():.2f} ± {1.96 * psi.std() / np.sqrt(n):.2f}")

In production, fit nuisance models with cross-fitting to avoid overfitting bias. This is what Double Machine Learning automates.

Double Machine Learning with EconML

from econml.dml import LinearDML
from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier

est = LinearDML(
    model_y=RandomForestRegressor(n_estimators=200, min_samples_leaf=20),
    model_t=RandomForestClassifier(n_estimators=200, min_samples_leaf=20),
    discrete_treatment=True,
    cv=5,  # cross-fitting folds
    random_state=0,
)
est.fit(df["Y"], df["T"], X=None, W=df[["age", "income"]])
print(est.ate_inference())

Step 3: Model the Causal Graph with DoWhy

DoWhy enforces a four-step workflow: model, identify, estimate, refute.

from dowhy import CausalModel

model = CausalModel(
    data=df,
    treatment="T",
    outcome="Y",
    common_causes=["age", "income"],
)

# Identify the estimand via the backdoor criterion
estimand = model.identify_effect(proceed_when_unidentifiable=True)

# Estimate with propensity score stratification
estimate = model.estimate_effect(
    estimand,
    method_name="backdoor.propensity_score_stratification",
)
print(estimate.value)

# Refute: a placebo treatment should yield an effect near zero
placebo = model.refute_estimate(
    estimand, estimate, method_name="placebo_treatment_refuter"
)
print(placebo)

# Refute: adding a random confounder should not move the estimate
random_cc = model.refute_estimate(
    estimand, estimate, method_name="random_common_cause"
)
print(random_cc)

Diagnostics Checklist

• Overlap: plot propensity score distributions by arm. Thin tails signal positivity violations.
• Covariate balance: compute standardized mean differences after weighting. Target |SMD| < 0.1.
• Placebo tests: estimate effects on outcomes the treatment cannot affect.
• Sensitivity analysis: quantify how strong an unmeasured confounder must be to erase the result (E-values, Rosenbaum bounds).

When You Cannot Adjust for Everything: Quasi-Experimental Methods

Unmeasured confounding breaks every method above. These designs exploit structure in how treatment was assigned.

A minimal DiD in regression form:

# panel columns: unit, post (0/1), treated (0/1), Y
did = smf.ols("Y ~ treated * post", data=panel).fit(
    cov_type="cluster", cov_kwds={"groups": panel["unit"]}
)
# The interaction coefficient is the DiD estimate of the ATT
print(did.params["treated:post"])

Always cluster standard errors at the unit level. Check parallel trends with an event-study plot of pre-period coefficients.

Heterogeneous Effects and Uplift Modeling

Average effects hide who benefits. CATE estimation targets individual-level lift, which lets you treat only users who respond.

from econml.metalearners import TLearner
from sklearn.ensemble import GradientBoostingRegressor

# T-learner: separate outcome models per arm; CATE = mu1(x) - mu0(x)
tl = TLearner(models=GradientBoostingRegressor())
tl.fit(df["Y"], df["T"], X=df[["age", "income"]])
cate = tl.effect(df[["age", "income"]])

Evaluate CATE models with the Qini curve or AUUC on held-out randomized data. Standard metrics like RMSE do not apply, because the true individual effect is never observed.

Real-World Use Cases

Marketing: Does the Discount Reduce Churn?

Customers who receive retention offers are usually those the model flagged as high risk. A naive comparison makes discounts look harmful. Use AIPW or DML with pre-treatment features (tenure, usage, plan type). Then fit a CATE model to find the “persuadables” and exclude “sure things” and “lost causes.”

Product: Measuring a Feature with Staggered Rollout

A feature launches in different regions in different weeks. Use DiD with staggered-adoption estimators (Callaway-Sant’Anna or Sun-Abraham). Standard two-way fixed effects can produce biased estimates when effects vary over time.

Pricing: A Test You Cannot Randomize

Randomizing prices may be legally or reputationally risky. Use regression discontinuity around a policy threshold, or synthetic control for a single-market price change.

A/B Test Rescue: Noncompliance

Only 60% of users assigned to treatment actually use the feature. Use the random assignment as an instrument to estimate the LATE (complier effect), rather than comparing users and non-users.

Common Pitfalls

• Post-treatment bias: controlling for variables measured after treatment can block part of the effect.
• Collider bias: conditioning on a common effect creates fake associations.
• Positivity violations: extrapolating into regions with no treated (or no control) units.
• Overfitting nuisance models: use cross-fitting with flexible learners.
• Treating feature importance as causal: SHAP values explain model predictions, not intervention effects.
• Ignoring uncertainty: report confidence intervals and bootstrap where analytic variance is unavailable.

Workflow Summary

Frequently Asked Questions

What is the difference between correlation and causation in data science?

Correlation measures statistical association in observed data. Causation measures how an outcome changes when you intervene on a variable. Confounders can produce strong correlation with zero causal effect.

Can machine learning models estimate causal effects?

Yes, but not by default. Predictive models optimize accuracy, not unbiased effect estimation. Methods like Double Machine Learning, causal forests, and meta-learners use ML for nuisance functions while preserving valid causal identification.

How do I do causal inference without an A/B test?

Use observational methods whose assumptions fit your data: propensity score weighting or AIPW when you can measure all confounders, difference-in-differences with panel data, instrumental variables with a valid instrument, or regression discontinuity with a cutoff rule.

Which Python libraries are best for causal inference?

DoWhy provides the model-identify-estimate-refute workflow. EconML offers DML, DR-Learner, and causal forests. CausalML specializes in uplift modeling. statsmodels covers regression, IV, and DiD baselines.