Predict House Prices With Linear Regression: A Complete Project

A house price model needs three things: clean tabular data, a baseline you can explain, and an error metric you trust. Linear regression delivers all three in under 50 lines of Python. This guide builds a complete price predictor on the California Housing dataset (20,640 rows, 8 features). It covers the math, preprocessing, training, diagnostics, regularization, and saving the model for reuse.

Quick Takeaways

  • Linear regression fits a weighted sum of features to price by minimizing MSE. It trains in milliseconds and every coefficient is interpretable.
  • A plain LinearRegression baseline on this dataset scores R² ≈ 0.58 and RMSE ≈ 0.75 (about $75,000) on a held-out 20% test split.
  • Log-transforms, outlier clipping, and Ridge regularization improve the baseline modestly. Large gains need non-linear models.
  • Always report RMSE, MAE, and R² on unseen data, never on training data.
Model Library Class Training Time Interpretability Typical R² (this dataset)
Linear Regression (OLS) LinearRegression < 50 ms Very high ~0.58
Ridge (L2) Ridge(alpha=1.0) < 50 ms Very high ~0.58
Lasso (L1) Lasso(alpha=0.001) < 200 ms High (sparse) ~0.58
Gradient Boosting HistGradientBoostingRegressor 1-3 s Low ~0.83

R² values are approximate and vary with split and preprocessing. Run the code below to reproduce them.

How Linear Regression Predicts a House Price

The model assumes price is a weighted sum of features plus a bias term:

ŷ = β₀ + β₁x₁ + β₂x₂ + … + βₙxₙ

Each βᵢ is a coefficient. If MedInc (median income, in $10,000s) has a coefficient of 0.44, then one extra unit of income raises predicted price by 0.44 × $100,000 = $44,000, holding all other features constant.

Training finds the coefficients that minimize Mean Squared Error (MSE):

MSE = (1/n) Σ (yᵢ − ŷᵢ)²

Ordinary Least Squares (OLS) solves this in closed form with the normal equation:

β = (XᵀX)⁻¹ Xᵀy

Scikit-learn uses a numerically stable SVD-based solver instead of inverting the matrix directly. That keeps it reliable when features are correlated.

The Five OLS Assumptions

Assumption What It Means How to Check
Linearity Price relates linearly to features Residuals vs. fitted plot
Independence Rows don’t influence each other Data collection review
Homoscedasticity Error variance is constant Residual spread plot
Normal residuals Errors are roughly Gaussian Q-Q plot
No multicollinearity Features aren’t near-duplicates Variance Inflation Factor (VIF)

Real housing data violates several of these. That is fine for a baseline, as long as you know which assumptions break and why.

Project Setup

Install the stack. Python 3.10+ is recommended.

pip install numpy pandas scikit-learn matplotlib joblib

The dataset has these columns:

Feature Meaning Unit
MedInc Median household income $10,000s
HouseAge Median house age Years
AveRooms Average rooms per household Count
AveBedrms Average bedrooms per household Count
Population Block group population People
AveOccup Average occupants per household Count
Latitude / Longitude Block group location Degrees
MedHouseVal (target) Median house value $100,000s

One caveat: the target is capped at 5.0 ($500,000). Many expensive blocks pile up at that ceiling, which limits how well any linear model can fit the top of the market.

Step 1: Load and Inspect the Data

import numpy as np
import pandas as pd
from sklearn.datasets import fetch_california_housing

data = fetch_california_housing(as_frame=True)
df = data.frame

print(df.shape)            # (20640, 9)
print(df.isna().sum().sum())  # 0 missing values
print(df.describe().T[["mean", "std", "min", "max"]])

Check three things before modeling:

  1. Missing values. This dataset has none. Real listings data usually does, so use SimpleImputer(strategy="median").
  2. Skew. Population, AveRooms, and AveOccup have extreme right tails.
  3. Outliers. AveOccup reaches over 1,000 in a few rows, which is a data artifact, not a real household.

Step 2: Build the Baseline Model

Split first. Compute every statistic from the training set only, to avoid data leakage.

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

X = df.drop(columns="MedHouseVal")
y = df["MedHouseVal"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

baseline = LinearRegression().fit(X_train, y_train)
pred = baseline.predict(X_test)

rmse = np.sqrt(mean_squared_error(y_test, pred))
print(f"MAE : {mean_absolute_error(y_test, pred):.3f}")  # ~0.533
print(f"RMSE: {rmse:.3f}")                               # ~0.746
print(f"R²  : {r2_score(y_test, pred):.3f}")             # ~0.576

An RMSE of 0.75 means typical predictions miss by roughly $75,000. An R² of 0.58 means the model explains about 58% of price variance.

Read the Coefficients

coefs = pd.Series(baseline.coef_, index=X.columns).sort_values()
print(coefs)

Expect MedInc to carry a strong positive weight. Latitude and Longitude carry large negative weights. They capture geography linearly, which is a poor fit for a map with coastal clusters.

Step 3: Improve the Features

Linear models only learn straight-line relationships. You supply the curvature through feature engineering.

def engineer(X: pd.DataFrame, caps: dict) -> pd.DataFrame:
    X = X.copy()
    for col, cap in caps.items():          # clip outliers using TRAIN quantiles
        X[col] = X[col].clip(upper=cap)
    X["Population"] = np.log1p(X["Population"])
    X["BedroomRatio"] = X["AveBedrms"] / X["AveRooms"]
    return X

caps = {c: X_train[c].quantile(0.99) for c in ["AveRooms", "AveBedrms", "AveOccup"]}

X_train_fe = engineer(X_train, caps)
X_test_fe = engineer(X_test, caps)
Transformation Target Problem Why It Helps
Clip at 99th percentile Extreme AveOccup values Stops single rows from dragging coefficients
log1p(Population) Right-skewed counts Makes the relationship closer to linear
Bedroom ratio Raw counts are collinear Encodes layout quality in one number

Step 4: Scale and Regularize With a Pipeline

OLS doesn’t need scaling. Ridge and Lasso do, because their penalties treat all coefficients equally. A Pipeline keeps scaling inside cross-validation folds.

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge, Lasso
from sklearn.model_selection import cross_val_score

models = {
    "OLS":   make_pipeline(StandardScaler(), LinearRegression()),
    "Ridge": make_pipeline(StandardScaler(), Ridge(alpha=1.0)),
    "Lasso": make_pipeline(StandardScaler(), Lasso(alpha=0.001)),
}

for name, m in models.items():
    scores = cross_val_score(
        m, X_train_fe, y_train, cv=5, scoring="neg_root_mean_squared_error"
    )
    print(f"{name:6s} CV RMSE: {-scores.mean():.3f} ± {scores.std():.3f}")

Ridge adds an L2 penalty (α · Σβ²) that shrinks coefficients and stabilizes correlated features. Lasso adds an L1 penalty (α · Σ|β|) that can drive weak coefficients to exactly zero, which gives you feature selection for free.

Tune alpha automatically:

from sklearn.linear_model import RidgeCV

ridge = make_pipeline(
    StandardScaler(),
    RidgeCV(alphas=np.logspace(-3, 3, 13), cv=5),
).fit(X_train_fe, y_train)

print(ridge[-1].alpha_)   # selected regularization strength

On this dataset, expect only small gains from regularization. The ceiling is the model family, not overfitting.

Step 5: Diagnose the Residuals

A single R² number hides structure. Plot the errors.

import matplotlib.pyplot as plt

final = models["Ridge"].fit(X_train_fe, y_train)
pred = final.predict(X_test_fe)
resid = y_test - pred

fig, ax = plt.subplots(1, 2, figsize=(11, 4))
ax[0].scatter(pred, resid, s=4, alpha=0.3)
ax[0].axhline(0, color="red")
ax[0].set(xlabel="Predicted", ylabel="Residual", title="Residuals vs. Fitted")
ax[1].hist(resid, bins=60)
ax[1].set(title="Residual Distribution")
plt.tight_layout(); plt.show()
Pattern in Plot Diagnosis Fix
Funnel shape (spread grows with price) Heteroscedasticity Model log(price) instead
Diagonal band at high prices Target cap at $500K Drop capped rows or use a censored model
Curved residual trend Non-linearity Add polynomial or interaction terms
Clusters by region Missing geography Add location bins or k-means cluster IDs

To fix the funnel, train on np.log(y) and invert predictions with np.exp. This also stops the model from predicting negative prices.

Step 6: Save and Reuse the Model

import joblib

joblib.dump({"model": final, "caps": caps}, "house_price_model.joblib")

bundle = joblib.load("house_price_model.joblib")
new_home = pd.DataFrame([{
    "MedInc": 6.5, "HouseAge": 20, "AveRooms": 6.0, "AveBedrms": 1.0,
    "Population": 1500, "AveOccup": 3.0, "Latitude": 37.8, "Longitude": -122.3,
}])

row = engineer(new_home, bundle["caps"])
price = bundle["model"].predict(row)[0] * 100_000
print(f"Estimated value: ${price:,.0f}")

Store the clipping caps next to the model. Without them, inference preprocessing won’t match training.

Evaluation Metrics Compared

Metric Formula Idea Units Best For
MAE Mean absolute error $ Typical miss, robust to outliers
RMSE √MSE $ Penalizing large misses
R² 1 − SS_res / SS_tot Unitless Comparing against a mean-only predictor
MAPE Mean % error % Explaining error to non-technical teams

Report RMSE and MAE together. A large gap between them signals a few very bad predictions.

Real-World Use Cases

Pricing Sanity Check for Listings

Real estate platforms use linear models as a transparent first-pass estimate. An agent can see that “+1 bedroom ≈ +$X” and challenge it. That auditability is why regulated lenders still keep linear models in the stack.

Tax Assessment Baselines

Assessors fit linear models per neighborhood to flag parcels whose assessed value deviates by more than 2 standard deviations from the prediction.

Benchmark for Complex Models

Teams train gradient boosting or neural nets, then compare them to the linear baseline. If the complex model beats it by only 2%, it doesn’t earn its deployment cost.

Feature Discovery

Lasso’s zeroed-out coefficients reveal which of 50+ candidate features (school ratings, crime indices, transit distance) carry signal.

When Linear Regression Falls Short

Limitation Symptom Better Option
Spatial effects Residuals cluster by region Gradient boosting or geographic features
Interactions (income × location) R² plateaus near 0.6 Tree ensembles
Capped or censored targets Bias at the top of the range Tobit or quantile regression
Heavy outliers Unstable coefficients Huber regression (HuberRegressor)

For a quick upgrade path, swap in HistGradientBoostingRegressor using the same split. It typically reaches R² ≈ 0.83 on this dataset, at the cost of coefficient-level interpretability.

Strategic FAQ

Is linear regression good for predicting house prices?

Yes, as a baseline. It trains instantly, explains every prediction, and typically reaches R² ≈ 0.55-0.65 on raw housing features. Tree-based models usually outperform it when location and interaction effects dominate.

What is a good R² score for a house price model?

It depends on the market and the features. On California Housing, 0.58 is a typical linear baseline and 0.80+ is typical for boosted trees. Compare R² against a baseline, and always check RMSE in dollars.

Should I scale features for linear regression?

Plain OLS doesn’t need scaling because coefficients adjust automatically. Ridge and Lasso do, since their penalties compare coefficient sizes directly. Use StandardScaler inside a Pipeline to avoid leakage.

How do I stop negative price predictions?

Train on log(price) and apply np.exp() to predictions. Exponentiated outputs are always positive. This also reduces heteroscedasticity in the residuals.

Hot this week

AI and Copyright: The Lawsuits, the Rulings, and What They Mean for Creators

Track every major AI copyright lawsuit in 2026: Anthropic's $1.5B deal, NYT v. OpenAI, Ross, Midjourney. See what it means for your work. Read now.

Build Your First Spam Email Classifier

Build a spam email classifier in Python with TF-IDF and Naive Bayes. Copy the code, hit 98%+ accuracy, and ship your first ML model today.

Best AI Music and Voice Generators in 2026

Compare Suno v6, ElevenLabs Music, Udio, Eleven v4, Cartesia and Fish Audio on price, rights and latency. Pick your tool and start creating today.

Train/Test Splits Done Right: Best Practices and Common Mistakes

Avoid data leakage with proper train/test splits. Compare random, stratified, group and time-series methods with Python code. Fix your pipeline today.

AI Engineer vs ML Engineer vs Data Scientist: What’s the Difference?

AI engineer vs ML engineer vs data scientist: compare skills, tools, and roles side by side. Find your best-fit career path and start learning today.

Topics

AI and Copyright: The Lawsuits, the Rulings, and What They Mean for Creators

Track every major AI copyright lawsuit in 2026: Anthropic's $1.5B deal, NYT v. OpenAI, Ross, Midjourney. See what it means for your work. Read now.

Build Your First Spam Email Classifier

Build a spam email classifier in Python with TF-IDF and Naive Bayes. Copy the code, hit 98%+ accuracy, and ship your first ML model today.

Best AI Music and Voice Generators in 2026

Compare Suno v6, ElevenLabs Music, Udio, Eleven v4, Cartesia and Fish Audio on price, rights and latency. Pick your tool and start creating today.

Train/Test Splits Done Right: Best Practices and Common Mistakes

Avoid data leakage with proper train/test splits. Compare random, stratified, group and time-series methods with Python code. Fix your pipeline today.

AI Engineer vs ML Engineer vs Data Scientist: What’s the Difference?

AI engineer vs ML engineer vs data scientist: compare skills, tools, and roles side by side. Find your best-fit career path and start learning today.

Chain-of-Thought, Few-Shot, and Role Prompts: When Each Technique Helps

Chain-of-thought, few-shot, and role prompts each fix different failures. Learn when each works, when it backfires, and copy the templates.

Outliers: How to Detect and Handle Them Without Losing Signal

Learn outlier detection with Z-score, IQR, Isolation Forest, and LOF. Copy the Python code and handle outliers without losing real signal.

Hotend and Extruder Basics: How Filament Becomes Plastic Layers

Learn how a hotend and extruder turn filament into layers. Compare direct vs Bowden, nozzles, and temps, then tune yours. Read the full guide.

Related Articles

Popular Categories