A house price model needs three things: clean tabular data, a baseline you can explain, and an error metric you trust. Linear regression delivers all three in under 50 lines of Python. This guide builds a complete price predictor on the California Housing dataset (20,640 rows, 8 features). It covers the math, preprocessing, training, diagnostics, regularization, and saving the model for reuse.
Quick Takeaways
- Linear regression fits a weighted sum of features to price by minimizing MSE. It trains in milliseconds and every coefficient is interpretable.
- A plain
LinearRegressionbaseline on this dataset scores R² ≈ 0.58 and RMSE ≈ 0.75 (about $75,000) on a held-out 20% test split. - Log-transforms, outlier clipping, and Ridge regularization improve the baseline modestly. Large gains need non-linear models.
- Always report RMSE, MAE, and R² on unseen data, never on training data.
| Model | Library Class | Training Time | Interpretability | Typical R² (this dataset) |
|---|---|---|---|---|
| Linear Regression (OLS) | LinearRegression |
< 50 ms | Very high | ~0.58 |
| Ridge (L2) | Ridge(alpha=1.0) |
< 50 ms | Very high | ~0.58 |
| Lasso (L1) | Lasso(alpha=0.001) |
< 200 ms | High (sparse) | ~0.58 |
| Gradient Boosting | HistGradientBoostingRegressor |
1-3 s | Low | ~0.83 |
R² values are approximate and vary with split and preprocessing. Run the code below to reproduce them.
How Linear Regression Predicts a House Price
The model assumes price is a weighted sum of features plus a bias term:
ŷ = β₀ + β₁x₁ + β₂x₂ + … + βₙxₙ
Each βᵢ is a coefficient. If MedInc (median income, in $10,000s) has a coefficient of 0.44, then one extra unit of income raises predicted price by 0.44 × $100,000 = $44,000, holding all other features constant.
Training finds the coefficients that minimize Mean Squared Error (MSE):
MSE = (1/n) Σ (yᵢ − ŷᵢ)²
Ordinary Least Squares (OLS) solves this in closed form with the normal equation:
β = (XᵀX)⁻¹ Xᵀy
Scikit-learn uses a numerically stable SVD-based solver instead of inverting the matrix directly. That keeps it reliable when features are correlated.
The Five OLS Assumptions
| Assumption | What It Means | How to Check |
|---|---|---|
| Linearity | Price relates linearly to features | Residuals vs. fitted plot |
| Independence | Rows don’t influence each other | Data collection review |
| Homoscedasticity | Error variance is constant | Residual spread plot |
| Normal residuals | Errors are roughly Gaussian | Q-Q plot |
| No multicollinearity | Features aren’t near-duplicates | Variance Inflation Factor (VIF) |
Real housing data violates several of these. That is fine for a baseline, as long as you know which assumptions break and why.
Project Setup
Install the stack. Python 3.10+ is recommended.
pip install numpy pandas scikit-learn matplotlib joblib
The dataset has these columns:
| Feature | Meaning | Unit |
|---|---|---|
MedInc |
Median household income | $10,000s |
HouseAge |
Median house age | Years |
AveRooms |
Average rooms per household | Count |
AveBedrms |
Average bedrooms per household | Count |
Population |
Block group population | People |
AveOccup |
Average occupants per household | Count |
Latitude / Longitude |
Block group location | Degrees |
MedHouseVal (target) |
Median house value | $100,000s |
One caveat: the target is capped at 5.0 ($500,000). Many expensive blocks pile up at that ceiling, which limits how well any linear model can fit the top of the market.
Step 1: Load and Inspect the Data
import numpy as np
import pandas as pd
from sklearn.datasets import fetch_california_housing
data = fetch_california_housing(as_frame=True)
df = data.frame
print(df.shape) # (20640, 9)
print(df.isna().sum().sum()) # 0 missing values
print(df.describe().T[["mean", "std", "min", "max"]])
Check three things before modeling:
- Missing values. This dataset has none. Real listings data usually does, so use
SimpleImputer(strategy="median"). - Skew.
Population,AveRooms, andAveOccuphave extreme right tails. - Outliers.
AveOccupreaches over 1,000 in a few rows, which is a data artifact, not a real household.
Step 2: Build the Baseline Model
Split first. Compute every statistic from the training set only, to avoid data leakage.
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
X = df.drop(columns="MedHouseVal")
y = df["MedHouseVal"]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
baseline = LinearRegression().fit(X_train, y_train)
pred = baseline.predict(X_test)
rmse = np.sqrt(mean_squared_error(y_test, pred))
print(f"MAE : {mean_absolute_error(y_test, pred):.3f}") # ~0.533
print(f"RMSE: {rmse:.3f}") # ~0.746
print(f"R² : {r2_score(y_test, pred):.3f}") # ~0.576
An RMSE of 0.75 means typical predictions miss by roughly $75,000. An R² of 0.58 means the model explains about 58% of price variance.
Read the Coefficients
coefs = pd.Series(baseline.coef_, index=X.columns).sort_values()
print(coefs)
Expect MedInc to carry a strong positive weight. Latitude and Longitude carry large negative weights. They capture geography linearly, which is a poor fit for a map with coastal clusters.
Step 3: Improve the Features
Linear models only learn straight-line relationships. You supply the curvature through feature engineering.
def engineer(X: pd.DataFrame, caps: dict) -> pd.DataFrame:
X = X.copy()
for col, cap in caps.items(): # clip outliers using TRAIN quantiles
X[col] = X[col].clip(upper=cap)
X["Population"] = np.log1p(X["Population"])
X["BedroomRatio"] = X["AveBedrms"] / X["AveRooms"]
return X
caps = {c: X_train[c].quantile(0.99) for c in ["AveRooms", "AveBedrms", "AveOccup"]}
X_train_fe = engineer(X_train, caps)
X_test_fe = engineer(X_test, caps)
| Transformation | Target Problem | Why It Helps |
|---|---|---|
| Clip at 99th percentile | Extreme AveOccup values |
Stops single rows from dragging coefficients |
log1p(Population) |
Right-skewed counts | Makes the relationship closer to linear |
| Bedroom ratio | Raw counts are collinear | Encodes layout quality in one number |
Step 4: Scale and Regularize With a Pipeline
OLS doesn’t need scaling. Ridge and Lasso do, because their penalties treat all coefficients equally. A Pipeline keeps scaling inside cross-validation folds.
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge, Lasso
from sklearn.model_selection import cross_val_score
models = {
"OLS": make_pipeline(StandardScaler(), LinearRegression()),
"Ridge": make_pipeline(StandardScaler(), Ridge(alpha=1.0)),
"Lasso": make_pipeline(StandardScaler(), Lasso(alpha=0.001)),
}
for name, m in models.items():
scores = cross_val_score(
m, X_train_fe, y_train, cv=5, scoring="neg_root_mean_squared_error"
)
print(f"{name:6s} CV RMSE: {-scores.mean():.3f} ± {scores.std():.3f}")
Ridge adds an L2 penalty (α · Σβ²) that shrinks coefficients and stabilizes correlated features. Lasso adds an L1 penalty (α · Σ|β|) that can drive weak coefficients to exactly zero, which gives you feature selection for free.
Tune alpha automatically:
from sklearn.linear_model import RidgeCV
ridge = make_pipeline(
StandardScaler(),
RidgeCV(alphas=np.logspace(-3, 3, 13), cv=5),
).fit(X_train_fe, y_train)
print(ridge[-1].alpha_) # selected regularization strength
On this dataset, expect only small gains from regularization. The ceiling is the model family, not overfitting.
Step 5: Diagnose the Residuals
A single R² number hides structure. Plot the errors.
import matplotlib.pyplot as plt
final = models["Ridge"].fit(X_train_fe, y_train)
pred = final.predict(X_test_fe)
resid = y_test - pred
fig, ax = plt.subplots(1, 2, figsize=(11, 4))
ax[0].scatter(pred, resid, s=4, alpha=0.3)
ax[0].axhline(0, color="red")
ax[0].set(xlabel="Predicted", ylabel="Residual", title="Residuals vs. Fitted")
ax[1].hist(resid, bins=60)
ax[1].set(title="Residual Distribution")
plt.tight_layout(); plt.show()
| Pattern in Plot | Diagnosis | Fix |
|---|---|---|
| Funnel shape (spread grows with price) | Heteroscedasticity | Model log(price) instead |
| Diagonal band at high prices | Target cap at $500K | Drop capped rows or use a censored model |
| Curved residual trend | Non-linearity | Add polynomial or interaction terms |
| Clusters by region | Missing geography | Add location bins or k-means cluster IDs |
To fix the funnel, train on np.log(y) and invert predictions with np.exp. This also stops the model from predicting negative prices.
Step 6: Save and Reuse the Model
import joblib
joblib.dump({"model": final, "caps": caps}, "house_price_model.joblib")
bundle = joblib.load("house_price_model.joblib")
new_home = pd.DataFrame([{
"MedInc": 6.5, "HouseAge": 20, "AveRooms": 6.0, "AveBedrms": 1.0,
"Population": 1500, "AveOccup": 3.0, "Latitude": 37.8, "Longitude": -122.3,
}])
row = engineer(new_home, bundle["caps"])
price = bundle["model"].predict(row)[0] * 100_000
print(f"Estimated value: ${price:,.0f}")
Store the clipping caps next to the model. Without them, inference preprocessing won’t match training.
Evaluation Metrics Compared
| Metric | Formula Idea | Units | Best For |
|---|---|---|---|
| MAE | Mean absolute error | $ | Typical miss, robust to outliers |
| RMSE | √MSE | $ | Penalizing large misses |
| R² | 1 − SS_res / SS_tot | Unitless | Comparing against a mean-only predictor |
| MAPE | Mean % error | % | Explaining error to non-technical teams |
Report RMSE and MAE together. A large gap between them signals a few very bad predictions.
Real-World Use Cases
Pricing Sanity Check for Listings
Real estate platforms use linear models as a transparent first-pass estimate. An agent can see that “+1 bedroom ≈ +$X” and challenge it. That auditability is why regulated lenders still keep linear models in the stack.
Tax Assessment Baselines
Assessors fit linear models per neighborhood to flag parcels whose assessed value deviates by more than 2 standard deviations from the prediction.
Benchmark for Complex Models
Teams train gradient boosting or neural nets, then compare them to the linear baseline. If the complex model beats it by only 2%, it doesn’t earn its deployment cost.
Feature Discovery
Lasso’s zeroed-out coefficients reveal which of 50+ candidate features (school ratings, crime indices, transit distance) carry signal.
When Linear Regression Falls Short
| Limitation | Symptom | Better Option |
|---|---|---|
| Spatial effects | Residuals cluster by region | Gradient boosting or geographic features |
| Interactions (income × location) | R² plateaus near 0.6 | Tree ensembles |
| Capped or censored targets | Bias at the top of the range | Tobit or quantile regression |
| Heavy outliers | Unstable coefficients | Huber regression (HuberRegressor) |
For a quick upgrade path, swap in HistGradientBoostingRegressor using the same split. It typically reaches R² ≈ 0.83 on this dataset, at the cost of coefficient-level interpretability.
Strategic FAQ
Is linear regression good for predicting house prices?
Yes, as a baseline. It trains instantly, explains every prediction, and typically reaches R² ≈ 0.55-0.65 on raw housing features. Tree-based models usually outperform it when location and interaction effects dominate.
What is a good R² score for a house price model?
It depends on the market and the features. On California Housing, 0.58 is a typical linear baseline and 0.80+ is typical for boosted trees. Compare R² against a baseline, and always check RMSE in dollars.
Should I scale features for linear regression?
Plain OLS doesn’t need scaling because coefficients adjust automatically. Ridge and Lasso do, since their penalties compare coefficient sizes directly. Use StandardScaler inside a Pipeline to avoid leakage.
How do I stop negative price predictions?
Train on log(price) and apply np.exp() to predictions. Exponentiated outputs are always positive. This also reduces heteroscedasticity in the residuals.



