A spam filter is the cleanest first machine learning project you can build. The data is text, the labels are binary, and the result is useful the moment it runs. In this guide you will train a working classifier with scikit-learn, measure it with the right metrics, and save it for reuse. The whole pipeline fits in about 40 lines of Python.
Quick Takeaways
- Pipeline: raw text → TF-IDF vectors (unigrams + bigrams) → Multinomial Naive Bayes or Logistic Regression.
- Dataset: the UCI SMS Spam Collection, 5,574 messages, about 13% spam. Expect 97–99% accuracy and ~0.90+ spam F1 on a held-out test set.
- Metric that matters: precision on spam. A false positive means a real message lands in the junk folder.
- Runtime: training takes under a second on a laptop CPU. No GPU, no API key.
| Model | Training Speed | Typical Accuracy | Interpretability | Best For |
|---|---|---|---|---|
| Multinomial Naive Bayes | Fastest (<1 s) | ~97–98% | High (word probabilities) | Baseline, small data |
| Logistic Regression | Fast (~1 s) | ~98–99% | High (coefficients) | Best classic accuracy |
| Fine-tuned DistilBERT (66M params) | Minutes on GPU | ~99% | Low | Obfuscated or adversarial spam |
Figures are typical results on the SMS Spam Collection with an 80/20 split. Your numbers will vary slightly by seed.
How a Spam Classifier Works
A classifier maps a message to one of two labels: spam (1) or ham (0). Models cannot read words, so you convert text to numbers first.
The classic recipe has three steps:
- Tokenize. Split text into words (and optionally word pairs).
- Vectorize. Turn each message into a sparse vector. Each dimension is one vocabulary term.
- Classify. Learn which terms push a message toward spam.
Why TF-IDF Beats Raw Word Counts
TF-IDF (term frequency × inverse document frequency) weights each term by how distinctive it is:
tfidf(t, d) = tf(t, d) × log(N / df(t))
Words like “the” appear everywhere and get a near-zero weight. Words like “winner” or “claim” appear mostly in spam and get a high weight. The result is a cleaner signal than a plain bag of words.
Why Naive Bayes Works So Well Here
Naive Bayes applies Bayes’ theorem and assumes words are independent given the class. That assumption is wrong for English. It still works, because spam detection depends on the presence of a few telltale words rather than on grammar.
P(spam | words) ∝ P(spam) × ∏ P(word | spam)
Step 1: Set Up Your Environment
You need Python 3.9+ and three libraries.
pip install scikit-learn pandas joblib
Step 2: Load and Inspect the Data
The SMS Spam Collection is a standard benchmark. It is tab-separated with no header row.
import pandas as pd
url = "https://raw.githubusercontent.com/justmarkham/pycon-2016-tutorial/master/data/sms.tsv"
df = pd.read_csv(url, sep="\t", header=None, names=["label", "message"])
print(df.shape) # (5572, 2)
print(df["label"].value_counts()) # ham ~4825, spam ~747
df["y"] = (df["label"] == "spam").astype(int)
The classes are imbalanced: about 87% ham. A model that always predicts “ham” would score 87% accuracy and catch zero spam. That is why you will not rely on accuracy alone.
Step 3: Split the Data Correctly
Use a stratified split so the train and test sets keep the same spam ratio.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
df["message"], df["y"],
test_size=0.2,
random_state=42,
stratify=df["y"],
)
Fit the vectorizer on the training set only. If you fit it on all the data, test vocabulary leaks into training and your scores come out too high. A scikit-learn Pipeline prevents this mistake automatically.
Step 4: Build and Train the Pipeline
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
nb_model = Pipeline([
("tfidf", TfidfVectorizer(
lowercase=True,
stop_words="english",
ngram_range=(1, 2), # unigrams + bigrams
min_df=2, # drop terms seen in <2 messages
sublinear_tf=True, # use 1 + log(tf) to damp repeats
)),
("clf", MultinomialNB(alpha=0.1)), # alpha = Laplace smoothing
])
nb_model.fit(X_train, y_train)
Key Hyperparameters
| Parameter | Value | What It Does |
|---|---|---|
ngram_range |
(1, 2) | Captures phrases like “free entry” and “call now” |
min_df |
2 | Removes one-off typos and noise |
sublinear_tf |
True | Stops repeated words from dominating |
alpha |
0.1 | Smoothing for unseen words. Lower is less forgiving |
stop_words |
“english” | Removes about 318 common words |
Step 5: Evaluate With the Right Metrics
from sklearn.metrics import classification_report, confusion_matrix
preds = nb_model.predict(X_test)
print(classification_report(y_test, preds, target_names=["ham", "spam"]))
print(confusion_matrix(y_test, preds))
Read the output in this order:
- Precision (spam): of everything flagged as spam, how much really was? Low precision means legitimate messages get blocked.
- Recall (spam): of all real spam, how much did you catch? Low recall means junk reaches the inbox.
- F1: the harmonic mean of the two.
- Confusion matrix: the raw counts of false positives and false negatives.
For email, precision comes first. Losing a job offer to the spam folder costs more than seeing one extra ad.
Step 6: Try a Stronger Model
Swap in Logistic Regression and compare. Setting class_weight="balanced" makes the model pay more attention to the rare spam class.
from sklearn.linear_model import LogisticRegression
lr_model = Pipeline([
("tfidf", TfidfVectorizer(
stop_words="english", ngram_range=(1, 2),
min_df=2, sublinear_tf=True,
)),
("clf", LogisticRegression(
max_iter=1000, C=10, class_weight="balanced",
)),
])
lr_model.fit(X_train, y_train)
print(classification_report(y_test, lr_model.predict(X_test),
target_names=["ham", "spam"]))
Naive Bayes vs. Logistic Regression
| Factor | Naive Bayes | Logistic Regression |
|---|---|---|
| Assumption | Features independent | Linear decision boundary |
| Handles correlated words | Poorly | Well |
| Probability calibration | Overconfident | Better calibrated |
| Tuning needed | alpha only |
C, class_weight |
| Typical spam F1 | ~0.91 | ~0.93 |
Step 7: Inspect What the Model Learned
A spam filter you cannot explain is hard to trust. Pull out the strongest spam signals from the logistic regression weights.
import numpy as np
vec = lr_model.named_steps["tfidf"]
clf = lr_model.named_steps["clf"]
terms = np.array(vec.get_feature_names_out())
top_spam = terms[np.argsort(clf.coef_[0])[-15:]][::-1]
print(top_spam) # expect: 'txt', 'free', 'claim', 'prize', 'call now' ...
If you see strange terms, such as a sender name that only appears in spam, you have found dataset bias. Fix it before shipping.
Step 8: Save the Model and Predict on New Messages
import joblib
joblib.dump(lr_model, "spam_classifier.joblib")
# Later, in any script:
model = joblib.load("spam_classifier.joblib")
def classify(text: str, threshold: float = 0.5) -> dict:
prob = model.predict_proba([text])[0, 1]
return {"label": "spam" if prob >= threshold else "ham",
"spam_probability": round(float(prob), 3)}
print(classify("WINNER! Claim your free prize now, text WIN to 80085"))
print(classify("Are we still on for lunch at noon?"))
The threshold is your control knob. Raise it to 0.8 for fewer false positives. Lower it to 0.3 to catch more spam.
Real-World Workflows and Use Cases
1. Tuning the Threshold for a Business Goal
| Scenario | Threshold | Trade-off |
|---|---|---|
| Personal inbox | 0.5–0.6 | Balanced |
| Customer support inbox | 0.8–0.9 | Never block a real customer |
| Comment moderation | 0.3–0.4 | Catch more junk, review a queue |
2. Moving Beyond SMS to Email
SMS is short and informal. Real email adds headers, HTML, and attachments. Adapt the pipeline in three ways:
- Strip HTML with
BeautifulSoupbefore vectorizing. - Add features: link count, ALL-CAPS ratio, sender domain, and
Reply-Tomismatches. - Retrain on email data such as the Enron-Spam or SpamAssassin corpora. SMS-trained models transfer poorly.
Combine text and numeric features with ColumnTransformer.
3. Handling Concept Drift
Spammers adapt. A model trained today decays over months. Run this loop:
- Log every prediction and a sample of user corrections (“Report spam” clicks).
- Retrain monthly on the fresh labels.
- Track spam precision and recall on a rolling window. Alert if either drops more than 3 points.
4. When to Use a Transformer Instead
Choose DistilBERT or a similar model when spammers use obfuscation (“fr3e m0ney”), mixed languages, or paraphrased phishing. Expect higher accuracy with higher cost: you need a GPU for training and 10–50× slower inference than TF-IDF models. For most first projects, the classical pipeline is enough.
Common Mistakes to Avoid
- Judging by accuracy alone. With 87% ham, accuracy hides failures. Check precision and recall.
- Fitting the vectorizer before splitting. This causes data leakage. Use a
Pipeline. - Skipping cross-validation. Run
cross_val_score(model, X, y, cv=5, scoring="f1")to confirm your score is stable. - Ignoring duplicates. The dataset contains repeated messages. Duplicates across train and test inflate your results. Run
df.drop_duplicates()for a stricter test. - Deploying without a threshold review. The default 0.5 rarely fits your cost of errors.
FAQ
What is the best algorithm for a spam email classifier?
Multinomial Naive Bayes is the best starting point because it is fast and strong on small text datasets. Logistic Regression with TF-IDF usually adds 1–2 points of F1. Transformers like DistilBERT help only on obfuscated or adversarial spam.
How much data do I need to train a spam classifier?
A few thousand labeled messages gives solid results. The SMS Spam Collection has 5,574 messages and reaches about 98% accuracy. For production email filtering, use tens of thousands of labeled examples, and keep adding new ones.
How accurate is a Naive Bayes spam filter?
On the SMS Spam Collection, Naive Bayes typically scores 97–98% accuracy and a spam F1 of about 0.90–0.93. Accuracy alone overstates quality on imbalanced data, so check precision and recall as well.
Do I need deep learning to detect spam?
No. TF-IDF with a linear model or Naive Bayes handles most spam. Deep learning pays off when spam is heavily obfuscated, multilingual, or crafted to evade keyword filters. It costs far more compute.



