Product Analytics 101: Metrics, Funnels, and Retention

Product analytics answers three questions: what users do, where they drop off, and whether they come back. Metrics quantify behavior, funnels expose friction between steps, and retention shows whether your product creates lasting value. This guide covers all three with Pandas implementations you can run on any event-level dataset.

Quick Takeaways

• Metrics fall into acquisition, activation, engagement, and revenue groups. Pick one North Star metric and a small set of input metrics that drive it.
• Funnels measure step-to-step conversion rate across an ordered sequence of events. The largest relative drop is your first optimization target.
• Retention tracks the share of a cohort that returns over time. A flattening curve signals product-market fit; a curve that decays to zero signals churn.
• All three derive from one table: an event log with user_id, event_name, and timestamp.

Step 1: Build the Event Data Foundation

Every analysis starts with a clean event schema. Use one row per user action.

Enforce a tracking plan before analysis. Inconsistent names (Signup, sign_up, signed_up) silently corrupt every downstream number.

import pandas as pd
import numpy as np

# Load the event log
events = pd.read_csv("events.csv", parse_dates=["timestamp"])

# Normalize event names and drop exact duplicates (common with client-side retries)
events["event_name"] = events["event_name"].str.lower().str.strip()
events = events.drop_duplicates(subset=["user_id", "event_name", "timestamp"])

# Remove internal/test traffic and null user IDs
events = events[events["user_id"].notna()]

# Derive date columns used in every later step
events["date"] = events["timestamp"].dt.date
events["week"] = events["timestamp"].dt.to_period("W").dt.start_time
events["month"] = events["timestamp"].dt.to_period("M").dt.start_time

print(events.shape, events["user_id"].nunique())

Step 2: Define and Calculate Core Product Metrics

Choose a North Star Metric

A North Star metric captures the value users receive. Examples: weekly messages sent (messaging app), nights booked (marketplace), weekly active projects (SaaS tool). Revenue is a lagging outcome, not a North Star.

The Metric Framework

Calculate DAU, WAU, MAU, and Stickiness

Stickiness is DAU divided by MAU. A value near 0.20 is typical for many SaaS products; consumer social apps often exceed 0.50.

# Define "active" as any qualifying event (exclude passive events like email_opened)
active_events = events[~events["event_name"].isin(["email_opened", "push_received"])]

dau = active_events.groupby("date")["user_id"].nunique().rename("dau")
wau = active_events.groupby("week")["user_id"].nunique().rename("wau")
mau = active_events.groupby("month")["user_id"].nunique().rename("mau")

# Rolling 30-day MAU aligned to each day (avoids calendar-month artifacts)
daily_users = active_events.groupby("date")["user_id"].apply(set)
rolling_mau = daily_users.rolling(30).apply(
    lambda s: len(set().union(*s)), raw=False
) if False else None  # see optimized version below

# Efficient rolling MAU using a user-date matrix
user_day = (
    active_events.assign(active=1)
    .drop_duplicates(["user_id", "date"])
    .pivot_table(index="date", columns="user_id", values="active", fill_value=0)
)
rolling_mau = (user_day.rolling(30, min_periods=1).max().sum(axis=1)).rename("mau_30d")

stickiness = (dau / rolling_mau).rename("stickiness")

For tables with millions of users, the pivot approach exhausts memory. Use SQL window functions or PySpark countDistinct over a 30-day range instead.

# PySpark equivalent for large event tables
from pyspark.sql import functions as F, Window

w = Window.orderBy(F.col("date").cast("timestamp").cast("long")).rangeBetween(-29 * 86400, 0)
mau_spark = (
    df.select("date", "user_id").distinct()
      .withColumn("mau_30d", F.approx_count_distinct("user_id").over(w))
)

Metric Hygiene Rules

1. Define every metric in a shared dictionary: numerator, denominator, filters, time zone.
2. Use ratios alongside absolute counts. Counts grow with traffic; ratios reveal quality.
3. Segment before concluding. Aggregates hide opposing trends (Simpson’s paradox).
4. Separate input metrics (features shipped, onboarding completion) from output metrics (revenue, retention).

Step 3: Build and Analyze Funnels

A funnel is an ordered series of events a user must complete. Step conversion equals users completing step n+1 divided by users completing step n.

Funnel Design Principles

• Choose 3 to 7 steps. Longer funnels bury insight.
• Set a conversion window (e.g., 7 days from first step). Without one, late completers inflate rates.
• Decide strict versus loose ordering. Strict funnels require events in exact sequence; loose funnels only require each event to occur after the prior one.

Funnel Implementation in Pandas

FUNNEL_STEPS = ["visit", "signup_completed", "project_created", "invite_sent", "subscription_started"]
WINDOW = pd.Timedelta(days=7)

def build_funnel(events: pd.DataFrame, steps: list, window: pd.Timedelta) -> pd.DataFrame:
    """Loose-ordered funnel with a conversion window anchored on step 1."""
    ev = events[events["event_name"].isin(steps)].sort_values(["user_id", "timestamp"])

    # First occurrence of each step per user
    first_ts = (
        ev.groupby(["user_id", "event_name"])["timestamp"].min()
          .unstack("event_name")
          .reindex(columns=steps)
    )

    # A user passes step i only if they passed step i-1, in order, inside the window
    reached = pd.DataFrame(index=first_ts.index)
    reached[steps[0]] = first_ts[steps[0]].notna()
    for i in range(1, len(steps)):
        prev, curr = steps[i - 1], steps[i]
        in_order = first_ts[curr] >= first_ts[prev]
        in_window = (first_ts[curr] - first_ts[steps[0]]) <= window
        reached[curr] = reached[prev] & in_order & in_window

    counts = reached.sum()
    out = pd.DataFrame({"users": counts})
    out["step_conversion"] = (counts / counts.shift(1)).round(3)
    out["overall_conversion"] = (counts / counts.iloc[0]).round(3)
    return out

funnel = build_funnel(events, FUNNEL_STEPS, WINDOW)
print(funnel)

Interpret the Output

Look at the largest relative drop, not the largest absolute one. A step falling from 40% to 20% (50% relative loss) deserves attention before a step falling from 90% to 80%.

Then segment the funnel by platform, acquisition channel, and plan. A mobile-only drop at signup_completed points to a form usability bug, not a messaging problem.

# Compare funnel conversion by acquisition channel
user_channel = events.drop_duplicates("user_id").set_index("user_id")["channel"]

results = {}
for ch, users in user_channel.groupby(user_channel).groups.items():
    sub = events[events["user_id"].isin(users)]
    results[ch] = build_funnel(sub, FUNNEL_STEPS, WINDOW)["overall_conversion"]

print(pd.DataFrame(results))

Test Whether a Funnel Difference Is Real

Use a two-proportion z-test before acting on a gap between segments.

from statsmodels.stats.proportion import proportions_ztest

# successes and sample sizes for two segments at the final step
count = np.array([320, 255])
nobs = np.array([4000, 4100])
stat, pval = proportions_ztest(count, nobs)
print(f"z = {stat:.2f}, p = {pval:.4f}")

Step 4: Measure Retention with Cohorts

Retention measures how many users from a starting cohort remain active over subsequent periods. It is the single strongest signal of product-market fit.

Three Retention Definitions

Pick the definition that matches natural usage frequency. Measuring day-1 retention for a tax-filing product misrepresents it.

Build a Cohort Retention Matrix

# 1. Assign each user to a cohort by their first active week
first_seen = (
    active_events.groupby("user_id")["timestamp"].min()
    .dt.to_period("W").dt.start_time.rename("cohort_week")
)

# 2. Join cohort onto activity and compute weeks since signup
activity = active_events[["user_id", "week"]].drop_duplicates().merge(first_seen, on="user_id")
activity["week_number"] = ((activity["week"] - activity["cohort_week"]).dt.days // 7)

# 3. Count unique returning users per cohort and week offset
cohort_counts = (
    activity.groupby(["cohort_week", "week_number"])["user_id"]
    .nunique().unstack(fill_value=0)
)

# 4. Convert to retention rates using week-0 cohort size
cohort_size = cohort_counts[0]
retention = cohort_counts.divide(cohort_size, axis=0).round(3)

print(retention.iloc[:, :9])

Visualize the Retention Heatmap

import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(12, 6))
sns.heatmap(retention.iloc[:, :12], annot=True, fmt=".0%", cmap="YlGnBu", vmin=0, vmax=1)
plt.title("Weekly Cohort Retention")
plt.xlabel("Weeks Since First Activity")
plt.ylabel("Cohort Week")
plt.tight_layout()
plt.show()

Read the Retention Curve

Read the matrix diagonally to spot calendar events (a bug, outage, or release) that hit all cohorts in the same week.

Compare Retention Across Behaviors

Identify the aha-moment by comparing retention for users who did an action early versus those who did not.

# Did users who created a project in week 0 retain better at week 4?
early_action = events[
    (events["event_name"] == "project_created")
].merge(first_seen, on="user_id")
early_action = early_action[
    (early_action["timestamp"] - early_action["cohort_week"]) < pd.Timedelta(days=7)
]["user_id"].unique()

week4_active = activity.loc[activity["week_number"] == 4, "user_id"].unique()
all_users = first_seen.index

df = pd.DataFrame({"user_id": all_users})
df["did_action"] = df["user_id"].isin(early_action)
df["retained_w4"] = df["user_id"].isin(week4_active)

print(df.groupby("did_action")["retained_w4"].mean())

Correlation is not causation. Users who create projects may simply be more motivated. Validate the aha-moment with an A/B test that nudges the behavior.

Step 5: Connect Metrics, Funnels, and Retention

Treat the three as one system.

A common pattern: improving the signup funnel raises new users but lowers retention, because you admitted lower-intent users. Always check retention after funnel changes.

Real-World Use Cases

Use Case 1: SaaS Onboarding Optimization

A project-management tool sees 12% overall funnel conversion from visit to paid. Segmenting reveals a steep drop between signup_completed and project_created for users from paid social. The team adds a template gallery to onboarding. Activation rises, and week-4 retention of the new cohort improves by several points, confirming the change attracted engaged users rather than casual ones.

Use Case 2: Mobile Game Retention

A game studio tracks day-1, day-7, and day-30 retention. Day-1 is healthy but day-7 collapses. Funnel analysis of in-game events shows players stall at level 5. Rebalancing that level’s difficulty shifts the day-7 retention curve upward in subsequent cohorts.

Use Case 3: E-commerce Checkout Leakage

A strict funnel (product_view → add_to_cart → checkout_started → payment_submitted → order_completed) shows the largest loss at checkout_started. Splitting by device exposes a payment-form defect on one mobile browser. The fix recovers revenue without touching acquisition spend.

Common Pitfalls

Frequently Asked Questions

What are the most important product analytics metrics?

Start with one North Star metric, then track activation rate, DAU/MAU stickiness, and cohort retention. Add ARPU or conversion to paid once usage patterns are stable. Fewer, well-defined metrics outperform dashboards with dozens of numbers.

What is a good retention rate?

It depends on category and usage frequency. Consumer apps often see day-30 retention below 10%, while B2B SaaS products target 70% or higher monthly retention for paying accounts. The shape matters more than the level: a curve that flattens means a stable core user base exists.

What is the difference between a funnel and a cohort analysis?

A funnel measures conversion through an ordered sequence of steps within a time window. A cohort analysis groups users by a shared start point and tracks their behavior over time. Funnels diagnose where users drop; cohorts show whether they come back.

How do I choose a conversion window for a funnel?

Base it on the natural decision time of your product. Impulse purchases suit 1 hour to 1 day, onboarding flows 7 days, and B2B sales cycles 30 days or more. Plot the cumulative distribution of time-to-convert and choose the point where it plateaus.