Using ChatGPT, Claude, and Gemini for Data Analysis: A Practical Guide

A large language model will happily compute a mean, write a groupby, and explain a p-value. It will also, if you let it, invent a column that does not exist. The difference between a reliable AI-assisted analysis and a confident wrong one is workflow design: what you send, what you ask for, and how you verify the output. This guide covers how ChatGPT, Claude, and Gemini fit into a real analytics process, where each tool tends to work best, and how to keep every number auditable.

Quick Takeaways

• Use LLMs as analysts’ assistants, not oracles. They accelerate EDA, code generation, SQL drafting, and interpretation. They do not replace validation.
• Send schema and summaries, not raw sensitive data. A compact data brief gives better answers and reduces privacy risk.
• Prefer tool-backed execution. Analyses run in a code sandbox are checkable. Numbers produced by “mental math” are not.
• Verify every claim. Recompute headline figures in your own environment before they reach a dashboard or a stakeholder.

Feature names and availability change often. Check each vendor’s current documentation before committing to a workflow.

How LLMs Fit Into the Data Analysis Lifecycle

An analysis has five stages. LLMs add the most value in some and the least in others.

Modeling and statistical inference carry the highest risk. A model can suggest a t-test on data that violates its assumptions, or recommend accuracy on a severely imbalanced target. You supply the judgment.

Step 1: Build a Data Brief Before You Prompt

Pasting a full CSV into a chat wastes tokens and can expose personal data. Send a structured brief instead. It tells the model the schema, types, missingness, and value ranges.

import pandas as pd

def build_data_brief(df: pd.DataFrame, max_cats: int = 5) -> str:
    """Create a compact, privacy-conscious summary of a DataFrame for LLM prompts."""
    lines = [f"Rows: {len(df):,} | Columns: {df.shape[1]}", ""]

    for col in df.columns:
        s = df[col]
        null_pct = s.isna().mean() * 100  # share of missing values
        base = f"- {col} ({s.dtype}), nulls: {null_pct:.1f}%, unique: {s.nunique():,}"

        if pd.api.types.is_numeric_dtype(s):
            # Numeric columns: range and central tendency only
            base += (f", min: {s.min():.3g}, median: {s.median():.3g}, "
                     f"max: {s.max():.3g}")
        else:
            # Categorical columns: top categories only, never full values
            top = s.value_counts().head(max_cats).index.tolist()
            base += f", top values: {top}"
        lines.append(base)

    return "\n".join(lines)

df = pd.read_csv("customers.csv")
print(build_data_brief(df))

Drop or mask identifiers (emails, names, account numbers) before you generate the brief. For regulated data, follow your organization’s policy on approved tools and data-retention settings.

Step 2: Write Prompts That Produce Auditable Work

Vague prompts produce vague analysis. A strong prompt states the objective, the data context, the constraints, and the output format.

ROLE: You are a senior data analyst.
OBJECTIVE: Identify the top 3 drivers of customer churn.
DATA BRIEF: [paste output of build_data_brief]
CONSTRAINTS:
- Use pandas and scikit-learn only.
- Do not assume columns that are not listed in the brief.
- State every statistical assumption you rely on.
OUTPUT:
1. A numbered analysis plan.
2. Complete, runnable Python code.
3. A list of checks I should run to validate the result.

Three habits improve results across all three assistants:

1. Ask for the plan first. Review the approach before any code is written.
2. Forbid assumptions explicitly. “Do not assume columns that are not listed” cuts hallucinated fields sharply.
3. Request validation steps. Models are good at listing how their own output could be wrong.

Step 3: Use Code Execution, Not Mental Arithmetic

When a model computes a result inside a code sandbox, you can inspect the script. When it estimates a result in prose, you cannot. For anything numeric, ask the model to run code and show it.

Use the table below to choose an execution path.

For datasets beyond a sandbox’s file limits, have the model write PySpark or SQL, and execute it in your own warehouse or cluster.

Step 4: Validate Every Headline Number

Treat LLM output as a pull request: review it. A lightweight verification harness catches most errors.

import pandas as pd
import numpy as np

def verify_claim(df: pd.DataFrame, group_col: str, value_col: str,
                 claimed: dict, tol: float = 0.01) -> pd.DataFrame:
    """Recompute grouped means and compare against values claimed by an LLM.

    claimed: {group_name: claimed_mean}
    tol: relative tolerance (1% by default)
    """
    actual = df.groupby(group_col)[value_col].mean()  # ground truth
    rows = []
    for group, claimed_val in claimed.items():
        actual_val = actual.get(group, np.nan)
        # np.isclose uses relative tolerance to flag meaningful mismatches
        ok = np.isclose(actual_val, claimed_val, rtol=tol)
        rows.append({"group": group, "claimed": claimed_val,
                     "actual": round(actual_val, 4), "match": ok})
    return pd.DataFrame(rows)

report = verify_claim(
    df, group_col="plan_type", value_col="monthly_spend",
    claimed={"basic": 42.10, "pro": 87.55}
)
print(report)

Add three standing checks to every analysis:

• Row-count reconciliation. Confirm counts before and after each filter or join.
• Duplicate-key detection. Many-to-many joins inflate totals silently.
• Leakage review. Check that no feature encodes the target (e.g., a “cancellation_date” column in a churn model).

Algorithm and Method Selection: Where LLMs Help and Where They Mislead

Models explain trade-offs well when you ask directly. Use that to pressure-test a choice.

That last row catches many beginners. A random split on temporal data leaks future information into training.

Practical Example: Predicting Customer Churn with Python

A subscription business wants to know who will cancel next month. The prompt from Step 2 yields code like the following. Review it line by line before trusting it.

import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import roc_auc_score

df = pd.read_csv("customers.csv")
y = df["churned"]                      # binary target
X = df.drop(columns=["churned", "customer_id"])  # drop target and identifier

num_cols = X.select_dtypes(include="number").columns
cat_cols = X.select_dtypes(exclude="number").columns

# Separate preprocessing per type; fit on training folds only to prevent leakage
preprocess = ColumnTransformer([
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")),
                      ("scale", StandardScaler())]), num_cols),
    ("cat", Pipeline([("impute", SimpleImputer(strategy="most_frequent")),
                      ("ohe", OneHotEncoder(handle_unknown="ignore"))]), cat_cols),
])

model = Pipeline([("prep", preprocess),
                  ("clf", GradientBoostingClassifier(random_state=42))])

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42  # stratify preserves class ratio
)

# 5-fold cross-validated AUC-ROC on the training set
cv_auc = cross_val_score(model, X_train, y_train, cv=5, scoring="roc_auc")
print(f"CV AUC-ROC: {cv_auc.mean():.3f} ± {cv_auc.std():.3f}")

model.fit(X_train, y_train)
test_auc = roc_auc_score(y_test, model.predict_proba(X_test)[:, 1])
print(f"Holdout AUC-ROC: {test_auc:.3f}")

Review points: the Pipeline keeps imputation and scaling inside cross-validation, stratify=y protects the class ratio, and the customer_id drop prevents the model from memorizing identifiers. If the cross-validated and holdout scores diverge widely, investigate leakage or overfitting.

More Real-World Use Cases

SQL generation for a revenue dashboard. Paste your table schemas and ask for a query computing monthly recurring revenue by cohort. Then test it against a month you already know the answer for.

Cleaning messy survey exports. Describe the columns and ask for a pandas cleaning script that standardizes categories, parses dates, and flags impossible values. Run it on a sample and diff the output.

Missing data in sensor streams. Ask the model to compare forward-fill, interpolation, and model-based imputation for a given sampling rate. Require it to state how each choice biases downstream aggregates.

Explaining a model to stakeholders. Feed it SHAP summary values (not raw data) and ask for a plain-language narrative. Check that every claim maps to a plotted value.

Choosing Between ChatGPT, Claude, and Gemini

No single assistant wins on every task, and rankings shift with each model release. Pick by workflow fit, then test on your own data.

Run a small bake-off. Take one real analysis task, give all three the same data brief and prompt, and score them on correctness, code quality, assumption handling, and verbosity.

Limitations to Plan Around

• Hallucinated columns and functions. Mitigate with explicit schema briefs and by running code immediately.
• Statistical misapplication. Models may select tests without checking assumptions. Ask them to justify each choice.
• Non-determinism. The same prompt can return different code. Save the final script, not the chat.
• Scale limits. Sandboxes cap file size and memory. Move heavy workloads to your own compute.
• Privacy. Anonymize inputs and follow your organization’s data-handling policy.

Frequently Asked Questions

Can ChatGPT, Claude, or Gemini replace a data analyst?

No. They speed up coding, EDA, and explanation, but they lack business context and accountability. Analysts still frame questions, validate results, and own the conclusions.

Which AI is best for data analysis: ChatGPT, Claude, or Gemini?

It depends on your workflow. ChatGPT and Claude both offer strong code-based analysis, and Gemini integrates tightly with Google Workspace and BigQuery. Test each on a representative task from your own data, because rankings change with every release.

Is it safe to upload company data to an AI chatbot?

Only under your organization’s approved terms. Remove personal identifiers, share summaries instead of raw records where possible, and review each vendor’s data-retention and training settings.

How do I stop an LLM from making up numbers?

Require code execution for every calculation, forbid assumed columns in your prompt, and recompute headline figures yourself. Treat any number without a traceable script as unverified.