Streamlit and Gradio: Ship an AI Demo in 30 Minutes

Most AI prototypes die in a notebook. A stakeholder can’t click a notebook, and a user can’t test one. Streamlit and Gradio fix that. Each turns a Python function into a shareable web app in under 50 lines. This guide builds the same streaming chatbot in both frameworks, compares them on the specs that matter, and gets you to a public URL.

Quick Takeaways

• Gradio is the faster path for a single model with typed inputs and outputs. gr.ChatInterface gives you a streaming chat UI in about 10 lines.
• Streamlit is the better fit when the demo is a multi-step app: sidebars, tabs, file uploads, session state, charts.
• Both deploy free to a public URL: Gradio on Hugging Face Spaces, Streamlit on Streamlit Community Cloud.
• Never ship an API key in the client. Use environment variables locally and platform secrets in production.

The 30-Minute Plan

Step 1: Set Up the Environment (5 Minutes)

Use Python 3.10 or newer. Isolate dependencies in a virtual environment.

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install --upgrade streamlit gradio anthropic
export ANTHROPIC_API_KEY="sk-ant-..."   # Windows: setx ANTHROPIC_API_KEY "sk-ant-..."

Pin versions before you deploy:

pip freeze | grep -iE "^(streamlit|gradio|anthropic)==" > requirements.txt

Step 2: Write the Model Layer Once

Both frameworks call the same function. Keep the LLM logic in its own file so you can swap UIs without touching it.

# llm.py
import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

MODEL = "claude-sonnet-5-5"
SYSTEM = "You are a concise technical assistant. Answer in under 150 words."

def stream_reply(messages: list[dict]):
    """messages: [{"role": "user"|"assistant", "content": str}, ...]"""
    with client.messages.stream(
        model=MODEL,
        max_tokens=1024,          # output cap per reply
        temperature=0.3,          # lower = more deterministic
        system=SYSTEM,
        messages=messages,
    ) as stream:
        for text in stream.text_stream:
            yield text

Three settings matter here:

• max_tokens=1024 caps cost and latency per turn.
• temperature=0.3 keeps demo answers stable. Raise it toward 0.8 for creative tasks.
• Streaming cuts perceived latency. The first token appears in a fraction of the time a full response takes.

Step 3: Build the Gradio Version (8 Minutes)

gr.ChatInterface handles the chat window, history, retry, and streaming. Your function receives the new message and the history. Yield the growing string to stream.

# app_gradio.py
import gradio as gr
from llm import stream_reply

def _text(content) -> str:
    # Gradio 6 can pass content as a string or a list of parts
    if isinstance(content, str):
        return content
    return "".join(p.get("text", "") for p in content if isinstance(p, dict))

def respond(message, history):
    msgs = [{"role": m["role"], "content": _text(m["content"])} for m in history]
    msgs.append({"role": "user", "content": _text(message)})
    partial = ""
    for chunk in stream_reply(msgs):
        partial += chunk
        yield partial

demo = gr.ChatInterface(
    fn=respond,
    title="AI Demo Assistant",
    description="Ask a technical question. Replies stream live.",
    examples=["Explain RAG in 3 sentences", "What is a KV cache?"],
)

if __name__ == "__main__":
    demo.launch()

Run it:

python app_gradio.py

Open http://127.0.0.1:7860. Add share=True to demo.launch() for a temporary public link that lasts about 1 week. Use it for quick stakeholder reviews, not production.

Gradio for Non-Chat Models

Chat is one pattern. For classifiers, image models, or audio, use gr.Interface with typed components:

import gradio as gr

def sentiment(text: str) -> dict:
    # Replace with a real model call
    return {"positive": 0.82, "negative": 0.18}

gr.Interface(
    fn=sentiment,
    inputs=gr.Textbox(label="Review text", lines=4),
    outputs=gr.Label(num_top_classes=2),
).launch()

Every Gradio app also exposes an API automatically. Call it from code with gradio_client, and recent releases can serve the app as an MCP tool for AI agents.

Step 4: Build the Streamlit Version (7 Minutes)

Streamlit reruns your script top to bottom on every interaction. You manage chat history yourself with st.session_state.

# app_streamlit.py
import streamlit as st
from llm import stream_reply

st.set_page_config(page_title="AI Demo Assistant", page_icon="🤖")
st.title("AI Demo Assistant")

with st.sidebar:
    st.header("Settings")
    st.caption("Model: claude-sonnet-5-5")
    if st.button("Clear chat"):
        st.session_state.messages = []
        st.rerun()

if "messages" not in st.session_state:
    st.session_state.messages = []

for m in st.session_state.messages:
    with st.chat_message(m["role"]):
        st.markdown(m["content"])

if prompt := st.chat_input("Ask a technical question"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        reply = st.write_stream(stream_reply(st.session_state.messages))
    st.session_state.messages.append({"role": "assistant", "content": reply})

Run it:

streamlit run app_streamlit.py

st.write_stream consumes any generator, renders tokens live, and returns the full string. That return value is what you store in history.

Cache Expensive Resources

If your demo loads a local model or an embedding index, cache it. Otherwise every rerun reloads it.

@st.cache_resource
def load_embedder():
    from sentence_transformers import SentenceTransformer
    return SentenceTransformer("all-MiniLM-L6-v2")   # 384-dim embeddings

Use @st.cache_resource for models and clients. Use @st.cache_data for serializable results like DataFrames.

Streamlit vs Gradio: Architecture Trade-Offs

Rule of thumb: choose Gradio when the model is the product. Choose Streamlit when the app around the model is the product.

Step 5: Deploy in 3 Minutes

Gradio on Hugging Face Spaces

1. Create a new Space and select the Gradio SDK.
2. Upload app_gradio.py renamed to app.py, plus llm.py and requirements.txt.
3. Open Settings → Variables and secrets and add ANTHROPIC_API_KEY as a secret.

The Space builds and serves on a public URL.

Streamlit on Community Cloud

1. Push app_streamlit.py, llm.py, and requirements.txt to a GitHub repo.
2. Sign in at share.streamlit.io and click New app.
3. Select the repo and entry file. Under Advanced settings → Secrets, paste:

ANTHROPIC_API_KEY = "sk-ant-..."

Streamlit exposes secrets as environment variables, so os.environ["ANTHROPIC_API_KEY"] in llm.py works unchanged. You can also read them with st.secrets["ANTHROPIC_API_KEY"].

Real-World Use Cases and Templates

1. Internal RAG Demo for Stakeholders

Add a file uploader, chunk the text, embed it, and ground answers in retrieved passages. This is the standard way teams show how retrieval reduces hallucinations.

uploaded = st.file_uploader("Upload a .txt file", type=["txt"])
if uploaded:
    text = uploaded.read().decode("utf-8")
    chunks = [text[i:i+800] for i in range(0, len(text), 700)]   # 800 chars, 100 overlap
    st.session_state.chunks = chunks

Use 800-character chunks with 100-character overlap as a starting point. Tune on your documents.

2. Prompt Comparison Tool

Let reviewers test two system prompts side by side. Streamlit’s st.columns(2) or Gradio’s gr.Row() makes this a 20-line build. Log both outputs and let reviewers vote.

3. Model Bake-Off

Expose a dropdown of models and a temperature slider. Show latency and token counts under each answer. Teams use this to pick a model on evidence instead of opinion.

model = st.selectbox("Model", ["claude-sonnet-5-5", "claude-haiku-5-5"])
temp = st.slider("Temperature", 0.0, 1.0, 0.3, 0.1)

4. Prompt Template for Demo Assistants

You are a {role} for {audience}.
Rules:
- Answer in at most {max_words} words.
- If the question is outside {domain}, say so and stop.
- Cite the provided context. If the answer is not in it, say "Not in the provided documents."

The last rule is the cheapest hallucination guard you can add to a demo.

Production Checklist Before You Share the Link

Which One Should You Pick?

Pick Gradio if you need a chat or model demo live in 10 minutes, want an API for free, or plan to host on Hugging Face where your audience already is.

Pick Streamlit if the demo needs navigation, multiple pages, charts, authentication, or business logic around the model. Its script-first model is also easier for data teams to read.

Ship both if you can. The shared llm.py pattern above makes the second UI a 7-minute job.

FAQ

Is Gradio or Streamlit better for an AI chatbot demo?

Gradio is faster for a plain chatbot because gr.ChatInterface provides history, streaming, and retry in one call. Streamlit is better when the chatbot sits inside a larger app with sidebars, uploads, and charts.

Can I deploy a Streamlit or Gradio app for free?

Yes. Gradio apps deploy free to Hugging Face Spaces. Streamlit apps deploy free to Streamlit Community Cloud from a GitHub repo. Both free tiers sleep after inactivity and have limited CPU and memory.

How do I keep my API key safe in a public demo?

Store the key as a platform secret, such as Spaces secrets or Streamlit Cloud secrets, and read it from an environment variable. Never hard-code it or commit it to Git. Add a spend limit and max_tokens cap to contain abuse.

How do I stream LLM responses in Streamlit and Gradio?

In Streamlit, pass a generator to st.write_stream. In Gradio, write a function that yields the growing response string and pass it to gr.ChatInterface. Both pair with the Anthropic SDK’s messages.stream method.