AI vocabulary grows faster than most people can track. One model announcement can mention MoE routing, 4-bit quantization, KV caching, and DPO alignment in a single paragraph. This glossary defines 70 terms in plain language, adds the technical specs engineers need, and shows how the concepts connect in real systems.
Quick Takeaways
- 70 terms, A to Z. Each entry has a one-line definition you can use immediately.
- Concepts cluster into five layers: architecture, training, prompting, retrieval, and deployment. Learn the cluster, not just the word.
- Three adaptation methods matter most: prompting, RAG, and fine-tuning. The comparison table below shows when to use each.
- Settings control behavior. Temperature, top-p, and max tokens change output quality, as the code example shows.
| Layer | Core Terms | Why It Matters |
|---|---|---|
| Architecture | Transformer, Attention, MoE, Mamba | Determines speed, cost, and context length |
| Training | Pretraining, RLHF, DPO, LoRA | Determines model behavior and safety |
| Prompting | Few-shot, CoT, System Prompt | Cheapest way to improve output |
| Retrieval | Embeddings, Vector DB, RAG | Reduces hallucinations with fresh data |
| Deployment | Quantization, KV Cache, Latency | Determines cost and user experience |
How to Use This Glossary
Terms are grouped alphabetically in five tables. Numbers run 1 to 70 so you can reference them. Bold text marks metrics, parameters, and algorithms worth remembering.
Terms 1–12: Agents to Cosine Similarity
| # | Term | Plain-English Definition |
|---|---|---|
| 1 | Agent | An LLM-driven system that plans steps, calls tools, and acts toward a goal with limited human input. |
| 2 | Alignment | Training a model so its outputs match human intent and safety rules. |
| 3 | Attention | A mechanism that weights how much each token should influence every other token. Compute cost grows quadratically with sequence length in standard form. |
| 4 | Autoregressive Model | A model that generates one token at a time, each conditioned on all previous tokens. |
| 5 | Benchmark | A standardized test (e.g., MMLU, HumanEval, SWE-bench) used to compare models. |
| 6 | Bias | Systematic skew in outputs caused by training data or design choices. |
| 7 | BPE (Byte-Pair Encoding) | A tokenization algorithm that merges frequent character pairs into subword units. |
| 8 | Chain-of-Thought (CoT) | A prompting technique that asks the model to reason step by step before answering. |
| 9 | Chunking | Splitting documents into pieces (often 200–1,000 tokens) before embedding them for retrieval. |
| 10 | Context Window | The maximum tokens a model can process in one request, covering input and output. Modern models range from 128K to over 1M tokens. |
| 11 | Constitutional AI | An Anthropic method where a model critiques and revises its own outputs against written principles. |
| 12 | Cosine Similarity | A score from -1 to 1 measuring the angle between two vectors. Higher means more semantically similar. |
Terms 13–29: Decoder to Guardrails
| # | Term | Plain-English Definition |
|---|---|---|
| 13 | Decoder | The transformer component that generates output tokens. GPT-style and Claude-style models are decoder-only. |
| 14 | Diffusion Model | A generative model that creates images by gradually removing noise from random data. |
| 15 | Distillation | Training a small “student” model to mimic a larger “teacher” model. |
| 16 | DPO (Direct Preference Optimization) | An alignment method that trains on preferred vs. rejected answers without a separate reward model. |
| 17 | Embedding | A numeric vector representing meaning. Common sizes are 768, 1,536, and 3,072 dimensions. |
| 18 | Encoder | The transformer component that converts input into rich representations. BERT is encoder-only. |
| 19 | Epoch | One full pass through the training dataset. |
| 20 | Evals | Custom tests measuring model performance on your specific task. |
| 21 | Few-Shot Prompting | Including 2–10 examples in the prompt so the model copies the pattern. |
| 22 | Fine-Tuning | Continuing training on task-specific data to change model behavior or style. |
| 23 | Foundation Model | A large model pretrained on broad data and adapted to many downstream tasks. |
| 24 | Function Calling | A feature where the model returns structured arguments for your code to execute, such as get_weather(city="Austin"). |
| 25 | GAN (Generative Adversarial Network) | Two networks compete: a generator makes fakes, a discriminator spots them. |
| 26 | Generative AI | Systems that create new text, images, audio, code, or video. |
| 27 | Gradient Descent | The optimization algorithm that adjusts weights to reduce loss step by step. |
| 28 | Grounding | Anchoring answers in verifiable sources, such as retrieved documents or search results. |
| 29 | Guardrails | Rules and filters that block unsafe, off-topic, or malformed inputs and outputs. |
Terms 30–45: Hallucination to Mixture of Experts
| # | Term | Plain-English Definition |
|---|---|---|
| 30 | Hallucination | A confident but false output. RAG and grounding reduce it. |
| 31 | Hyperparameter | A setting chosen before training, such as learning rate or batch size. |
| 32 | Inference | Running a trained model to produce outputs. |
| 33 | In-Context Learning | Learning a task from examples in the prompt, with no weight updates. |
| 34 | Instruction Tuning | Fine-tuning on instruction-response pairs so the model follows commands. |
| 35 | Jailbreak | A prompt crafted to bypass a model’s safety training. |
| 36 | Knowledge Cutoff | The date after which a model has no training data. |
| 37 | KV Cache | Stored key/value attention tensors that avoid recomputation during generation. It speeds up decoding but uses significant GPU memory. |
| 38 | Latency | Response delay. Teams track time to first token (TTFT) and tokens per second. |
| 39 | LLM (Large Language Model) | A transformer trained on massive text corpora with billions of parameters. |
| 40 | LoRA (Low-Rank Adaptation) | A parameter-efficient fine-tuning method that trains small adapter matrices (rank r = 8–64) instead of all weights. |
| 41 | Loss Function | A formula scoring how wrong a prediction is. Cross-entropy is standard for language models. |
| 42 | MCP (Model Context Protocol) | An open standard for connecting models to external tools and data sources. |
| 43 | Mixture of Experts (MoE) | An architecture that routes each token to a few specialized sub-networks, so only part of the model runs per token. |
| 44 | Multimodal | Handling more than one data type, such as text, images, audio, and video. |
| 45 | Mamba | A state space model architecture with linear-time scaling in sequence length, offered as an alternative to attention. |
Terms 46–59: Neural Network to Reranking
| # | Term | Plain-English Definition |
|---|---|---|
| 46 | Neural Network | Layers of connected nodes that learn patterns by adjusting weights. |
| 47 | NLP (Natural Language Processing) | The field covering machine understanding and generation of human language. |
| 48 | Open-Weight Model | A model whose trained weights are downloadable (e.g., Llama, Mistral). Training data and code may stay closed. |
| 49 | Overfitting | Memorizing training data so well that performance drops on new data. |
| 50 | Parameter | A learned numeric value in the network. Model size is quoted in billions (7B, 70B, 405B). |
| 51 | Perplexity | A metric of how surprised a model is by text. Lower is better. |
| 52 | Pretraining | The first, expensive training phase on vast unlabeled data using next-token prediction. |
| 53 | Prompt Engineering | Designing inputs to get reliable, high-quality outputs. |
| 54 | Prompt Injection | An attack where malicious text in the input hijacks the model’s instructions. |
| 55 | Quantization | Reducing numeric precision (FP16 → INT8 → 4-bit) to shrink memory use and speed up inference with some accuracy loss. |
| 56 | RAG (Retrieval-Augmented Generation) | Retrieving relevant documents at query time and feeding them to the model as context. |
| 57 | Reasoning Model | A model trained to spend extra “thinking” tokens working through problems before answering. |
| 58 | RLHF | Reinforcement Learning from Human Feedback. Humans rank outputs, and the model is optimized toward preferred ones. |
| 59 | Reranking | A second-pass model that reorders retrieved results by relevance before they reach the LLM. |
Terms 60–70: Self-Supervised Learning to Zero-Shot
| # | Term | Plain-English Definition |
|---|---|---|
| 60 | Self-Supervised Learning | Creating training labels from the data itself, such as predicting the next word. |
| 61 | Structured Output | Forcing responses into a schema such as JSON for reliable parsing. |
| 62 | Synthetic Data | Training or evaluation data generated by models instead of collected from people. |
| 63 | System Prompt | Top-level instructions that set role, tone, and rules for a conversation. |
| 64 | Temperature | A sampling setting, typically 0.0–1.0, controlling randomness. Low is deterministic. High is creative. |
| 65 | Token | A unit of text, roughly ¾ of an English word on average. Pricing and limits are counted in tokens. |
| 66 | Top-p (Nucleus Sampling) | Samples only from the smallest token set whose cumulative probability reaches p (e.g., 0.9). |
| 67 | Transformer | The attention-based architecture introduced in 2017 that underpins modern LLMs. |
| 68 | Vector Database | A database optimized for storing embeddings and running nearest-neighbor search (e.g., pgvector, Pinecone, Qdrant). |
| 69 | Weights | The learned parameters that encode a model’s knowledge. |
| 70 | Zero-Shot Learning | Performing a task from instructions alone, with no examples. |
How These Terms Connect: Adapting a Model to Your Task
The most practical decision in applied AI is how to adapt a model. Three methods cover most cases.
| Method | Changes Weights? | Best For | Trade-Offs |
|---|---|---|---|
| Prompt Engineering | No | Formatting, tone, quick prototypes | Limited by context window; costs tokens every call |
| RAG | No | Fresh or private knowledge, citations | Needs chunking, embeddings, vector DB, and reranking |
| Fine-Tuning (LoRA) | Yes (adapters) | Consistent style, narrow domain skills | Needs labeled data; risks overfitting |
Rule of thumb: start with prompting, add RAG when facts matter, and fine-tune only when behavior still falls short.
Architecture Comparison: Transformer vs. Mamba vs. MoE
| Feature | Transformer (Dense) | Mixture of Experts | Mamba (SSM) |
|---|---|---|---|
| Scaling with length | Quadratic | Quadratic (attention layers) | Linear |
| Active params per token | All | A fraction | All |
| Maturity | Highest | High | Emerging |
| Key advantage | Proven quality | Large capacity at lower compute | Long-sequence efficiency |
MoE and Mamba are not mutually exclusive. Hybrid designs exist that mix attention, state space layers, and expert routing.
Hands-On: Controlling Output With Core Parameters
Three settings shape most API behavior: temperature, max_tokens, and the system prompt. This example uses the Anthropic Python SDK.
pip install anthropic
export ANTHROPIC_API_KEY="your-key-here"
import anthropic
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY from env
response = client.messages.create(
model="claude-sonnet-5-5", # model string
max_tokens=500, # caps output length
temperature=0.2, # low = consistent, factual answers
system="You are a precise AI tutor. Answer in under 80 words.",
messages=[
{"role": "user", "content": "Explain KV cache to a beginner."}
],
)
print(response.content[0].text)
print(response.usage.input_tokens, response.usage.output_tokens) # token metrics
Tuning guide:
| Task | Temperature | Notes |
|---|---|---|
| Data extraction, classification | 0.0–0.2 | Pair with structured output |
| Q&A over documents (RAG) | 0.1–0.3 | Keeps answers grounded |
| Brainstorming, copywriting | 0.7–1.0 | Higher variety |
Adjust either temperature or top-p, not both. Changing both makes results harder to predict.
Practical Workflows and Real-World Use Cases
Prompt Template: Few-Shot Classification
System: Classify support tickets as BILLING, BUG, or FEATURE. Reply with the label only.
Ticket: "I was charged twice this month." -> BILLING
Ticket: "The export button crashes the app." -> BUG
Ticket: "Please add dark mode." -> FEATURE
Ticket: "{{user_ticket}}" ->
This uses in-context learning with three examples. Set temperature 0 and max_tokens 5 for stable, cheap outputs.
Enterprise RAG Pipeline to Reduce Hallucinations
- Chunk source documents into 500-token segments with a 50-token overlap.
- Embed each chunk (e.g., 1,536 dimensions) and store it in a vector database.
- Retrieve the top 20 matches by cosine similarity at query time.
- Rerank to the best 5 chunks.
- Generate with a system prompt that says: “Answer only from the provided context. If missing, say you don’t know.”
- Evaluate with evals that check faithfulness and citation accuracy.
Agent Workflow: Research Assistant
An agent receives a goal, then loops: plan, call a tool via function calling or MCP, observe the result, and decide the next step. Add guardrails that limit allowed tools, cap loop iterations, and sanitize retrieved text against prompt injection.
Cost and Speed Optimization
Teams serving models at scale combine 4-bit quantization to fit models on fewer GPUs, KV caching to cut repeated computation, and distillation to move high-volume tasks to smaller models. Track TTFT and tokens per second alongside quality evals so optimizations don’t silently degrade answers.
FAQ
What is the difference between AI, machine learning, and deep learning?
AI is the broad goal of machines performing tasks that need intelligence. Machine learning is a subset where systems learn from data. Deep learning is a subset of ML that uses multi-layer neural networks. LLMs are deep learning models.
What is the difference between RAG and fine-tuning?
RAG gives a model external knowledge at query time without changing its weights. Fine-tuning changes the weights to alter behavior or style. Use RAG for changing facts and citations. Use fine-tuning for consistent tone or specialized skills.
What is a token in AI?
A token is a chunk of text a model reads or writes, often a word fragment. In English, one token averages about ¾ of a word. Context limits and API pricing are measured in tokens.
What causes AI hallucinations, and how can I reduce them?
Models predict plausible text, not verified truth, so gaps in knowledge can produce confident errors. Reduce hallucinations with RAG, grounding instructions, low temperature, structured output, and evals that test factual accuracy.
