Artificial Intelligence (AI)
The field of building machines that perform tasks normally requiring human intelligence.
210 terms · 15 categories
The complete glossary
All AI terms in English (the standard usage) with simple definitions.
Fundamentals
10 termsMachine Learning (ML)
A subfield of AI where a system learns from data rather than being explicitly programmed.
Deep Learning
ML based on neural networks with many layers.
Narrow AI
AI specialized in a single task (e.g. playing chess).
AGI (Artificial General Intelligence)
A hypothetical AI capable of performing any intellectual task a human can.
Used in: ch.16
Superintelligence
An AI that would vastly surpass human intelligence across all domains.
Symbolic AI
A classical approach based on explicit rules and formal logic.
Model
A mathematical representation learned from data.
Inference
The phase where a trained model makes predictions.
Training
The phase where the model learns from data.
Learning Types
10 termsSupervised Learning
Learning from labeled data (input → known output).
Unsupervised Learning
Learning without labels; the model finds hidden structure in data.
Self-supervised Learning
The model generates its own labels from the data itself.
Used in: ch.06
Reinforcement Learning
Learning by trial and error with rewards.
Used in: ch.08
Transfer Learning
Reusing a model trained on one task for a different but related task.
Few-shot Learning
Learning from very few examples.
Zero-shot Learning
Succeeding at a task without having seen any examples during training.
Meta-learning
'Learning to learn' — training a model to adapt quickly to new tasks.
Federated Learning
Distributed training without centralizing data.
Continual Learning
The model learns new tasks without forgetting previous ones.
Neural Networks
17 termsNeuron / Unit
A basic computation unit in a network, inspired by biological neurons.
Layer
A set of neurons processed in parallel.
Weights
Parameters that scale the connections between neurons.
Bias
A constant term added to the weighted sum.
Activation Function
A non-linear function applied to a neuron's output.
ReLU
A very common activation function: max(0, x).
Logits
The raw scores produced by the final layer, before softmax turns them into probabilities.
Softmax
Converts a vector into a probability distribution.
Forward Pass
Computing the output from a given input.
Backpropagation
Computing gradients to update the weights.
Used in: ch.06
Gradient Descent
An algorithm that adjusts weights to minimize error.
Loss Function
Measures the error between a prediction and the ground truth.
Cross-Entropy Loss
The standard loss function for classification tasks.
Optimizer
An algorithm that drives weight updates (Adam, SGD, etc.).
Learning Rate
The step size during gradient descent.
Used in: ch.06
Epoch
One full pass through the training dataset.
Batch
A subset of data processed together.
Architectures
20 termsFeedforward Network
A network where information flows in one direction only.
MLP (Multi-Layer Perceptron)
A classic feedforward network with dense layers.
CNN (Convolutional Neural Network)
Specialized for images using convolutional filters.
RNN (Recurrent Neural Network)
Processes sequences by maintaining an internal state.
Used in: ch.04
LSTM
An improved RNN designed to remember information over long sequences.
Used in: ch.04
Transformer
An attention-based architecture dominant in AI since 2017.
Used in: ch.01, ch.02, ch.04, ch.05, ch.07, ch.08, ch.10, ch.15, ch.17, ch.18, ch.20, ch.21
Self-Attention
Each position in a sequence computes its relationship to every other position in the same sequence.
Used in: ch.01, ch.02, ch.03, ch.04, ch.05, ch.07, ch.08, ch.09, ch.10, ch.14, ch.15, ch.17, ch.18, ch.21
Cross-Attention
Attention where the queries come from one sequence and the keys/values from another — text looking at an image, a decoder looking at the encoder.
Multi-Head Attention
Multiple attention mechanisms running in parallel.
Encoder-Decoder
A classic architecture for translation and generation tasks.
Used in: ch.15
Decoder-only / Autoregressive
The architecture of modern LLMs: a single stack of blocks that generates one token at a time, each token seeing only the previous ones.
Positional Encoding
Injects word order information into a Transformer.
Used in: ch.05
RoPE
A modern rotary positional encoding scheme, common in LLMs.
Used in: ch.05
Residual Connection
A shortcut connection that skips over layers to stabilize training.
Layer Normalization
Normalization applied across a layer to stabilize training.
Mixture of Experts (MoE)
An architecture where only a few sub-networks ('experts') are activated for each token, which cuts the amount of computation.
Used in: ch.05
Diffusion Model
A generative model that learns to denoise data (image, audio, text, etc.) step by step.
GAN (Generative Adversarial Network)
Two competing networks: a generator and a discriminator.
Vision Transformer (ViT)
A Transformer architecture applied to images.
State Space Model / Mamba
An alternative architecture to the Transformer for long sequences.
Natural Language Processing
14 termsNLP
The field of processing and understanding human language with machines.
Used in: ch.15
Tokenization
Splitting text into units called tokens.
Token
The basic unit processed by a model (word, subword, or character).
Used in: ch.01, ch.02, ch.03, ch.04, ch.05, ch.06, ch.07, ch.08, ch.09, ch.10, ch.11, ch.12, ch.13, ch.14, ch.15, ch.17, ch.18, ch.19, ch.20, ch.21
BPE (Byte Pair Encoding)
A widely used tokenization algorithm.
Used in: ch.02
WordPiece / SentencePiece
WordPiece: another tokenization algorithm. SentencePiece: a library that implements BPE and Unigram.
Used in: ch.02
Vocabulary
The set of all tokens known to a model.
Embedding
A dense vector representation of a word or token.
Used in: ch.01, ch.03, ch.04, ch.05, ch.10, ch.11, ch.15, ch.21
Cosine Similarity
Measures how close two vectors are by the angle between them, ignoring their length.
Word2Vec
An older technique for producing word embeddings.
Used in: ch.03
Contextual Embedding
An embedding that depends on the surrounding context.
Named Entity Recognition
Identifying named entities in text (people, places, etc.).
Sentiment Analysis
Detecting the emotional tone of a text.
Speech Recognition (ASR)
Converting spoken audio into text.
Text-to-Speech (TTS)
Converting text into spoken audio.
Used in: ch.15
Language Models & Generative AI
38 termsLanguage Model
A model that predicts the probability of a sequence of words.
LLM (Large Language Model)
A massive language model trained on trillions of tokens.
Used in: ch.01, ch.02, ch.03, ch.04, ch.05, ch.06, ch.07, ch.08, ch.09, ch.10, ch.11, ch.12, ch.13, ch.15, ch.16, ch.17, ch.18, ch.20, ch.21
Foundation Model
A general-purpose model reusable across many tasks.
Generative AI
AI capable of producing content (text, images, audio, etc.).
Pre-training
Initial training on large amounts of raw data.
Fine-tuning
Adapting a pre-trained model for a specific task.
Used in: ch.01, ch.06, ch.08, ch.09, ch.11, ch.13, ch.14, ch.19, ch.20
Instruction Tuning
Fine-tuning to follow human instructions.
RLHF
Refining a model using human feedback.
DPO (Direct Preference Optimization)
A simplified alternative to RLHF.
Constitutional AI
An alignment method based on a set of guiding principles.
Used in: ch.08
Prompt
The input text given to a model.
Used in: ch.01, ch.07, ch.08, ch.09, ch.10, ch.11, ch.12, ch.17, ch.18, ch.19, ch.20, ch.21
System Prompt
General instructions given to the model before the conversation begins.
Prompt Engineering
The art of crafting effective prompts.
Chain-of-Thought (CoT)
Prompting the model to reason step by step.
Self-Consistency
Generating several reasoning chains for the same question, then keeping the most frequent answer.
Used in: ch.12
Few-shot Prompting
Providing a few examples within the prompt.
In-Context Learning
The model's ability to learn from examples provided in the prompt.
Context Window
The amount of text a model can process at once.
Temperature
Controls the randomness of outputs (low = predictable, high = varied; 0 = almost deterministic).
Top-k Sampling
Sampling from the k most probable tokens.
Top-p / Nucleus Sampling
Sampling from the smallest set of tokens whose cumulative probability reaches p (e.g. 0.9).
Greedy Decoding
Always choosing the most probable next token.
Used in: ch.07
Beam Search
Exploring several candidate sequences in parallel.
Used in: ch.07
Hallucination
The model generates false but plausible-sounding information.
RAG (Retrieval-Augmented Generation)
The model retrieves external documents before generating a response.
Chunking
Splitting documents into right-sized pieces before indexing them — too large and the useful passage gets buried, too small and the meaning is lost.
Reranking
A second pass that reorders the retrieved documents with a finer model, so the truly relevant ones rise to the top.
Used in: ch.10
Vector Database
A database that stores and searches embeddings by similarity.
Semantic Search
Searching by meaning rather than exact keyword match.
Reasoning Model
An LLM optimized to reason at length before answering.
Thinking Tokens
Reasoning tokens a model generates before its answer — an internal draft, often hidden or summarized.
Used in: ch.17
Test-time Compute
Compute spent at answering time rather than during training: letting a model think longer improves its answers on hard problems.
Used in: ch.17
Multimodal Model
A model that handles multiple input types (text, image, audio, etc.).
Vision-Language Model
A model that understands both text and images.
Text-to-Image
Generating an image from a text description.
LoRA
A lightweight fine-tuning technique based on low-rank matrix decomposition.
Used in: ch.14
QLoRA
LoRA combined with quantization to reduce memory usage.
PEFT
A family of parameter-efficient fine-tuning methods.
Used in: ch.14
Agents & Tools
8 termsAI Agent
An AI system that pursues a goal through multiple steps and tools.
Tool Use / Function Calling
The model's ability to call external functions.
Used in: ch.11
ReAct
A pattern that alternates reasoning and action.
Used in: ch.11
Planning
An agent's ability to decompose a goal into subtasks.
Multi-Agent System
Multiple agents that collaborate or coordinate.
MCP (Model Context Protocol)
A standard protocol for connecting tools to an LLM.
Used in: ch.11
Computer Use
An agent's ability to operate a computer.
Used in: ch.11
Agentic AI
AI that acts autonomously within an environment.
Data
11 termsDataset
A collection of data used to train or evaluate a model.
Training / Validation / Test Set
Data splits for training, tuning, and evaluation.
Label / Ground Truth
The correct answer associated with a training example.
Annotation
The act of assigning labels to data.
Data Augmentation
Creating variants of existing data (rotations, paraphrases, etc.) to enrich training.
Synthetic Data
Data generated artificially, for example by an AI.
Feature
An input variable used by the model.
Used in: ch.20
Feature Engineering
Manually creating relevant input variables.
One-Hot Encoding
Encoding a category as a binary vector.
Data Drift
A gradual shift in data distribution compared to training time.
Data Leakage
When test information leaks into training, causing inflated results.
Training & Optimization
13 termsHyperparameter
A parameter set before training (learning rate, batch size, etc.).
Cross-Validation
Evaluating a model with multiple data splits.
Overfitting
The model memorizes training data and generalizes poorly.
Used in: ch.06
Underfitting
The model is too simple to capture the signal in the data.
Regularization
Techniques that prevent overfitting.
Used in: ch.06
Dropout
Randomly disabling neurons during training.
Early Stopping
Stopping training when validation error starts increasing.
Vanishing Gradient
Gradients become too small to train early layers effectively.
Catastrophic Forgetting
The model forgets old tasks when learning new ones.
Distillation
Transferring knowledge from a large model into a smaller one.
Used in: ch.21
Pruning
Removing redundant weights to make the model smaller.
Quantization
Reducing the numerical precision of weights.
Scaling Laws
Relationships between model size, data, compute, and performance.
Evaluation & Metrics
12 termsAccuracy
The percentage of correct predictions.
Precision
Of all positive predictions, how many are actually correct.
Recall
Of all the genuinely positive cases, how many the model finds.
F1 Score
The harmonic mean of precision and recall.
Confusion Matrix
A table comparing predictions against ground truth.
ROC / AUC
A curve and area under the curve for evaluating a classifier.
Perplexity
A measure of a language model's uncertainty (lower is better).
BLEU / ROUGE
Metrics for evaluating machine translation and summarization.
FID
A measure of the quality of generated images.
Benchmark
A standardized test for comparing models (MMLU-Pro, SWE-bench, GPQA, etc.).
Used in: ch.16
LLM-as-a-Judge
Using an LLM to grade another model's answers — fast and cheap, but with biases of its own.
Used in: ch.16
Red Teaming
Adversarial testing to find weaknesses in a model.
Reinforcement Learning
8 termsAgent
An entity that makes decisions within an environment.
Environment
The world in which an agent operates.
Policy
The agent's strategy for choosing actions given a state.
Reward
A numerical signal indicating how good an action was.
Q-Learning
An RL algorithm that learns to estimate the value of each action in each state (the Q function).
PPO
A widely used RL algorithm, especially in RLHF.
Exploration vs Exploitation
The trade-off between trying new actions and using known good ones.
Reward Hacking
The agent finds unintended ways to maximize the reward signal.
Used in: ch.11
Computer Vision
9 termsComputer Vision
The AI field concerned with processing images and video.
Image Classification
Assigning a category label to an image.
Object Detection
Locating and classifying objects within an image.
Semantic Segmentation
Labeling every pixel in an image with a class.
OCR
Recognizing text in an image.
CLIP
A model that aligns images and text in a shared embedding space — the bridge that lets image generators understand a prompt.
YOLO / R-CNN
Classic object detection model architectures.
SAM (Segment Anything)
A universal segmentation model.
NeRF
A learned 3D representation of a scene from 2D images.
Safety, Alignment & Ethics
15 termsAI Alignment
Ensuring AI systems pursue goals that match human values.
AI Safety
The field of making AI reliable, controllable, and harmless.
Adversarial Attack
An input crafted to fool a model.
Jailbreak
Bypassing the safety guardrails of an LLM.
Used in: ch.08
Prompt Injection
Malicious instructions hidden in content the model processes (a web page, a document) to hijack its behavior.
Used in: ch.12
Bias (algorithmic)
Systematic errors in data or model predictions.
Fairness
Equity in algorithmic decisions.
Explainability
The ability to understand why a model made a given decision.
Mechanistic Interpretability
Reverse-engineering the internal circuits of a network to understand the algorithms it has learned.
Sparse Autoencoder (SAE)
A network that decomposes a model's activations into sparse features that can be interpreted one by one.
Used in: ch.20
Differential Privacy
A formal method for protecting individual privacy in data.
Deepfake
AI-generated synthetic media designed to appear real.
Misalignment
A gap between what the model actually pursues and what its designers intended.
Reward Model
A model that predicts how humans would rate a response.
AI Governance
The frameworks and regulations for overseeing AI development.
Infrastructure & MLOps
16 termsGPU
A graphics processing unit, widely used for AI workloads.
TPU
A specialized AI accelerator chip designed by Google.
CUDA
Nvidia's parallel computing platform for GPU programming.
FLOPS
Floating point operations per second — a measure of compute.
Distributed Training
Training spread across multiple machines.
Data Parallelism
Same weights, different data on each GPU.
Model Parallelism
The model is split across multiple GPUs.
PyTorch / TensorFlow / JAX
The main deep learning frameworks.
Hugging Face
A platform for models, datasets, and ML tools.
ONNX
A standard format for exchanging models between frameworks.
Edge AI
AI running on embedded or on-device hardware.
KV Cache
The keys and values already computed during generation, kept in memory: avoids reprocessing the whole context for every new token.
Speculative Decoding
A small model drafts several tokens ahead and the large one validates them in one pass — same output, lower latency.
Latency
The response time of a model.
Used in: ch.18
Throughput
The number of inferences processed per second.
MLOps
DevOps practices applied to machine learning.
Emerging Concepts
9 termsEmergent Abilities
Capabilities that seem to appear abruptly beyond a certain model scale (a debated phenomenon).
Used in: ch.19
Scaling
Increasing model size, data, or compute to improve performance.
Open Weights
A model whose weights are publicly available.
Frontier Model
A model at the cutting edge of current capabilities.
Small Language Model (SLM)
A compact language model optimized for efficiency.
Human-in-the-Loop
Including humans in the learning or decision-making process.
World Model
An agent's internal model for predicting how the environment evolves.
Embodied AI
AI situated in a physical body, such as a robot.
Model Card
A document describing a model's capabilities, limitations, and biases.