Tokenization & Attention
Tokenize Your Text
LLMs operate on tokens, not raw words. Try your own sentence; click a token to highlight its attention row/column.
Attention Visualization
Word Embeddings
Words in Space
Embeddings place words as points in a high‑dimensional space. Here we project to 2D; similar words appear nearby. Drag your cursor over points for details; click to reveal 3 nearest neighbors.
Semantic Clusters
Words with similar meanings form clusters in the embedding space.
Positive Cluster
Negative Cluster
Positional Encoding
Transformers use sinusoidal or learned signals that encode token positions. Adjust settings to see how frequency and max length affect the waves.
Model Training Journey
Pretraining Phase
Absorbs billions of tokens from corpora
Learns syntax & semantics
Internalizes broad knowledge
Fine‑tuning Phase
Focuses on target tasks
Refines weights
Improves accuracy
Making Models Helpful & Safe
Reinforcement Learning from Human Feedback (RLHF) aligns model behavior with preferences. Try a mini preference comparison below.
Model Response
Generates an answer
Human Feedback
Raters prefer better answers
Aligned Model
Learns a reward model
Candidate A: Overfitting is when a model memorizes the training data so well that it fails to generalize to unseen data.
Candidate B: Overfitting is when a model is over and fits the training data and then is worse on test data because it is over.
Magic Prompting
Try Prompting!
Craft your prompt. Use the tools to estimate tokens and copy quickly.
Model Response
Set a Role
Provide Examples
Be Clear
Retrieval‑Augmented Generation (RAG)
RAG retrieves relevant passages and feeds them to the generator. Use the demo to search a tiny in‑page knowledge base.
Retrieve
Find relevant chunks
Augment
Attach to the prompt
Generate
Answer with citations
Fighting Hallucinations
Hallucination
Model makes up incorrect information
Verified Response
Model provides accurate, checked information
Verify a Claim
Model Optimization
Knowledge Distillation
Transfer knowledge from large models to smaller, efficient ones.
Teacher Model
Student Model
Model Quantization
Reduce precision to shrink size and speed up inference.
Model Compression
Model Evaluation
Perplexity
How surprised the model is
BLEU
Translation quality
Task Accuracy
Performance on tasks