Loading your workspace…
Loading your workspace…
Go deeper into mechanisms, mathematics, implementations, and research. Free references complement the six-stage Study path.
Preparing the 3D view…
Move from a line to a plane to a volume.
Preparing the 3D view…
Append keys and values across layers.
Preparing the 3D view…
Explore rotary coordinates at different frequencies.
Gemma 4 E2B
Separate per-layer embeddings, stored weights and attention patterns in a small multimodal model.
6 min read Visual inside
Qwen3.8-27B
Trace the 64-layer hybrid: recurrent Gated DeltaNet state alongside full-attention KV caches.
7 min read Visual inside
DeepSeek-V4
Distinguish active experts from stored parameters, then separate compression from sparse attention.
7 min read Visual inside
T5 Relative-Position Bias
Follow a signed token offset into a learned attention-bias bucket. See why the encoder and decoder allocate the same 32 slots differently.
7 min read Visual inside
Attention Mechanisms
Turn compatibility scores into attention weights, then combine values. Explore the geometry, normalization, and limits of the mechanism.
9 min read Visual inside
Batch Normalization
Normalize activations across a batch, apply a learned scale and shift, and understand why training and evaluation behave differently.
8 min read Visual inside
Transformer Architecture
Follow attention, feed-forward layers, normalization, and residual connections through a Transformer block.
9 min read Visual inside
Backpropagation
Trace local derivatives backward through a computation and connect the chain rule to a manual implementation.
9 min read Visual inside
Positional Encoding
Compare positional signals at different frequencies, then connect absolute encodings and RoPE to attention.
7 min read Visual inside
Recurrent Neural Networks (RNNs, LSTMs, GRUs)
Trace state through time and compare a plain RNN with gated LSTM and GRU cells. Learn what gating helps, what it does not guarantee, and when recurrence is useful.
9 min read Visual inside
Bias-Variance Tradeoff
Separate systematic error from sensitivity to the training sample. Read an error decomposition without assuming every model follows a U-shaped curve.
8 min read Visual inside
Logistic Regression
Connect a linear score to a sigmoid probability, a cross-entropy loss, and a decision threshold.
8 min read Visual inside
Principal Component Analysis (PCA)
Compare centered data with its projection onto a principal direction. Understand what variance retention does and does not preserve.
8 min read Visual inside
Gradient Boosting & XGBoost
See one residual-fitting update, then connect the additive model to gradient-based boosting for other losses.
8 min read Visual inside
LoRA and PEFT
Follow a frozen weight path and a trainable low-rank correction. Understand parameter savings, scaling, and merging.
8 min read Visual inside
RLHF (Reinforcement Learning from Human Feedback)
Distinguish demonstrations, preference comparisons, reward models, and policy updates. Compare SFT, PPO, and DPO visuals.
9 min read Visual inside
KV Cache
See which token projections caching saves, what attention must still read, and how memory use grows with context.
8 min read Visual inside
Tokenization & BPE
Follow a pair of vocabulary merges and distinguish learning merge rules from applying a tokenizer to new text.
7 min read Visual inside
Embeddings
Compare vectors by direction and magnitude, and connect embedding geometry to retrieval and representation learning.
7 min read Visual inside
Retrieval-Augmented Generation (RAG)
Follow retrieval into a grounded prompt, then examine chunking, ranking, evidence quality, and generation failures.
8 min read Visual inside
Temperature, Top-k & Top-p Sampling
Watch temperature reshape probabilities and distinguish that transformation from top-k, top-p, and sampling.
7 min read Visual inside
Frontier LLM Post-training and Reasoning
Separate the pretrained base, post-training objectives, and inference-time reasoning budget. Public model reports show task-specific gains but rarely reveal a complete training recipe.
10 min read Visual inside
PufferLib and PPO Engineering
Understand what PPO optimizes and what PufferLib changes in the surrounding training system. Keep throughput claims tied to a version, workload, and measurement.
10 min read Visual inside
Choosing an RL Algorithm
Start with the decision problem and the data you can collect. Use action space, online interaction, replay, safety, and compute to narrow the algorithm family before tuning it.
9 min read Visual inside