ATOD: teacher guidance that changes with training
The Studio preset exposes the schedule and turn-weighting mechanism on synthetic inputs. It does not reproduce the training runs.
Explore the interactive mechanism ↗Read on alphaXiv ↗From the paper to the mechanism
See what changes.
Understand why.
Follow the learning signal from an example, a preference, a teacher, or a reward. These small, inspectable models expose the calculations behind the animation.
Supervision ≠ reinforcement learning
SFT learns from demonstrations. DPO learns preferences. Distillation matches a teacher. Policy-gradient methods use reward. Explore them side by side without treating them as interchangeable.
Interactive mechanism
Supervised fine-tuning
Choose the demonstrated response. Watch cross-entropy move probability toward it, one gradient update at a time.
Explore the mechanism ↗Interactive mechanism
Direct preference optimization
Pick a preferred and rejected response. Compare their log-probability ratios against a frozen reference.
Explore the mechanism ↗Interactive mechanism
Knowledge distillation
Reveal a teacher’s softened distribution and watch a student learn its relative preferences.
Explore the mechanism ↗Interactive mechanism
PPO clipping
Change the clip range and inspect where the policy surrogate stops rewarding movement in the same direction.
Explore the mechanism ↗Interactive mechanism
Group-relative optimization
Score six candidate responses, normalize their rewards, and trace the clipped policy update.
Explore the mechanism ↗Interactive mechanism
Policy-gradient bandit
A seeded agent samples actions, receives their rewards, and updates its categorical policy with REINFORCE.
Explore the mechanism ↗Recent paper · August 2026 revision
Teacher-to-reward guidance
Explore ATOD’s annealing and turn-level gates using editable synthetic signals.
Explore the mechanism ↗Further reading · discovered through alphaXiv
Where the ideas are going.
Distilled Reinforcement Learning for LLM Post-training
This July 2026 paper investigates selective teacher signals inside reward-based learning. It is further reading, rather than an implemented training algorithm in Studio.
Discover on alphaXiv ↗Read the primary paper ↗