PPO clipping

Change the clip range and inspect where the policy surrogate stops rewarding movement in the same direction.

Preparing the preview…

Made to be adapted.

The initial categorical distribution is the old policy. Each of the six outcomes is treated as one fixed rollout observation with a supplied advantage.

A six-outcome mathematical model with exact updates. No language model is being trained.

Export formats: PNG / HTML
Editable in Studio: Explanation, appearance, and model inputs
Access: Free with attribution. No account needed.
Reference: Read the source

Loading your workspace…

PPO clipping

Made to be adapted.

Another angle on the idea.

Group-relative optimization

Knowledge distillation

PPO clipping

Made to be adapted.

Another angle on the idea.

Group-relative optimization

Knowledge distillation