If you've used Linux, you've undoubtedly experienced these problems, so why not take a look?
SDPG is the main contribution. It extends GRPO with an exact per-token forward KL between the actor (without privileged context) and itself conditioned on privileged context c: ...