Finetuning Sessions – Week 6 Office Hours

Here's the recording from yesterday's live session, covering the Week 6 content of the Finetuning Sessions.

During office hours, we focused on:

• Why PPO's four-model setup becomes an infrastructure bottleneck and how GRPO eliminates the critic

• How GRPO works: group-based advantage estimation and grading on a curve

• The difference between GRPO and RLVR — optim…