Research notes, engineering logs, reading group takeaways, and the occasional reflection.
Reflections on the core insight behind CATPO — why structured natural language critiques at each tree node provide fundamentally better optimization gradients than scalar reward signals from the leaf. Drawing parallels to how humans debug code: we don't just rate the output, we reason about intermediate steps…
research · rl · alignmentHow we reduced wall-clock time per epoch by 23% on our shared GPU cluster. This post covers the unglamorous but important stuff: gradient accumulation tuning, mixed-precision scheduling quirks, DataLoader bottlenecks, and why your training is secretly I/O bound…
infrastructure · gpu · systemsNotes from our VLG internal presentation on how Chinchilla-style scaling laws interact with evolutionary pruning. If you plan to prune a model to 50% sparsity, should you train a larger model with less data, or a smaller model with more data?
reading group · scaling · pruningReplicating Anthropic's indirect object identification circuit discovery on GPT-2 Small and Pythia 410M. What survives across model families, what doesn't, and what that tells us about the universality of learned circuits…
interpretability · circuits · transformersShort-form updates — what I shipped, read, presented, or learned.
Submitted CATPO to COLM 2026.
Pipeline optimization pass complete — 23% wall-clock reduction on multi-node cluster.
Deployed automated GPU health-check scripts across VLG compute pool.
Presented evolutionary pruning framework at VLG reading group.
CATPO shows 12-point win-rate improvement vs. DPO with 40% fewer rollouts.
"Natural Selection of LLMs via Pruning" accepted at AAAI 2026 Student Abstract.
Finalized CATPO prototype — evaluation pipeline + AlpacaEval-2 benchmarks wired up.
Started mech interp experiments — IOI circuit analysis on GPT-2 Small & Pythia 410M.
Set up TransformerLens + nnsight for activation patching experiments.
Joined VLG at IIT Roorkee as a core research member.
Started B.Tech in Data Science & AI at IIT Roorkee.