← All work
Hindsight
A replay engine that finds the decisive failing step in an agent run and turns it into SFT and DPO training data.
89–100%
of failed runs: fatal step pinpointed
~5%
log-heuristic baseline
~80%
fewer rollouts
What it does
Agent failures usually have one decisive step: the point after which the run could no longer succeed. Hindsight replays failed runs to find that step, then converts it into training data in both SFT and DPO form.
It pinpointed the fatal step in 89–100% of failed runs, versus about 5% for log heuristics, and needed roughly 80% fewer rollouts to do it.
Stack
Python · FastAPI · SQLite · PyTorch · HuggingFace Transformers · NumPy · SciPy · Pytest
Honest notes
These numbers come from a scripted policy, not a live model. The code is private for now; I'm happy to walk through it.