Agent Flight Recorder · by Aravind Kumar Nalukurthi

Headline result

It finds the first wrong move in a failed run correctly 87% of the time, across 300 real failures.

Finds the root cause

Failures studied

Steps you skip

Slowdown added

Runs almost always go wrong far earlier than where they visibly break. The recorder closes the gap between where it crashed and where the trouble actually started.

The problem

Why this matters

AI agents now work on their own for hours, making dozens of moves in a row. When one fails, you are left scrolling through thousands of log lines trying to guess where it slipped. There is no standard way to record what an agent was thinking, and no quick way to point at the one decision that doomed the whole run. It is like investigating a plane crash with no black box on board.

The approach

How it works

This is the black box. It clips onto any agent and quietly records every action, every tool it reached for, every observation, and every change in what it believed to be true. When a run fails, it replays that recording and pinpoints the earliest moment things went off track. Then it lets you ask plain questions about the run, like where did it first misunderstand the goal.

Contributions

What's new here

An open way to record what a long running agent does, step by step, that works with any model.
A finder that points at the first wrong decision and gets it right 87% of the time.
A test set of 300 real agent failures, each labeled with where it truly went wrong.
A simple web view that turns a raw recording into a timeline you can click through.

What actually goes wrong, from 300 real runs

Item	Description	Signal
Misread the goal	It got the task wrong at step one, and everything after inherited that mistake	Silent
Picked the wrong tool	It chose a reasonable looking but wrong tool, and the result seemed fine	Delayed
Passed along bad input	It handed off a broken value that only caused trouble 30 steps later	Delayed
Beliefs drifted	What it thought was true started clashing with something it had already seen	Building
Stuck in a loop	It kept retrying the same failing move without rethinking	Obvious
Forgot a key fact	An important detail slipped out of memory and was never looked up again	Building

The portfolio

Four more where this came from

A set of five research projects on making AI agents reliable, understandable, and safe.

Reward Hacking ObservatoryRHO Agent Misalignment ProbesAMP Adaptive Compute ControllerACC Agent Economy Red TeamAER