Find the flap that caused the storm

When a team of AI agents fails a task, Butterfly finds the agent that made the decisive mistake, the step where it happened, and, when the graph links them, the chain of steps that carried it to the failure.

Try Butterfly See the results

A real failed run from the Who&When benchmark Running

184 real multi-agent failures Graph engine: FalkorDB Benchmark: Who&When (ICML 2025) No LLM needed to build the graph

Debugging a failed agent run took hours of scrolling. Butterfly takes one click and shows its work.

Multi-agent systems fail quietly. One agent misreads a page early on, the others build on it, and the failure only shows at the end. Butterfly reads the run as a graph of who relied on whom, and walks back from the failure to the step where it started.

Measured on 184 real failures

Same model for every method, no answers in any prompt, scored against the benchmark's human labels.

34.8%

Finds the decisive step

as often as the strongest whole-log method (33.2%)

62.5%

Names the agent at fault

against 53.8% for the whole-log method

51%

Fewer LLM tokens

58% fewer on long runs of up to 130 steps

28.3%

With no LLM at all

the graph alone finds the step

See the full benchmark Paired tests: step accuracy is a tie (p = 0.78); agent accuracy is better (p = 0.04).

How it works

Turn the run into a graph

Every message becomes a step. When a later step reuses a URL, file, number or name from an earlier one, follows its instruction, or reads its output, Butterfly links them. No LLM involved.

Let FalkorDB rank the suspects

Graph queries score every step by how much of the run builds on it, how many new facts it introduced, and how early it happened. Most of the work ends here.

One LLM call, then the chain

A model reads only the top suspects and picks the decisive one. A shortest-path query then looks for the chain from the failure back to the mistake, so the answer can be checked when the graph links them.

Built on FalkorDB

One graph per run. Each failure is its own isolated graph, created and queried in milliseconds.
Graph algorithms as tools. Breadth-first search measures how far a mistake spreads; shortest paths link the failure back to it.
Safe by default. Every read is a read-only query with a timeout, and every import is safe to re-run.
Answers you can check. When the graph links a mistake to the failure, the link is a Cypher path, not a paragraph.

// The proof: from the failure back to the suspect
MATCH (f:Step {is_final: true}), (s:Step {idx: $idx})
WITH shortestPath((f)-[:DEPENDS_ON*]->(s)) AS p
RETURN [n IN nodes(p) | n.idx] AS steps

// How much of the run builds on each suspect
MATCH (s:Step {candidate: true})
CALL algo.BFS(s, 200, 'FEEDS') YIELD nodes
RETURN s.idx, size(nodes) AS downstream

See the butterfly in a real failure

Pick any of the 184 failed runs, watch the graph rank the suspects, and read the proof step by step.

Try Butterfly