Analyzing Coding Agent Task Performance Through Behavioral Traces

Written by

in

Researchers analyzed 143,209 execution traces from 125 agent configurations and 117 different models to compare the search, code-editing, and testing behaviors of coding agents. They found that agents with similar solution rates may still exhibit significantly different behavioral patterns; when task outcomes, token counts, and tool names are excluded, aggregated behavioral features can also identify corresponding agent combinations across different code repositories and programming languages. The team also released a deterministic parser, the complete annotation and analysis pipeline, and an annotated subset of traces.

Source:Hugging Face 社区论文精选 · Read Original

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *