MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks 2025

MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks 2025(arxiv.org)

3 points by janandonly 13 days ago | 1 comment

dima853 13 days ago |

How do you handle partial success in the benchmark? If an agent picks the wrong tool mid-chain but fixes it with backtracking, does it still count as a full success?

dima853 13 days ago |

How do you handle partial success in the benchmark? If an agent picks the wrong tool mid-chain but fixes it with backtracking, does it still count as a full success?