SWE-rebench: Over 21,000 Open Tasks for SWE LLMs(huggingface.co) |
SWE-rebench: Over 21,000 Open Tasks for SWE LLMs(huggingface.co) |
One common limitation in such datasets is that they usually don’t have many tasks, and they come from only a small number of repositories. For example, in the original SWE-bench there are 2,000+ tasks from just 18 repos. This mostly happens because researchers install each project manually and then collect the tasks.
We automated and scaled this process, so we were able to collect 21,000+ tasks from over 3,400 repositories.