RepoFactory Scales Long‑Horizon Coding Agents with Automated Environments
Researchers introduced RepoFactory, an automated pipeline that turns code repositories into training environments.
Source: Grafy Research LabRepoFactory, a new automated pipeline described in a pre‑print released on November 17, 2026, converts software repositories into training environments for long‑horizon coding agents. The work, authored by Yang Tian, Tianyu Zheng, Xiyuan Yang, Yaxin Du, Chengran Yang, Bo Zhao, Jian Yang and Bryan Dai, addresses a bottleneck that has limited the scalability of agents that must reason over many sequential coding steps.
What problem does RepoFactory aim to solve?
Training agents that can solve complex programming tasks often requires environments that provide reliable, verifiable feedback after each action. Existing pipelines typically follow a linear workflow: collect data, build a static environment, train the model, then run a final validation. While this confirms that an agent can complete a benchmark, it offers little insight into how the training process could be improved. According to the paper, the lack of timely proxy signals makes it difficult to gauge the value of intermediate training iterations, especially when tasks span dozens or hundreds of coding steps.
How does the environment evolution loop work?
RepoFactory introduces an "environment evolution loop" that continuously synthesises new training environments from real‑world code bases. Rather than fixing a single set of repositories, the pipeline periodically selects, mutates, and recompiles code snippets to create fresh challenges. Each iteration produces a verifiable outcome – for example, a passing test suite – which serves as an immediate feedback signal. This loop enables recursive self‑improvement (RSI) without the costly full‑model rollouts traditionally required for long‑horizon tasks. The authors argue that the loop reduces feedback latency and lowers the computational budget needed for each training cycle.
What are the key components of RepoFactory?
The paper outlines three core modules:
- Repository Harvesting – an automated crawler that pulls public repositories from platforms such as GitHub, filters them for relevance, and extracts build scripts and test cases.
- Environment Synthesis – a transformer‑based model that rewrites code, injects bugs, or adds new API calls to generate novel problem instances while preserving syntactic correctness.
- Verification Engine – a lightweight sandbox that executes the synthesized code against its test suite, producing a binary success/failure signal that can be fed back to the training algorithm. These components operate in a closed loop, allowing the system to scale the diversity of environments as training progresses.
Why does this matter for developers and researchers?
Long‑horizon coding agents are essential for tasks such as automated refactoring, end‑to‑end software synthesis, and AI‑assisted debugging. However, the high cost of generating meaningful training data has slowed progress. By automating the creation of verifiable environments, RepoFactory promises to:
- Accelerate iteration – developers can obtain immediate feedback on whether a training change improves agent performance.
- Increase diversity – the pipeline can explore a far larger space of code patterns than hand‑crafted benchmarks.
- Reduce compute waste – proxy signals replace expensive downstream evaluations, freeing resources for larger model experiments. For research teams, the approach offers a reproducible way to benchmark RSI strategies on realistic code bases rather than synthetic toy problems.
How can Grafy users experiment with RepoFactory concepts?
While Grafy does not yet host a dedicated RepoFactory service, the platform provides several tools that let creators explore similar workflows:
- Grafy Code (/code) lets you write, run, and test code snippets in a browser‑based sandbox, mirroring the verification engine used in RepoFactory.
- Grafy Reader (/reader) enables you to upload the arXiv PDF, search for specific sections, and ask the model to summarise the methodology.
- Grafy Graph (/graph) can track the evolution of prompts and outputs as you iteratively modify repository‑derived tasks, giving you a visual history of your experiments. By chaining these apps, a developer can manually replicate parts of the environment evolution loop: harvest a repository, mutate the code with the AI chat, execute tests in the coding studio, and record results in a graph. This hands‑on approach helps teams validate the paper’s claims before committing to a full implementation.
What are the limitations and future directions?
The authors acknowledge that the current pipeline relies on well‑structured test suites; projects without comprehensive tests may produce noisy feedback. Additionally, the mutation model can occasionally generate code that compiles but does not reflect realistic developer intent. Future work outlined in the paper includes expanding the repository pool to private code bases, integrating static analysis tools for richer proxy signals, and exploring multi‑agent curricula where several agents co‑evolve within the same environment loop.
Where to read the full study?
The complete pre‑print is available on arXiv at https://arxiv.org/abs/2611.21152. It provides detailed experiments, ablation studies, and code release plans for the RepoFactory pipeline.
Frequently asked questions
How does RepoFactory differ from traditional static benchmark datasets?
RepoFactory continuously generates fresh coding challenges from live repositories, whereas static benchmarks are fixed collections of problems. The dynamic nature provides ongoing feedback and prevents agents from overfitting to a narrow set of tasks.
Can I use RepoFactory with open‑source models hosted on OpenRouter?
The paper does not specify model dependencies, but the pipeline is model‑agnostic. Users can plug in any compatible coding model, including those accessible through OpenRouter, as long as the model can accept the synthesized code and produce actionable outputs.
Is there a Grafy‑native way to automate repository harvesting?
Grafy does not include an out‑of‑the‑box repository crawler. However, you can combine Grafy Code with external scripts (run locally or in the cloud) to pull repositories, then feed the results into the Coding Studio for mutation and testing.
Source: Grafy Research Lab, “From Repositories to Training Environments: Scaling Long‑Horizon Coding Agents with Environment Evolution Loop”, arXiv, 2026‑11‑17


