.png)
Microsoft Research released Orchard, an open-source framework for training and evaluating AI agents across task types. The core pitch is straightforward: one shared environment layer, reusable across software engineering, browser navigation, computer use, and personal-assistant workflows, so researchers can spend less time rebuilding harnesses and more time comparing agent recipes.
Orchard is not just a benchmark suite. It is an open foundation for agentic modeling research that packages the environment side of the stack into a stable substrate, then lets different training and evaluation recipes plug into it.
That matters because agent work is usually bottlenecked by infrastructure fragmentation. SWE tasks, GUI tasks, and assistant-style workflows tend to end up with separate data formats, rollout code, reward logic, and evaluation harnesses. Orchard’s design tries to remove that duplication so the same infrastructure can support multiple domains without each project reimplementing the plumbing.
Microsoft Research says that reuse is a key enabler for smaller models: if the environment and evaluation stack stay consistent, the model gets the benefit of better training signal and less systems overhead rather than relying on brute-force scale.
The most concrete technical claim in the paper is the training recipe behind Orchard-SWE. Microsoft Research distilled 107K trajectories from MiniMax-M2.5 and Qwen3.5-397B, then used credit-assignment SFT to learn from productive segments of otherwise unresolved trajectories. That is the right instinct for long-horizon agent data: full-trajectory supervision is often too sparse, and the useful signal sits inside partial progress rather than only at terminal success.
They pair that with Balanced Adaptive Rollout for RL, which is the other part of the story that actually matters in practice. For agent training, rollout quality and diversity are often as important as optimizer choice. If the framework can keep rollouts balanced across task difficulty and failure modes, it becomes easier to improve policies without overfitting to a narrow slice of “easy” interactions.
Starting from Qwen3-30B-A3B-Thinking, Orchard-SWE reaches 64.3% on SWE-bench Verified after SFT and 67.5% after SFT+RL. Microsoft positions that as a new state of the art among open-source models of comparable size. For ML teams, the important signal is not just the benchmark number; it is that the framework is enabling a relatively modest model to close a gap that usually gets attributed to scale alone.
Orchard is not limited to code editing. The repository and paper also describe GUI and personal-assistant settings, which makes the framework more interesting than a narrow SWE benchmark harness.
The GUI track includes 3,070 multimodal browser-navigation rollouts, and the reported results show the same training stack transferring to browser-style interaction. In the paper, Orchard-GUI posts improvements after SFT+RL, with stronger gains on long-horizon benchmarks. That is a useful signal because long-horizon GUI tasks are where agent frameworks tend to break first: the evaluation stack is brittle, state drift is hard to diagnose, and reward assignment gets noisy fast.
The personal-assistant side is even more telling. Orchard-Claw targets assistant agents and reportedly achieves 59.6% pass@3 on Claw-Eval with only 0.2K synthetic tasks, rising to 73.9% when paired with a stronger ZeroClaw harness. The small synthetic dataset requirement suggests the environment abstraction is doing real work; if you can bootstrap performance from a tiny task set, the framework is probably capturing enough structure to make downstream adaptation practical.
For teams building agent systems, Orchard is interesting if you care about repeatable research rather than one-off demos. I would evaluate it along three axes.
First, I would look at environment fidelity and task transfer. If the same substrate really spans SWE, browser use, and assistant workflows without hidden domain-specific hacks, that is a meaningful infrastructure win.
Second, I would inspect how trajectory reuse affects training efficiency. Credit-assignment SFT on productive segments is a strong idea, but only if the segment selection logic is robust and not quietly baking in bias toward certain task patterns.
Third, I would compare the evaluation stack against your internal harnesses. The value of a framework like this is not just published scores; it is whether it gives you a stable way to run ablations, swap policies, and compare rollouts across domains without rewriting the surrounding system every time.
Orchard looks most useful as a research substrate for teams that want to iterate on agent recipes across several task families while keeping the environment and evaluation layer fixed. If Microsoft keeps the framework open and the recipes reproducible, it could become a credible reference stack for agentic AI experiments rather than just another benchmark wrapper.
