Grafy News
Try Grafy free

ARCHER Boosts Small Language Models with Executable Reasoning

Researchers unveil ARCHER, a two‑stage framework that turns free‑form rationales into executable traces, raising small model math accuracy by up to 9 points.

Illustration for the news story: ARCHER Boosts Small Language Models with Executable ReasoningSource: Grafy Research Lab

Small language models (SLMs) have long been attractive for on‑device, low‑cost, and privacy‑preserving AI, yet they still stumble on math problems that demand stepwise decomposition, exact calculation, and verification. A new paper from the Grafy Research Lab introduces ARCHER – Action‑trace Reinforcement for Calculator‑grounded Hierarchical Executable Reasoning – and shows it lifts sub‑10B models by up to nine percentage points on standard math benchmarks.

What is ARCHER and why does it matter?

ARCHER is a two‑stage training pipeline designed to teach SLMs to generate executable reasoning trajectories instead of merely fluent natural‑language explanations. According to the paper, the approach targets three pain points of existing small‑model methods: (1) free‑form chain‑of‑thought rationales mix reasoning decisions with hidden arithmetic, (2) tool‑augmented systems give models access to calculators but do not teach when or how to call them, and (3) pure outcome‑only reinforcement signals provide weak credit assignment for multi‑step problems. By converting rationales into typed action traces that include calculator calls, deterministic observations, verification steps, and final answers, ARCHER gives SLMs a concrete supervision target that can be audited and reproduced.

How does ARCHER’s two‑stage training work?

The pipeline consists of TRACE (Stage 1) and HiGRA (Stage 2).

Stage 1 – TRACE compiles legacy chain‑of‑thought data into typed action traces. A rule‑based compiler scans each rationale, detects arithmetic spans, normalizes expressions, and replaces them with explicit calculator‑call actions followed by the observed result. The output is a sequence such as:

1. Identify variables
2. CALL_CALC(3*7) → 21
3. Use result in next step …
4. VERIFY(21 < 30) → true
5. FINAL_ANSWER(42)

These traces become the supervised target for a student policy that learns to emit the same action language.

Stage 2 – HiGRA refines the TRACE policy with asymmetric hierarchical reinforcement learning. Rewards are grouped into families – answer correctness, tool grounding, process discipline, and trace alignment – and are conditionally gated so that a later family only contributes when earlier families are satisfied. For example, the tool‑grounding reward is only evaluated if the model actually issued a calculator call; the process‑discipline reward checks that observations are used consistently; and the trace‑alignment reward measures how closely the generated trace matches the compiled reference. The authors train using a policy‑gradient method called GRPO, which respects the hierarchical gating.

What results does ARCHER achieve on benchmark tests?

The authors evaluate ARCHER on seven sub‑10B backbones across eight mathematical reasoning benchmarks, totalling 40 model‑benchmark pairs. Across the five standard backbones, ARCHER improves the macro‑average accuracy by 5.51 percentage points over the strongest tool‑augmented baseline. Notable gains include:

  • +8.95 pp on the MATH‑500 subset, a challenging collection of competition‑style problems.
  • +5.33 pp on AIME‑2024 and +5.34 pp on AIME‑2025. The method also outperforms reasoning‑specialized 7B models that had previously undergone reinforcement‑learning or distillation, indicating that executable grounding adds value even after heavy reasoning‑focused training. The paper reports that ARCHER’s improvements are most pronounced on formal, multi‑step benchmarks, where precise arithmetic and verification are essential.

How does ARCHER compare to prior tool‑augmented or reinforcement‑learning methods?

Earlier work falls into three broad camps:

  1. Rationale distillation – models imitate free‑form explanations (e.g., Wei et al., 2022). ARCHER replaces that imitation with concrete action sequences, eliminating the hidden arithmetic problem.
  2. Tool‑augmented reasoning – systems expose calculators or APIs but leave the policy for when to call them undefined (e.g., program‑aided approaches). ARCHER’s TRACE stage explicitly trains the policy to emit well‑formed tool calls, and HiGRA rewards correct usage.
  3. Process‑level reinforcement learning – methods reward intermediate steps but often suffer from noisy or misaligned signals. ARCHER’s hierarchical reward families provide priority‑ordered credit, ensuring that the model first learns to get the answer right, then to use tools correctly, and finally to follow disciplined processes. By unifying these strands, ARCHER achieves consistent gains across model families and demonstrates that executable supervision is a scalable way to improve small models without inflating parameter counts.

How to explore ARCHER concepts with Grafy

While Grafy does not yet host a dedicated ARCHER trainer, creators and researchers can experiment with the underlying ideas using existing tools:

  • Grafy Reader lets you upload the ARCHER arXiv PDF, ask natural‑language questions, and retrieve specific sections such as the TRACE compiler details or HiGRA reward formulas.
  • Grafy Chat enables you to prompt a small open‑source model (e.g., a 7B LLM) to output a toy action trace for a simple arithmetic problem, then manually invoke the built‑in calculator tool to see the observation and verification steps in action. These workflows give a hands‑on feel for executable reasoning without needing a full reinforcement‑learning loop. For teams interested in building a full ARCHER‑style pipeline, the paper’s open‑source code (linked in the appendix) can be integrated with Grafy’s Code studio to prototype the compiler and reward functions.

Source: https://arxiv.org/abs/2611.18494

Frequently asked questions

Does ARCHER teach a small model to decide when to use a calculator?

Yes. In Stage 1 the compiler inserts explicit CALL_CALC actions, and in Stage 2 the hierarchical reward gives credit only when the model actually issues a calculator call, reinforcing the decision‑making behavior.

Is the improvement from ARCHER limited to math benchmarks?

The reported experiments focus on mathematical reasoning datasets such as MATH‑500 and AIME. The authors note that the framework is general‑purpose and could be adapted to any domain where tool‑grounded, stepwise execution is beneficial.

Can I use Grafy to run the full ARCHER training pipeline?

Grafy does not currently provide a turnkey ARCHER trainer, but you can explore the paper with Reader, prototype trace generation with Chat, and develop custom training scripts in Code. The open‑source repository linked in the paper contains the necessary components for a self‑hosted implementation.


Source: Grafy Research Lab — ARCHER: From Rationale Imitation to Executable Reasoning in Small Language Models