Semantic Context Handoff Cuts Tokens and Boosts Long-Horizon Agent Accuracy
Researchers introduce Semantic Context Handoff (SCH), a state‑evidence separation method that trims token usage by 43% while raising Pass@1 on SWE‑bench.
Source: Grafy Research LabTraditional long‑horizon language‑model agents keep the entire interaction history in each prompt, which quickly inflates inference cost and can drown out the most relevant information. The new Semantic Context Handoff (SCH) framework, presented by Guolong Zheng and colleagues, proposes a different balance: keep a concise, mutable representation of the agent’s current understanding and store the raw evidence separately for on‑demand retrieval.
What is Semantic Context Handoff and why does it matter?
SCH introduces a state–evidence separation abstraction. The semantic state is a lightweight snapshot of the agent’s current interpretation of the task, including constraints, hypotheses, and progress markers. The evidence archive holds the full list of action–observation pairs that led to that state. When the agent moves to the next step, it receives the semantic state plus any evidence the controller deems relevant, while the bulk of older observations stays out of the prompt. This design reduces the size of the prompt without discarding the ability to re‑examine past data, addressing a key weakness of prior compression‑only approaches that permanently lose evidence.
How does SCH separate state and evidence?
At each interaction cycle, the language model outputs two items: a sparse state delta that updates the semantic state, and an environment action to be executed. A deterministic controller then:
- Applies the delta to the stored semantic state.
- Executes the action in the environment.
- Appends the resulting action–observation pair to the evidence archive.
- Selects a subset of archived evidence that is likely to be useful for the next decision and bundles it with the updated state for the next prompt. This loop is illustrated in Figure 1, where the LLM receives the current semantic state together with relevant evidence, returns a delta and an action, and the controller manages the archive.
What performance gains were reported?
In a controlled 150‑task study on the SWE‑bench coding benchmark, the authors found that enabling evidence recovery lifted the mean Pass@1 score from 80.0% to 86.9% while keeping token consumption and wall‑clock time essentially unchanged. In a full‑system comparison, SCH achieved the same 86.9% Pass@1 but used 43% fewer tokens and 41% less execution time than the strongest baseline that relied on full‑history prompting. Across multiple models and benchmarks, SCH consistently matched or outperformed existing methods, confirming that the ability to bring back non‑local evidence improves both effectiveness and efficiency.
How does SCH differ from existing context‑compression techniques?
Prior work such as LongLLMLingua, ReSum, and ACON focuses on trimming the prompt by summarizing or selectively retaining recent interactions. Those methods reduce cost but risk losing evidence that later becomes relevant, leading to irreversible mistakes. SCH, by contrast, never discards raw evidence; it merely moves it out of the active prompt until a later step signals that the evidence should be revisited. This revision‑aware recovery avoids the “summary‑only” failure mode where a hypothesis cannot be re‑examined because its supporting observations have been compressed away.
How can Grafy users explore SCH concepts?
While Grafy does not yet provide a built‑in SCH engine, creators can experiment with the underlying ideas using existing tools:
- Grafy Code – Build a simple loop that stores each action‑observation pair in a JSON file (the evidence archive) and maintains a separate variable for the semantic state. Use the code studio to run the loop and measure token usage with different prompt sizes.
- Grafy Chat – Prompt the chat model with a compact state description and ask it to request specific pieces of evidence from the archive, mimicking the on‑demand retrieval step.
- Grafy Reader – Upload the full SWE‑bench paper and let the Reader extract the sections on evidence recovery and state deltas, then ask follow‑up questions to deepen understanding. These workflows let developers prototype the state–evidence split and evaluate its impact on token budgets before a dedicated SCH module becomes available.
Limitations and future directions
The authors acknowledge that SCH still depends on a deterministic controller to decide which evidence to surface, which may become a bottleneck in highly dynamic environments. Additionally, the current implementation assumes a single, linear archive; extending the approach to hierarchical or multi‑agent settings remains an open research question. Future work could integrate learned retrieval models to automate evidence selection while preserving the zero‑retrieval‑call guarantee that SCH currently offers.
Source: https://arxiv.org/abs/2611.24357
Frequently asked questions
How does Semantic Context Handoff improve token efficiency?
SCH keeps only a compact semantic state in the active prompt and moves older action‑observation records to an external archive. By avoiding the repeated inclusion of the full interaction history, the prompt size shrinks, leading to up to 43% fewer tokens per task while still allowing the model to pull back evidence when needed.
Does SCH require additional retrieval‑model calls to fetch evidence?
No. The evidence archive is addressed directly by the controller, which selects relevant records based on provenance links stored with the semantic state. This means SCH does not invoke separate retrieval models, preserving inference speed.
Can SCH be applied to domains beyond software engineering, such as multimodal agents?
The authors designed SCH as a general abstraction for any long‑horizon agent that accumulates action‑observation data. While the paper’s experiments focus on coding benchmarks, the state–evidence separation could be adapted to multimodal workflows, robotics, or dialogue systems where historical evidence may become relevant after many steps.
Source: Grafy Research Lab — Semantic Context Handoff with On-Demand Evidence for Long-Horizon Agents


