Browser‑Based LLM Detects Silent Order Errors with Zero Data Leakage
A fine‑tuned 0.8B‑parameter model runs entirely in a web page, spotting hidden order mistakes without sending any text off‑device.
Source: Grafy Research LabA new research demo shows a 0.8 billion‑parameter language model that parses and validates typed orders entirely inside a visitor’s browser tab, preventing any text from leaving the device. According to the Grafy Research Lab paper (arXiv:2611.18284), the system highlights a subtle failure mode – silent errors – where the model outputs a well‑formed order that was never actually placed.
What problem does the demo address?
Traditional web‑based form assistants either reject malformed output or, worse, silently produce a plausible‑looking result that never matches the user’s intent. The authors call the latter a silent error because downstream systems accept the clean output, masking the mistake. In domains like food delivery or ticket reservation, such hidden mismatches can lead to costly refunds or unhappy customers.
How does the model work inside the browser?
The team fine‑tuned a compact 0.8 B‑parameter transformer on a dataset of order‑to‑structure mappings. Using llama.cpp compiled to WebAssembly, the model runs client‑side without any server round‑trip. As a user types, the page continuously parses the input into a structured order object. Each ticket is then scored with the PIZZA benchmark’s matcher, which the authors ported to TypeScript for on‑device execution. Because all computation stays in the browser, no personal data is transmitted.
What are silent errors and why are they dangerous?
A silent error occurs when the model produces a syntactically correct order that differs from the user’s actual request. Unlike outright parsing failures, these outputs pass validation checks, so downstream order‑fulfillment pipelines treat them as genuine. The paper demonstrates that even with a modest model size, silent errors appear regularly, especially when the input deviates slightly from training patterns. Detecting them requires a secondary verification step – the PIZZA matcher – which flags mismatches between the inferred structure and the raw text.
How was the demo built and can it be reproduced?
The authors released a public web demo (link in the paper) that loads the WebAssembly module on demand. The workflow is:
- Load the compiled llama.cpp binary.
- Initialize the fine‑tuned 0.8 B model.
- Capture keystrokes and feed them to the model in real time.
- Convert the model’s token output into a JSON order representation.
- Run the TypeScript‑based PIZZA matcher to assign a confidence score. All of these steps are open‑source, enabling developers to embed a similar privacy‑preserving parser in any web app.
Why does this matter for creators and developers using Grafy?
Grafy’s suite already supports on‑device AI through several apps. While the demo itself is a custom web page, the underlying concepts map directly to existing tools:
- Grafy Chat (/chat) lets you experiment with multiple models side‑by‑side, making it easy to compare a tiny on‑device model against larger cloud alternatives.
- Grafy Reader (/reader) can ingest the arXiv paper and let you ask questions about the methodology without leaving the platform. Developers can prototype similar parsers using the Grafy Code studio, which supports JavaScript/TypeScript execution and can import WebAssembly modules. By combining these apps, you can build privacy‑first form assistants that run entirely in the user’s browser, mirroring the research demo’s approach.
What are the next steps for this line of research?
The authors suggest two avenues:
- Model scaling – exploring whether larger, yet still efficient, models reduce silent‑error rates without sacrificing on‑device performance.
- Benchmark extensions – enhancing the PIZZA matcher to cover more complex order domains (e.g., multi‑item combos, custom modifiers) and integrating it with other client‑side validation libraries. Future work may also investigate hybrid schemes where a lightweight on‑device model performs a first pass, and only ambiguous cases are escalated to a secure server for deeper analysis.
Frequently asked questions
How does the browser‑based model keep user data private?
All processing happens client‑side using WebAssembly, so the raw order text never leaves the user’s device. Only the model’s internal tokens are used locally, and the final structured order is validated in‑browser before any network request.
What distinguishes a silent error from a regular parsing failure?
A regular failure produces malformed output that downstream systems reject outright. A silent error yields syntactically correct output that appears valid, allowing the mistake to propagate unnoticed until a secondary check, like the PIZZA matcher, flags the inconsistency.
Can I try the demo without building anything myself?
Yes. The paper provides a public URL to the live demo, where you can type orders and watch the model’s structured output and confidence scores in real time. For deeper exploration, you can load the source code and run it locally using the same WebAssembly build.


