ThinkingCap trims Gemma-4 reasoning tokens 39% at flat accuracy

BottleCap AI has published ThinkingCap-gemma-4-26B-A4B-it, a retrained version of Google's gemma-4-26B-A4B-it that, according to the company, uses 38.9% fewer thinking tokens on average across twelve benchmarks while giving up 0.15 percentage points of accuracy. For teams paying per token, that is the trade worth understanding: the reasoning trace is the bill, and a shorter trace at the same answer quality is a cheaper answer.

What did BottleCap AI release?

ThinkingCap is BottleCap AI's family of token-efficient reasoning models, and this is the fifth release. According to the company's announcement, it follows ThinkingCap-Qwen3.6-27B, ThinkingCap-Qwen3.8-27B, ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B and ThinkingCap: Qwen3.8-Flash-Next, and it is the first built on a Google model — the third model family the company has applied the technique to, after Qwen and NVIDIA. The post surfaced on the Hacker News front page.

The base model is Google's sparse mixture of experts: 26 billion parameters with about 4 billion active per token. The ThinkingCap build ships as bottlecapai/ThinkingCap-gemma-4-26B-A4B-it with GGUF builds alongside, and BottleCap describes it as a drop-in replacement that inherits Google's recommended sampling preset unchanged and runs on the same serving stack, vLLM. A PRO tier is offered separately, which the company says uses up to 87.6% fewer thinking tokens and 62% on average for enterprise deployments.

How much thinking did it cut?

The average mean thinking-token count falls from 8,773 to 5,398 across the twelve benchmarks. BottleCap reports that as a 38.9% average reduction, and is careful about a distinction worth keeping: 38.9% is the mean of the twelve per-benchmark reductions, while the ratio of the two token totals is 38.5%.

The cuts are not evenly spread. LongBench v2 falls 69.1%, AA-LCR 59.3% and MMLU-Pro 49.1%, while IFBench falls 13.7% and τ²-bench 20.8%. BottleCap's explanation is that the shallow rows are the tasks least about thinking at length — following a list of format constraints, and applying a fixed customer-service policy through the right tool calls.

Two side effects matter more than the average. Truncated responses, where the reasoning never closes and the answer scores zero, drop from 4.35% to 0.72%. Looping, caught by a compression-based detector, drops from 3.64% of the base model's responses to 0.76%; on the three maths benchmarks where looping is worst, it falls from 5.5%, 8.6% and 17.0% to 0.9%, 1.2% and 4.5%.

What did it cost in accuracy?

Averaged over the twelve benchmarks, accuracy moves from 75.18% to 75.03%, a change of -0.15 percentage points. BottleCap calls that the smallest accuracy cost of any ThinkingCap so far, and notes it is smaller than the run-to-run spread on most of the suite.

By task family, instruction following and agentic gains 0.65pp and maths gains 0.39pp; code is flat at -0.02pp; knowledge and reasoning costs 0.51pp and long-context and multimodal 0.92pp. On eight of the twelve individual benchmarks the gap sits inside the run-to-run spread, and five come out ahead of the original model. Four move beyond their intervals: IFBench at +1.83pp in BottleCap's favour, GPQA-Diamond at -0.96pp, MMMLU at -0.66pp and AA-LCR at -4.00pp. The company names AA-LCR as the one real cost in the suite.

How much should you trust the numbers?

The methodology is unusually explicit for a vendor post. Both models ran in the same harness on vLLM, with thinking enabled, temperature 1.0, top_p 0.95 and top_k 64, and generation capped at 65,536 tokens per response. The base model was measured by BottleCap rather than quoted from published figures. Accuracy is the mean across seeds with a 95% t-interval: 64 seeds on the AIME and HMMT maths sets, 32 on GPQA-Diamond and LiveCodeBench, 16 on RealWorldQA, IFBench and τ²-bench, 8 on AA-LCR and LongBench v2, and 4 on the two large multiple-choice sets.

Two caveats. This is a single vendor's own harness, with no independent replication mentioned. And the AA-LCR figure is the kind of number that deserves attention from anyone whose workload is long-context and retrieval-heavy.

What does this mean in practice?

Three things. Cost: thinking tokens are billed tokens, so a 39% average reduction in reasoning length is a direct cut to the line item that dominates reasoning-model inference. Latency: shorter traces mean faster responses, which matters for interactive agents and for anything running under a timeout. Stability: fewer truncated and looping responses means fewer retries and fewer dead ends inside agent loops.

BottleCap also plots accuracy against token budget, which is the question a deployment actually asks: for a given budget per question, how much accuracy do you get? Its curve sits above the original model's through the middle of the range and converges once the budget is wide enough that neither model is constrained.

The catch is variance. The average hides a wide spread — long-context work saves the most and loses the most, while format-following work saves little. Test on your own traffic before swapping.

Where this fits in Grafy

Grafy runs many cloud models side by side, so the practical way to test a claim like this is to hold a prompt constant and compare. In Grafy Chat you can make exactly the comparison BottleCap makes — same question, different thinking budget — and read the reasoning traces next to each other.

For coding work, Grafy Code matches the benchmark that held steadiest: LiveCodeBench is flat at -0.02pp while using 27.6% fewer thinking tokens, which makes a coding assistant the least risky place to try a token-efficient reasoning model. For long documents, Reader is where the savings are largest and the risk is real: the long-context and multimodal family averages a 54.4% cut, but AA-LCR is the one benchmark where BottleCap reports a genuine accuracy loss, at -4.00pp.

At the app layer, the same idea already appears in Grafy's lowest-cost model routing — spend the cheapest model that can answer. ThinkingCap makes one model cheaper; routing picks a cheaper model. Whether a ThinkingCap build is reachable through any of Grafy's model providers is worth checking in the model picker inside Chat before planning around it. If it is not listed, the method still transfers: run your own prompts, count the reasoning, compare the answers.

Frequently asked questions

What is ThinkingCap-gemma-4-26B-A4B-it?

It is BottleCap AI's retrained version of Google's gemma-4-26B-A4B-it, a sparse mixture-of-experts model with 26 billion parameters and about 4 billion active per token. According to the company, the training targets one thing: spend fewer tokens reaching the answer while preserving knowledge, reasoning ability, answer quality, instruction following and safety behaviour. It ships with GGUF builds and is described as a drop-in replacement under Google's recommended sampling preset.

Does thinking less actually cost accuracy?

On BottleCap's own benchmarks, barely. Averaged over twelve tests, accuracy moves from 75.18% to 75.03%, a 0.15 percentage-point drop, and eight of the twelve gaps fall inside the run-to-run spread of the seeds. Five benchmarks improve. The exception is AA-LCR, a long-context retrieval test, which loses 4.00 points — the one result the company itself flags as a real cost. Treat the average as the headline and AA-LCR as the warning label.

How should I evaluate a token-efficient reasoning model myself?

Replicate the setup rather than trusting the headline. Use one harness for both models, keep sampling identical, and run several seeds per benchmark, because reasoning models vary run to run at temperature 1.0. Report accuracy as a mean with a confidence interval, and track thinking tokens, truncated responses and looping alongside accuracy. BottleCap also plots accuracy against token budget, which is the view that matters in production: the same accuracy for a smaller budget.

---

Source: Hacker News — front page — Gemma-4-26B-A4B with 39-69% less thinking at same accuracy