Grafy News
Try Grafy free

Thinking Machines launches Inkling 2 Mini for local AI reasoning

Thinking Machines Lab released Inkling 2 Mini, a looped transformer model designed for local deployment on laptops without sacrificing reasoning power.

Illustration for the news story: Thinking Machines launches Inkling 2 Mini for local AI reasoningSource: VentureBeat — AI

Thinking Machines Lab has released Inkling 2 Mini, a new open-weights multimodal model designed to provide high-level reasoning capabilities on consumer hardware. By utilizing a novel "looped transformer" architecture, the model aims to challenge the necessity of massive cloud-based infrastructure for complex agentic tasks.

How does the looped transformer architecture work?

Standard transformer architectures typically process data in a single, linear pass through a sequence of layers. If a task requires deep reasoning, these models often must generate long, text-based "chains of thought" to compensate, which increases latency and cost. According to VentureBeat, Inkling 2 Mini avoids this by cycling data through its middle layers multiple times per token.

The model is a 24-layer, decoder-only sparse Mixture-of-Experts (MoE) system. In this configuration, the first six layers (the prelude) and the final six layers execute once. However, the middle 12 layers are run three times using shared weights. This process results in 48 effective layer applications per token and 5.4 billion effective active parameters, even though the total parameter count is 26.4 billion. To maintain coherence during these iterations, the developers implemented attention residuals, allowing each pass to access a gated message from the previous iteration.

What are the performance benchmarks for Inkling 2 Mini?

Despite its compact size and ~20GB memory footprint, Inkling 2 Mini demonstrates competitive performance in specialized domains. The model is engineered to run offline on a 32GB MacBook Pro or a single desktop GPU.

In software engineering benchmarks, the model shows significant strength. According to VentureBeat, it leads sub-100B open-weights models on DeepSWE v1.1 with a score of 48.2% and on SWE-bench Pro v2 Hard with 77.9%. It also achieved 85.3% on DeepSearchQA. When measured on Stanford's intelligence-per-joule benchmark, the model scored 0.703, outperforming both Qwen3.6-35B-A3B and Gemma-4-26B-A4B.

However, the model is not a universal replacement for larger dense models. Because factual recall is closely tied to total parameter count, Inkling 2 Mini trails denser competitors like Qwen3.8-27B in areas such as the AA-Omniscience v1.1 benchmark. It also shows lower scores in multi-turn chat and complex multi-constraint instruction following compared to its larger peers.

Is the model safe for local deployment?

Internal reasoning that does not manifest as written text has raised questions regarding model oversight. To address this, Thinking Machines released a companion variable-loop checkpoint, TML-d24-varloop-research, to assist in studying mechanistic interpretability.

In preliminary monitorability tests, the company found that while looping allowed the model to extend its reasoning horizon—increasing from 16 seconds at one pass to 37 seconds at three passes—the reasoning traces remained human-legible. Furthermore, the model showed no evidence of covert sabotage on the LinuxArena control benchmark, suggesting that internal "thinking" does not necessarily obscure the model's intent from human observers.

How can developers use Inkling 2 Mini?

Inkling 2 Mini is released under the Apache 2.0 license, making it highly accessible for both research and commercial applications. This permissive license allows users to modify, fine-tune, and redistribute the model without the heavy restrictions found in some other open-weight agreements.

To optimize performance on consumer silicon, the team used specific precision techniques during training. The first 20 layers were trained in 4-bit NVFP4 precision, while the final four layers used 8-bit MXFP8 precision. Because the weights are shipped in these exact formats, developers can deploy the model without the accuracy loss typically associated with post-training quantization. Early community support has already provided quantized GGUF builds and support for runtimes like llama.cpp and Ollama.

How does this impact the AI development landscape?

The release marks a significant shift toward "local-first" agentic AI. By proving that a 26B parameter model can handle complex tasks like building a flight simulator or managing autonomous simulations on a laptop, Thinking Machines is providing a blueprint for high-performance, low-cost AI deployment.

For developers working on autonomous agents or coding assistants, this architecture offers a way to reduce reliance on expensive API calls while maintaining the reasoning depth required for sophisticated software engineering. The ability to run such a model in "airplane mode" suggests a future where high-intelligence agents are truly portable and privacy-centric.

For those looking to apply these types of agentic coding capabilities in a managed environment, Grafy Code provides a dedicated AI coding studio. While Grafy utilizes various high-performance cloud models to power its workspace, the principles of efficient, agentic reasoning seen in Inkling 2 Mini are central to the workflows supported in our coding and development tools.

Frequently asked questions

What is the primary architectural difference in Inkling 2 Mini?

Inkling 2 Mini uses a looped transformer architecture instead of a standard linear pass. It executes its middle 12 layers three times per token, which allows the model to perform deeper reasoning and achieve more effective active parameters without increasing the total storage size or parameter count.

Can Inkling 2 Mini run on a standard laptop?

Yes, the model is specifically engineered for local deployment. It is designed to run offline on hardware such as a 32GB MacBook Pro or a single desktop GPU, making it highly accessible for developers who want to run powerful agentic models without a constant cloud connection.

What license is used for Inkling 2 Mini?

The model is released under the Apache 2.0 license. This is a highly permissive open-source license that allows researchers and commercial enterprises to use, modify, and redistribute the model in proprietary products without paying royalties, provided the original copyright notice is included.


Source: VentureBeat — AI — Thinking Machines wows with Inkling 2 Mini looped transformer model that runs locally on laptops