Clover-35B-A3B-Preview Sets Benchmark for Diffusion Language Models
Clover-35B-A3B-Preview, a 35 billion‑parameter diffusion language model that activates just 3 billion weights.
Source: Grafy Research LabClover-35B-A3B-Preview is a new diffusion‑based language model introduced in a technical report released on 19 November 2026. The model contains a total of 35 billion parameters but activates only about 3 billion of them for any given generation step, making it a mixture‑of‑experts (MoE) system that scales size while keeping compute modest.
The authors adapt Gated Delta Networks to the diffusion setting. Each generation block reads a recurrent memory that stores tokens generated earlier, and it combines this memory with a bidirectional softmax attention over the tokens inside the current block. The historical memory readout and the current‑block tokens compete in a shared softmax distribution, while a learnable block‑size calibration adjusts how many tokens belong to the block. An additional right‑context convolution branch, initialized to zero, provides a lightweight pathway for future context.
In experiments the model, referred to as “Clover”, was evaluated against seven other diffusion language models. The report states that Clover achieved the best scores across the tested metrics, demonstrating that the gated‑delta diffusion approach can improve both fluency and factual consistency, especially when the model is asked to reason step‑by‑step in “thinking‑mode” generation.
Developers who want to explore Clover have a few practical options. If the model weights are released publicly (the report links to a code repository and weight files), they can be downloaded and loaded into a custom inference environment. Grafy’s Code studio can be used to set up a Python environment with the necessary libraries (e.g., PyTorch, transformers) and run inference scripts provided by the authors. For users who prefer not to manage the infrastructure, the model may become available through model‑hosting services such as OpenRouter; in that case Grafy Chat can be pointed at the model’s endpoint to experiment with prompt‑to‑output behavior directly in the browser.
The report also highlights a “thinking‑mode” where the model is instructed to generate a chain of internal reasoning before producing the final answer. This mode leverages the recurrent memory and the gated‑delta mechanism to keep earlier reasoning steps accessible throughout the generation, which can be valuable for tasks that require multi‑step deduction or planning.
Relevance for Grafy users
While Grafy does not yet host Clover as a built‑in model, the platform offers tools that let creators and researchers work with it. The Reader app can ingest the technical report PDF, allowing you to ask natural‑language questions about the architecture, training data, or evaluation results. The Chat interface can be configured to query a hosted instance of Clover, enabling rapid prototyping of prompts that use thinking‑mode generation. Finally, the Graph app can help you map the flow of tokens between memory, attention, and convolution branches, visualising how the model’s internal state evolves during generation.
Overall, Clover‑35B‑A3B‑Preview demonstrates that diffusion‑based language models can benefit from expert routing and gated‑delta memory, opening new possibilities for high‑quality, compute‑efficient text generation. Researchers and developers interested in next‑generation reasoning capabilities should consider testing the model in their own pipelines.
Source: https://arxiv.org/abs/2611.24636
Frequently asked questions
How many parameters does Clover-35B-A3B-Preview use during generation?
Clover has a total of 35 billion parameters, but its mixture‑of‑experts design activates only about 3 billion of them for each generation step, reducing the compute required while preserving model capacity.
What is “thinking‑mode” generation in Clover?
Thinking‑mode asks the model to produce an explicit chain of reasoning before the final answer. The recurrent memory and gated‑delta attention keep earlier reasoning steps available, helping the model maintain context across multiple inference steps.
Can I run Clover on Grafy without external hosting?
Grafy does not currently provide a built‑in Clover endpoint. You can upload the model weights to your own server or a cloud service and then connect Grafy Chat to that endpoint, or use Grafy Code to run inference locally.
Source: Grafy Research Lab — Clover-35B-A3B-Preview Technical Report


