Ant Ling releases Ling-3.2-flash multimodal MoE model

Ant Ling has announced the release of Ling-3.2-flash, a new native multimodal model designed with a Mixture-of-Experts (MoE) architecture. This release introduces a significant scale of parameters alongside a massive context window, aiming to handle diverse data types within a single framework.

What are the technical specifications of Ling-3.2-flash?

According to a report from Reddit ML, Ling-3.2-flash is built on a Mixture-of-Experts architecture that utilizes a total of 560 billion parameters. However, the model is optimized for efficiency, activating only 25 billion parameters during inference. A standout feature of this model is its ability to process a maximum context window of up to 1 million tokens.

The model is described as natively multimodal, meaning its documentation specifies that it can accept and process text, image, and video inputs directly. This integration allows the model to reason across different media formats within its large context window.

How did the model perform in evaluations?

The release documentation provides several benchmark scores to demonstrate the model's capabilities across various domains. In mathematical and visual reasoning, the model achieved a MathVision (w/CI) score of 96.80. For complex reasoning tasks, it recorded a score of 30.12 on the Humanity's Last Exam-MM (HLE-MM) multimodal subset.

In specialized testing environments, the model showed the following results:

• Document and Data Analysis: It scored 92.21 on OmniDocBench 1.5 and 85.15 on CharXiv_RQ. It also recorded a score of 22.25 on the GDP.pdf benchmark.
• Visual and Web Intelligence: The model reached 85.86 on Vision2Web and 85.63 on AndroidWorld. It also scored 69.85 on BabyVision.
• Technical and Medical Benchmarks: On BlenderBench, it scored 67.77. In medical contexts, MedXpertQA-MM received a score of 79.93, while AntBench-Medical reported a correctness of 99.22% and a comprehensiveness of 89.52%.
• General Benchmarking: The model achieved 18.00 / 43.71 on ZEROBench / ZEROBench(Sub).

Why does a Mixture-of-Experts architecture matter?

A Mixture-of-Experts (MoE) architecture is a method used to scale model capacity without a proportional increase in the computational cost required for every single query. By having a large total parameter count (560B in this case) but only activating a fraction of them (25B) for any given task, the model can theoretically possess vast knowledge while maintaining faster inference speeds and lower energy requirements than a dense model of the same total size.

When combined with native multimodality, this means the model can switch between "expert" pathways to handle a text prompt, a video frame, or a complex mathematical image with high efficiency. The 1M token context window further enables the model to maintain these connections over very long documents or extended video sequences.

How can researchers use these multimodal capabilities?

The ability to ingest video and large-scale text simultaneously opens doors for deeper automated research. Researchers can feed entire series of papers or long-form video lectures into the model to extract nuanced connections.

For those looking to analyze large datasets or dense academic papers, you can explore these concepts using Grafy Reader. While we do not currently host the Ling-3.2-flash model specifically, the Reader app allows you to upload PDFs and papers to conduct AI-driven research, search, and questioning, mirroring the type of high-context document analysis that models like Ling-3.2-flash are designed to facilitate.

Frequently asked questions

What is the total parameter count of Ling-3.2-flash?

Ling-3.2-flash features a total of 560 billion parameters. However, because it uses a Mixture-of-Experts (MoE) architecture, it only activates 25 billion parameters for any given task, which helps optimize the computational efficiency of the model during use.

Does Ling-3.2-flash support video input?

Yes, Ling-3.2-flash is a native multimodal model. According to its official documentation, the model is capable of processing text, image, and video inputs, allowing it to reason across different types of media within its 1 million token context window.

What is the context window size for this model?

The maximum context window for Ling-3.2-flash is 1 million tokens. This large capacity allows the model to process and retain a significant amount of information, such as very long documents or extensive video files, in a single session.

---

Source: Reddit ML — Ant Ling introduces Ling-3.2-flash: 560B/25B native multimodal MoE with 1M context [N]