AntEmbed Introduces Dual‑Mode Multimodal Embeddings for Faster, More Precise Retrieval

AntEmbed, a new dual‑mode multimodal embedding model, was announced by a team from Soochow University and Ant Group on 13 Nov 2026. The model claims to unite the speed of discriminative embeddings with the fine‑grained semantic understanding of generative embeddings, offering a single retriever that can handle both large‑scale low‑latency queries and complex, reasoning‑heavy searches.

What problem does AntEmbed aim to solve?

Current multimodal retrieval systems fall into two camps. Discriminative embeddings directly encode raw inputs into vectors, delivering low encoding overhead and fast lookup, but they often miss subtle semantic cues needed for nuanced queries. Generative embeddings, by contrast, generate intermediate reasoning (often called Chain‑of‑Thought, or CoT) that captures query intent and relevance evidence, leading to richer representations at the cost of higher inference latency. According to the paper, “the primary representation paradigm for large‑scale online retrieval” is discriminative, yet “generative embeddings … provide richer semantic representations for complex queries.” AntEmbed seeks to combine these strengths in a unified embedding space, eliminating the need for separate indexes or multiple models.

How does AntEmbed work?

The authors describe a two‑stage training pipeline.

1. Multi‑positive contrastive learning – For each query, AntEmbed treats the original candidate and a reasoning‑enhanced version of the same candidate as semantically equivalent views. Both views are placed into the positive set for the query, and a single predefined embedding token is used to produce a vector. By pulling together the two views, the model aligns discriminative and generative embeddings within the same space.
2. Cross‑mode self‑distillation – After the first stage, the reasoning‑enhanced branch serves as a teacher. The model distills fine‑grained semantic information into the original (discriminative) branch through two mechanisms: candidate similarity‑distribution distillation and query‑representation alignment. This lets the discriminative embeddings inherit reasoning benefits without incurring additional inference steps.

The paper also introduces confusion‑aware CoT construction, a process that adds intent‑focused and confusion‑aware semantic evidence to the reasoning‑enhanced view, ensuring the generated content directly supports retrieval.

What performance gains does AntEmbed deliver?

AntEmbed was evaluated on the MMEB‑v2 benchmark, a suite covering image retrieval, visual‑document retrieval, and other multimodal tasks. The 8‑billion‑parameter version (AntEmbed‑8B) achieved a state‑of‑the‑art overall score of 81.5 with its generative embeddings, outperforming existing generative and discriminative baselines of comparable size. While the discriminative embeddings lag slightly behind the generative ones, they remain competitive and surpass several generative baselines, demonstrating that the self‑distillation step successfully narrows the performance gap.

Figure 1 in the paper (referenced in the abstract) shows AntEmbed consistently beating competing models across domains, and Figure 2 illustrates how treating original and reasoning‑enhanced inputs as a single positive set reduces the representation gap that prior dual‑token approaches (e.g., UME‑R1) suffered from.

Why does this matter for real‑world retrieval systems?

Large‑scale services—search engines, e‑commerce catalogues, multimedia archives—must balance two competing demands: throughput (fast encoding and lookup) and precision (understanding nuanced user intent). Traditional pipelines either sacrifice speed for richer semantics or maintain speed at the expense of relevance. AntEmbed’s unified space means a single index can serve both needs. In practice, a system could default to discriminative embeddings for the majority of queries, switching to the reasoning‑enhanced view only when a query is flagged as complex. The cross‑mode self‑distillation also reduces the storage overhead that earlier dual‑token designs required, because only one embedding token per item is stored.

How can Grafy users experiment with AntEmbed?

While Grafy does not yet host AntEmbed as a built‑in model, the research can be explored through existing tools:

• Grafy Reader (/reader) lets you upload the PDF, run keyword search, and ask follow‑up questions about the methodology or results.
• Grafy Chat (/chat) can be used to discuss the paper with a multimodal model, generate prompts that mimic the CoT reasoning step, or prototype a simplified version of AntEmbed’s reasoning‑enhanced view.
• Grafy Graph (/graph) enables you to map the two‑stage training flow—contrastive learning ➜ self‑distillation—as a visual prompt graph, helping you design your own pipeline or adapt the ideas to other domains.

By combining these apps, creators, developers, and researchers can prototype a lightweight version of AntEmbed, test multi‑positive contrastive loss on their own data, and evaluate whether cross‑mode distillation improves their retrieval quality.

What are the next steps and open questions?

The authors note two primary avenues for future work. First, scaling the model beyond 8 B parameters could further close the gap between discriminative and generative embeddings. Second, extending the confusion‑aware CoT generation to additional modalities (e.g., audio or video) may broaden the applicability of the dual‑mode approach. Open questions remain around how to efficiently trigger the reasoning‑enhanced path only for queries that truly benefit, and how to integrate AntEmbed with existing vector‑search engines that expect a single embedding per document.

To explore this yourself, open Reader: read, search and question PDFs and papers with AI.

Frequently asked questions

How does AntEmbed differ from earlier dual‑token models like UME‑R1?

AntEmbed treats the original and reasoning‑enhanced inputs as semantically equivalent positives within a single contrastive loss, using one embedding token. UME‑R1 uses separate tokens for discriminative and generative modes, resulting in lower similarity between the two views and requiring distinct indexes.

Does AntEmbed eliminate the need for separate generative embeddings at inference time?

Yes. After cross‑mode self‑distillation, the discriminative embeddings inherit the fine‑grained semantics of the generative branch, allowing the system to rely on a single embedding token for both fast retrieval and nuanced matching.

Can the AntEmbed training pipeline be applied to domains beyond vision‑language, such as audio or text‑only retrieval?

The paper focuses on multimodal vision‑language inputs, but the core ideas—multi‑positive contrastive learning and cross‑mode self‑distillation—are modality‑agnostic. Researchers could adapt the confusion‑aware CoT construction to other data types, provided a suitable reasoning generator is available.

---

Source: Grafy Research Lab — AntEmbed: Dual-Mode Multimodal Embedding Model for Efficient and Fine-Grained Retrieval