Grafy News
Try Grafy free

IntentFlow introduces intent-driven multimodal search for Xiaohongshu

Researchers released IntentFlow, a real-time multimodal search system that uses image-text queries and explicit intent modeling to improve note retrieval.

Illustration for the news story: IntentFlow introduces intent-driven multimodal search for XiaohongshuSource: Grafy Research Lab

What is IntentFlow and why does it matter?

IntentFlow is a real‑time multimodal search system designed for the Chinese user‑generated content platform Xiaohongshu. According to the Grafy Research Lab paper, the system accepts combined image‑text queries and explicitly models the user's retrieval intent, rather than relying solely on visual or topical similarity. This matters because UGC communities contain massive, loosely structured notes where relevance is often intent‑dependent; a better alignment between query and note can surface content that would otherwise be missed.

How does intent‑driven multimodal search differ from traditional visual or textual search?

Traditional multimodal search pipelines typically retrieve candidates based on visual similarity or keyword overlap, then optionally rerank them. The authors point out that such an approach fails when the most relevant notes are not visually similar to the query image or do not share obvious keywords. By introducing an explicit intent model, IntentFlow aligns both the candidate retrieval stage and the reranking stage to a consistent relevance criterion that reflects what the user actually wants to find.

What architecture does IntentFlow use to model user intent?

The paper describes a two‑stage framework. First, a retrieval module generates a broad candidate set from a large index of notes. Second, an intent‑modeling component evaluates each candidate against the inferred user intent derived from the combined image‑text query. This alignment ensures that notes missing from the initial visual shortlist can still be considered if they match the intent signal. The authors emphasize that the intent model is trained jointly with the retrieval engine so that both stages share a common relevance definition.

What were the key findings of the study?

In their 29‑page technical report (including seven figures), the researchers demonstrate that simply reranking visual results does not recover all intent‑relevant notes. By contrast, IntentFlow’s explicit intent modeling improves recall of relevant notes without sacrificing precision. The experiments on Xiaohongshu data show that notes aligned with the user's retrieval intent are often absent from the initial candidate pool, confirming the need for intent‑aware retrieval.

How can creators and developers experiment with intent‑driven multimodal search using Grafy?

Grafy offers several tools that let you explore the concepts behind IntentFlow without writing custom code. You can upload the arXiv PDF to Grafy Reader and ask the AI to summarize the architecture, extract the intent‑modeling algorithm, or generate a visual diagram of the pipeline. With Grafy Chat you can converse with multiple models side‑by‑side to probe how different prompts affect retrieval intent. If you want to prototype prompt‑based retrieval workflows, the Grafy Graph app lets you build a branching graph of image‑text queries, model outputs, and reranking steps, mirroring IntentFlow’s two‑stage process. Finally, Grafy Creator can generate synthetic image‑text queries to test your own retrieval pipelines.

Where to learn more and what to try next?

The full technical report is available on arXiv (https://arxiv.org/abs/2611.20448). Readers interested in deeper technical details should consult the 29‑page document, which includes implementation specifics and evaluation metrics. Within Grafy, you can start by loading the paper into Reader, asking Chat to explain the intent‑modeling loss function, and then using Graph to experiment with alternative retrieval strategies. This hands‑on approach lets developers and researchers apply IntentFlow’s ideas to their own multimodal datasets.

Source: https://arxiv.org/abs/2611.20448


Source: Grafy Research Lab — IntentFlow: An Intent-Driven Real-Time Multimodal Search System with Image-Text Queries at Xiaohongshu