Shopify launches SID-LLM, a unified generative model for search and recommendation

Shopify unveiled a new large language model, SID‑LLM, that treats product search and recommendation as a single generative task: producing short, content‑derived semantic IDs. The approach promises faster, more relevant product discovery across a catalog that now spans billions of items.

What is SID‑LLM and how does it generate semantic IDs?

SID‑LLM stands for Semantic ID Large Language Model. Instead of returning a ranked list of product vectors, the model predicts a sequence of tokens that uniquely identify products—"semantic IDs" (SIDs). Each SID is created by quantizing a product’s content embedding into a compact codebook, so related items share token fragments. During inference, the model receives a prompt (a query, a buyer profile, or interaction history) and autoregressively or in parallel emits the SID tokens. Constrained decoding then maps those tokens back to eligible products in the live catalog before downstream ranking. This formulation lets a single model handle three retrieval scenarios: non‑personalized search, personalized search, and recommendation without separate pipelines (according to the arXiv paper).

How does generative semantic‑ID retrieval differ from traditional methods?

Traditional e‑commerce retrieval stacks combine lexical matching (e.g., BM25), dense embedding search, and two‑tower candidate generators. Lexical match excels at exact brand or SKU terms but suffers from vocabulary mismatch. Dense retrieval encodes queries and products into a shared vector space, enabling approximate nearest‑neighbour lookup, yet it still treats the query and product representations independently. SID‑LLM flips the script: the model directly generates identifiers conditioned on the full context, allowing the language backbone to reason over the entire prompt while sharing token sub‑structures across similar products. This reduces the need for separate embedding indexes and enables a unified training objective that aligns search and recommendation signals.

What performance improvements did Shopify see in production?

Shopify evaluated SID‑LLM both as a stand‑alone retriever and as a component of its live search and recommendation stacks. Offline benchmarks showed that the model matched the quality of the existing multi‑model stack on standard metrics such as OneDCG for search and recommendation recall. In online A/B tests, integrating SID‑LLM yielded statistically significant lifts in click‑through and purchase recall while keeping request latency within the strict sub‑100 ms budget required for billions of daily queries. The paper notes that multi‑token prediction paired with a lightweight ranker further improved retrieval quality and reduced latency compared with a pure autoregressive baseline.

How is the model trained and refined?

Training proceeds in three phases. First, a tokenizer is built by quantizing product embeddings into a codebook; design choices such as codebook size and collision handling are explored in the paper’s Section 3. Next, SID‑LLM is jointly pre‑trained on a mixture of catalog attribute data, historical search sessions, recommendation interaction streams, and general‑domain replay data. This joint pre‑training aligns the language model with both product content and user behavior. After the base model is ready, the authors apply supervised fine‑tuning on task‑specific signals, followed by preference optimisation using reinforcement learning (offline CE + DPO and GRPO) to boost judged relevance. Finally, task‑specific expert models are distilled into the unified SID‑LLM, preserving recommendation quality without harming search performance.

How can creators and researchers explore similar ideas with Graf AI?

While Grafy does not yet host a built‑in semantic‑ID tokenizer, the concepts can be examined using two of its core apps. The Reader app lets you upload the SID‑LLM paper and interactively query its sections, making it easy to dive into the tokenizer construction, training recipe, and evaluation methodology. For hands‑on experimentation, you can use Chat to prototype a simple generative retriever: feed a product description as context and ask the model to output a short token‑like identifier, then resolve it against a small catalog stored in a spreadsheet or CSV via the Files tools. This workflow mirrors the SID‑LLM pipeline on a manageable scale and helps you understand the trade‑offs between token generation and traditional vector search.

Frequently asked questions

How does SID‑LLM handle a catalog that changes multiple times per day?

SID‑LLM relies on an eligibility‑constrained decoding step that consults a live catalog index at inference time. When products are added or removed, the index is refreshed, ensuring generated SIDs always map to currently available items. The paper’s deployment section describes how this index is kept in sync with Shopify’s rapid catalog turnover.

Does reinforcement learning improve search relevance for all query types?

According to the authors, reinforcement learning (using offline CE + DPO and GRPO) primarily boosts judged relevance for personalized and non‑personalized search. The gains are measured on standard relevance metrics and were confirmed in live traffic, though the improvement for pure lexical queries is smaller because those already perform well with term matching.

Can the semantic‑ID approach be applied to domains outside e‑commerce?

The underlying idea—generating compact identifiers that encode content semantics—does not depend on product data. Any large item catalog with rich textual or visual attributes (e.g., news articles, multimedia assets, or scientific papers) could benefit from a similar generative retrieval model, provided a suitable tokenizer and eligibility constraints are defined.

---

Source: https://arxiv.org/abs/2611.08804

---

Source: Arxiv IR Recent — Generative Search and Recommendation at Shopify