Grafy News
Try Grafy free

MOSAIC Sets New Standard for Multimodal Molecular Generation

A new transformer model, MOSAIC, can generate any combination of 3D structures, UV.

Illustration for the news story: MOSAIC Sets New Standard for Multimodal Molecular GenerationSource: Grafy Research Lab

What is MOSAIC and why it matters?

The research team behind Lila Sciences introduced MOSAIC, a multimodal flow‑matching transformer that jointly models 3‑dimensional molecular geometry and three types of optical spectra (UV, IR, Raman). According to the arXiv pre‑print, the model can take any subset of these modalities as input and generate the remaining ones, a capability the authors call any‑to‑any generation. This matters because experimental chemistry often yields several coupled measurements from a single costly experiment; MOSAIC extracts supervision from every measurement instead of discarding it, potentially reducing the amount of data needed to train high‑quality generative models.

How does any‑to‑any multimodal flow matching work?

MOSAIC builds on two recent ideas. First, it encodes molecular structures with the UAE‑3D autoencoder, turning atom types, bond types and Cartesian coordinates into a sequence of continuous latent tokens. Second, it treats UV, IR and Raman spectra as tokenised spectral sequences. All four token streams are fed into a MMDiT (multimodal DiT) architecture trained with a rectified‑flow objective. During training the model randomly masks each modality, designating the masked part as the target and the unmasked part as the condition. This random masking creates many supervised denoising tasks from a single experiment, allowing the same network to learn how to reconstruct any modality from any other.

How does MOSAIC perform compared to existing models?

On the QM9S benchmark – a version of the QM9 dataset enriched with simulated UV, IR and Raman spectra – MOSAIC achieved 88.8 % molecular graph recovery. This outperformed the strongest spectra‑to‑structure baseline by 23.2 % and also surpassed a specialist structure‑to‑spectra model in spectral generation quality. The authors note a 5 % multimodal advantage for structure recovery when training with any‑to‑any objectives versus structure‑only training. In addition to higher accuracy, MOSAIC offered faster inference than comparable diffusion‑based approaches.

What new capabilities does MOSAIC enable for scientific research?

Because MOSAIC can accept arbitrary subsets of measurements, it supports several tasks that previously required separate models:

  • Bidirectional generation – generate spectra from a structure or infer a structure from spectra.
  • Missing‑modality completion – fill in absent UV, IR or Raman data when only a subset is measured.
  • Joint generation – produce a consistent set of structure and all three spectra in a single pass.
  • Spectral retrieval – match experimentally observed spectra to generated candidates, enabling molecular identification without explicit structure input.

The paper demonstrates the retrieval use‑case by conditioning on an experimental IR spectrum and searching the generated library for the most likely molecular structure, showing that MOSAIC can serve as a front‑end for spectral database searches.

How can Grafy users explore or apply MOSAIC concepts?

While MOSAIC itself is not a built‑in Grafy app, creators, developers and researchers can interact with the underlying ideas through Grafy Reader. By uploading the arXiv PDF (or the supplementary material) into the Reader, users can:

  1. Ask the chat‑backed AI to summarise specific sections, such as the flow‑matching loss formulation.
  2. Extract tables of results and compare them with other models using the built‑in spreadsheet view.
  3. Generate custom prompts that query the model about how to adapt the any‑to‑any training scheme to other scientific domains (e.g., materials science or proteomics).

For developers interested in prototyping a similar system, the Grafy Code studio provides a sandbox where Python scripts can call open‑source diffusion or flow‑matching libraries, experiment with random masking strategies, and visualise generated spectra using the built‑in plot widgets. Pairing Code with Reader lets teams iterate quickly: ingest the paper, extract the algorithmic pseudo‑code, implement a minimal version, and test on a small molecular dataset.

For creators who want to visualise MOSAIC’s output, the Creator app can turn generated 3‑D structures into rendered images or short videos, while the Graph tool can map the sequence of conditioning‑target masks as a branching diagram, illustrating how a single experiment yields multiple supervised tasks.


Source: https://arxiv.org/abs/2611.22098

Frequently asked questions

How does random masking create multiple supervised tasks?

Random masking selects a random subset of modalities to hide during each training step. The hidden part becomes the target while the visible part serves as the condition, so a single experiment can be used to train the model to predict any modality from any other.

Does MOSAIC require the number of atoms to be known in advance?

No. The authors built a variable‑atom‑count version of the model that infers the atom count during generation, allowing the system to handle molecules of differing sizes without a fixed prior.

Can MOSAIC be used with real experimental spectra, not just simulated ones?

The paper reports a proof‑of‑concept retrieval experiment on experimental IR spectra, showing that the model can accept real data as a condition and generate matching structures. Further validation on larger experimental datasets would be needed for production use.


Source: Grafy Research Lab — Every Measurement, Every Direction, All at Once: Multimodal Flow Matching for Molecules and Spectra