DISCO Enables Exact Conditional Sampling Without Diffusion

DISCO, a new diffusion‑free score‑matching technique, lets practitioners draw exact conditional samples from generative models, a task that has long troubled diffusion‑based approaches. The method learns only one joint score field, simplifying conditioning and improving probabilistic reasoning.

---

What problem does DISCO aim to solve?

Traditional score‑based generative models rely on diffusion training, which learns a continuum of time‑indexed score fields for increasingly noisy versions of the data. While this yields high‑quality unconditional samples, conditioning becomes cumbersome because the conditioning information exists only for clean data, not for the noisy intermediate states needed during generation. Existing conditional samplers—gradient guidance, replacement heuristics, and particle‑based methods—often produce biased results, especially on simple toy distributions, as shown in Figure 1 of the paper.

DISCO removes the need for a whole family of noisy scores. By training a single score for a slightly perturbed data distribution, the exact conditional score can be obtained directly, making conditioning a straightforward operation.

How does DISCO differ from diffusion‑based training?

Diffusion models broaden the region where the score is learned by gradually adding Gaussian noise and training a separate score network for each noise level. DISCO, in contrast, constructs an objective from a mixture of importance‑weighted Fisher divergences across several proposal distributions. The resulting loss resembles denoising score matching but targets only the perturbed data distribution rather than a full diffusion trajectory. Consequently, only one neural network $s_{\theta}(x)$ is optimized, yet it still benefits from the support‑broadening effect that diffusion provides.

The key theoretical insight is that the conditional score of the joint model can be expressed analytically once the joint score is known. Standard MCMC samplers (e.g., Metropolis‑adjusted Langevin dynamics) can then draw samples from the exact conditional without resorting to heuristic guidance.

Does DISCO sacrifice sample quality?

In the authors' experiments, DISCO matches state‑of‑the‑art diffusion models on standard image benchmarks in terms of Frechet Inception Distance and visual fidelity. More importantly, when evaluating conditional sampling—both on low‑dimensional Gaussian mixtures and high‑dimensional image datasets—DISCO’s samples align closely with the true conditionals, whereas diffusion‑based heuristics show noticeable bias. Repeated resampling tests on images demonstrate that DISCO’s conditional samples remain stable, indicating a faithful underlying data representation.

How can developers and researchers try DISCO?

The authors provide an open‑source code repository (https://anonymous.4open.science/r/disco-code/) containing training scripts, model checkpoints, and sampling utilities. To experiment with DISCO, one can:

1. Clone the repository and install the required Python dependencies.
2. Use the supplied training loop to fit the single‑score network on a chosen dataset (e.g., CIFAR‑10 or a custom image collection).
3. Apply the provided tempered Sequential Monte Carlo (SMC) sampler or standard MCMC to draw conditional samples by fixing the observed variables in the joint score.
4. Compare results against diffusion baselines using the same evaluation metrics.

Developers familiar with PyTorch or JAX will find the implementation compatible with existing score‑matching pipelines, requiring only the replacement of the multi‑scale diffusion loss with the DISCO objective.

Where does this fit within Grafy?

For creators and researchers who want to explore DISCO without setting up a full development environment, Grafy’s Reader app offers a convenient way to ingest the arXiv PDF, ask questions, and obtain concise explanations of the method. By uploading the paper to /reader, users can:

• Query the model for step‑by‑step derivations of the DISCO objective.
• Generate summaries that compare DISCO to diffusion heuristics.
• Extract code snippets and have the AI explain how to adapt them to their own data.

If you prefer to prototype the sampling algorithm, Grafy’s Code studio (/code) lets you run the provided Python notebooks in a cloud‑based notebook environment, with access to the same models used in the paper. This way, you can experiment with conditional generation on image or tabular data directly within Grafy’s workspace.

---

Frequently asked questions

How does DISCO obtain the exact conditional score?

DISCO learns a joint score $s_{\theta}(x) = \nabla_x \log p_{\theta}(x)$. By fixing the observed variables in $x$ and computing the gradient with respect to the remaining variables, the resulting vector field is the exact conditional score of the learned joint distribution. Standard MCMC samplers can then target this conditional density without approximations.

Does DISCO work for high‑dimensional data such as images?

Yes. The paper reports experiments on image datasets using an unconstrained score parameterization. While the conditional score is exact only at the population optimum, empirical results show that DISCO’s image samples are comparable to diffusion models and its conditionals remain unbiased under repeated resampling.

Can DISCO replace diffusion models for unconditional generation?

DISCO achieves comparable unconditional sample quality to diffusion models, but its primary advantage lies in reliable conditional sampling. For tasks where only unconditional generation is needed, diffusion models remain a solid choice; however, DISCO offers a unified framework that handles both unconditional and conditional generation with a single score network.

---

Source: Arxiv ML2 Recent — Diffusion-free Score Matching for Exact Conditional Sampling