Back to feed

Towards demystifying the creativity of diffusion models

The latest research from Google

Jul 15, 2026

7/15/2026

Score Smoothing From Training-Time Approximation Drives Diffusion Model Novelty

Towards demystifying the creativity of diffusion models · The latest research from Google

Science, Technology & Innovation · Jul 15, 2026

The paper argues diffusion-model novelty is a deterministic consequence of training-time approximation: neural-network regularization and imperfect fit 'smooth' the learned denoising (score) function so generation interpolates between training examples instead of reproducing them, making creativity an interpolation effect and a controllable tradeoff with memorization.


7/15/2026

Score Smoothing Emerges From Both Explicit Regularizers And Implicit Optimization Dynamics, Making Diffusion Model Behavior More Stable Across Implementations.

Towards demystifying the creativity of diffusion models · The latest research from Google

Science, Technology & Innovation · Jul 15, 2026

The paper argues that score smoothing in neural nets arises not only from explicit regularizers (like weight decay) but also from implicit regularization of gradient-based optimization, making smoother denoising fields—and thus creative behavior in diffusion models—an intrinsic, stable outcome of training that platform designers should shape rather than assume they can remove by tweaking a single penalty.


7/15/2026

Higher Weight Decay Smooths The Score And Tunes Memorization Versus Novelty In Diffusion Models

Towards demystifying the creativity of diffusion models · The latest research from Google

Science, Technology & Innovation · Jul 15, 2026

In a 1-D two-point toy problem, stronger weight decay when training two-layer ReLU score networks smooths a sharp sign-switching “cliff” in the denoising field so central particles slow and settle in an interpolation zone, giving direct mechanistic evidence that regularization controls memorization versus novelty generation.


7/15/2026

Anisotropic Score Smoothing Enables Diffusion Models To Be Sharp And Novel Without Memorizing Training Examples

Towards demystifying the creativity of diffusion models · The latest research from Google

Science, Technology & Innovation · Jul 15, 2026

Anisotropic (direction-dependent) score smoothing—stronger along manifold tangents but minimal toward the manifold—lets diffusion models avoid collapsing onto training examples while still moving efficiently from noise onto the meaningful image manifold, explaining how they can be both sharp and novel and suggesting architectures that preserve this asymmetry could improve novelty without blur.