The latest research from Google
Jul 15, 2026
Towards demystifying the creativity of diffusion models · The latest research from Google
Science, Technology & Innovation · Jul 15, 2026
The paper argues diffusion-model novelty is a deterministic consequence of training-time approximation: neural-network regularization and imperfect fit 'smooth' the learned denoising (score) function so generation interpolates between training examples instead of reproducing them, making creativity an interpolation effect and a controllable tradeoff with memorization.
Towards demystifying the creativity of diffusion models · The latest research from Google
Science, Technology & Innovation · Jul 15, 2026
The paper argues that score smoothing in neural nets arises not only from explicit regularizers (like weight decay) but also from implicit regularization of gradient-based optimization, making smoother denoising fields—and thus creative behavior in diffusion models—an intrinsic, stable outcome of training that platform designers should shape rather than assume they can remove by tweaking a single penalty.
Towards demystifying the creativity of diffusion models · The latest research from Google
Science, Technology & Innovation · Jul 15, 2026
In a 1-D two-point toy problem, stronger weight decay when training two-layer ReLU score networks smooths a sharp sign-switching “cliff” in the denoising field so central particles slow and settle in an interpolation zone, giving direct mechanistic evidence that regularization controls memorization versus novelty generation.
Towards demystifying the creativity of diffusion models · The latest research from Google
Science, Technology & Innovation · Jul 15, 2026
Anisotropic (direction-dependent) score smoothing—stronger along manifold tangents but minimal toward the manifold—lets diffusion models avoid collapsing onto training examples while still moving efficiently from noise onto the meaningful image manifold, explaining how they can be both sharp and novel and suggesting architectures that preserve this asymmetry could improve novelty without blur.