Diffusion Model

A diffusion model is a generative AI system that creates new data — typically images — by learning to reverse a noise-adding process. It starts with pure random noise and gradually refines it into a coherent output. DALL-E, Midjourney, and Stable Diffusion are all built on diffusion models.

How diffusion works

The training process has two phases:

1. Forward process (adding noise) — take a real image and gradually add random noise over many steps until it becomes pure static. This is automatic and requires no learning.
2. Reverse process (removing noise) — train a neural network to reverse each step, predicting and removing the noise that was added. This is where the learning happens.

Once trained, the model can start from pure random noise and iteratively denoise it into a realistic image. The genius is that by conditioning this process on a text description, the model generates images that match the prompt.

Why diffusion models won

Earlier generative approaches like GANs (generative adversarial networks) produced impressive results but were notoriously difficult to train — prone to mode collapse and training instability. Diffusion models are more stable, produce higher-quality outputs, and offer better control over the generation process.

Key concepts

• Denoising steps — more steps produce higher-quality images but take longer to generate
• Guidance scale — controls how strongly the output adheres to the text prompt (higher values are more faithful but less creative)
• Latent diffusion — performing the diffusion process in a compressed latent space rather than pixel space, dramatically reducing computational cost. This is what makes Stable Diffusion practical.
• ControlNet — additional conditioning that lets you guide generation with sketches, depth maps, or poses

Beyond images

Diffusion models now generate audio (music, speech), video, 3D models, and even molecular structures for drug discovery. The core principle — learn to denoise — applies across data types.