q(x_t|x_{t-1}) adds Gaussian noise at each step
Image: FDV, CC BY-SA 4.0, via Wikimedia Commons
Diffusion model
q(x_t|x_{t-1}) adds Gaussian noise at each step
The forward diffusion process in diffusion models involves adding Gaussian noise incrementally to transform data into a simpler distribution. This process gradually corrupts the original data, making it easier to learn the reverse process during training. The noise addition at each step follows a Markov chain, ensuring that each step depends only on the previous one.
Example
Consider an image initially clean. During the forward diffusion process, Gaussian noise is added at each time step, gradually transforming the image into a noisy, blurred version. This transformation follows a Markov chain, where each step's noise addition depends solely on the image's state at the previous step.
Understanding the forward diffusion process is crucial for grasping how diffusion models learn to generate new data samples by reversing the noise addition process. This knowledge is fundamental for developing and improving generative models in machine learning.
Related concepts
Langevin dynamics does: adds noise to gradient descent to sample from a distribution
Langevin dynamics adds noise to gradient descent to sample from a distribution
the reverse process learns: p_θ(x_{t-1}|x_t)
The reverse process learns: p_θ(x_{t-1}|x_t) — denoising one step at a time
Stable Diffusion
Stable Diffusion generates images from text descriptions
the momentum term does: v_t = βv_{t-1} + ∇L, accumulates gradient direction
Momentum term accelerates convergence in the gradient direction
Lyapunov exponents measure: rate of divergence of nearby trajectories in a dynamical system
Lyapunov exponents measure the rate of divergence of nearby trajectories in a dynamical system
Cross-entropy H(p,q) = -Σ p(x) log q(x) measures how well q approximates p
Cross-entropy H(p,q) = -Σ p(x) log q(x) quantifies approximation quality between distributions p and q
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews