Batch normalization formula: Y = (X - μ) / σ * γ + β
Image: Typhoon_Mawar_2005_computer_simulation.gif: Atmoz derivative work: Atmoz (talk), CC BY-SA 3.0, via Wikimedia Commons
Batch normalization
Batch normalization formula: Y = (X - μ) / σ * γ + β
Batch normalization adjusts the inputs to each layer in a neural network. It re-centers them around zero and re-scaales them to a standard size, making training faster and more stable.
Example
If X = [1, 2, 3], μ = 2, and σ = 1, then Y = [(1-2)/1, (2-2)/1, (3-2)/1] * γ + β = [-1, 0, 1] * γ + β = [-1*γ + β, 0*γ + β, 1*γ + β].
Understanding the formula for batch normalization is crucial for implementing this technique correctly in neural networks.
Related concepts
Normalization (machine learning)
L2 normalization equation: x_i' = x_i / ||x||_2
Normal distribution
Normal distribution PDF formula
Mean squared error
Mean squared error (MSE) formula: MSE = (1/n) * Σ(y_i - ŷ_i)²
Expected value
Expected value formula: E[X] = Σ [x * P(x)]
convolution (f * g)(t) = ∫f(τ)g(t-τ)dτ
(f * g)(t) = ∫f(τ)g(t-τ)dτ
Mahalanobis distance
Mahalanobis distance formula: D^2 = (x - μ)'Σ^(-1)(x - μ)
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews