Lyapunov exponents measure: rate of divergence of nearby trajectories in a dynamical system

Langevin dynamics does: adds noise to gradient descent to sample from a distribution

Langevin dynamics adds noise to gradient descent to sample from a distribution

Chaos theory

Butterfly effect demonstrates sensitive dependence on initial conditions

Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits

Entropy H = -Σ p(x) log₂ p(x) quantifies uncertainty in a system

The elastic net combines L1 and L2: λ₁|w| + λ₂w² gives both sparsity and stability

Elastic net: λ₁|w| + λ₂w² enforces sparsity and stability simultaneously

Curvature

Curvature measures the angular rate of change of the direction of the tangent line per unit distance along the curve

Cross-entropy H(p,q) = -Σ p(x) log q(x) measures how well q approximates p

Cross-entropy H(p,q) = -Σ p(x) log q(x) quantifies approximation quality between distributions p and q