The elastic net combines L1 and L2: λ₁|w| + λ₂w² gives both sparsity and stability

ill-conditioned matrices cause numerical instability: small input changes → large output changes

Ill-conditioned matrices amplify input perturbations, leading to significant output variability

Regularization (mathematics)

L1 regularization results in sparse solutions

non-convex loss landscapes are hard: many local minima and saddle points

Non-convex loss landscapes are hard due to many local minima and saddle points

Ordinary least squares

OLS minimizes squared differences

Vanishing gradient problem

Residual connections help by allowing gradient flow through the skip connection

the L1 norm is not differentiable at zero

The L1 norm is not differentiable at zero because the absolute value function has a kink at zero