saddle points are more common than local minima in high dimensions

non-convex loss landscapes are hard: many local minima and saddle points

Non-convex loss landscapes are hard due to many local minima and saddle points

List of unsolved problems in mathematics

Random points in high dimensions are nearly equidistant due to the uniform distribution of volume in high-dimensional space

SGD with momentum escapes local minima better than vanilla SGD

SGD with momentum adds velocity to escape shallow local minima faster

Manifold hypothesis

High-dimensional data lies on lower-dimensional manifolds

random projection to O(log n/ε²) dimensions preserves pairwise distances within 1±ε

Random projection reduces dimensionality while preserving pairwise distances within ε² due to the Johnson-Lindenstrauss lemma

Inner product space

Inner product space generalizes Euclidean geometry