Non-convex loss landscapes are hard due to many local minima and saddle points
Image: Bjørn Christian Tørrissen, CC BY-SA 3.0, via Wikimedia Commons
Non-convex loss landscapes are hard due to many local minima and saddle points
saddle points are more common than local minima in high dimensions
Saddle points arise due to mixed partial derivatives being zero, leading to more complex curvature in high dimensions
Convex optimization
Convex functions have only one global minimum
t-SNE preserves local structure
t-SNE preserves local structure by converting distances to probabilities and minimizing Kullback-Leibler divergence
the Johnson-Lindenstrauss lemma says
Random projection reduces dimensionality while approximately preserving pairwise distances
Graph (abstract data type)
Time complexity of BFS and DFS: O(V + E)
random projection to O(log n/ε²) dimensions preserves pairwise distances within 1±ε
Random projection reduces dimensionality while preserving pairwise distances within ε² due to the Johnson-Lindenstrauss lemma
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews