Why the curse of dimensionality makes nearest neighbor search unreliable

How does the curse of dimensionality affect the performance and accuracy of clustering algorithms in high-dimensional datasets?

High-dimensional data can lead to sparse clusters, reducing clustering accuracy due to increased distance between points

Time complexity of binary search: O(log n) — halves search space each step

Binary search reduces search space by half with each iteration, achieving O(log n) complexity

Why L1 distance is called Manhattan distance — grid-like paths

L1 distance mimics grid-like city blocks, hence "Manhattan" distance

Why ALiBi allows length extrapolation better than learned position embeddings

ALiBi uses fixed-length position encodings, enabling efficient length extrapolation without model retraining

What consistent hashing does: minimizes remapping when nodes join/leave

Consistent hashing minimizes data redistribution during nodes' addition or removal

Why non-convex loss landscapes are hard: many local minima and saddle points

Non-convex landscapes have numerous local minima and saddle points, complicating optimization