the dot product measures alignment: it equals |a||b|cos(θ)

Cosine similarity

Cosine similarity formula: cos(θ) = (A · B) / (||A|| ||B||)

List of algorithms

Cosine similarity measures the angle between vectors, not their magnitude

cosine similarity works better than Euclidean distance in high dimensions

Cosine similarity measures orientation, not magnitude, making it more robust to irrelevant dimensions in high-dimensional spaces

cosine similarity is preferred over dot product for normalized embeddings

Cosine similarity measures orientation, not magnitude, making it ideal for normalized embeddings

Euclidean geometry

Euclidean distance measures absolute position in space

sinusoidal position encoding works: each dimension has a different frequency

Sinusoidal position encoding assigns unique frequencies to each dimension, enabling the model to distinguish positions effectively