List of algorithms

Cosine similarity measures the angle between vectors, not their magnitude

Cosine similarity is a metric used to measure the similarity between two vectors by comparing the direction of the vectors, not their magnitude. This is particularly useful in high-dimensional spaces where the magnitude of vectors can be very large and less informative.

Example

Consider two vectors A = [1, 2, 3] and B = [2, 4, 6]. The cosine similarity between A and B is calculated as the dot product of A and B divided by the product of their magnitudes. In this case, the dot product is 1*2 + 2*4 + 3*6 = 28, and the magnitudes are sqrt(1^2 + 2^2 + 3^2) = sqrt(14) and sqrt(2^2 + 4^2 + 6^2) = sqrt(56). The cosine similarity is 28 / (sqrt(14) * sqrt(56)) = 0.5.

Cosine similarity is important because it focuses on the direction of vectors, making it useful for comparing the similarity of high-dimensional data, such as text embeddings or image features, where the magnitude of vectors can vary significantly.

Related concepts

cosine similarity is preferred over dot product for normalized embeddings

Cosine similarity measures orientation, not magnitude, making it ideal for normalized embeddings

cosine similarity works better than Euclidean distance in high dimensions

Cosine similarity measures orientation, not magnitude, making it more robust to irrelevant dimensions in high-dimensional spaces

Cosine similarity

Cosine similarity formula: cos(θ) = (A · B) / (||A|| ||B||)

ALiBi allows length extrapolation better than learned position embeddings

ALiBi uses relative positional encoding, avoiding fixed-size embeddings, enabling better handling of variable-length sequences

non-convex loss landscapes are hard: many local minima and saddle points

Non-convex loss landscapes are hard due to many local minima and saddle points

Euclidean geometry

Euclidean distance measures absolute position in space

One email a day: 5 concepts + the 5 stories that matter →

Swipe through 100 ML concepts daily

Open TickerNews