HNSW is an efficient ANN search algorithm
Hierarchical navigable small world
HNSW is an efficient ANN search algorithm
The multi-layered graph structure of HNSW allows for both rough and detailed searches, optimizing the balance between speed and accuracy. This hierarchical method significantly reduces the computational complexity compared to traditional methods that compare the query with every item individually.
HNSW's efficiency and scalability make it ideal for large-scale vector data searches, significantly improving search performance in applications like image and document retrieval.
Related concepts
UMAP is faster than t-SNE
UMAP is faster due to approximate nearest neighbors and cross-entropy optimization
Attention Is All You Need
O(n) complexity for long sequences
MoE models have more parameters but similar compute cost
MoE models distribute parameters across k experts, reducing active experts' compute cost
Retrieval-augmented generation
RAG enables LLMs to access new information without retraining
non-convex loss landscapes are hard: many local minima and saddle points
Non-convex loss landscapes are hard due to many local minima and saddle points
paged attention (vLLM) improves serving throughput
Paged attention (vLLM) improves serving throughput by reducing latency through non-contiguous KV-cache pages, enabling faster data retrieval
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews