Product Quantization for Vector Similarity Search (+ Python)
Vector similarity search can require huge amounts of memory. Indexes containing 1M dense vectors (a small dataset in today’s world) will often require several GBs of memory to store. When building recommendation systems or semantic search engines, this is not acceptable. The problem of excessive memory usage is exasperated by high-dimensional data, and with ever-increasing dataset sizes, this can very quickly become unmanageable. Product quantization (PQ) is a popular method for dramatically compressing high-dimensional vectors to use 97% less memory, and for making nearest-neighbor search speeds 5.5x faster in our tests. A composite IVF+PQ index speeds up the search by another 16.5x without affecting accuracy, for a whopping total speed increase of 92x compared to non-quantized indexes. 🌲 Pinecone article: https://www.pinecone.io/learn/product... 🤖 70% Discount on the NLP With Transformers in Python course: https://bit.ly/3DFvvY5 🎉 Sign-up For New Articles Every Week on Medium! / membership 👾 Discord: / discord 🕹️ Free AI-Powered Code Refactoring with Sourcery: https://sourcery.ai/?utm_source=YouTu...
Faiss - Vector Compression with PQ and IVFPQ (in Python)
3 Vector-based Methods for Similarity Search (TF-IDF, BM25, SBERT)
Product quantization in Faiss and from scratch
Vector Search & Approximate Nearest Neighbors (ANN) | FAISS (HNSW & IVF)
Uniform Manifold Approximation and Projection (UMAP) | Dimensionality Reduction Techniques (5/5)
Locality Sensitive Hashing (LSH) for Search with Shingling + MinHashing (Python)
AlphaFold - The Most Useful Thing AI Has Ever Done
Vector Database Search - Hierarchical Navigable Small Worlds (HNSW) Explained
HNSW for Vector Search Explained and Implemented with Faiss (Python)
How To Think SO CLEARLY People Assume You're A Genius
Choosing Indexes for Similarity Search (Faiss in Python)
Argentinien – Österreich Highlights | Gruppe J, FIFA WM 2026 | sportstudio
Variational Autoencoders
![[CVPR20 Tutorial] Billion-scale Approximate Nearest Neighbor Search](https://i.ytimg.com/vi/SKrHs03i08Q/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLBqoZLVkbvkp0ltUGJwdD_RFtVZKw)
[CVPR20 Tutorial] Billion-scale Approximate Nearest Neighbor Search
Group theory, abstraction, and the 196,883-dimensional monster
Latent Space Visualisation: PCA, t-SNE, UMAP | Deep Learning Animated
Faiss - Introduction to Similarity Search
The Strange Math That Predicts (Almost) Anything
nDCG: the evaluation metric you've (probably) never heard of
