Apple proposes amortized maximum inner product search to speed up vector retrieval

Apple’s Machine Learning Research team describes a method to amortize the cost of maximum inner product search (MIPS), a core subroutine in many machine learning pipelines that selects the best-matching vector from a database for a given query.

The proposed approach trains neural networks to directly predict MIPS solutions instead of computing them repeatedly for each query, assuming queries are drawn from a known distribution over a fixed key database.

The authors frame MIPS as a support function of the key set, motivating two models: SupportNet, an input-convex neural network that regresses the support function and can act as a cluster router to guide queries to relevant database partitions, and KeyNet, a vector-valued network that regresses the optimal key and can be used as a drop-in replacement for the original query in off-the-shelf indexing pipelines.

In experiments on the BEIR benchmark using document embeddings, SupportNets and KeyNets improved IVF match rates while reducing compute as measured by FLOPs, the number of probes, or wall-clock time.