Faster KNN search in Manticore: 2-pass HNSW, batched distances, and AVX-512

Manticore's KNN search engine, built on top of hnswlib, has undergone significant optimizations. Specifically, the company modified hnswlib's core search loop to restructure neighbor traversal, distance function calls, and CPU memory hierarchy interactions. These changes, combined with new AVX-512 distance implementations in the columnar library, target inefficient memory access patterns, redundant data loads, and indirect function call overhead. The optimizations include compile-time distance function specialization using C++ templates, 2-pass neighbor processing with prefetching, and batched distance computation.

Manticore's optimizations position the company to better compete in the vector search market, where efficient and scalable similarity search is crucial. The use of AVX-512 instructions, which process 16 floats per iteration, provides a significant boost in performance. Manticore now ships three library variants: baseline, AVX2, and AVX-512, with automatic CPU capability detection and library loading. This strategic move enables Manticore to differentiate itself from competitors, such as those using hnswlib directly, by providing a more optimized and efficient KNN search solution.

The implications of these optimizations are substantial, particularly for applications relying heavily on KNN search. With improved performance and efficiency, Manticore's customers can expect faster query execution times and better scalability. However, to maintain this competitive edge, Manticore must continue to monitor and adapt to evolving CPU architectures and emerging trends in vector search. Additionally, the company should focus on ensuring seamless integration and compatibility of its optimized KNN search engine with existing infrastructure and applications.