Just brute force your embeddings

Discussion (4 Comments)Read Original on HackerNews

emschwartz•about 4 hours ago

This works especially well if your embedding model was trained to perform well with quantized embeddings. Binary + hamming distance = incredibly fast.

This post is from 2024 but I wrote about using this technique in https://emschwartz.me/binary-vector-embeddings-are-so-cool/

softwaredoug•about 4 hours ago

Hamming w/xor+popcount is the only thing I can make numpy do faster than float32 dot products :)

int8s, float16s are all fairly slow. I suppose it’s because BLAS does float32/64 very fast.

emschwartz•about 3 hours ago

Yeah that makes sense. I took the optimizations of my hamming distance library to a bit of an insane level. I wrote this about the first round https://emschwartz.me/unnecessary-optimization-in-rust-hammi... The second round brought the best result down to 0.8 nanoseconds per comparison on x86 with SIMD (for a batch of 1000).

Separately, I’m a huge fan of your writing about search! It’s been very helpful while I’ve been building https://scour.ing.

softwaredoug•about 3 hours ago

Thank you! <3

Just brute force your embeddings

⚡ Community Insights

Discussion (4 Comments)Read Original on HackerNews