Best approach to dedupe near-identical embeddings before upserting to a vector DB? Cosine threshold feels brittle across domains.
Thread
We bucket by a coarse LSH signature first, then only do cosine within a bucket — way more stable than a global threshold.
