Scalable late interaction vectors in Elasticsearch: Bit Vectors

In this notebook, we will be looking at how to convert late interaction vectors to bit vectors to

1. Save siginificant disk space
2. Lower query latency

We will also look at how we can use hamming distance to speed our queries up even further.
This notebook builds on part 1 where we downloaded the images, created ColPali vectors and saved them to disk. Please execute this notebook before trying the techniques in this notebook.

Also check out our accompanying blog post on Scaling Late Interaction Models for more context on this notebook.

This is the key part of this notebook. We use the to_bit_vectors() function to convert our vectors into bit vectors.
The function is simple in essence. Values > 0 are converted to 1, values < 0 are converted to 0. We then convert our array of 0s and 1s to a hex string, that represents our bit vector.
So don't be surprised that the values that we will be indexing look like strings and not arrays as before. This is intended!

Learn more about bit vectors and hamming distance in our blog about this topic.

Here we are defining our mapping for our Elasticsearch index. Note how we set the element_type parameter to bit to inform Elasticsearch that we will be indexing bit vectors in this field.

Above we have seen how to query our data using the maxSimInvHamming(...) function.
We can also just pass the full fidelity col pali vector and use the maxSimDotProduct(...) function for asymmetric similarity between the vectors.