Vector search with LanceDB Cloud and LlamaIndex

Credentials

Copy and paste the project name and the api key from your project page. These will be used later to connect to LanceDB Cloud

You can also set the LANCEDB_API_KEY as an environment variable. More details can be found here.

Since we will be using OPENAI API, let us set the OPENAI API KEY as well.

Installing dependencies

Importing libraries

Download the data

Load the documents stored in the data/paul_graham/ using the SimpleDirectoryReader:

Store data in LanceDB Cloud

Let's connect to LanceDB so we can store our documents, It requires 0 setup !

Create an index

And thats it! We're all setup. The next step is to run some queries, let's try a few:

Query the index

We can now ask questions using the created index. Filtering can be enabled via MetadataFilters or use native lance where clause.

Let's use LanceDB filters(SQL like) directly via the where clause :

Append data to the index

You can also add data to an existing index