Notebooks
E
Elastic Vector Search Implementation Guide Api

Vector Search Implementation Guide Api

openai-chatgptlangchain-pythonchatgptgenaielasticsearchelasticopenaiAIchatlogvectordatabasevector-search-implementation-guide-apiPythonsearchgenaistacksupporting-blog-contentvectorelasticsearch-labslangchainapplications
alph-notebooks/elasticsearch-labs / vector_search_implementation_guide_api.ipynb

Simplified Vector Search (kNN) Implementation Guide

Loading the Embedding Model

Loading embedding model: sentence-transformers/all-distilroberta-v1

Loading code borrowed from elasticsearch-labs NLP text search example notebook

[ ]
[ ]
[ ]
[ ]

Ingest pipeline setup

[48]
{'acknowledged': True}

Index Mapping / Template setup

[49]
{'acknowledged': True}

Indexing Data

[50]
[51]
ObjectApiResponse({'_shards': {'total': 2, 'successful': 1, 'failed': 0}})

Querying Data

Approximate k-nearest neighbor (kNN)

[52]
[{'_id': 'PoHEcpIB5JwEUwVjEs6E',
  '_index': 'my_vector_index-01',
  '_score': 0.7825787,
  '_source': {'ml': {'inference': {}},
              'my_metadata': 'The Dude',
              'my_text': "Hey, careful, man, there's a beverage here!"}}]

Hybrid Searching (kNN + BM25) with RRF

[53]
[{'_id': 'QYHEcpIB5JwEUwVjEs6E',
  '_index': 'my_vector_index-01',
  '_score': 1.8082356,
  'fields': {'my_metadata': ['Walter Sobchak'],
             'my_text': ['What do you mean brought it bowling, Dude?']}},
 {'_id': 'QoHEcpIB5JwEUwVjEs6E',
  '_index': 'my_vector_index-01',
  '_score': 1.2366624,
  'fields': {'my_metadata': ['Walter Sobchak'],
             'my_text': ['Donny was a good bowler, and a good man. He was one '
                         'of us. He was a man who loved the outdoors... and '
                         'bowling, and as a surfer he explored the beaches of '
                         'Southern California, from La Jolla to Leo Carrillo '
                         'and... up to... Pismo']}}]

Filtering

[55]
[{'_id': 'PoHEcpIB5JwEUwVjEs6E',
  '_index': 'my_vector_index-01',
  '_score': 0.59394693,
  'fields': {'my_metadata': ['The Dude'],
             'my_text': ["Hey, careful, man, there's a beverage here!"]}}]

Aggregrations

and Select fields returned

[56]
[{'_id': 'QYHEcpIB5JwEUwVjEs6E',
  '_index': 'my_vector_index-01',
  '_score': 0.7433834,
  'fields': {'my_metadata': ['Walter Sobchak'],
             'my_text': ['What do you mean brought it bowling, Dude?']}},
 {'_id': 'PoHEcpIB5JwEUwVjEs6E',
  '_index': 'my_vector_index-01',
  '_score': 0.6028075,
  'fields': {'my_metadata': ['The Dude'],
             'my_text': ["Hey, careful, man, there's a beverage here!"]}}]