Building Multilingual Rag With Elastic And Mistral
openai-chatgptlangchain-pythonchatgptgenaielasticsearchelasticbuilding-multilingual-rag-with-elastic-and-mistralopenaiAIchatlogvectordatabasePythonsearchgenaistacksupporting-blog-contentvectorelasticsearch-labslangchainapplications
Export
Building RAG with Elastic and Mistral
This notebook is a hands-on demonstration of how to create a multilingual RAG system. The following steps were extracted from the article "Building RAG with Elastic and Mistral".
Install Packages and Import Necessary Modules
[ ]
Collecting elasticsearch==8.14
Downloading elasticsearch-8.14.0-py3-none-any.whl (480 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 480.2/480.2 kB 2.7 MB/s eta 0:00:00
Collecting mistralai
Downloading mistralai-0.4.2-py3-none-any.whl (20 kB)
Collecting elastic-transport<9,>=8.13 (from elasticsearch==8.14)
Downloading elastic_transport-8.13.1-py3-none-any.whl (64 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.5/64.5 kB 7.5 MB/s eta 0:00:00
Collecting httpx<1,>=0.25 (from mistralai)
Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 9.1 MB/s eta 0:00:00
Collecting orjson<3.11,>=3.9.10 (from mistralai)
Downloading orjson-3.10.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.1/141.1 kB 5.6 MB/s eta 0:00:00
Requirement already satisfied: pydantic<3,>=2.5.2 in /usr/local/lib/python3.10/dist-packages (from mistralai) (2.8.2)
Requirement already satisfied: urllib3<3,>=1.26.2 in /usr/local/lib/python3.10/dist-packages (from elastic-transport<9,>=8.13->elasticsearch==8.14) (2.0.7)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from elastic-transport<9,>=8.13->elasticsearch==8.14) (2024.7.4)
Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai) (3.7.1)
Collecting httpcore==1.* (from httpx<1,>=0.25->mistralai)
Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 6.8 MB/s eta 0:00:00
Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai) (3.7)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai) (1.3.1)
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.25->mistralai)
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 6.2 MB/s eta 0:00:00
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai) (0.7.0)
Requirement already satisfied: pydantic-core==2.20.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai) (2.20.1)
Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai) (4.12.2)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx<1,>=0.25->mistralai) (1.2.1)
Installing collected packages: orjson, h11, elastic-transport, httpcore, elasticsearch, httpx, mistralai
Successfully installed elastic-transport-8.13.1 elasticsearch-8.14.0 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 mistralai-0.4.2 orjson-3.10.6
Declaring Variables
This code will create inputs where you can enter your credentials. Here you can learn how to retrieve your Elasticsearch credentials: Finding Your Cloud ID
[ ]
Elastic Cloud ID: ·········· Elastic Api Key: ·········· MISTRAL Api key: ··········
Instance a Elasticsearch client
[ ]
Creating embeddings endpoint
[ ]
Embedding endpoint created successfully.
Creating Mappings
[ ]
Indexing documents
[ ]
[ ]
[ ]
Data indexed successfully.
Retrieving documents
[ ]
{
"took": 48,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.9155389,
"hits": [
{
"_index": "multilingual-mistral",
"_id": "1",
"_score": 0.9155389,
"_source": {
"super_body": {
"inference": {
"inference_id": "multilingual-embeddings",
"model_settings": {
"similarity": "cosine",
"element_type": "float",
"task_type": "text_embedding",
"dimensions": 384
}
},
"text": "\n _Support Ticket #EN1234_\n **Subject**: Calendar sync not working with Google Calendar\n\n **Description**:\n I'm having trouble syncing my project deadlines with Google Calendar. Whenever I try to sync, I get an error message saying \"Unable to connect to external calendar service.\"\n\n **Resolution**:\n The issue was resolved by following these steps:\n 1. Go to Settings > Integrations\n 2. Disconnect the Google Calendar integration\n 3. Clear browser cache and cookies\n 4. Reconnect the Google Calendar integration\n 5. Authorize the app again in Google's security settings\n\n The sync should now work correctly. If problems persist, ensure that third-party cookies are enabled in your browser settings.\n "
}
}
},
{
"_index": "multilingual-mistral",
"_id": "2",
"_score": 0.90574336,
"_source": {
"super_body": {
"inference": {
"inference_id": "multilingual-embeddings",
"model_settings": {
"similarity": "cosine",
"element_type": "float",
"task_type": "text_embedding",
"dimensions": 384
}
},
"text": "\n _Support-Ticket #DE5678_\n **Betreff**: Datei-Upload funktioniert nicht\n\n **Beschreibung**:\n Ich kann keine Dateien mehr in meine Projekte hochladen. Jedes Mal, wenn ich es versuche, bleibt der Ladebalken bei 99% stehen und dann erscheint eine Fehlermeldung.\n\n **L\u00f6sung**:\n Das Problem wurde durch folgende Schritte gel\u00f6st:\n 1. \u00dcberpr\u00fcfen Sie die Dateigr\u00f6\u00dfe. Die maximale Uploadgr\u00f6\u00dfe betr\u00e4gt 100 MB.\n 2. Deaktivieren Sie vor\u00fcbergehend den Virenschutz oder die Firewall.\n 3. Versuchen Sie, die Datei im Inkognito-Modus hochzuladen.\n 4. Wenn das nicht funktioniert, leeren Sie den Browser-Cache und die Cookies.\n 5. Als letzten Ausweg, versuchen Sie einen anderen Browser zu verwenden.\n\n In den meisten F\u00e4llen lag das Problem an zu gro\u00dfen Dateien oder an Interferenzen durch Sicherheitssoftware. Nach Anwendung dieser Schritte sollte der Upload funktionieren.\n "
}
}
}
]
}
}
Answering the question
Now we will use Mistral to answer the question.
[ ]
[ ]
## Question:
Hola, estoy teniendo problemas para ocupar su aplicación, estoy teniendo problemas para sincronizar mi calendario, y encima al intentar subir un archivo me da error.
## Related knowledge:
_Support Ticket #EN1234_
**Subject**: Calendar sync not working with Google Calendar
**Description**:
I'm having trouble syncing my project deadlines with Google Calendar. Whenever I try to sync, I get an error message saying "Unable to connect to external calendar service."
**Resolution**:
The issue was resolved by following these steps:
1. Go to Settings > Integrations
2. Disconnect the Google Calendar integration
3. Clear browser cache and cookies
4. Reconnect the Google Calendar integration
5. Authorize the app again in Google's security settings
The sync should now work correctly. If problems persist, ensure that third-party cookies are enabled in your browser settings.
_Support-Ticket #DE5678_
**Betreff**: Datei-Upload funktioniert nicht
**Beschreibung**:
Ich kann keine Dateien mehr in meine Projekte hochladen. Jedes Mal, wenn ich es versuche, bleibt der Ladebalken bei 99% stehen und dann erscheint eine Fehlermeldung.
**Lösung**:
Das Problem wurde durch folgende Schritte gelöst:
1. Überprüfen Sie die Dateigröße. Die maximale Uploadgröße beträgt 100 MB.
2. Deaktivieren Sie vorübergehend den Virenschutz oder die Firewall.
3. Versuchen Sie, die Datei im Inkognito-Modus hochzuladen.
4. Wenn das nicht funktioniert, leeren Sie den Browser-Cache und die Cookies.
5. Als letzten Ausweg, versuchen Sie einen anderen Browser zu verwenden.
In den meisten Fällen lag das Problem an zu großen Dateien oder an Interferenzen durch Sicherheitssoftware. Nach Anwendung dieser Schritte sollte der Upload funktionieren.
The answer is on point!
[ ]
Hola, me alegra que te hayas comunicado con nosotros. Parece que hay dos problemas distintos. En cuanto a la sincronización del calendario, puedes seguir estos pasos para resolver el problema: 1. Ve a Configuración > Integraciones 2. Desconecta la integración del Calendario de Google 3. Borra la caché y las cookies del navegador 4. Vuelve a conectar la integración del Calendario de Google 5. Autoriza de nuevo la aplicación en la configuración de seguridad de Google Si sigues teniendo problemas, asegúrate de que las cookies de terceros están habilitadas en la configuración de tu navegador. En cuanto al problema de subir un archivo, hay varias cosas que puedes probar: 1. Comprueba el tamaño del archivo. El tamaño máximo de carga es de 100 MB. 2. Desactiva temporalmente el antivirus o el cortafuegos. 3. Intenta cargar el archivo en modo incógnito. 4. Si eso no funciona, borra la caché y las cookies del navegador. 5. Como último recurso, prueba a usar un navegador diferente. En la mayoría de los casos, el problema se debe a archivos demasiado grandes o a interferencias causadas por software de seguridad. Al seguir estos pasos, deberías poder cargar el archivo correctamente. ¡Espero que esto te ayude a resolver tus problemas! Si tienes alguna otra pregunta, no dudes en preguntar.
Deleting
Finally, we can delete the resources used to prevent them from consuming resources.
[ ]
<ipython-input-20-f7fb7887af9e>:2: DeprecationWarning: Passing transport options in the API method is deprecated. Use 'Elasticsearch.options()' instead. es_client.indices.delete(index='multilingual-mistral', ignore=[400, 404]) <ipython-input-20-f7fb7887af9e>:5: DeprecationWarning: Passing transport options in the API method is deprecated. Use 'Elasticsearch.options()' instead. es_client.inference.delete_model(inference_id='multilingual_embeddings', ignore=[400, 404])
ObjectApiResponse({'acknowledged': True, 'pipelines': [], 'indexes': []})