LanceDB Main

Main

agentsllmsvector-databaselancedbgptopenaiAImultimodal-aitutorialsmachine-learningembeddingsfine-tuningdeep-learninggpt-4-visionllama-indexragAdvace_RAG_LlamaParsermultimodallangchainlancedb-recipes

alph-notebooks/lancedb-recipes / main.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

Langchain & Llamaindex RAG comparison

This notebook compares Langchain & Llamaindex for understand which method is best extraction of table & text from PDF in the following

Here we have covered

Langchain RAG
Llamaindex RAG
Langchain wiht llamaparser
Llamaindex with llamaparser

from above this method will get idea about which is best method for table extraction for the following data used

[ ]

Collecting llama-index
  Downloading llama_index-0.10.37-py3-none-any.whl (6.8 kB)
Collecting llama-index-core
  Downloading llama_index_core-0.10.37.post1-py3-none-any.whl (15.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.4/15.4 MB 40.6 MB/s eta 0:00:00
Collecting llama-index-embeddings-openai
  Downloading llama_index_embeddings_openai-0.1.9-py3-none-any.whl (6.0 kB)
Collecting llama-parse
  Downloading llama_parse-0.4.3-py3-none-any.whl (7.7 kB)
Collecting llama-index-agent-openai<0.3.0,>=0.1.4 (from llama-index)
  Downloading llama_index_agent_openai-0.2.5-py3-none-any.whl (13 kB)
Collecting llama-index-cli<0.2.0,>=0.1.2 (from llama-index)
  Downloading llama_index_cli-0.1.12-py3-none-any.whl (26 kB)
Collecting llama-index-indices-managed-llama-cloud<0.2.0,>=0.1.2 (from llama-index)
  Downloading llama_index_indices_managed_llama_cloud-0.1.6-py3-none-any.whl (6.7 kB)
Collecting llama-index-legacy<0.10.0,>=0.9.48 (from llama-index)
  Downloading llama_index_legacy-0.9.48-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 52.0 MB/s eta 0:00:00
Collecting llama-index-llms-openai<0.2.0,>=0.1.13 (from llama-index)
  Downloading llama_index_llms_openai-0.1.19-py3-none-any.whl (11 kB)
Collecting llama-index-multi-modal-llms-openai<0.2.0,>=0.1.3 (from llama-index)
  Downloading llama_index_multi_modal_llms_openai-0.1.6-py3-none-any.whl (5.8 kB)
Collecting llama-index-program-openai<0.2.0,>=0.1.3 (from llama-index)
  Downloading llama_index_program_openai-0.1.6-py3-none-any.whl (5.2 kB)
Collecting llama-index-question-gen-openai<0.2.0,>=0.1.2 (from llama-index)
  Downloading llama_index_question_gen_openai-0.1.3-py3-none-any.whl (2.9 kB)
Collecting llama-index-readers-file<0.2.0,>=0.1.4 (from llama-index)
  Downloading llama_index_readers_file-0.1.22-py3-none-any.whl (36 kB)
Collecting llama-index-readers-llama-parse<0.2.0,>=0.1.2 (from llama-index)
  Downloading llama_index_readers_llama_parse-0.1.4-py3-none-any.whl (2.5 kB)
Requirement already satisfied: PyYAML>=6.0.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (6.0.1)
Requirement already satisfied: SQLAlchemy[asyncio]>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (2.0.30)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (3.9.5)
Collecting dataclasses-json (from llama-index-core)
  Downloading dataclasses_json-0.6.6-py3-none-any.whl (28 kB)
Collecting deprecated>=1.2.9.3 (from llama-index-core)
  Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
Collecting dirtyjson<2.0.0,>=1.0.8 (from llama-index-core)
  Downloading dirtyjson-1.0.8-py3-none-any.whl (25 kB)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (2023.6.0)
Collecting httpx (from llama-index-core)
  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 9.9 MB/s eta 0:00:00
Collecting jsonpath-ng<2.0.0,>=1.6.0 (from llama-index-core)
  Downloading jsonpath_ng-1.6.1-py3-none-any.whl (29 kB)
Collecting llamaindex-py-client<0.2.0,>=0.1.18 (from llama-index-core)
  Downloading llamaindex_py_client-0.1.19-py3-none-any.whl (141 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.9/141.9 kB 14.4 MB/s eta 0:00:00
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (3.3)
Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (3.8.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (1.25.2)
Collecting openai>=1.1.0 (from llama-index-core)
  Downloading openai-1.30.1-py3-none-any.whl (320 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 320.6/320.6 kB 26.6 MB/s eta 0:00:00
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (2.0.3)
Requirement already satisfied: pillow>=9.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (9.4.0)
Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (2.31.0)
Requirement already satisfied: spacy<4.0.0,>=3.7.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (3.7.4)
Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (8.3.0)
Collecting tiktoken>=0.3.3 (from llama-index-core)
  Downloading tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 54.8 MB/s eta 0:00:00
Requirement already satisfied: tqdm<5.0.0,>=4.66.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (4.66.4)
Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (4.11.0)
Collecting typing-inspect>=0.8.0 (from llama-index-core)
  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)
Requirement already satisfied: wrapt in /usr/local/lib/python3.10/dist-packages (from llama-index-core) (1.14.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core) (4.0.3)
Collecting ply (from jsonpath-ng<2.0.0,>=1.6.0->llama-index-core)
  Downloading ply-3.11-py2.py3-none-any.whl (49 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.6/49.6 kB 5.7 MB/s eta 0:00:00
Requirement already satisfied: beautifulsoup4<5.0.0,>=4.12.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index) (4.12.3)
Collecting pypdf<5.0.0,>=4.0.1 (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index)
  Downloading pypdf-4.2.0-py3-none-any.whl (290 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 290.4/290.4 kB 27.4 MB/s eta 0:00:00
Collecting striprtf<0.0.27,>=0.0.26 (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index)
  Downloading striprtf-0.0.26-py3-none-any.whl (6.9 kB)
Requirement already satisfied: pydantic>=1.10 in /usr/local/lib/python3.10/dist-packages (from llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core) (2.7.1)
Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core) (3.7.1)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core) (2024.2.2)
Collecting httpcore==1.* (from httpx->llama-index-core)
  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 10.0 MB/s eta 0:00:00
Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core) (3.7)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core) (1.3.1)
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx->llama-index-core)
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 6.2 MB/s eta 0:00:00
Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core) (8.1.7)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core) (1.4.2)
Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core) (2023.12.25)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.1.0->llama-index-core) (1.7.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index-core) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index-core) (2.0.7)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.2.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (8.2.3)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (0.9.4)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (6.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (3.1.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (67.7.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (24.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core) (3.4.0)
Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core) (3.0.3)
Collecting mypy-extensions>=0.3.0 (from typing-inspect>=0.8.0->llama-index-core)
  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json->llama-index-core)
  Downloading marshmallow-3.21.2-py3-none-any.whl (49 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.3/49.3 kB 4.4 MB/s eta 0:00:00
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core) (2023.4)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core) (2024.1)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->llama-index-core) (1.2.1)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4<5.0.0,>=4.12.3->llama-index-readers-file<0.2.0,>=0.1.4->llama-index) (2.5)
Requirement already satisfied: language-data>=1.2 in /usr/local/lib/python3.10/dist-packages (from langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core) (1.2.0)
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10->llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core) (0.6.0)
Requirement already satisfied: pydantic-core==2.18.2 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10->llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core) (2.18.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->llama-index-core) (1.16.0)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core) (0.1.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from weasel<0.4.0,>=0.1.0->spacy<4.0.0,>=3.7.1->llama-index-core) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->spacy<4.0.0,>=3.7.1->llama-index-core) (2.1.5)
Requirement already satisfied: marisa-trie>=0.7.7 in /usr/local/lib/python3.10/dist-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core) (1.1.1)
Installing collected packages: striprtf, ply, dirtyjson, pypdf, mypy-extensions, marshmallow, jsonpath-ng, h11, deprecated, typing-inspect, tiktoken, httpcore, httpx, dataclasses-json, openai, llamaindex-py-client, llama-index-legacy, llama-index-core, llama-parse, llama-index-readers-file, llama-index-llms-openai, llama-index-indices-managed-llama-cloud, llama-index-embeddings-openai, llama-index-readers-llama-parse, llama-index-multi-modal-llms-openai, llama-index-cli, llama-index-agent-openai, llama-index-program-openai, llama-index-question-gen-openai, llama-index
Successfully installed dataclasses-json-0.6.6 deprecated-1.2.14 dirtyjson-1.0.8 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 jsonpath-ng-1.6.1 llama-index-0.10.37 llama-index-agent-openai-0.2.5 llama-index-cli-0.1.12 llama-index-core-0.10.37.post1 llama-index-embeddings-openai-0.1.9 llama-index-indices-managed-llama-cloud-0.1.6 llama-index-legacy-0.9.48 llama-index-llms-openai-0.1.19 llama-index-multi-modal-llms-openai-0.1.6 llama-index-program-openai-0.1.6 llama-index-question-gen-openai-0.1.3 llama-index-readers-file-0.1.22 llama-index-readers-llama-parse-0.1.4 llama-parse-0.4.3 llamaindex-py-client-0.1.19 marshmallow-3.21.2 mypy-extensions-1.0.0 openai-1.30.1 ply-3.11 pypdf-4.2.0 striprtf-0.0.26 tiktoken-0.7.0 typing-inspect-0.9.0
Collecting llama-index-postprocessor-flag-embedding-reranker
  Downloading llama_index_postprocessor_flag_embedding_reranker-0.1.3-py3-none-any.whl (3.0 kB)
Requirement already satisfied: llama-index-core<0.11.0,>=0.10.35 in /usr/local/lib/python3.10/dist-packages (from llama-index-postprocessor-flag-embedding-reranker) (0.10.37.post1)
Requirement already satisfied: PyYAML>=6.0.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (6.0.1)
Requirement already satisfied: SQLAlchemy[asyncio]>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.0.30)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.9.5)
Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.6.6)
Requirement already satisfied: deprecated>=1.2.9.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.2.14)
Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.0.8)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2023.6.0)
Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.27.0)
Requirement already satisfied: jsonpath-ng<2.0.0,>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.6.1)
Requirement already satisfied: llamaindex-py-client<0.2.0,>=0.1.18 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.1.19)
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.3)
Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.8.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.25.2)
Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.30.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.0.3)
Requirement already satisfied: pillow>=9.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (9.4.0)
Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.31.0)
Requirement already satisfied: spacy<4.0.0,>=3.7.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.7.4)
Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (8.3.0)
Requirement already satisfied: tiktoken>=0.3.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.7.0)
Requirement already satisfied: tqdm<5.0.0,>=4.66.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (4.66.4)
Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (4.11.0)
Requirement already satisfied: typing-inspect>=0.8.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.9.0)
Requirement already satisfied: wrapt in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.14.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (4.0.3)
Requirement already satisfied: ply in /usr/local/lib/python3.10/dist-packages (from jsonpath-ng<2.0.0,>=1.6.0->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.11)
Requirement already satisfied: pydantic>=1.10 in /usr/local/lib/python3.10/dist-packages (from llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.7.1)
Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.7.1)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2024.2.2)
Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.0.5)
Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.7)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.3.1)
Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.14.0)
Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (8.1.7)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.4.2)
Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2023.12.25)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.1.0->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.7.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.0.7)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.2.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (8.2.3)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.9.4)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (6.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.1.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (67.7.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (24.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.4.0)
Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.0.3)
Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect>=0.8.0->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.0.0)
Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (3.21.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2023.4)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2024.1)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.2.1)
Requirement already satisfied: language-data>=1.2 in /usr/local/lib/python3.10/dist-packages (from langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.2.0)
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10->llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.6.0)
Requirement already satisfied: pydantic-core==2.18.2 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10->llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.18.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.16.0)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.1.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from weasel<0.4.0,>=0.1.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (2.1.5)
Requirement already satisfied: marisa-trie>=0.7.7 in /usr/local/lib/python3.10/dist-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama-index-postprocessor-flag-embedding-reranker) (1.1.1)
Installing collected packages: llama-index-postprocessor-flag-embedding-reranker
Successfully installed llama-index-postprocessor-flag-embedding-reranker-0.1.3
Collecting git+https://github.com/FlagOpen/FlagEmbedding.git
  Cloning https://github.com/FlagOpen/FlagEmbedding.git to /tmp/pip-req-build-wmws0zv2
  Running command git clone --filter=blob:none --quiet https://github.com/FlagOpen/FlagEmbedding.git /tmp/pip-req-build-wmws0zv2
  Resolved https://github.com/FlagOpen/FlagEmbedding.git to commit 95b873d9ac923bca47436efeae39ca4559970210
  Preparing metadata (setup.py) ... done
Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from FlagEmbedding==1.2.9) (2.2.1+cu121)
Requirement already satisfied: transformers>=4.33.0 in /usr/local/lib/python3.10/dist-packages (from FlagEmbedding==1.2.9) (4.40.2)
Collecting datasets (from FlagEmbedding==1.2.9)
  Downloading datasets-2.19.1-py3-none-any.whl (542 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 542.0/542.0 kB 6.6 MB/s eta 0:00:00
Collecting accelerate>=0.20.1 (from FlagEmbedding==1.2.9)
  Downloading accelerate-0.30.1-py3-none-any.whl (302 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 302.6/302.6 kB 10.6 MB/s eta 0:00:00
Collecting sentence_transformers (from FlagEmbedding==1.2.9)
  Downloading sentence_transformers-2.7.0-py3-none-any.whl (171 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 171.5/171.5 kB 9.6 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.20.1->FlagEmbedding==1.2.9) (1.25.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.20.1->FlagEmbedding==1.2.9) (24.0)
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.20.1->FlagEmbedding==1.2.9) (5.9.5)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.20.1->FlagEmbedding==1.2.9) (6.0.1)
Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.20.1->FlagEmbedding==1.2.9) (0.20.3)
Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.20.1->FlagEmbedding==1.2.9) (0.4.3)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->FlagEmbedding==1.2.9) (3.14.0)
Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->FlagEmbedding==1.2.9) (4.11.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->FlagEmbedding==1.2.9) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->FlagEmbedding==1.2.9) (3.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->FlagEmbedding==1.2.9) (3.1.4)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->FlagEmbedding==1.2.9) (2023.6.0)
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
Collecting nvidia-curand-cu12==10.3.2.106 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
Collecting nvidia-nccl-cu12==2.19.3 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl (166.0 MB)
Collecting nvidia-nvtx-cu12==12.1.105 (from torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
Requirement already satisfied: triton==2.2.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->FlagEmbedding==1.2.9) (2.2.0)
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.6.0->FlagEmbedding==1.2.9)
  Using cached nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (21.1 MB)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.33.0->FlagEmbedding==1.2.9) (2023.12.25)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers>=4.33.0->FlagEmbedding==1.2.9) (2.31.0)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.33.0->FlagEmbedding==1.2.9) (0.19.1)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.33.0->FlagEmbedding==1.2.9) (4.66.4)
Requirement already satisfied: pyarrow>=12.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->FlagEmbedding==1.2.9) (14.0.2)
Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets->FlagEmbedding==1.2.9) (0.6)
Collecting dill<0.3.9,>=0.3.0 (from datasets->FlagEmbedding==1.2.9)
  Downloading dill-0.3.8-py3-none-any.whl (116 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.3/116.3 kB 15.4 MB/s eta 0:00:00
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets->FlagEmbedding==1.2.9) (2.0.3)
Collecting xxhash (from datasets->FlagEmbedding==1.2.9)
  Downloading xxhash-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 194.1/194.1 kB 11.2 MB/s eta 0:00:00
Collecting multiprocess (from datasets->FlagEmbedding==1.2.9)
  Downloading multiprocess-0.70.16-py310-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.8/134.8 kB 16.2 MB/s eta 0:00:00
Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets->FlagEmbedding==1.2.9) (3.9.5)
Collecting huggingface-hub (from accelerate>=0.20.1->FlagEmbedding==1.2.9)
  Downloading huggingface_hub-0.23.0-py3-none-any.whl (401 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 401.2/401.2 kB 14.4 MB/s eta 0:00:00
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from sentence_transformers->FlagEmbedding==1.2.9) (1.2.2)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sentence_transformers->FlagEmbedding==1.2.9) (1.11.4)
Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from sentence_transformers->FlagEmbedding==1.2.9) (9.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->FlagEmbedding==1.2.9) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->FlagEmbedding==1.2.9) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->FlagEmbedding==1.2.9) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->FlagEmbedding==1.2.9) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->FlagEmbedding==1.2.9) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->FlagEmbedding==1.2.9) (4.0.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.33.0->FlagEmbedding==1.2.9) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.33.0->FlagEmbedding==1.2.9) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.33.0->FlagEmbedding==1.2.9) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.33.0->FlagEmbedding==1.2.9) (2024.2.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.6.0->FlagEmbedding==1.2.9) (2.1.5)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->FlagEmbedding==1.2.9) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->FlagEmbedding==1.2.9) (2023.4)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->FlagEmbedding==1.2.9) (2024.1)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence_transformers->FlagEmbedding==1.2.9) (1.4.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence_transformers->FlagEmbedding==1.2.9) (3.5.0)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.6.0->FlagEmbedding==1.2.9) (1.3.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->datasets->FlagEmbedding==1.2.9) (1.16.0)
Building wheels for collected packages: FlagEmbedding
  Building wheel for FlagEmbedding (setup.py) ... done
  Created wheel for FlagEmbedding: filename=FlagEmbedding-1.2.9-py3-none-any.whl size=165917 sha256=24688c17b3bc6214be93c7bef77d4b9baacde749336b83f600188a704c6d8cad
  Stored in directory: /tmp/pip-ephem-wheel-cache-45wml86h/wheels/41/cf/a5/5dee96ed64e5aaffe5aa3d583828258fdefed9a305db6e7f48
Successfully built FlagEmbedding
Installing collected packages: xxhash, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, dill, nvidia-cusparse-cu12, nvidia-cudnn-cu12, multiprocess, huggingface-hub, nvidia-cusolver-cu12, datasets, sentence_transformers, accelerate, FlagEmbedding
  Attempting uninstall: huggingface-hub
    Found existing installation: huggingface-hub 0.20.3
    Uninstalling huggingface-hub-0.20.3:
      Successfully uninstalled huggingface-hub-0.20.3
Successfully installed FlagEmbedding-1.2.9 accelerate-0.30.1 datasets-2.19.1 dill-0.3.8 huggingface-hub-0.23.0 multiprocess-0.70.16 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvjitlink-cu12-12.4.127 nvidia-nvtx-cu12-12.1.105 sentence_transformers-2.7.0 xxhash-3.4.1
Collecting llama-index-vector-stores-lancedb
  Downloading llama_index_vector_stores_lancedb-0.1.3-py3-none-any.whl (4.1 kB)
Collecting lancedb<0.6.0,>=0.5.1 (from llama-index-vector-stores-lancedb)
  Downloading lancedb-0.5.7-py3-none-any.whl (115 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.1/115.1 kB 3.2 MB/s eta 0:00:00
Requirement already satisfied: llama-index-core<0.11.0,>=0.10.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-vector-stores-lancedb) (0.10.37.post1)
Collecting deprecation (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb)
  Downloading deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pylance==0.9.18 (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb)
  Downloading pylance-0.9.18-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (21.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.6/21.6 MB 14.3 MB/s eta 0:00:00
Collecting ratelimiter~=1.0 (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb)
  Downloading ratelimiter-1.2.0.post0-py3-none-any.whl (6.6 kB)
Collecting retry>=0.9.2 (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb)
  Downloading retry-0.9.2-py2.py3-none-any.whl (8.0 kB)
Requirement already satisfied: tqdm>=4.27.0 in /usr/local/lib/python3.10/dist-packages (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (4.66.4)
Requirement already satisfied: pydantic>=1.10 in /usr/local/lib/python3.10/dist-packages (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (2.7.1)
Requirement already satisfied: attrs>=21.3.0 in /usr/local/lib/python3.10/dist-packages (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (23.2.0)
Collecting semver>=3.0 (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb)
  Downloading semver-3.0.2-py3-none-any.whl (17 kB)
Requirement already satisfied: cachetools in /usr/local/lib/python3.10/dist-packages (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (5.3.3)
Requirement already satisfied: pyyaml>=6.0 in /usr/local/lib/python3.10/dist-packages (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (6.0.1)
Requirement already satisfied: click>=8.1.7 in /usr/local/lib/python3.10/dist-packages (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (8.1.7)
Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (2.31.0)
Collecting overrides>=0.7 (from lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb)
  Downloading overrides-7.7.0-py3-none-any.whl (17 kB)
Requirement already satisfied: pyarrow>=12 in /usr/local/lib/python3.10/dist-packages (from pylance==0.9.18->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (14.0.2)
Requirement already satisfied: numpy>=1.22 in /usr/local/lib/python3.10/dist-packages (from pylance==0.9.18->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (1.25.2)
Requirement already satisfied: SQLAlchemy[asyncio]>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2.0.30)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.9.5)
Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.6.6)
Requirement already satisfied: deprecated>=1.2.9.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.2.14)
Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.0.8)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2023.6.0)
Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.27.0)
Requirement already satisfied: jsonpath-ng<2.0.0,>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.6.1)
Requirement already satisfied: llamaindex-py-client<0.2.0,>=0.1.18 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.1.19)
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.3)
Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.8.1)
Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.30.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2.0.3)
Requirement already satisfied: pillow>=9.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (9.4.0)
Requirement already satisfied: spacy<4.0.0,>=3.7.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.7.4)
Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (8.3.0)
Requirement already satisfied: tiktoken>=0.3.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.7.0)
Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (4.11.0)
Requirement already satisfied: typing-inspect>=0.8.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.9.0)
Requirement already satisfied: wrapt in /usr/local/lib/python3.10/dist-packages (from llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.14.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.3.1)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (4.0.3)
Requirement already satisfied: ply in /usr/local/lib/python3.10/dist-packages (from jsonpath-ng<2.0.0,>=1.6.0->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.11)
Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.7.1)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2024.2.2)
Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.0.5)
Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.7)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.3.1)
Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.14.0)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.4.2)
Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2023.12.25)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.1.0->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.7.0)
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (0.6.0)
Requirement already satisfied: pydantic-core==2.18.2 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (2.18.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (2.0.7)
Requirement already satisfied: decorator>=3.4.2 in /usr/local/lib/python3.10/dist-packages (from retry>=0.9.2->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb) (4.4.2)
Collecting py<2.0.0,>=1.4.26 (from retry>=0.9.2->lancedb<0.6.0,>=0.5.1->llama-index-vector-stores-lancedb)
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 kB 11.8 MB/s eta 0:00:00
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.2.2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (8.2.3)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.9.4)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (6.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.1.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (67.7.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (24.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.10/dist-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.4.0)
Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.0.3)
Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect>=0.8.0->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.0.0)
Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (3.21.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2023.4)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2024.1)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.2.1)
Requirement already satisfied: language-data>=1.2 in /usr/local/lib/python3.10/dist-packages (from langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.2.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.16.0)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.1.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from weasel<0.4.0,>=0.1.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (2.1.5)
Requirement already satisfied: marisa-trie>=0.7.7 in /usr/local/lib/python3.10/dist-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.1->llama-index-vector-stores-lancedb) (1.1.1)
Installing collected packages: ratelimiter, semver, py, overrides, deprecation, retry, pylance, lancedb, llama-index-vector-stores-lancedb
Successfully installed deprecation-2.1.0 lancedb-0.5.7 llama-index-vector-stores-lancedb-0.1.3 overrides-7.7.0 py-1.11.0 pylance-0.9.18 ratelimiter-1.2.0.post0 retry-0.9.2 semver-3.0.2

[ ]

Download the PDF (contains both tables & text)

[ ]

1. Langchain with Q&A on PDF

[ ]

'The net loss value attributable to Uber Technologies, Inc. for the period was $5.9 billion, compared to $108 million in the same period the previous year. This represents a significant increase in net loss year-over-year.'

[ ]

"I don't know."

[ ]

"I don't have detailed charts of intangible assets, net as of December 31, 2021 and March 31, 2022."

FOR QUERY 2 & QUERY 3, We didn't got any output.

LETS TRY TO DO IT WITH LLAMAINDEX

2 . Llamaindex with Q&A on PDF

[ ]

The net loss attributable to Uber Technologies, Inc. was $5.9 billion in the current period, compared to a net loss of $108 million in the same period last year.

[ ]

The detailed charts of intangible assets, net as of December 31, 2021 and March 31, 2022 are as follows:

**As of December 31, 2021:**
- Consumer, Merchant and other relationships: $1,574 million
- Developed technology: $653 million
- Trade names and trademarks: $175 million
- Patents: $8 million
- Other: $2 million
- Total Intangible assets: $2,412 million

**As of March 31, 2022:**
- Consumer, Merchant and other relationships: $1,494 million
- Developed technology: $599 million
- Trade names and trademarks: $167 million
- Patents: $7 million
- Other: $2 million
- Total Intangible assets: $2,269 million

[ ]

Adjusted EBITDA for 2021 was a loss of $359 million, while for 2022 it improved to $168 million. Interest expense for the period increased from $115 million in 2021 to $129 million in 2022.

3 Llamaparser with Langchain on PDF

we are simply saving all llamaparser output in .md file & based on that we are doing Q& A. there are better methods also to add llamaparser with Langchain but lets do this experiment

[ ]

Started parsing the file under job_id 6aef2258-0e5c-4796-8ea1-f82e817cb542

[ ]


  0%|          | 0/1 [00:00<?, ?it/s][nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /root/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.

100%|██████████| 1/1 [00:12<00:00, 12.93s/it]

[ ]

Document(page_content='Document\n\nUNITED STATES SECURITIES AND EXCHANGE COMMISSION Washington, D.C. 20549\n\nFORM 10-Q\n\n(Mark One)\n\nQUARTERLY REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934 For the quarterly period ended March 31, 2022\n\nCommission File Number: 001-38902\n\nUBER TECHNOLOGIES, INC.\n\n(Exact name of registrant as specified in its charter)\n\nDelaware 45-2647441\n\n(State or other jurisdiction of incorporation or organization) 1515 3rd Street (I.R.S. Employer Identification No.)\n\nSan Francisco, California 94158\n\n(Address of principal executive offices, including zip code) (415) 612-8582\n\n(Registrant’s telephone number, including area code)\n\nSecurities registered pursuant to Section 12(b) of the Act:\n\nTitle of each class Trading Symbol(s) Name of each exchange on which registered Common Stock, par value $0.00001 per share UBER New York Stock Exchange\n\nIndicate by check mark whether the registrant (1) has filed all reports required to be filed by Section 13 or 15(d) of the Securities Exchange Act of 1934 during the preceding 12 months (or for such shorter period that the registrant was required to file such reports), and (2) has been subject to such filing requirements for the past 90 days. Yes ☒ No ☐\n\nIndicate by check mark whether the registrant has submitted electronically every Interactive Data File required to be submitted pursuant to Rule 405 of Regulation S-T (§232.405 of this chapter) during the preceding 12 months (or for such shorter period that the registrant was required to submit such files). Yes ☒ No ☐\n\nIndicate by check mark whether the registrant is a large accelerated filer, an accelerated filer, a non-accelerated filer, a smaller reporting company, or an emerging growth company. See the definitions of “large accelerated filer,” “accelerated filer,” “smaller reporting company,” and “emerging growth company” in Rule 12b-2 of the Exchange Act.', metadata={'source': '/content/data/output.md'})

[ ]

'The net loss attributable to Uber Technologies, Inc. for the first quarter of 2022 was $5.9 billion, compared to a net loss of $108 million in the same period in 2021. This represents a significant increase in net loss compared to the previous year.'

[ ]

'The Cash paid for Income taxes, net of refunds is not specifically mentioned in the provided context.'

[ ]

'Detailed charts of intangible assets, net as of December 31, 2021 and March 31, 2022, are as follows:\n\nAs of December 31, 2021:\n- Consumer, Merchant and other relationships: $1,494 million\n- Developed technology: $599 million\n- Trade names and trademarks: $167 million\n- Patents: $7 million\n- Other: $2 million\n\nAs of March 31, 2022:\n- Consumer, Merchant and other relationships: $1,574 million\n- Developed technology: $653 million\n- Trade names and trademarks: $175 million\n- Patents: $8 million\n- Other: $2 million'

Till now, We are not getting all answers but now lets try the llamaparser with Llamaindex to see the results

4. Llamaparse with Llamaindex on PDF

[ ]

Started parsing the file under job_id 69ea99de-dd19-46db-86d5-e03d2027a7ce

/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(

Adjusted EBITDA improved by $527 million from a loss of $359 million in 2021 to $168 million in 2022. Interest expense increased by an immaterial amount.

[ ]

| |Gross Carrying Value|Accumulated Amortization|Net Carrying Value|Useful Life - Years|
|---|---|---|---|---|
|Consumer, Merchant and other relationships|$1,868|$(294)|$1,574|9|
|Developed technology|$922|$(269)|$653|5|
|Trade names and trademarks|$222|$(47)|$175|6|
|Patents|$15|$(7)|$8|7|
|Other|$5|$(3)|$2|0|

| |Gross Carrying Value|Accumulated Amortization|Net Carrying Value|Useful Life - Years|
|---|---|---|---|---|
|Consumer, Merchant and other relationships|$1,856|$(362)|$1,494|9|
|Developed technology|$924|$(325)|$599|5|
|Trade names and trademarks|$222|$(55)|$167|6|
|Patents|$15|$(8)|$7|6|
|Other|$5|$(3)|$2|0|

[ ]


***********New pdf+ lancedb ***********
The cash paid for income taxes, net of refunds, was $22 million for the three months ended March 31, 2021, and $41 million for the three months ended March 31, 2022.

WOW 🤩 the answers are more clear & better compaired to other methods .thats power of Llamaparser .this Llamaparser with LlamaIndex is doing quite well on table & text data from PDF.