Langchain Simple Pdf
Multilanguage RAG filtering by multiple PDFs with Langchain and OpenAi
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. langchain-openai 0.3.27 requires langchain-core<1.0.0,>=0.3.66, but you have langchain-core 1.1.3 which is incompatible. langchain 0.3.26 requires langchain-core<1.0.0,>=0.3.66, but you have langchain-core 1.1.3 which is incompatible. langchain 0.3.26 requires langchain-text-splitters<1.0.0,>=0.3.8, but you have langchain-text-splitters 1.0.0 which is incompatible. Note: you may need to restart the kernel to use updated packages. Requirement already satisfied: langchain-openai in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (0.3.27) Requirement already satisfied: tiktoken in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (0.9.0) Requirement already satisfied: langchain in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (0.3.26) Requirement already satisfied: pypdf in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (5.7.0) Collecting langchain-core<1.0.0,>=0.3.66 (from langchain-openai) Using cached langchain_core-0.3.80-py3-none-any.whl.metadata (3.2 kB) Requirement already satisfied: openai<2.0.0,>=1.86.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain-openai) (1.93.0) Requirement already satisfied: regex>=2022.1.18 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from tiktoken) (2024.11.6) Requirement already satisfied: requests>=2.26.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from tiktoken) (2.32.5) Collecting langchain-text-splitters<1.0.0,>=0.3.8 (from langchain) Using cached langchain_text_splitters-0.3.11-py3-none-any.whl.metadata (1.8 kB) Requirement already satisfied: langsmith>=0.1.17 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain) (0.4.4) Requirement already satisfied: pydantic<3.0.0,>=2.7.4 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain) (2.11.7) Requirement already satisfied: SQLAlchemy<3,>=1.4 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain) (2.0.41) Requirement already satisfied: PyYAML>=5.3 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain) (6.0.2) Requirement already satisfied: tenacity!=8.4.0,<10.0.0,>=8.1.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain-core<1.0.0,>=0.3.66->langchain-openai) (9.1.2) Requirement already satisfied: jsonpatch<2.0.0,>=1.33.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain-core<1.0.0,>=0.3.66->langchain-openai) (1.33) Requirement already satisfied: typing-extensions<5.0.0,>=4.7.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain-core<1.0.0,>=0.3.66->langchain-openai) (4.14.1) Requirement already satisfied: packaging<26.0.0,>=23.2.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langchain-core<1.0.0,>=0.3.66->langchain-openai) (24.2) Requirement already satisfied: httpx<1,>=0.23.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langsmith>=0.1.17->langchain) (0.28.1) Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langsmith>=0.1.17->langchain) (3.10.18) Requirement already satisfied: requests-toolbelt<2.0.0,>=1.0.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langsmith>=0.1.17->langchain) (1.0.0) Requirement already satisfied: zstandard<0.24.0,>=0.23.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from langsmith>=0.1.17->langchain) (0.23.0) Requirement already satisfied: anyio<5,>=3.5.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from openai<2.0.0,>=1.86.0->langchain-openai) (4.9.0) Requirement already satisfied: distro<2,>=1.7.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from openai<2.0.0,>=1.86.0->langchain-openai) (1.9.0) Requirement already satisfied: jiter<1,>=0.4.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from openai<2.0.0,>=1.86.0->langchain-openai) (0.10.0) Requirement already satisfied: sniffio in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from openai<2.0.0,>=1.86.0->langchain-openai) (1.3.1) Requirement already satisfied: tqdm>4 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from openai<2.0.0,>=1.86.0->langchain-openai) (4.67.1) Requirement already satisfied: annotated-types>=0.6.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from pydantic<3.0.0,>=2.7.4->langchain) (0.7.0) Requirement already satisfied: pydantic-core==2.33.2 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from pydantic<3.0.0,>=2.7.4->langchain) (2.33.2) Requirement already satisfied: typing-inspection>=0.4.0 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from pydantic<3.0.0,>=2.7.4->langchain) (0.4.1) Requirement already satisfied: charset_normalizer<4,>=2 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from requests>=2.26.0->tiktoken) (3.4.2) Requirement already satisfied: idna<4,>=2.5 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from requests>=2.26.0->tiktoken) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from requests>=2.26.0->tiktoken) (2.5.0) Requirement already satisfied: certifi>=2017.4.17 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from requests>=2.26.0->tiktoken) (2025.6.15) Requirement already satisfied: httpcore==1.* in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from httpx<1,>=0.23.0->langsmith>=0.1.17->langchain) (1.0.9) Requirement already satisfied: h11>=0.16 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->langsmith>=0.1.17->langchain) (0.16.0) Requirement already satisfied: jsonpointer>=1.9 in /Users/dudanogueira/dev/weaviate/recipes/.env/lib/python3.13/site-packages (from jsonpatch<2.0.0,>=1.33.0->langchain-core<1.0.0,>=0.3.66->langchain-openai) (3.0.0) Using cached langchain_core-0.3.80-py3-none-any.whl (450 kB) Using cached langchain_text_splitters-0.3.11-py3-none-any.whl (33 kB) Installing collected packages: langchain-core, langchain-text-splitters Attempting uninstall: langchain-core Found existing installation: langchain-core 1.1.3 Uninstalling langchain-core-1.1.3: Successfully uninstalled langchain-core-1.1.3 Attempting uninstall: langchain-text-splitters Found existing installation: langchain-text-splitters 1.0.0 Uninstalling langchain-text-splitters-1.0.0: Successfully uninstalled langchain-text-splitters-1.0.0 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. langchain-classic 1.0.0 requires langchain-core<2.0.0,>=1.0.0, but you have langchain-core 0.3.80 which is incompatible. langchain-classic 1.0.0 requires langchain-text-splitters<2.0.0,>=1.0.0, but you have langchain-text-splitters 0.3.11 which is incompatible. langchain-community 0.4.1 requires langchain-core<2.0.0,>=1.0.1, but you have langchain-core 0.3.80 which is incompatible. Successfully installed langchain-core-0.3.80 langchain-text-splitters-0.3.11 [notice] A new release of pip is available: 25.0 -> 25.3 [notice] To update, run: pip install --upgrade pip Note: you may need to restart the kernel to use updated packages.
you must have a valid key for OpenAi in OPENAI_API_KEY environment variable
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"warning","log_level_env":"","msg":"log level not recognized, defaulting to info","time":"2025-12-10T20:41:44-03:00"}
{"action":"startup","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"Feature flag LD integration disabled: could not locate WEAVIATE_LD_API_KEY env variable","time":"2025-12-10T20:41:44-03:00"}
{"action":"startup","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","default_vectorizer_module":"none","level":"info","msg":"the default vectorizer modules is set to \"none\", as a result all new schema classes without an explicit vectorizer setting, will use this vectorizer","time":"2025-12-10T20:41:44-03:00"}
{"action":"startup","auto_schema_enabled":{},"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"auto schema enabled setting is set to \"\u0026{\u003cnil\u003e {{{} {0 0}} 0 0 {{} 0} {{} 0}} true}\"","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"No resource limits set, weaviate will use all available memory and CPU. To limit resources, set LIMIT_RESOURCES=true","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"module offload-s3 is enabled","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","flag_key":"collection-retrieval-strategy","level":"info","msg":"feature flag instantiated","time":"2025-12-10T20:41:44-03:00","tool":"feature_flag","value":"LeaderOnly"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"warning","msg":"Multiple vector spaces are present, GraphQL Explore and REST API list objects endpoint module include params has been disabled as a result.","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"open cluster service","servers":{"Embedded_at_8079":53928},"time":"2025-12-10T20:41:44-03:00"}
{"address":"192.168.28.206:53929","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"starting cloud rpc server ...","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"starting raft sub-system ...","time":"2025-12-10T20:41:44-03:00"}
{"address":"192.168.28.206:53928","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"tcp transport","tcpMaxPool":3,"tcpTimeout":10000000000,"time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"loading local db","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"local DB successfully loaded","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"schema manager loaded","n":0,"time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","metadata_only_voters":false,"msg":"construct a new raft node","name":"Embedded_at_8079","time":"2025-12-10T20:41:44-03:00"}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","index":1077,"level":"info","msg":"initial configuration","servers":"[[{Suffrage:Voter ID:Embedded_at_8079 Address:192.168.28.30:52799}]]","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","last_snapshot_index":0,"last_store_applied_index_on_start":1057,"level":"info","msg":"raft node constructed","raft_applied_index":0,"raft_last_index":1077,"time":"2025-12-10T20:41:44-03:00"}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","follower":{},"leader-address":"","leader-id":"","level":"info","msg":"entering follower state","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","hasState":true,"level":"info","msg":"raft init","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"attempting to join","remoteNodes":{"Embedded_at_8079":"192.168.28.206:53928"},"time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"attempted to join and failed","remoteNode":"192.168.28.206:53928","status":8,"time":"2025-12-10T20:41:44-03:00"}
{"action":"read_disk_use","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"warning","msg":"disk usage currently at 81.82%, threshold set to 80.00%","path":"/Users/dudanogueira/.local/share/weaviate","time":"2025-12-10T20:41:44-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"attempting to join","remoteNodes":{"Embedded_at_8079":"192.168.28.206:53928"},"time":"2025-12-10T20:41:45-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"attempted to join and failed","remoteNode":"192.168.28.206:53928","status":8,"time":"2025-12-10T20:41:45-03:00"}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","last-leader-addr":"","last-leader-id":"","level":"warning","msg":"heartbeat timeout reached, starting election","time":"2025-12-10T20:41:45-03:00"}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"entering candidate state","node":{},"term":238,"time":"2025-12-10T20:41:45-03:00"}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"pre-vote successful, starting election","refused":0,"tally":1,"term":238,"time":"2025-12-10T20:41:45-03:00","votesNeeded":1}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"election won","tally":1,"term":238,"time":"2025-12-10T20:41:45-03:00"}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","leader":{},"level":"info","msg":"entering leader state","time":"2025-12-10T20:41:45-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","last_store_log_applied_index":1057,"level":"info","log_index":1057,"log_name":"LogCommand","log_type":0,"msg":"reloading local DB as RAFT and local DB are now caught up","time":"2025-12-10T20:41:45-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"reload local db: update schema ...","time":"2025-12-10T20:41:45-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","index":"Collection1","level":"info","msg":"reload local index","time":"2025-12-10T20:41:45-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","index":"WikipediaLangChain","level":"info","msg":"reload local index","time":"2025-12-10T20:41:45-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"configured versions","server_version":"1.30.5","time":"2025-12-10T20:41:46-03:00","version":"1.30.5"}
{"action":"grpc_startup","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"grpc server listening at [::]:50050","time":"2025-12-10T20:41:46-03:00"}
{"action":"restapi_management","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"Serving weaviate at http://127.0.0.1:8079","time":"2025-12-10T20:41:46-03:00","version":"1.30.5"}
{"address":"192.168.28.206:53928","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"current Leader","time":"2025-12-10T20:41:46-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"attempting to join","remoteNodes":{"Embedded_at_8079":"192.168.28.206:53928"},"time":"2025-12-10T20:41:46-03:00"}
{"action":"raft","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","command":0,"level":"info","msg":"updating configuration","server-addr":"192.168.28.206:53928","server-id":"Embedded_at_8079","servers":"[[{Suffrage:Voter ID:Embedded_at_8079 Address:192.168.28.206:53928}]]","time":"2025-12-10T20:41:46-03:00"}
Client is Ready? True
Let's check our Client and Server Version:
Client: 4.15.4, Server: 1.30.5
Let's create our Collection beforehand
this will ensure the collection is created with a vectorizer and generative config. Make sure to use the same model while creating and passing the embeddings to langchain
{"action":"load_all_shards","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"error","msg":"failed to load all shards: context canceled","time":"2025-12-10T20:41:46-03:00"}
Now we have a Weaviate client! Let's read our 2 pdf files, brazil-wikipedia-article-text.pdf and netherlands-wikipedia-article-text.pdf
Then chunk them and ingest using Langchain.
{"action":"hnsw_prefill_cache_async","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"not waiting for vector cache prefill, running in background","time":"2025-12-10T20:41:47-03:00","wait_for_cache_prefill":false}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"Completed loading shard collection1_9lo68dI5wYDu in 1.971417ms","time":"2025-12-10T20:41:47-03:00"}
{"action":"hnsw_vector_cache_prefill","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","count":3000,"index_id":"main","level":"info","limit":1000000000000,"msg":"prefilled vector cache","time":"2025-12-10T20:41:47-03:00","took":133208}
{"action":"telemetry_push","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"telemetry started","payload":"\u0026{MachineID:85ee30ca-f2c2-4319-bb6e-eae79c18ffa2 Type:INIT Version:1.30.5 ObjectsCount:0 OS:darwin Arch:arm64 UsedModules:[generative-openai text2vec-openai] CollectionsCount:2}","time":"2025-12-10T20:41:47-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"warning","msg":"prop len tracker file /Users/dudanogueira/.local/share/weaviate/wikipedialangchain/enDIrImIqqRL/proplengths does not exist, creating new tracker","time":"2025-12-10T20:41:48-03:00"}
{"action":"hnsw_prefill_cache_async","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"not waiting for vector cache prefill, running in background","time":"2025-12-10T20:41:48-03:00","wait_for_cache_prefill":false}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"Created shard wikipedialangchain_enDIrImIqqRL in 1.165833ms","time":"2025-12-10T20:41:48-03:00"}
{"action":"hnsw_vector_cache_prefill","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","count":1000,"index_id":"main","level":"info","limit":1000000000000,"msg":"prefilled vector cache","time":"2025-12-10T20:41:48-03:00","took":36167}
GOT 247 chunks for Brazil GOT 274 chunks for Netherlands
lets first use our collection
let's count how many objects we have in total
AggregateReturn(properties={}, total_count=521)
Now, how many objects we have per source?
netherlands-wikipedia-article-text.pdf 274 brazil-wikipedia-article-text.pdf 247
Langchain added some metadata, like source page. Let's get one object.
dict_keys(['creationdate', 'page_label', 'total_pages', 'source', 'page', 'title', 'creator', 'text', 'producer'])
netherlands-wikipedia-article-text.pdf 2.0 "Frisian freedom"), which resented the imposition of the feudal system. Around 1000 AD, due to several agricultural developments, the economy started to develop at a fast pace, and the higher productivity allowed workers to farm more land or become tradesmen. Towns grew around monasteries and castles, and a
Let's ask in French, a content in English
lets do a RAG directly using only Weaviate
Objects matched: 8 Text The traditional food of Brazil is diverse and varies by region, reflecting the country's mix of indigenous and immigrant populations. Some notable examples include: - **Feijoada**: Considered the national dish of Brazil, it is a hearty stew of black beans with pork or beef. - **Beiju**: A type of tapioca pancake. - **Feijão Tropeiro**: A dish made with beans, eggs, and manioc flour. - **Vatapá**: A creamy dish made from bread, shrimp, coconut milk, and peanuts. - **Moqueca**: A fish stew made with coconut milk and palm oil. Common meals often consist of rice and beans, typically served with beef, salad, french fries, and a fried egg. Popular snacks include **pastel** (fried pastry), **coxinha** (chicken croquette), and **pão de queijo** (cheese bread). Desserts such as **brigadeiros** (chocolate fudge balls) and **bolo de rolo** (roll cake with guava paste) are also popular. The national beverage is coffee, and **cachaça**, a liquor made from sugar cane, is the main ingredient in the famous cocktail **Caipirinha**.
Objects used for this generative search
#### page: 13.0 #### Cuisine Brazilian cuisine varies greatly by region, reflecting the country's varying mix of indigenous and immigrant populations. This has created a national cuisine marked by the preservation of regional differences. Examples are Feijoada, considered the country's national dish; and regional foods such as beiju, feijão tropeiro, vatapá, moqueca, polenta (from Italian cuisine) and #### page: 13.0 #### flour (farofa). Fried potatoes, fried cassava, fried banana, fried meat and fried cheese are very often eaten in lunch and served in most typical restaurants. Popular snacks are pastel (a fried pastry); coxinha (a variation of chicken croquete); pão de queijo (cheese bread and cassava flour / tapioca); pamonha (corn and milk paste); esfirra (a variation of Lebanese #### page: 13.0 #### acarajé (from African cuisine). The national beverage is coffee and cachaça is Brazil's native liquor. Cachaça is distilled from sugar cane and is the main ingredient in the national cocktail, Caipirinha. A typical meal consists mostly of rice and beans with beef, salad, french fries and a fried egg. Often, it is mixed with cassava #### page: 13.0 #### pastry); kibbeh (from Arabic cuisine); empanada (pastry) and empada, little salt pies filled with shrimps or heart of palm. Brazil has a variety of desserts such as brigadeiros (chocolate fudge balls), bolo de rolo (roll cake with goiabada), cocada (a coconut sweet), beijinhos (coconut truffles and clove) and Romeu e Julieta (cheese with goiabada). Peanuts are used to #### page: 13.0 #### make paçoca, rapadura and pé-de-moleque. Local common fruits like açaí, cupuaçu, mango, papaya, cocoa, cashew, guava, orange, lime, passionfruit, pineapple, and hog plum are turned in juices and used to make chocolates, ice pops and ice cream. Cinema The Brazilian film industry began in the late 19th century, during the early days of the Belle Époque. While there were
Note that we used a filter, so the content will be searched and generated only for that specific pdf. Let's change the filter to the second pdf file.
The traditional food of the Netherlands includes several notable dishes. One popular item is **kibbeling**, which consists of small chunks of battered white fish and has become a national fast food. Another traditional dish is **lekkerbek**. The typical Dutch dinner traditionally consists of potatoes, a portion of meat, and seasonal vegetables. In terms of pastries, the **Vlaai** from Limburg and the **Moorkop** and **Bossche Bol** from Brabant are well-known. Additionally, **worstenbroodje**, which is a roll with a sausage of ground beef, is a popular savory pastry. For sweets, **stroopwafel** is a famous cookie that contains a lot of butter and sugar, often filled with something like almond paste, known as **gevulde koek**. The traditional alcoholic beverages include beer and **Jenever**.
And of course, we can use different filters, and get different content for our questions.
A common cultural aspect of Brazil and the Netherlands is their rich and diverse culinary traditions, which reflect a blend of indigenous and immigrant influences. In Brazil, the cuisine varies greatly by region, showcasing a mix of indigenous ingredients and dishes influenced by African, Portuguese, and other European cultures, with Feijoada being a notable national dish. Similarly, the Netherlands has a culinary heritage that includes traditional foods like cheese and Dutch pastries, as well as influences from its colonial past, particularly from Indonesia and Suriname. Both countries celebrate their unique culinary identities, which are shaped by their historical interactions and cultural exchanges.
Using Langchain to query data and answer questions
Up until now, we used Langchain to ingest our data, and we queried Weaviate directly.
Now, let's use Langchain also to query. If you noticed, after ingesting our data, langchain will return us a vectorstore.
We can use that vector store, or initiate a new one. Let's initiate a new one, passing an empty docs []
We can now search our data
{'title': 'Netherlands - Wikipedia Text Only, Convert to PDF', 'source': 'netherlands-wikipedia-article-text.pdf', 'page': 14.0, 'uuid': 'a5a9b02b-9e59-4d6a-aac1-71bc0775d8cb'}
widely available and typical for the region.
Kibbeling
, once a local delicacy consisting of small chunks of battered white fish, has
become a national fast food, just as lekkerbek.
The Southern Dutch cuisine consists of the cuisines of the Dutch provinces of North Brabant and Limburg and the Flemish Region in
###############
{'title': 'Netherlands - Wikipedia Text Only, Convert to PDF', 'source': 'netherlands-wikipedia-article-text.pdf', 'page': 14.0, 'uuid': '40651f32-4456-4737-8bce-97fc149f95c7'}
amount of fish. The various dried sausages, belonging to the metworst-family of Dutch sausages are found throughout this region.
Also smoked sausages are common, of which (
Gelderse
)
rookworst
is the most renowned. Larger sausages are often eaten
alongside
stamppot
,
hutspot
or
zuurkool
(sauerkraut); whereas smaller ones are often eaten as a street food. The provinces are
###############
{'title': 'Netherlands - Wikipedia Text Only, Convert to PDF', 'source': 'netherlands-wikipedia-article-text.pdf', 'page': 14.0, 'uuid': '637a5fa6-7fbd-40d8-b79d-651cdded43f6'}
cream, custard or fruits. Cakes, such as the
Vlaai
from Limburg and the
Moorkop
and
Bossche Bol
from Brabant, are typical
pastries. Savoury pastries also occur, with the
worstenbroodje
(a roll with a sausage of ground beef, literally translates into sausage
bread) being the most popular. The traditional alcoholic beverage of the region is beer. There are many local brands, ranging from
###############
{'title': 'Netherlands - Wikipedia Text Only, Convert to PDF', 'source': 'netherlands-wikipedia-article-text.pdf', 'page': 14.0, 'uuid': '03b44657-bebb-4042-b87f-7538504fd992'}
(in its modern form) and
Zeeuwse bolus
are
good examples. Cookies are also produced in great number and tend to contain a lot of butter and sugar, like
stroopwafel
, as well
as a filling of some kind, mostly almond, like
gevulde koek
. The traditional alcoholic beverages of this region are beer (strong pale
lager) and
Jenever
###############
{'title': 'Brazil - Wikipedia Text Only, Convert to PDF', 'source': 'brazil-wikipedia-article-text.pdf', 'page': 13.0, 'uuid': '7d62a5f2-690d-411a-8e4a-62921dfe3952'}
flour (farofa). Fried potatoes, fried cassava, fried banana, fried meat and fried cheese are very often eaten in lunch and
served in most typical restaurants. Popular snacks are pastel (a fried pastry); coxinha (a variation of chicken croquete); pão
de queijo (cheese bread and cassava flour / tapioca); pamonha (corn and milk paste); esfirra (a variation of Lebanese
###############
Filter by a property
the property source is automatically added by LangChain.
More on how to add multiple operands and nested filters
[Document(metadata={'creationdate': datetime.datetime(2023, 10, 31, 22, 3, 6, tzinfo=datetime.timezone.utc), 'page_label': '14', 'total_pages': 16.0, 'source': 'brazil-wikipedia-article-text.pdf', 'page': 13.0, 'title': 'Brazil - Wikipedia Text Only, Convert to PDF', 'creator': 'wkhtmltopdf 0.12.2.1', 'producer': 'Qt 4.8.6'}, page_content='flour (farofa). Fried potatoes, fried cassava, fried banana, fried meat and fried cheese are very often eaten in lunch and\nserved in most typical restaurants. Popular snacks are pastel (a fried pastry); coxinha (a variation of chicken croquete); pão\nde queijo (cheese bread and cassava flour / tapioca); pamonha (corn and milk paste); esfirra (a variation of Lebanese'), Document(metadata={'producer': 'Qt 4.8.6', 'page_label': '14', 'total_pages': 16.0, 'source': 'brazil-wikipedia-article-text.pdf', 'page': 13.0, 'title': 'Brazil - Wikipedia Text Only, Convert to PDF', 'creator': 'wkhtmltopdf 0.12.2.1', 'creationdate': datetime.datetime(2023, 10, 31, 22, 3, 6, tzinfo=datetime.timezone.utc)}, page_content="Cuisine\nBrazilian cuisine varies greatly by region, reflecting the country's varying mix of indigenous and immigrant populations. This\nhas created a national cuisine marked by the preservation of regional differences. Examples are Feijoada, considered the\ncountry's national dish; and regional foods such as beiju, feijão tropeiro, vatapá, moqueca, polenta (from Italian cuisine) and"), Document(metadata={'producer': 'Qt 4.8.6', 'page_label': '14', 'total_pages': 16.0, 'source': 'brazil-wikipedia-article-text.pdf', 'page': 13.0, 'title': 'Brazil - Wikipedia Text Only, Convert to PDF', 'creator': 'wkhtmltopdf 0.12.2.1', 'creationdate': datetime.datetime(2023, 10, 31, 22, 3, 6, tzinfo=datetime.timezone.utc)}, page_content='pastry); kibbeh (from Arabic cuisine); empanada (pastry) and empada, little salt pies filled with shrimps or heart of palm.\nBrazil has a variety of desserts such as brigadeiros (chocolate fudge balls), bolo de rolo (roll cake with goiabada), cocada (a\ncoconut sweet), beijinhos (coconut truffles and clove) and Romeu e Julieta (cheese with goiabada). Peanuts are used to')]
You can also do some question and answering
{"action":"read_disk_use","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"warning","msg":"disk usage currently at 81.82%, threshold set to 80.00%","path":"/Users/dudanogueira/.local/share/weaviate","time":"2025-12-10T20:42:14-03:00"}
Brazil's traditional food varies by region, with notable dishes including Feijoada, beiju, and vatapá. A typical meal often consists of rice and beans served with beef, salad, and fried eggs. Popular snacks include coxinha and pão de queijo.
{"action":"restapi_management","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"Shutting down... ","time":"2025-12-10T20:42:17-03:00","version":"1.30.5"}
{"action":"restapi_management","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"Stopped serving weaviate at http://127.0.0.1:8079","time":"2025-12-10T20:42:17-03:00","version":"1.30.5"}
{"action":"telemetry_push","build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"telemetry terminated","payload":"\u0026{MachineID:85ee30ca-f2c2-4319-bb6e-eae79c18ffa2 Type:TERMINATE Version:1.30.5 ObjectsCount:2 OS:darwin Arch:arm64 UsedModules:[generative-openai text2vec-openai] CollectionsCount:2}","time":"2025-12-10T20:42:17-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"closing raft FSM store ...","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"shutting down raft sub-system ...","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"transferring leadership to another server","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","error":"cannot find peer","level":"error","msg":"transferring leadership","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"closing raft-net ...","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"closing log store ...","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"closing data store ...","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"closing loaded database ...","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"closing raft-rpc client ...","time":"2025-12-10T20:42:21-03:00"}
{"build_git_commit":"62dcafac32","build_go_version":"go1.24.3","build_image_tag":"HEAD","build_wv_version":"1.30.5","level":"info","msg":"closing raft-rpc server ...","time":"2025-12-10T20:42:21-03:00"}