Main
agentsllmsvector-databasemultimodal_clip_diffusiondblancedbgptopenaiAImultimodal-aimachine-learningembeddingsfine-tuningexamplesdeep-learninggpt-4-visionllama-indexragmultimodallangchainlancedb-recipes
Export
Multimodal search using CLIP
Installing all dependencies
[11]
⏬ Downloading https://github.com/jaimergp/miniforge/releases/download/24.11.2-1_colab/Miniforge3-colab-24.11.2-1_colab-Linux-x86_64.sh... 📦 Installing... 📌 Adjusting configuration... 🩹 Patching environment... ⏲ Done in 0:00:11 🔁 Restarting kernel...
[1]
Channels:
- conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done
Solving environment: \ | done
==> WARNING: A newer version of conda exists. <==
current version: 24.11.2
latest version: 25.7.0
Please update conda by running
$ conda update -n base -c conda-forge conda
## Package Plan ##
environment location: /usr/local/envs/py310
added / updated specs:
- python=3.10
The following packages will be downloaded:
package | build
---------------------------|-----------------
bzip2-1.0.8 | hda65f42_8 254 KB conda-forge
ca-certificates-2025.8.3 | hbd8a1cb_0 151 KB conda-forge
ld_impl_linux-64-2.44 | h1423503_1 660 KB conda-forge
libexpat-2.7.1 | hecca717_0 73 KB conda-forge
libffi-3.4.6 | h2dba641_1 56 KB conda-forge
libgcc-15.1.0 | h767d61c_5 805 KB conda-forge
libgcc-ng-15.1.0 | h69a702a_5 29 KB conda-forge
libgomp-15.1.0 | h767d61c_5 437 KB conda-forge
liblzma-5.8.1 | hb9d3cd8_2 110 KB conda-forge
libnsl-2.0.1 | hb9d3cd8_1 33 KB conda-forge
libsqlite-3.50.4 | h0c1763c_0 911 KB conda-forge
libuuid-2.41.1 | he9a06e4_0 36 KB conda-forge
ncurses-6.5 | h2d0b736_3 871 KB conda-forge
openssl-3.5.3 | h26f9b46_0 3.0 MB conda-forge
pip-25.2 | pyh8b19718_0 1.1 MB conda-forge
python-3.10.18 |hd6af730_0_cpython 23.9 MB conda-forge
readline-8.2 | h8c095d6_2 276 KB conda-forge
setuptools-80.9.0 | pyhff2d567_0 731 KB conda-forge
tk-8.6.13 |noxft_hd72426e_102 3.1 MB conda-forge
tzdata-2025b | h78e105d_0 120 KB conda-forge
------------------------------------------------------------
Total: 36.6 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
_openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-2_gnu
bzip2 conda-forge/linux-64::bzip2-1.0.8-hda65f42_8
ca-certificates conda-forge/noarch::ca-certificates-2025.8.3-hbd8a1cb_0
ld_impl_linux-64 conda-forge/linux-64::ld_impl_linux-64-2.44-h1423503_1
libexpat conda-forge/linux-64::libexpat-2.7.1-hecca717_0
libffi conda-forge/linux-64::libffi-3.4.6-h2dba641_1
libgcc conda-forge/linux-64::libgcc-15.1.0-h767d61c_5
libgcc-ng conda-forge/linux-64::libgcc-ng-15.1.0-h69a702a_5
libgomp conda-forge/linux-64::libgomp-15.1.0-h767d61c_5
liblzma conda-forge/linux-64::liblzma-5.8.1-hb9d3cd8_2
libnsl conda-forge/linux-64::libnsl-2.0.1-hb9d3cd8_1
libsqlite conda-forge/linux-64::libsqlite-3.50.4-h0c1763c_0
libuuid conda-forge/linux-64::libuuid-2.41.1-he9a06e4_0
libxcrypt conda-forge/linux-64::libxcrypt-4.4.36-hd590300_1
libzlib conda-forge/linux-64::libzlib-1.3.1-hb9d3cd8_2
ncurses conda-forge/linux-64::ncurses-6.5-h2d0b736_3
openssl conda-forge/linux-64::openssl-3.5.3-h26f9b46_0
pip conda-forge/noarch::pip-25.2-pyh8b19718_0
python conda-forge/linux-64::python-3.10.18-hd6af730_0_cpython
readline conda-forge/linux-64::readline-8.2-h8c095d6_2
setuptools conda-forge/noarch::setuptools-80.9.0-pyhff2d567_0
tk conda-forge/linux-64::tk-8.6.13-noxft_hd72426e_102
tzdata conda-forge/noarch::tzdata-2025b-h78e105d_0
wheel conda-forge/noarch::wheel-0.45.1-pyhd8ed1ab_1
Downloading and Extracting Packages:
python-3.10.18 | 23.9 MB | : 0% 0/1 [00:00<?, ?it/s]
tk-8.6.13 | 3.1 MB | : 0% 0/1 [00:00<?, ?it/s]
openssl-3.5.3 | 3.0 MB | : 0% 0/1 [00:00<?, ?it/s]
pip-25.2 | 1.1 MB | : 0% 0/1 [00:00<?, ?it/s]
libsqlite-3.50.4 | 911 KB | : 0% 0/1 [00:00<?, ?it/s]
ncurses-6.5 | 871 KB | : 0% 0/1 [00:00<?, ?it/s]
libgcc-15.1.0 | 805 KB | : 0% 0/1 [00:00<?, ?it/s]
setuptools-80.9.0 | 731 KB | : 0% 0/1 [00:00<?, ?it/s]
ld_impl_linux-64-2.4 | 660 KB | : 0% 0/1 [00:00<?, ?it/s]
libgomp-15.1.0 | 437 KB | : 0% 0/1 [00:00<?, ?it/s]
readline-8.2 | 276 KB | : 0% 0/1 [00:00<?, ?it/s]
bzip2-1.0.8 | 254 KB | : 0% 0/1 [00:00<?, ?it/s]
ca-certificates-2025 | 151 KB | : 0% 0/1 [00:00<?, ?it/s]
tzdata-2025b | 120 KB | : 0% 0/1 [00:00<?, ?it/s]
liblzma-5.8.1 | 110 KB | : 0% 0/1 [00:00<?, ?it/s]
libexpat-2.7.1 | 73 KB | : 0% 0/1 [00:00<?, ?it/s]
libffi-3.4.6 | 56 KB | : 0% 0/1 [00:00<?, ?it/s]
libuuid-2.41.1 | 36 KB | : 0% 0/1 [00:00<?, ?it/s]
libnsl-2.0.1 | 33 KB | : 0% 0/1 [00:00<?, ?it/s]
... (more hidden) ...
libsqlite-3.50.4 | 911 KB | : 2% 0.017568447137567673/1 [00:00<00:06, 7.00s/it]
python-3.10.18 | 23.9 MB | : 0% 0.0006542580201315321/1 [00:00<04:31, 271.33s/it]
tk-8.6.13 | 3.1 MB | : 0% 0.004987209317899284/1 [00:00<00:35, 36.17s/it]
pip-25.2 | 1.1 MB | : 1% 0.013918149321082462/1 [00:00<00:12, 12.65s/it]
libsqlite-3.50.4 | 911 KB | : 100% 1.0/1 [00:00<00:00, 7.00s/it]
libsqlite-3.50.4 | 911 KB | : 100% 1.0/1 [00:00<00:00, 5.26it/s]
libsqlite-3.50.4 | 911 KB | : 100% 1.0/1 [00:00<00:00, 5.26it/s]
openssl-3.5.3 | 3.0 MB | : 85% 0.8514414226568393/1 [00:00<00:00, 4.37it/s]
pip-25.2 | 1.1 MB | : 100% 1.0/1 [00:00<00:00, 12.65s/it]
ncurses-6.5 | 871 KB | : 2% 0.018375108367605347/1 [00:00<00:14, 14.52s/it]
python-3.10.18 | 23.9 MB | : 8% 0.0837450265768361/1 [00:00<00:02, 2.74s/it]
openssl-3.5.3 | 3.0 MB | : 100% 1.0/1 [00:00<00:00, 4.37it/s]
ncurses-6.5 | 871 KB | : 100% 1.0/1 [00:00<00:00, 14.52s/it]
libgcc-15.1.0 | 805 KB | : 2% 0.019878887296755243/1 [00:00<00:17, 17.94s/it]
ld_impl_linux-64-2.4 | 660 KB | : 2% 0.02423510895740514/1 [00:00<00:14, 14.73s/it]
python-3.10.18 | 23.9 MB | : 30% 0.29899591520011015/1 [00:00<00:00, 1.05it/s]
setuptools-80.9.0 | 731 KB | : 2% 0.021880692532465797/1 [00:00<00:17, 17.53s/it]
ld_impl_linux-64-2.4 | 660 KB | : 100% 1.0/1 [00:00<00:00, 14.73s/it]
libgcc-15.1.0 | 805 KB | : 100% 1.0/1 [00:00<00:00, 17.94s/it]
setuptools-80.9.0 | 731 KB | : 100% 1.0/1 [00:00<00:00, 17.53s/it]
python-3.10.18 | 23.9 MB | : 47% 0.47237429053496616/1 [00:00<00:00, 1.29it/s]
libgomp-15.1.0 | 437 KB | : 4% 0.03663562268707445/1 [00:00<00:12, 12.87s/it]
readline-8.2 | 276 KB | : 100% 1.0/1 [00:00<00:00, 8.07s/it]
libgomp-15.1.0 | 437 KB | : 100% 1.0/1 [00:00<00:00, 12.87s/it]
ca-certificates-2025 | 151 KB | : 11% 0.10611261512156578/1 [00:00<00:04, 4.77s/it]
ca-certificates-2025 | 151 KB | : 100% 1.0/1 [00:00<00:00, 4.77s/it]
bzip2-1.0.8 | 254 KB | : 6% 0.06293284576766625/1 [00:00<00:07, 8.21s/it]
bzip2-1.0.8 | 254 KB | : 100% 1.0/1 [00:00<00:00, 8.21s/it]
tzdata-2025b | 120 KB | : 13% 0.1332379155552664/1 [00:00<00:03, 4.15s/it]
tzdata-2025b | 120 KB | : 100% 1.0/1 [00:00<00:00, 4.15s/it]
liblzma-5.8.1 | 110 KB | : 15% 0.1451272875440679/1 [00:00<00:03, 3.90s/it]
python-3.10.18 | 23.9 MB | : 63% 0.6274334413061392/1 [00:00<00:00, 1.37it/s]
libffi-3.4.6 | 56 KB | : 29% 0.2852715337871955/1 [00:00<00:01, 2.13s/it]
libffi-3.4.6 | 56 KB | : 100% 1.0/1 [00:00<00:00, 2.13s/it]
libexpat-2.7.1 | 73 KB | : 22% 0.2190052265041237/1 [00:00<00:02, 2.89s/it]
libexpat-2.7.1 | 73 KB | : 100% 1.0/1 [00:00<00:00, 2.89s/it]
libuuid-2.41.1 | 36 KB | : 44% 0.4417720495052175/1 [00:00<00:00, 1.46s/it]
libuuid-2.41.1 | 36 KB | : 100% 1.0/1 [00:00<00:00, 1.46s/it]
libnsl-2.0.1 | 33 KB | : 49% 0.4857252972043521/1 [00:00<00:00, 1.33s/it]
libnsl-2.0.1 | 33 KB | : 100% 1.0/1 [00:00<00:00, 1.33s/it]
openssl-3.5.3 | 3.0 MB | : 100% 1.0/1 [00:00<00:00, 4.37it/s]
... (more hidden) ...
python-3.10.18 | 23.9 MB | : 77% 0.7746414958357339/1 [00:00<00:00, 1.17it/s]
pip-25.2 | 1.1 MB | : 100% 1.0/1 [00:00<00:00, 1.37it/s]
python-3.10.18 | 23.9 MB | : 90% 0.9028760677815142/1 [00:00<00:00, 1.00s/it]
tk-8.6.13 | 3.1 MB | : 100% 1.0/1 [00:01<00:00, 2.86it/s]
ld_impl_linux-64-2.4 | 660 KB | : 100% 1.0/1 [00:01<00:00, 1.29s/it]
ld_impl_linux-64-2.4 | 660 KB | : 100% 1.0/1 [00:01<00:00, 1.29s/it]
libgcc-15.1.0 | 805 KB | : 100% 1.0/1 [00:01<00:00, 1.37s/it]
python-3.10.18 | 23.9 MB | : 100% 1.0/1 [00:01<00:00, 1.00s/it]
setuptools-80.9.0 | 731 KB | : 100% 1.0/1 [00:01<00:00, 1.73s/it]
setuptools-80.9.0 | 731 KB | : 100% 1.0/1 [00:01<00:00, 1.73s/it]
libgomp-15.1.0 | 437 KB | : 100% 1.0/1 [00:01<00:00, 1.77s/it]
libgomp-15.1.0 | 437 KB | : 100% 1.0/1 [00:01<00:00, 1.77s/it]
readline-8.2 | 276 KB | : 100% 1.0/1 [00:01<00:00, 1.82s/it]
readline-8.2 | 276 KB | : 100% 1.0/1 [00:01<00:00, 1.82s/it]
ca-certificates-2025 | 151 KB | : 100% 1.0/1 [00:01<00:00, 1.85s/it]
ca-certificates-2025 | 151 KB | : 100% 1.0/1 [00:01<00:00, 1.85s/it]
bzip2-1.0.8 | 254 KB | : 100% 1.0/1 [00:01<00:00, 1.86s/it]
bzip2-1.0.8 | 254 KB | : 100% 1.0/1 [00:01<00:00, 1.86s/it]
ncurses-6.5 | 871 KB | : 100% 1.0/1 [00:02<00:00, 2.29s/it]
ncurses-6.5 | 871 KB | : 100% 1.0/1 [00:02<00:00, 2.29s/it]
tzdata-2025b | 120 KB | : 100% 1.0/1 [00:02<00:00, 2.29s/it]
tzdata-2025b | 120 KB | : 100% 1.0/1 [00:02<00:00, 2.29s/it]
liblzma-5.8.1 | 110 KB | : 100% 1.0/1 [00:02<00:00, 2.31s/it]
liblzma-5.8.1 | 110 KB | : 100% 1.0/1 [00:02<00:00, 2.31s/it]
libexpat-2.7.1 | 73 KB | : 100% 1.0/1 [00:02<00:00, 2.36s/it]
libexpat-2.7.1 | 73 KB | : 100% 1.0/1 [00:02<00:00, 2.36s/it]
libffi-3.4.6 | 56 KB | : 100% 1.0/1 [00:02<00:00, 2.42s/it]
libffi-3.4.6 | 56 KB | : 100% 1.0/1 [00:02<00:00, 2.42s/it]
libuuid-2.41.1 | 36 KB | : 100% 1.0/1 [00:02<00:00, 2.56s/it]
libuuid-2.41.1 | 36 KB | : 100% 1.0/1 [00:02<00:00, 2.56s/it]
libnsl-2.0.1 | 33 KB | : 100% 1.0/1 [00:02<00:00, 2.60s/it]
libnsl-2.0.1 | 33 KB | : 100% 1.0/1 [00:02<00:00, 2.60s/it]
... (more hidden) ...
python-3.10.18 | 23.9 MB | : 100% 1.0/1 [00:03<00:00, 1.00s/it]
Preparing transaction: - \ done
Verifying transaction: / - \ | / - \ | / - \ | / done
Executing transaction: \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done
#
# To activate this environment, use
#
# $ conda activate py310
#
# To deactivate an active environment, use
#
# $ conda deactivate
[ ]
Requirement already satisfied: tantivy==0.20.1 in /usr/local/envs/py310/lib/python3.10/site-packages (0.20.1)
First run setup: Download data and pre-process
[5]
[ ]
--2024-02-25 04:51:11-- https://eto-public.s3.us-west-2.amazonaws.com/datasets/diffusiondb_lance.tar.gz Resolving eto-public.s3.us-west-2.amazonaws.com (eto-public.s3.us-west-2.amazonaws.com)... 3.5.79.102, 3.5.82.217, 3.5.77.120, ... Connecting to eto-public.s3.us-west-2.amazonaws.com (eto-public.s3.us-west-2.amazonaws.com)|3.5.79.102|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6121107645 (5.7G) [application/x-gzip] Saving to: ‘diffusiondb_lance.tar.gz’ diffusiondb_lance.t 100%[===================>] 5.70G 69.8MB/s in 88s 2024-02-25 04:52:39 (66.6 MB/s) - ‘diffusiondb_lance.tar.gz’ saved [6121107645/6121107645] diffusiondb_test/ diffusiondb_test/_versions/ diffusiondb_test/_latest.manifest diffusiondb_test/data/ diffusiondb_test/data/138fc0d8-a806-4b10-84f8-00dc381afdad.lance diffusiondb_test/_versions/1.manifest
Create / Open LanceDB Table
[6]
Create CLIP embedding function for the text
[7]
/usr/local/lib/python3.12/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: The secret `HF_TOKEN` does not exist in your Colab secrets. To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session. You will be able to reuse this secret in all of your notebooks. Please note that authentication is recommended but still optional to access public models or datasets. warnings.warn(
tokenizer_config.json: 0%| | 0.00/592 [00:00<?, ?B/s]
vocab.json: 0.00B [00:00, ?B/s]
merges.txt: 0.00B [00:00, ?B/s]
tokenizer.json: 0.00B [00:00, ?B/s]
special_tokens_map.json: 0%| | 0.00/389 [00:00<?, ?B/s]
config.json: 0.00B [00:00, ?B/s]
pytorch_model.bin: 0%| | 0.00/605M [00:00<?, ?B/s]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Fetching 1 files: 0%| | 0/1 [00:00<?, ?it/s]
model.safetensors: 0%| | 0.00/605M [00:00<?, ?B/s]
preprocessor_config.json: 0%| | 0.00/316 [00:00<?, ?B/s]
[8]
prompt: string ,seed: uint32 ,step: uint16 ,cfg: float ,sampler: string ,width: uint16 ,height: uint16 ,timestamp: timestamp[s] ,image_nsfw: float ,prompt_nsfw: float ,vector: fixed_size_list<item: float>[512] , child 0, item: float ,image: binary
Search functions for Gradio
[9]
Setup Gradio interface
[10]
/tmp/ipython-input-3232346804.py:23: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead. ).style(columns=[3], rows=[3], object_fit="contain", height="auto")
Colab notebook detected. To show errors in colab notebook, set debug=True in launch() IMPORTANT: You are using gradio version 3.41.2, however version 4.44.1 is available, please upgrade. -------- Running on public URL: https://82fe54bd5c2f7817ca.gradio.live This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
[ ]