Repository files navigation

Search photos on Unsplash using natural language descriptions. The search is powered by OpenAI'sCLIP model and theUnsplash Dataset.

_{Photos byRichard Burlton,Karl Anderson andXuecheng Chen onUnsplash.}

_{Photos byGenton Damian ,Anna Rozwadowska,Jude Beck onUnsplash.}

_{Photos bybruce mars,LOGAN WEAVER,Vasyl Skunziak onUnsplash.}

_{Photos byDalal Nizam andAnna Tremewan onUnsplash.}

OpenAI'sCLIP neural network is able to transform both images and text into the same latent space, where they can be compared using a similarity measure.

For this project, all photos from the fullUnsplash Dataset (almost 2M photos) were downloaded and processed with CLIP.

The pre-computed feature vectors for all images can then be used to find the best match to a natural language search query.

If you just want to play around with different queries jump to theColab notebook.

Before running any of the code, make sure to install all dependencies:

pip install -r requirements.txt

If you want to run all the code yourself open the Jupyter notebooks in the order they are numbered and follow the instructions there:

• 01-setup-clip.ipynb - setup the environment checking out and preparing the CLIP code.
• 02-download-unsplash-dataset.ipynb - download the photos from the Unsplash dataset
• 03-process-unsplash-dataset.ipynb - process all photos from the dataset with CLIP
• 04-search-image-dataset.ipynb - search for a photo in the dataset using natural language queries
• 09-search-image-api.ipynb - search for a photo using the Unsplash Search API and filter the results using CLIP.

NOTE: only the Lite version of the Unsplash Dataset is publicly available. If you want to use the Full version, you will need toapply for (free) access.

NOTE: searching for images using the Unsplash Search API doesn't require access to the Unsplash Dataset, but will probably deliver worse results.

This project was inspired by these projects:

• Beyond tags and entering the semantic search era on images with OpenAI CLIP byRamsri Goutham Golla
• Alph, The Sacred River byTravis Hoppe
• OpenAI's CLIP
• Unsplash