Repository files navigation

✅ Completely local, no external APIs
✅ Fits in a T4 GPU
✅ Supports ~20 file types
✅ Convert documents, multimedia, and web pages to high-quality structured markdown
✅ Table extraction, image extraction/captioning, audio/video transcription, web page crawling
✅ Easily deployable using Docker and Skypilot
✅ Colab friendly
✅ Interative UI powered by Gradio \

It's challenging to process data as it comes in different shapes and sizes. OmniParse aims to be an ingestion/parsing platform where you can ingest any type of data, such as documents, images, audio, video, and web content, and get the most structured and actionable output that is GenAI (LLM) friendly.

Note: The server only works on Linux-based systems. This is due to certain dependencies and system-specific configurations that are not compatible with Windows or macOS.To install OmniParse, you can usepip:

git clone https://github.com/adithya-s-k/omniparsecd omniparse

Create a Virtual Environment:

conda create --name omniparse-venv python=3.10conda activate omniparse-venv

Install Dependencies:

poetry install# orpip install -e.

To use OmniParse with Docker, execute the following commands:

1. Pull the OmniParse API Docker image from Docker Hub:
2. Run the Docker container, exposing port 8000:👉🏼Docker Image

docker pull savatar101/omniparse:0.1# if you are running on a gpudocker run --gpus all -p 8000:8000 savatar101/omniparse:0.1# elsedocker run -p 8000:8000 savatar101/omniparse:0.1

Alternatively, if you prefer to build the Docker image locally:Then, run the Docker container as follows:

docker build -t omniparse.# if you are running on a gpudocker run --gpus all -p 8000:8000 omniparse# elsedocker run -p 8000:8000 omniparse

Run the Server:

python server.py --host 0.0.0.0 --port 8000 --documents --media --web

• --documents: Load in all the models that help you parse and ingest documents (Surya OCR series of models and Florence-2).
• --media: Load in Whisper model to transcribe audio and video files.
• --web: Set up selenium crawler.

curl -X POST -F "file=@/path/to/document" http://localhost:8000/parse_document

curl -X POST -F "file=@/path/to/document.pdf" http://localhost:8000/parse_document/pdf

curl -X POST -F "file=@/path/to/presentation.ppt" http://localhost:8000/parse_document/ppt

curl -X POST -F "file=@/path/to/document.docx" http://localhost:8000/parse_document/docs

curl -X POST -F "file=@/path/to/image.jpg" http://localhost:8000/parse_media/image

curl -X POST -F "image=@/path/to/image.jpg" -F "task=Caption" -F "prompt=Optional prompt" http://localhost:8000/parse_media/process_image

curl -X POST -F "file=@/path/to/video.mp4" http://localhost:8000/parse_media/video

curl -X POST -F "file=@/path/to/audio.mp3" http://localhost:8000/parse_media/audio

curl -X POST -H "Content-Type: application/json" -d '{"url": "https://example.com"}' http://localhost:8000/parse_website

🦙 LlamaIndex | Langchain | Haystack integrations coming soon📚 Batch processing data⭐ Dynamic chunking and structured data extraction based on specified Schema
🛠️ One magic API: just feed in your file prompt what you want, and we will take care of the rest
🔧 Dynamic model selection and support for external APIs
📄 Batch processing for handling multiple files at once
📦 New open-source model to replace Surya OCR and Marker

Final goal: replace all the different models currently being used with a single MultiModel Model to parse any type of data and get the data you need.

OmniParse is licensed under the GPL-3.0 license. SeeLICENSE for more information.The project uses Marker under the hood, which has a commercial license that needs to be followed. Here are the details:

Marker and Surya OCR Models are designed to be as widely accessible as possible while still funding development and training costs. Research and personal usage are always allowed, but there are some restrictions on commercial usage.The weights for the models are licensed under cc-by-nc-sa-4.0. However, this restriction is waived for any organization with less than $5M USD in gross revenue in the most recent 12-month period AND less than $5M in lifetime VC/angel funding raised. To remove the GPL license requirements (dual-license) and/or use the weights commercially over the revenue limit, check out the options provided.Please refer toMarker for more Information about the License of the Model weights

This project builds upon the remarkableMarker project created byVik Paruchuri. We express our gratitude for the inspiration and foundation provided by this project. Special thanks toSurya-OCR andTexify for the OCR models extensively used in this project, and toCrawl4AI for their contributions.

Models being used:

• Surya OCR, Detect, Layout, Order, and Texify
• Florence-2 base
• Whisper Small

Thank you to the authors for their contributions to these models.