Repository files navigation

Original PDF

Docling-API	Marker

PyPDF	PyMuPDF4LLM

• Multiple Format Support: Converts various document types including:

  • PDF files
  • Microsoft Word documents (DOCX)
  • PowerPoint presentations (PPTX)
  • HTML files
  • Images (JPG, PNG, TIFF, BMP)
  • AsciiDoc files
  • Markdown files
  • CSV files

• Conversion Capabilities:

  • Text extraction and formatting
  • Table detection, extraction and conversion
  • Image extraction and processing
  • Multi-language OCR support (French, German, Spanish, English, Italian, Portuguese etc)
  • Configurable image resolution scaling

• API Endpoints:

  • Synchronous single document conversion
  • Synchronous batch document conversion
  • Asynchronous single document conversion with job tracking
  • Asynchronous batch conversion with job tracking

• Processing Modes:

  • CPU-only processing for standard deployments
  • GPU-accelerated processing for improved performance
  • Distributed task processing using Celery
  • Task monitoring through Flower dashboard

• Python 3.8 or higher
• Poetry (Python package manager)
• Redis server (for task queue)

curl -sSL https://install.python-poetry.org| python3 -

git clone https://github.com/drmingler/docling-api.gitcd docling-apipoetry install

Create a.env file in the project root:

REDIS_HOST=redis://localhost:6379/0ENV=development

Start Redis locally (install if not already installed):

brew install redisbrew services start redis

sudo apt-get install redis-serversudo service redis-server start

1. Start the FastAPI server:

poetry run uvicorn main:app --reload --port 8080

2. Start Celery worker (in a new terminal):

poetry run celery -A worker.celery_config worker --pool=solo -n worker_primary --loglevel=info

3. Start Flower dashboard for monitoring (optional, in a new terminal):

poetry run celery -A worker.celery_config flower --port=5555

1. Check if the API server is running:

curl http://localhost:8080/docs

2. Test Celery worker:

curl -X POST"http://localhost:8080/documents/convert" \  -H"accept: application/json" \  -H"Content-Type: multipart/form-data" \  -F"document=@/path/to/test.pdf"

3. Access monitoring dashboard:

• Openhttp://localhost:5555 in your browser to view the Flower dashboard

• The API documentation is available athttp://localhost:8080/docs
• Redis is used as both message broker and result backend for Celery tasks
• The service supports both synchronous and asynchronous document conversion
• For development, the server runs with auto-reload enabled

1. Clone the repository:

git clone https://github.com/drmingler/docling-api.gitcd docling-api

2. Create a.env file:

REDIS_HOST=redis://redis:6379/0ENV=production

To start the service using CPU-only processing, use the following command. You can adjust the number of Celery workers by specifying the --scale option. In this example, 1 worker will be created:

docker-compose -f docker-compose.cpu.yml up --build --scale celery_worker=1

For production, it is recommended to enable GPU acceleration, as it significantly improves performance. Use the command below to start the service with GPU support. You can also scale the number of Celery workers using the --scale option; here, 3 workers will be launched:

docker-compose -f docker-compose.gpu.yml up --build --scale celery_worker=3

The service will start the following components:

• API Server:http://localhost:8080
• Redis:http://localhost:6379
• Flower Dashboard:http://localhost:5556

Convert a single document immediately:

curl -X POST"http://localhost:8080/documents/convert" \  -H"accept: application/json" \  -H"Content-Type: multipart/form-data" \  -F"document=@/path/to/document.pdf" \  -F"extract_tables_as_images=true" \  -F"image_resolution_scale=4"

1. Submit a document for conversion:

curl -X POST"http://localhost:8080/conversion-jobs" \  -H"accept: application/json" \  -H"Content-Type: multipart/form-data" \  -F"document=@/path/to/document.pdf"

2. Check conversion status:

curl -X GET"http://localhost:8080/conversion-jobs/{job_id}" \  -H"accept: application/json"

Convert multiple documents asynchronously:

curl -X POST"http://localhost:8080/batch-conversion-jobs" \  -H"accept: application/json" \  -H"Content-Type: multipart/form-data" \  -F"documents=@/path/to/document1.pdf" \  -F"documents=@/path/to/document2.pdf"

• image_resolution_scale: Control the resolution of extracted images (1-4)
• extract_tables_as_images: Extract tables as images (true/false)
• CPU_ONLY: Build argument to switch between CPU/GPU modes

• Access the Flower dashboard to monitor Celery tasks and workers
• View task status, success/failure rates, and worker performance
• Monitor resource usage and task queues

The service uses a distributed architecture with the following components:

1. FastAPI application serving the REST API
2. Celery workers for distributed task processing
3. Redis as message broker and result backend
4. Flower for task monitoring and management
5. Docling for the file conversion

• GPU mode provides significantly faster processing for large documents
• CPU mode is suitable for smaller deployments or when GPU is not available
• Multiple workers can be scaled horizontally for increased throughput

The codebase is under MIT license. See LICENSE for more information

• Docling the state-of-the-art document conversion library by IBM
• FastAPI the web framework
• Celery for distributed task processing
• Flower for monitoring and management