unstructured-data

Dealing with all unstructured data, such as reverse image search, audio search, molecular search, video analysis, question and answer systems, NLP, etc.

python nlp deep-learning embeddings question-answering image-classification image-recognition image-search semantic-search unstructured-data audio-search rag milvus vector-database llm

UpdatedDec 15, 2025
Jupyter Notebook

instill-ai /instill-core

Star2.3k

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

python api cli golang open-source typescript ai pipeline etl developer-tools gpt hacktoberfest low-code no-code unstructured-data llm stable-diffusion generative-ai

UpdatedDec 5, 2025
Python

nomic-ai /nomic

Star1.9k

Nomic Developer API SDK

python clustering text embeddings topic-modeling duplicate-detection unstructured-data

UpdatedNov 11, 2025
Python

NanoNets /docext

Star1.8k

An on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)

nlp machine-learning ocr extraction document onprem document-analysis table-extraction unstructured-data rag onpremise llms vlms document-information-extraction ocr-onpremise document-data-extraction onprem-vision onprem-ocr llm-ocr ocr-benchmark

UpdatedAug 25, 2025
Python

shcherbak-ai /contextgem

Star1.7k

ContextGem: Effortless LLM extraction from documents

nlp ai text-analysis docx data-extraction contract-analysis legaltech docx2txt unstructured-data document-intelligence llm docx2md prompt-engineering llms generative-ai llm-framework llm-pipeline llm-extraction

UpdatedNov 16, 2025
Python

A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.

structured-data serving unstructured-data unified-sql vector-database mysql-compatibility hybrid-search embedding-search embedding-store key-value-distributed-store vector-ocean real-time-semantic-search

UpdatedDec 17, 2025
Java

yobix-ai /extractous

Star1.6k

Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.

nlp rust pdf machine-learning natural-language-processing ocr etl tika extraction docx data-pipelines pdf-parser unstructured unstructured-data rag etl-pipelines llm

UpdatedDec 21, 2024
Rust

lotus-data /lotus

Star1.5k

AI-Powered Data Processing: Use LOTUS to process all of your datasets with LLMs and embeddings. Enjoy up to 1000x speedups with fast, accurate query processing, that's as simple as writing Pandas code

python data pandas semantic-search unstructured-data llm ai-data-processing semantic-operators llm-data-processing llm-document-processing

UpdatedDec 11, 2025
Python

emcf /thepipe

Star1.5k

Get clean data from tricky documents, powered by vision-language models ⚡

python pdf web scraping openai document scrapers structured-data unstructured-data multimodal vision-transformer large-language-models vision-language-model

UpdatedOct 18, 2025
Python

tstanislawek /awesome-document-understanding

Star1.5k

A curated list of resources for Document Understanding (DU) topic

nlp pdf machine-learning natural-language-processing awesome ocr deep-learning information-extraction awesome-list pdf-documents document-analysis rpa unstructured-data robotic-process-automation document-layout-analysis document-understanding key-information-extraction document-ai document-intelligence intelligent-processing

UpdatedJun 2, 2023

amphi-ai /amphi-etl

Star1.3k

Visual Data Preparation and Transformation. Low-Code Python-based ETL.

data-science data etl data-analysis structured-data data-pipelines data-preparation unstructured-data datatransformation analytics-automation

UpdatedDec 8, 2025
TypeScript

Renumics /spotlight

Star1.2k

Interactively explore unstructured datasets from your dataframe.

audio machine-learning video computer-vision timeseries images exploratory-data-analysis data-visualization hacktoberfest meshes data-curation unstructured-data data-centric-ai

UpdatedDec 17, 2025
TypeScript

Open-Source-Legal /OpenContracts

Star1.1k

Enterprise-grade and API-first LLM workspace for unstructured documents, including data extraction, redaction, rights management, prompt playground, and more!

agent etl unstructured-data etl-pipeline vector-database llm prompt-engineering agentic-ai