Entity resolution

Star

Entity resolution (also known as data matching, data linkage, record linkage, and many other terms) is the task of finding entities in a dataset that refer to the same entity across different data sources (e.g., data files, books, websites, and databases). Entity resolution is necessary when joining different data sets based on entities that may or may not share a common identifier (e.g., database key, URI, National identification number), which may be due to differences in record shape, storage location, or curator style or preference.

Here are 191 public repositories matching this topic...

Language:All

Filter by language

All191 Python95 Jupyter Notebook34 Java14 R10 HTML7 C++3 Rust3 Scala3 JavaScript2 TypeScript2

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

dedupeio /dedupe

Star4.4k

🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

python clustering dedupe record-linkage python-library entity-resolution datamade dedupe-library de-duplicating

UpdatedJul 29, 2025
Python

moj-analytical-services /splink

Star1.8k

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

data-science spark record-linkage entity-resolution fuzzy-matching deduplication em-algorithm data-matching deduplicate-data duckdb uk-gov-data-science

UpdatedDec 17, 2025
Python

zinggAI /zingg

Star1.1k

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

data-science identity-resolution spark analytics dedupe entity-resolution ml snowflake fuzzy-matching master-data-management mdm databricks deduplication cdp datalake masterdata dataengineering fuzzymatch dataquality customer-data-platform

UpdatedDec 17, 2025
Java

J535D165 /recordlinkage

Sponsor

Star1k

A powerful and modular toolkit for record linkage and duplicate detection in Python

python machine-learning privacy dedupe record-linkage python-library entity-resolution similarity string-distance deduplication data-matching utrecht-university

UpdatedFeb 21, 2024
Python

JohnSnowLabs /nlu

Star948

1 line for thousands of State of The Art NLP models in hundreds of languages The fastest and most accurate way to solve text problems.

sentiment-analysis text-classification entity-resolution nlu transformers language-detection pandas named-entity-recognition text-summarization seq2seq lemmatizer spell-checker text-translation dependency-parsing sentiment-classifier natural-language-understanding sentence-embeddings t5 streamlit bert-embedding

UpdatedJan 28, 2025
Python

Picovoice /rhino

Star696

On-device Speech-to-Intent engine powered by deep learning

entity-resolution nlu voice-commands voice-recognition speech-recognition voice-control voice-command slot-filling voice-assistant natural-language-understanding slu vui on-device spoken-language-understanding voice-ui voice-user-interface intent-inference voice-command-control

UpdatedDec 15, 2025
Python

dedupeio /csvdedupe

Star432

🆔 Command line tool for deduplicating CSV files

cli dedupe record-linkage csv-files entity-resolution

UpdatedMar 31, 2020
Python

dedupeio /dedupe-examples

Star415

🆔 Examples for using the dedupe library

python dedupe record-linkage entity-resolution

UpdatedAug 10, 2024
Python

J535D165 /data-matching-software

Sponsor

Star396

A list of free data matching and record linkage software.

open-source machine-learning awesome record-linkage entity-resolution fuzzy-matching software awesome-list deduplication data-matching

UpdatedFeb 21, 2024

izuna385 /Entity-Linking-Recent-Trends

Star347

Recent trends of Entity Linking, Disambiguation, and Representation.

nlp natural-language-processing entity-resolution bert entity-linking entity-disambiguation entity-language-model entity-representation

UpdatedJun 26, 2021

microsoft /vert-papers

Star280

This repository contains code and datasets related to entity/knowledge papers from the VERT (Versatile Entity Recognition & disambiguation Toolkit) project, by the Knowledge Computing group at Microsoft Research Asia (MSRA).

nlp entity-resolution ml named-entity-recognition ner nlp-resources entity-linking unitrans entity-extraction grn entity-disambiguation language-understanding linkingpark bertel can-ner xl-ner cross-lingual-ner

UpdatedMar 16, 2024
Python

amazon-science /ReFinED

Star228

ReFinED is an efficient and accurate entity linking (EL) system.

nlp entity-resolution pytorch entity-linking entity-extraction

UpdatedDec 13, 2024
Python

scify /JedAIToolkit

Star221

An open source, high scalability toolkit in Java for Entity Resolution.

entity-resolution scalability blocking entity-matching

UpdatedJul 12, 2025
Java

maxharlow /csvmatch

Star190

🔎 Finds fuzzy matches between CSV files

csv record-linkage entity-resolution fuzzy-matching data-matching

UpdatedMar 26, 2025
Python

DerwenAI /strwythura

Star186

Strwythura: construct a knowledge graph from unstructured data sources, organized by results from entity resolution, implementing an enhanced GraphRAG approach, and also implementing an ontology pipeline plus context engineering for optimizing AI application outcomes within a specific domain

nlp machine-learning entity-resolution ontology knowledge-graph named-entity-recognition entity-linking relation-extraction unstructured-data entity-embedding semantic-layer textgraphs vector-database dspy graphrag semantic-expansion semantic-random-walk ontology-pipeline

UpdatedDec 16, 2025
Jupyter Notebook

zentity-io /zentity

Star164

Entity resolution for Elasticsearch.

elasticsearch identity-resolution entity-resolution elasticsearch-plugin gdpr address-matching entity-matching name-matching

UpdatedDec 16, 2025
Java

vintasoftware /entity-embed

Star159

PyTorch library for transforming entities like companies, products, etc. into vectors to support scalable Record Linkage / Entity Resolution using Approximate Nearest Neighbors.

python deep-learning record-linkage entity-resolution pytorch embeddings representation-learning deduplication entity-matching data-matching approximate-nearest-neighbors