data-profiling

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

metadata data-science data-validation snowflake data-catalog data-discovery dbt hacktoberfest data-quality-checks data-quality data-profiling metadata-management dataengineering dataquality data-governance data-lineage data-contracts data-observability datadiscovery data-collaboration

UpdatedMar 21, 2025
TypeScript

fbdesignpro /sweetviz

Star3k

Visualize and compare datasets, target values and associations, with one line of code.

python data-science machine-learning statistics pandas-dataframe exploratory-data-analysis eda pandas data-visualization exploration data-analysis data-exploration data-profiling

UpdatedAug 6, 2024
Python

sodadata /soda-core

Star2k

⚡ Data quality testing for the modern data stack (SQL, Spark, and Pandas)https://www.soda.io

python data-validation snowflake data-engineering dbt data-quality-checks data-quality data-profiling dataquality data-governance data-monitoring data-quality-monitoring pipeline-testing data-unit-tests data-testing data-contracts datatesting data-observability data-reliability data-quality-testing

UpdatedMar 17, 2025
Python

hi-primus /optimus

Star1.5k

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

data-science machine-learning spark bigdata data-transformation pyspark data-extraction data-analysis data-wrangling dask data-exploration data-preparation data-cleaning data-profiling data-cleansing big-data-cleaning data-cleaner cudf dask-cudf

UpdatedDec 2, 2024
Python

opendatadiscovery /odd-platform

Star1.3k

First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

metadata data-science oss bigdata alerting data-engineering data-catalog data-discovery data-platform data-exploration data-pipelines observability lineage data-quality data-profiling metadata-management data-governance data-lineage datacatalog data-observability

UpdatedFeb 19, 2025
Java

cleanlab /cleanvision

Star1.1k

Automatically find issues in image datasets and practice data-centric computer vision.

data-science computer-vision deep-learning data-validation exploratory-data-analysis image-classification image-generation image-segmentation image-analysis data-exploration image-quality data-quality data-profiling data-centric-ai

UpdatedApr 23, 2024
Python

datavane /datavines

Star595

Know your data better！Datavines is Next-gen Data Observability Platform, support metadata manage and data quality.

metadata data-science spark datascience data-engineering flink data-quality-checks data-quality data-profiling cleandata dataquality doris data-quality-monitoring data-quality-report data-profilers dataprofile data-quality-framework dataobservability

UpdatedFeb 20, 2025
Java

polyaxon /traceml

Star516

Engine for ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

tracking data-science statistics spark tensorflow plotly pandas data-visualization pytorch dataops matplotlib dask data-exploration pandas-summary dataframes data-quality-checks data-quality data-profiling explainable-ai mlops

UpdatedJan 5, 2025
Python

ing-bank /popmon

Star499

Monitor the stability of a Pandas or Spark dataframe ⚙︎

python data-science statistics spark monitoring jupyter ipython pandas data-analysis statistical-tests hacktoberfest data-profiling ing-bank covariate-shift statistical-process-control mlops drift-detection data-distributions population-monitoring dataset-shifts

UpdatedJan 24, 2025
Python

InfuseAI /piperider

Star485

Code review for data in dbt

python data-science continuous-integration reporting exploratory-data-analysis eda data-visualization code-review pull-requests dbt data-exploration data-pipeline data-quality data-profiling data-testing data-observability data-profiler data-reliability dbt-metrics

UpdatedJan 3, 2025
Python

polyaxon /haupt

Star454

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

visualization python tracking data-science machine-learning ui deep-learning jupyter tensorflow models plotly data-visualization pytorch bokeh matplotlib data-processing lineage data-profiling serving mlops

UpdatedMar 2, 2025
Python

Desbordante /desbordante-core

Star396

Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.

data-science data-mining exploratory-data-analysis tabular-data feature-selection data-engineering feature-extraction data-analytics knowledge-discovery data-wrangling data-preprocessing feature-engineering spreadsheets data-exploration data-mining-algorithms data-cleaning data-profiling anomaly-detection data-cleansing correlations

UpdatedMar 15, 2025
C++

databrickslabs /dqx

Star232

Databricks framework to validate Data Quality of pySpark DataFrames

spark spark-streaming databricks data-quality-checks data-quality data-profiling dlt data-quality-monitoring

UpdatedMar 19, 2025
Python

hi-primus /bumblebee

Star141

🚕 A spreadsheet-like data preparation web app that works over Optimus (Pandas, Dask, cuDF, Dask-cuDF, Spark and Vaex)

python gui gpu datasets dask optimus data-preparation data-cleaning data-profiling bumblebee prepare-data cudf dask-cudf

UpdatedJul 15, 2023
Vue

dqops /dqo

Star133

Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.

monitoring data-quality-checks data-quality data-profiling data-ops data-quality-measurement data-quality-monitoring data-quality-report data-observability

UpdatedJan 14, 2025
Java

DataKitchen /data-observability-installer

Star110

Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team is the first to know and the first to solve with visibility across and down your data estate. Save time with simple, fast data quality test generation and execution. Trust your data, tools, and systems end to end.