DQX (Databricks Labs)

DQX is a Databricks Labs framework for data quality checks on PySpark workloads, on streaming and standard DataFrames and tables. The README tagline: “Simplified Data Quality checking at Scale for PySpark Workloads”.

Repository snapshot (GitHub API, 2026-11-18)

ItemDetail
Repodatabrickslabs/dqx, Python
LicenceThe Databricks License, not an OSI open-source licence. Its text defines the agreement by reference to the Databricks Services terms, so read it before use outside Databricks.
Created2024-04-23
Activitylast push 2026-11-18; latest release v0.17.2 (2026-11-05)
Size487 stars, 168 forks

Capabilities (from the README)

  • 80+ built-in checks (null, range, regex, referential, aggregate, geo, PII) at row and dataset level, plus custom check functions.
  • Checks as code or as YAML or JSON; failures marked warning or error; invalid rows can be dropped, marked or quarantined.
  • Profiling and rule generation from existing data; AI-assisted rule suggestions from business descriptions (DSPy and Databricks Model Serving); Isolation Forest row anomaly detection.
  • Generates quality rules from Open Data Contract Standard contracts, including schema validation.
  • Summary metrics saved to Delta tables, a Lakeview quality dashboard, and Slack, Teams or webhook alerts.
  • Rules stored in files, Unity Catalog tables, Volumes or Lakebase.
  • DQX Studio (a no-code UI deployed as a Databricks App) and an MCP server exposing DQX to AI agents, with data access following Unity Catalog permissions.
  • Works for batch and Spark Structured Streaming, including Lakeflow Pipelines.

Where it fits against Soda Core

DQX is the Databricks-native choice: everything runs on Spark, Delta and Unity Catalog. It covers quality checks, contracts and dashboards in one framework, but ties you to Databricks. Soda Core is warehouse-neutral.

Sources

  • DQX README, its LICENSE file and GitHub API (2026-11-18)