Databricks gives you more native data quality than most platforms. It still leaves you writing the tests. The best tool for Databricks profiles every table, generates tests from what it finds, watches freshness, volume, and schema, and does all of it on your own SQL warehouse.
What Databricks gives you natively
Checked October 21, 2026:
Data quality monitoring has two parts. Anomaly detection learns each table’s commit pattern and flags stale or incomplete tables across a schema. Data profiling, formerly Lakehouse Monitoring, tracks null fractions, distributions, and drift. Both run on serverless compute and bill as serverless DBUs.
Table constraints enforce NOT NULL and CHECK on Delta tables. Primary key, foreign key, and unique constraints are informational only.
Lakeflow pipeline expectations warn, drop, or fail on rules attached to pipeline datasets.
The limit: anomaly detection watches tables, not values, and constraints and expectations are rules somebody wrote. Nothing generates a test suite from a profile.
Tools worth considering
| Tool | Profiling | Generated data tests | Observability monitors | Data quality dashboards | Runs on Databricks compute | Open source | Price basis |
|---|---|---|---|---|---|---|---|
| DataOps TestGen | Yes, 55 column characteristics | Yes, 34 test types | Freshness, volume, schema, custom metric | Scorecards, CDE score | Yes, SQL on your SQL warehouse | Apache 2.0 | Free for 1 user and 1 connection; Enterprise $100/user/mo + $100/connection/mo |
| Databricks data quality monitoring | Yes (data profiling) | No | Freshness, completeness | Yes | Yes, serverless | No | Serverless DBUs |
| Monte Carlo | Yes | Partial | Yes, ML-based | Yes | No, SaaS | No | Credits, per monitor (pricing) |
| Anomalo | Yes | Yes | Yes, ML-based | Partial | No, SaaS or self-host | No | On request |
| Soda | Yes | Partial | Yes | Yes | No, SaaS | Soda Core, Elastic License 2.0 | Free tier; Team $750/mo (pricing) |
Monte Carlo, Anomalo, and Soda ratings come from DataKitchen’s comparison pages, reviewed June 2026. Pricing comes from the vendor pages, checked October 21, 2026.
Who should pick what
Pick Monte Carlo or Anomalo if you want a managed service, ML anomaly detection, and a vendor Databricks already lists: both appear as Partner Connect partners with Unity Catalog support. That status matters to some procurement teams, and TestGen doesn’t have it. Budget accordingly: DataKitchen estimates Monte Carlo at $120,000 to $250,000 and Anomalo at $100,000 to $200,000 a year for 10 users and 1,000 tables (DataKitchen’s estimates, reviewed June 2026).
Pick TestGen if you want tests you didn’t write, run on compute you already pay for. A medallion lakehouse has thousands of columns per layer, and DataKitchen’s post on test coverage in a medallion architecture does the arithmetic on why hand-written suites never catch up. TestGen also gates a Databricks job: DataKitchen’s tripwire post puts a TestGen task between bronze-to-silver and silver-to-gold, and a failure stops the job before gold is written. The Databricks how-to covers setup.
TestGen connects to 14 databases, so the same suite follows a table if it moves. The core profiling, testing, and monitoring engine is free and open source. Enterprise adds team access, security, Commercial Pharma tests, source-data controls for AI, and support.
Documentation: Databricks quickstart, Access requirements, Manage connections, Observability Databricks agent.