Airflow runs tasks in order and stops when one fails. It doesn’t profile a table, write a test, or notice that a column went 30% null. So the question is which tool you call from Airflow, and the answer is the one that does the most work you’d otherwise write by hand.
What Airflow gives you natively
The common-sql provider (version 2.3.0, checked October 21, 2026) ships check operators. SQLColumnCheckOperator runs null, distinct, unique, min, and max checks on columns you name. SQLTableCheckOperator runs boolean statements you write. SQLThresholdCheckOperator fails the task when a query result falls outside your bounds.
The limit is authorship. Every check is one you thought of, typed into a DAG, and maintain. Nothing profiles the table, and nothing watches freshness or volume between runs.
Tools worth considering
| Tool | Profiling | Generated data tests | Observability monitors | Data quality dashboards | How it runs from Airflow | Open source | Price basis |
|---|---|---|---|---|---|---|---|
| DataOps TestGen | Yes, 55 column characteristics | Yes, 34 test types | Freshness, volume, schema, custom metric | Scorecards, CDE score | Task calls the REST API; SQL runs in your database | Apache 2.0 | Free for 1 user and 1 connection; Enterprise $100/user/mo + $100/connection/mo |
| Airflow SQL check operators | No | No | No | No | Operator runs SQL in your database | Apache 2.0 | Free |
| Great Expectations (GX Core) | Yes | No | Partial | No | GXValidateBatchOperator and two sibling operators | Apache 2.0 | Free |
| Soda | Partial (Soda Core) | No (Soda Core) | Partial (Soda Core) | Yes (Soda Cloud) | Python task runs a Soda scan and raises on failed checks | No; Soda Core is source-available under Elastic License 2.0 | Free tier; Team $750/mo (pricing) |
| Monte Carlo | Yes | Partial | Yes, ML-based | Yes | airflow-mcd callbacks and SimpleCircuitBreakerOperator | No | Credits, per monitor, on request (pricing) |
Capability ratings for GX, Soda, and Monte Carlo come from DataKitchen’s comparison pages, reviewed June 2026. Integration and pricing details come from the linked vendor pages, checked October 21, 2026. GX Cloud, the commercial product, shut down June 1, 2026; GX Core continues.
Who should pick what
Pick Monte Carlo if you want a managed service with ML anomaly detection and lineage, and the budget for it. DataKitchen’s Monte Carlo comparison estimates $120,000 to $250,000 a year for 10 users and 1,000 tables (DataKitchen’s estimate, reviewed June 2026). Its Airflow package is mature, and its circuit breaker is a real gate. Pick Soda or GX if your team prefers writing checks as code and owns that code.
Pick TestGen if you want coverage you didn’t type. It profiles the table, generates tests from the profile, and runs them as SQL in your database. Airflow calls it with one task, as the how-to shows. Add DataOps Observability’s Airflow agent and the DAG run and the test results land together, which is the point DataKitchen’s post on test coverage versus lineage makes about process lineage. Both products are open source, and DataKitchen explains why in Why We Open-Sourced Our Data Observability Products.
TestGen connects to 14 databases plus files. The core profiling, testing, and monitoring engine is free and open source. Enterprise adds team access, security, Commercial Pharma tests, source-data controls for AI, and support.
Documentation: TestGen API, Run profiling and tests, Observability Airflow agent, Data quality quickstarts.