A guide for data engineers, data quality professionals, and data team leaders. First published October 2025, updated September 2026 for the GX Cloud shutdown.
Your LLM just told the CEO that revenue is up 40% when it’s actually down. Your analytic engineers are vibe coding late into the night. Your predictive models, once your pride and joy, are degrading faster than you can debug them. And nobody trusts your standard reports. Welcome to 2026, where bad data doesn’t stop at a broken dashboard. It gets fed to a model that acts on it.
AI turns a data quality problem into a volume problem. A schema drift that used to mean one broken report now means thousands of wrong predictions before anyone looks. The validation you postponed trained your model to be confidently wrong at scale. Your engineers are spot-checking tables by hand while the models consume data faster than any human can read it. The executives who asked for “AI transformation” now want to know why the models produce nonsense. And the observability platform you bought? It tells you what broke after the model has already made 10,000 bad decisions.
Here is the bind. AI demands clean data at the exact moment when volume, source count, and complexity have put clean data out of reach for a team writing SQL tests by hand. The pipelines retrain hourly, pull from hundreds of sources, and decide in milliseconds. Your test suite is still written like it’s 2015. The enterprise vendors will take six figures to tell you what you already know: your data is broken.
This guide maps the open-source alternative. It defines the two categories, compares the tools, and explains what changed in 2026 when the best-known project in the category lost its commercial product in 30 days.
From Hope-Based Data to Observability-Driven Trust
Five years ago, most data teams ran on hope. They hoped the source systems behaved, that joins stayed aligned, and that nobody silently changed a schema. Testing, where it existed, was manual and reactive.
Then came the first wave of structured testing frameworks. Great Expectations brought the software habit of hand-written tests into analytics. That was the start of a shift: from “data should work” to “let’s prove it does.”
Great Expectations itself did not make it through 2026 intact. The company was acquired, and GX Cloud, its commercial product, shut down on June 1, 2026. Customers got about 30 days’ notice. The open-source GX Core project continues under Fivetran stewardship, so the framework that started the discipline is still there. The business built on top of it is not.
By 2020, the conversation had expanded to data observability: continuous monitoring of freshness, volume, schema, and anomalies in production. Closed-source tools such as Monte Carlo, Databand, and Acceldata made it mainstream.
Now, in 2026, the open-source projects have caught up. Soda Core, Elementary, and dbt tests, alongside DataKitchen’s own DataOps Data Quality TestGen and DataOps Observability, deliver what used to sit behind an enterprise license. Data quality testing and data observability are converging on one expectation: pipelines that prove their own reliability, the way modern software does.
Why Open Source Matters
Open source changes who holds the power in data tooling. You can read how a test runs, how a metric is computed, and how an alert fires. You can add your own test types or contribute them back. You can wire the tool into dbt, Airflow, or Dagster without waiting on a vendor’s integration roadmap. You can pick up connectors and checks that peers in other industries already wrote. And you can try all of it without a license conversation.
Open-source adoption has its own problems: fragmented tools, uneven test coverage, and thin governance. DataKitchen’s tools answer those by putting tests, monitoring, and scorecards under one process, so you keep the transparency and add the structure a production pipeline needs.
Mapping the Open-Source Data Quality and Observability Landscape
Open-source tools in this space fall into two categories: data quality software and data observability software. The last column in each table is new for 2026. It answers the question the GX Cloud shutdown taught everyone to ask: how much of the product do you get if the company behind it disappears tomorrow?
Open Source Data Quality Testing Software
| Open Source Data Quality Tool | Description | Comment | Open Source: Command Line / DSL | Open Source: UI | Auto-Generate Data Quality Tests | Full Product in Open Source? |
|---|---|---|---|---|---|---|
| Great Expectations: github.com/great-expectations/great_expectations | A Python-based framework to define “expectations” about your data (validations/tests/assertions), run them, document outcomes, and monitor data quality over time. | The framework that started the category; heavy to maintain. GX Core continues under Fivetran. The UI and scheduler were GX Cloud, which shut down June 1, 2026. | yes | no | no | partial |
| Soda Core: github.com/sodadata/soda-core | Open-source CLI + Python library for data quality testing: using SodaCL (checks language) to scan datasets for missing/invalid/unexpected values. | Really, a DSL (domain-specific language) to write data quality tests. The UI and alerting are Soda Cloud. | yes | no | no | partial |
| Deequ: github.com/awslabs/deequ | Scala library (Spark-based) by AWS Labs: define constraints/tests on DataFrames, compute metrics, detect anomalies/constraints violations at scale. | Very good for big-data / Spark environments; less so for lighter warehousing without Spark. | yes | no | no | yes |
| DataQualityDashboard: github.com/OHDSI/DataQualityDashboard | Focused on observational health data (OMOP CDM) but generalizable: runs systematic data quality checks (completeness, consistency) via defined check types. | DataQualityDashboard is an R package | yes | yes | no | yes |
| DQOps: github.com/dqops/dqo | Data quality monitoring platform: ~150 built-in table/column checks, dashboards, incident grouping, notifications for freshness/timeliness. | Their open-source version is limited to a few tables, so it works as a demo of the enterprise product rather than a deployment. | yes | yes | no | demo |
| SQLFluff: github.com/sqlfluff/sqlfluff | A dialect-flexible SQL linter and auto-formatter designed for ELT/DBT codebases; supports templating (Jinja, dbt), multi-dialect SQL. | Linting of SQL is NOT data quality, more like code quality | yes | no | no | yes |
| CSVPath: www.csvpath.org | A data file feeds a quality management framework centered around data identification, validation, and descriptive and lineage metadata production | A new player with a unique twist and a narrow focus on enterprise file feeds. | yes | yes | no | yes |
| DataOps Data Quality TestGen: github.com/DataKitchen/dataops-testgen | DataOps Data Quality TestGen delivers simple, fast data quality test generation and execution by data profiling, new dataset hygiene review, AI generation of data quality validation tests, ongoing testing of data refreshes, & continuous anomaly monitoring | Engineers don’t have time to write tests by hand. Full coverage of every table and column in one click. Same engine and UI in the open-source and enterprise editions. | yes | yes | yes | yes |
Open Source Data Observability Software
| Open Source Data Observability Tool | Description | Comment | Open Source: Command Line / DSL | Open Source: UI | Auto-Generate Data Quality Tests | Full Product in Open Source? |
|---|---|---|---|---|---|---|
| Elementary: github.com/elementary-data/elementary | ”dbt-native” data observability: a dbt package plus CLI that runs tests, tracks anomalies, generates a static report, and sends alerts. | Good fit for a dbt-plus-warehouse stack; lighter weight. The open-source package emits a static report; the live UI and alert routing are Elementary Cloud. | yes | static report | no | partial |
| OpenMetadata: github.com/open-metadata/OpenMetadata | A unified metadata platform: supports data discovery, lineage, governance, and includes data quality testing/test-suites, metrics, and dashboards. | Strong “observability + governance” angle; growing quickly. The full web UI ships in open source; Collate sells a hosted edition. | yes | yes | no | yes |
| ODD Platform: github.com/opendatadiscovery/odd-platform | Open-source data discovery and observability platform: includes a data quality dashboard, integration with DQ frameworks, and a metadata graph. | Less mature than some, but aligns with the observability story rather than pure DQ. Ships its own web UI. | yes | yes | no | yes |
| Prometheus: github.com/prometheus/prometheus | Time-series monitoring and alerting system for metrics collection and analysis. | Very mature; only partial data observability (logs/metrics), no task, subtasks, no schedules, no process lineage | yes | no | no | yes |
| Grafana: github.com/grafana/grafana | A visualization and dashboard platform for metrics/logs/traces integration. Used with Prometheus. | Extremely mature; default UI layer for IT observability stacks. | no | yes | no | yes |
| DataOps Observability: github.com/DataKitchen/data-observability-installer | DataOps Observability monitors every data journey, from data source to customer value and from any team development environment to production, across every tool, team, environment, and customer, so that problems are detected, localized, and understood immediately. | Locate the root causes, understand the impact, and prevent future problems. Provides a dashboard to show improvements in delivery speed and quality. | yes | yes | no (integrates with DataOps TestGen) | yes |
What the GX Cloud Shutdown Says About the Rest of This List
The Great Expectations story changes how you should read the two tables above. Most of these projects are not one product. They are an open-source core plus a commercial layer, and the commercial layer is usually where the UI, the scheduler, the alert routing, and the dashboards live. Soda and Elementary both pair an open-source core with a hosted cloud. DQOps limits its open-source edition to a handful of tables. Great Expectations put the whole operating experience in GX Cloud, and on June 1, 2026 that experience stopped existing. The Python framework survived. Everything a team had built on the paid product had 30 days to go somewhere else.
So the question to ask of every row is not “is it open source?” It is “what does the open-source edition do on its own, the day the company behind it gets acquired?”
If the answer is “run tests from the command line and write results to a folder,” you are one funding event away from rebuilding your data quality program. That is not a knock on any of these engineers. It is what venture-funded open source looks like from the inside: the free tier is the top of a funnel, and funnels get sold. We wrote TestGen the other way around. The open-source edition and the enterprise edition run the same engine, the same UI, the same test generation. And it comes from a company that has been profitable since 2013 with no investors. The 30-day email is the failure mode we built the company to avoid sending.
The Brutal Truth About Open Source Data Quality in the Age of AI
AI moves fast, and its appetite for data has broken the traditional approach to quality. While your engineers write test cases one SQL query at a time, bad data reaches the models at volumes nobody is checking. The math does not work. An AI pipeline needs full test coverage: two tests per column, three per table, across hundreds or thousands of tables, maintained as schemas change and data patterns shift. By the time you have written tests for table 10, tables one through nine have changed under you. And the LLM downstream is answering questions with whatever it was given.
The established open-source frameworks, Great Expectations, Soda Core, Deequ, and dbt tests, are solid engineering built for a slower time, when a human had room to think about test design. They demand setup, skill, and above all time, which is the thing an AI development cycle does not have. Models retrain daily. Source counts keep climbing. AI made data quality more necessary and harder to get by hand in the same year. Engineers burn out chasing a standard the tools cannot help them reach, and predictive models that used to work fail because the assumptions underneath them no longer hold.
What you need is not another framework that takes months to set up, or another dashboard that looks good and changes nothing. You need tools that fight AI with AI. Tools that generate broad test coverage in hours. Tools a non-engineer can use to add a check. Tools that let you show a stakeholder a problem before it lands instead of explaining it afterward. The hand-written approach to data quality is as dated as hand-coding HTML. That is the gap DataOps Data Quality TestGen and DataOps Observability were built to fill.
The AI Has to Be Able to Call the Tool
“Fight AI with AI” has a second half that most of this list misses. Your assistant can write a sonnet about the sales pipeline. It cannot tell you whether the sales table loaded this morning, because nothing on this list hands it that information. That is the job of the Model Context Protocol (MCP): an open standard that lets Claude, Cursor, Copilot, or an agent you build call a system’s functions directly, with real data and real history instead of whatever you pasted into a chat window.
“Has an MCP server” is not the bar. Complete is the bar. A read-only server that returns a quality score is a demo. If the assistant can see a failed test but cannot read the profile behind it, generate a replacement, run it, record the disposition, and set a monitor, then a human is still doing the loop by hand. The AI is narrating. TestGen’s MCP server covers the whole loop: 96 tools across eight stages, discover, profile, hygiene, test, run, monitor, score, and operate. Every tool runs under the connected person’s existing TestGen role, every tool that writes is marked as such, and anything the assistant changes lands in the UI and the logs like any other operation. It ships in the open-source edition under Apache 2.0, complete since the June 30, 2026 release.
So ask every vendor on this list two questions: how many MCP tools, and how many of them can write? A chat window bolted onto the product is not an answer. It composes with nothing, and it is not what your autonomous agent will call next year.
In 2026, a data quality tool your assistant cannot call is a dashboard.
The Future of Open-Source Data Reliability: DataOps
Automation and AI will push the line further. Machine-learning models already generate tests, detect anomalies, and predict failures. Soon, generated tests will cover far more ground than any team can author by hand. At the same time, shift-left testing, which embeds quality checks early in development, and shift-down testing, which connects quality results to leadership dashboards, will merge into one continuous feedback loop.
The teams that master that loop will stop checking quality after the fact. They will design for trust from the start. Open-source frameworks, run with DataOps discipline, are the foundation.
Conclusion
The open-source shift in data quality and data observability has already happened. You have access to transparent, community-built tools that match or beat proprietary systems. What remains is coordination: linking those tools into one automated process that tests, monitors, and reports on every dataset. And, after 2026, one more check: making sure the open-source edition you adopt is the whole product and not a trial.
TIP
Looking for Commercial Data Quality or Data Observability Software? Read The 2026 Landscape
That coordination layer is what DataKitchen provides. TestGen profiles your database and generates 120-plus tests per table set with one click, then monitors freshness, volume, schema, and drift as part of the open-source edition. The scorecards and dashboards ship with it, so you are not spending a quarter building your own.
The pricing is plain. The open-source edition is free, with no feature gates. TestGen Enterprise is $100 per user per month plus $100 per database connection per month, with unlimited data and events. It comes from a company that has been profitable since 2013 and has never taken outside investment. Venture-backed competitors pivot pricing or sunset products when their investors need them to, and GX Cloud customers got about 30 days’ notice in 2026. There is no exit clock running on TestGen.
More coverage, lower cost, and a vendor that will still be here. That is the whole pitch.
TIP
Explore DataKitchen’s open-source projects: DataOps TestGen and DataOps Observability
FAQ
What are the key points in this blog?
Open-source data quality software has split into two jobs: testing, which checks whether data content is correct, and observability, which watches tables over time for freshness, volume, and schema change. Most tools do one or the other. The post maps the main projects, including dbt tests, Soda Core, Elementary, DQOps, and DataOps TestGen, and argues that manual test authoring cannot keep pace with AI pipelines that retrain hourly.
What are the best open source data quality tools in 2026?
The main open-source options are dbt tests, Soda Core, Elementary, DQOps, and DataKitchen’s DataOps TestGen and DataOps Observability. They differ most in how tests get created. dbt, Soda, and Great Expectations expect you to write every rule by hand in YAML or SQL. DQOps ships roughly 150 built-in table and column checks. TestGen generates its tests from profiling results instead.
Is Great Expectations still available in 2026?
The open-source framework is. GX Core continues as an Apache 2.0 project, and Fivetran took over stewardship of the project in May 2026. The commercial product, GX Cloud, shut down on June 1, 2026 after the company was acquired, and customers got about 30 days to migrate. FICO holds the assets and is not selling it. If you run GX Core, keep running it. If you paid for GX Cloud, the UI, scheduling, and dashboards are gone.
What is the difference between data quality testing and data observability?
Testing checks data content against rules: column formats, value ranges, referential integrity. Observability tracks table-level patterns over time: whether a table updated on schedule, whether row counts moved unexpectedly, whether the schema shifted. A test compares your data to a number you typed. A monitor learns from history and flags deviation on its own. You need both, and most tools deliver only one.
Is open source data quality software good enough for production?
Yes, and several of these projects run in production at large organizations today. The real constraint is not capability but coverage: a tool that makes you hand-author every rule will never cover a 4,000-table warehouse, no matter how good the rule engine is. Pick on how tests get created and maintained rather than on the feature checklist.
Why has AI made data quality more urgent?
Because bad data now propagates into decisions automatically rather than stopping at a dashboard a human reads. An LLM given wrong numbers will state them confidently to whoever asked. Pipelines that retrain on a schedule consume errors before anyone reviews them. The failure mode changed from a wrong chart somebody notices to thousands of decisions nobody audits.
How much does open source data quality software cost to run?
The software is free, but the real costs are compute and staff time. Tests run as SQL inside your warehouse, so you pay your warehouse for that execution. The larger hidden cost is authoring: hand-writing rules for every column is engineering time that scales with your table count. Tools that generate tests from profiling shift that cost from people to machines.
Comparisons
DataKitchen TestGen vs the field
Head-to-head against every major data quality and observability vendor.
No vendors match. .
