Catch bad data before your customer does
Your dashboards run green while bad data slips through. TestGen is open source. It profiles every table, generates thousands of data quality tests, scores every column, and gives you a quality dashboard the team will actually look at. The DataOps way to data quality. No vendor lock-in.
Key points
- Dashboards run green while bad data slips through, because a pipeline that succeeded is not the same as data that is correct.
- TestGen is open source: it profiles every table, generates thousands of data quality tests, and scores the results.
- Hygiene issues surface at profile time, before anyone has written a test.
- A quality dashboard helps only if the team reads it, which means a score rather than raw test output.
- Continuous table monitoring catches the drift a one-time assessment is structurally unable to see.
Run open-source TestGen yourself
TestGen is open source and ships in a Docker container. Stand it up in 15 minutes against PostgreSQL, Snowflake, Redshift, BigQuery, Databricks, Oracle, SAP HANA, or any of the other supported databases. Point it at a schema. It profiles every column, scores the data, and generates the test suite from what it finds. A junior operator can run it against a demo cluster without help. No per-row pricing. No vendor lock-in. The source is on GitHub.
The DataOps way to data quality
Most teams write a data quality plan, schedule a quarterly review, and call it shipped. By the time the review happens the schema has changed three times and an analyst has filed a ticket. The DataOps way is the opposite: measure first, score, generate tests on the score, run the tests on every refresh, iterate. You get coverage in days instead of two quarters. Shift left to catch issues at the source where they cost a dollar a record to fix, not at the dashboard where they cost a hundred.
“So much of what we do involves business questions that are fire drills. Executives don't trust our analytics. DataKitchen enabled us to deliver over 10,000 data quality validation tests that run every release. Now, they trust us!”
Build a quality dashboard the team will read
Most quality dashboards die in a Confluence page nobody opens. TestGen builds a scorecard the team uses: a total score, a CDE-only score, and breakdowns by accuracy, completeness, consistency, timeliness, uniqueness, and validity. Drill from a scorecard into a dimension, into a column, into the failing test. Add custom scorecards for a specific table group, a critical pipeline, or a domain owner's slice of the catalog. The score updates with every profiling run. Nobody is updating it by hand at the end of the quarter.
Find hygiene issues at profile time
TestGen scans every column for hygiene issues before you ever write a test. Non-standard blank values. Quoted values stored as strings. Dates that haven't moved in two years. Columns that look like PII but aren't tagged. Each issue lands in a triage view with table, column, likelihood, and a one-line detail. Mark it definite, dismiss it, or push it back to the data engineer who owns the source. The dirty columns surface themselves.
Smart, continuous table monitoring
TestGen runs the suite on every refresh and watches for the things you didn't think to test for. Freshness, volume, schema, custom metrics. The Monitors page rolls anomalies up by table over a 14-day window so at a glance you see which tables drifted overnight and which broke. Drill into a table for a trend chart with anomaly markers. Notifications fire when a test fails.
See TestGen run against your data
Stand up open-source TestGen against your own schema in 15 minutes. Free. No vendor lock-in.
Frequently Asked Questions
Common questions about Data Quality
What is data quality testing?
Data quality testing evaluates rules against your rows and reports which ones failed. It covers formats, value ranges, referential integrity, uniqueness, and the business logic specific to your organization. The distinguishing feature is that a failure names the table, the column, and the failing rows rather than just reporting a score.
How many data quality tests do I need?
A working rule of thumb is a minimum of two tests per table plus coverage on every column, applied at every layer. On a few hundred tables that runs to thousands of tests, which is why hand-authoring stalls and why generating the mechanical majority from profiling is the only path to real coverage.
Can data quality tests be generated automatically?
Yes. TestGen profiles every table and column, analyzes 55 column characteristics, and derives roughly 120 validation tests from what it finds. That covers the mechanical checks. Business rules that depend on domain knowledge still need a human, which is where your team's time is actually worth spending.
Which databases does this work with?
Snowflake, Databricks SQL, Azure Synapse Analytics, Azure SQL Database, Microsoft SQL Server, Google BigQuery, Amazon Redshift, Amazon Aurora PostgreSQL, Oracle Database 12c and later, SAP HANA, PostgreSQL, and Salesforce Data 360, in both the open-source and enterprise editions. On-premises SQL Server is a supported database rather than a separate infrastructure project.
What does data quality testing cost?
The open-source edition is free under Apache 2.0 with no feature gates or usage limits. Tests run as SQL inside your own warehouse, so you pay your warehouse for that compute and nothing per table or per check. Enterprise pricing is flat rather than usage-based.
Where should data quality tests run?
Inside your warehouse, as close to the data as possible, and at more than one point in the pipeline. Source data, ingestion, between transformation steps as a pipeline gate, and in CI against code changes. The same test does a different job at each of those four points.