Open source data quality for Microsoft Purview teams
Microsoft Purview governs your data. TestGen writes the tests.
Purview data quality rules are configured one at a time. TestGen generates them. It profiles every column on SQL Server, Azure SQL, Synapse, and Fabric, and writes thousands of tests. Free and open source.
Install TestGen
Pick your platform. Free and open source.
# TestGen install for Mac, Linux.
# Download the latest installer.
$ curl -o dk-installer.py \
'https://raw.githubusercontent.com/DataKitchen/data-observability-installer/main/dk-installer.py'
# Run the install command.
$ python3 dk-installer.py tg install
# Full docs: docs.datakitchen.io/testgen/get-started/install-on-mac-linux/- Double-click
dk-installer.exein your Downloads folder. - If Windows shows a protection warning, choose More info, then Run anyway.
- Pick TestGen, then Docker or pip. The installer seeds demo data, opens the UI, and prints your login.
# Full docs: docs.datakitchen.io/testgen/get-started/install-on-windows/
- 55data profiling column characteristics
- 32data hygiene detectors
- 34test types auto-generated from the profile
- 19business rule and custom SQL test types
Purview and TestGen: two halves of the same job
Purview is the catalog and governance system of record. TestGen is the testing engine. Used together, governance gets a quality signal it can’t produce on its own.
- Purview does
- Catalog and search, lineage across databases, lakes, Fabric, and Power BI, business glossaries, sensitivity labels, access control, and domain ownership.
- TestGen adds
- Tests generated from a profile of 55 column characteristics, multi-table checks, and a Pass, Patch, Purge, or Push Back workflow with record-level detail.
- Where rules come from
- Both profile data and run checks. In Purview each rule is configured by hand; TestGen generates them from the profile.
- The arithmetic
- In DataKitchen’s benchmark, 20 tables and 1,000 columns came to 2,526 tests. TestGen does it in two steps. By hand, at 30 minutes a test, it is about 1,250 hours. The math.
- Where it runs
- In your environment, as read-only SQL inside the database. On-prem SQL Server and non-Microsoft platforms too.
- Price
- Open source is free. Enterprise is $100 per user and $100 per connection a month. See the full comparison.
Rules TestGen generates that you’d otherwise configure in Purview
The same data quality dimensions, with the tests written for you. Each card lists the TestGen checks generated from a profile of your data.
-
Completeness
Missing values, empty columns, placeholder values.
-
Uniqueness
Duplicate rows and keys that should be unique.
-
Validity
Formats, patterns, ranges and types.
-
Timeliness
Freshness and dates that make sense.
-
Consistency
Totals and references across tables.
-
Accuracy
Values and distributions that suddenly change.
What TestGen adds next to Purview
Everything below is in the open source release. Enterprise adds team access, security, Commercial Pharma tests, and support.
-
Auto-Generated Tests (documentation, opens in a new tab)
Where Purview has you configure each data quality rule, TestGen generates thousands of tests from a data profile of every column.
-
Data Profiling (documentation, opens in a new tab)
55 characteristics per column, the evidence behind every generated test, for the same tables Purview catalogs.
-
Hygiene Detection (documentation, opens in a new tab)
Finds problem rows that no rule was written for yet: placeholders, mixed types, truncation, stray spaces.
-
Anomaly Detection (documentation, opens in a new tab)
Freshness, volume, schema and custom-metric monitors on the tables Purview tracks, learned from their history.
-
Data Catalog (documentation, opens in a new tab)
Purview stays the enterprise catalog. TestGen’s catalog adds the test results, hygiene issues and quality scores for each table.
-
Quality Scoring (documentation, opens in a new tab)
Scores by data quality dimension and Critical Data Element, the quality signal governance can’t produce on its own.
-
MCP Server (the TestGen MCP cheat sheet)
Ask Claude, Copilot or Cursor what failed and see the rows behind it, through 112 MCP tools that run behind your firewall.
-
Open Source Core, UI Included
Free under Apache 2.0 for one connection, beside a Purview contract our comparison page estimates at $60K–$150K a year. Enterprise is $100 per user and per connection a month.
From DataKitchen on Purview and Microsoft data quality
- Answer
Does Microsoft Purview Data Quality work with on-prem SQL Server?
Purview Data Quality runs data quality scans on on-premises SQL Server but doesn't profile it, per Microsoft's supported-sources table. What that means in practice.
- Answer
How much does Microsoft Purview Data Quality cost compared with an open source tool?
Purview Data Quality bills per processing unit and per governed asset. Microsoft's November 2026 rates, a worked example, and how open source TestGen compares.
-
Blog How TestGen Complements Microsoft Purview for Enterprise Data Quality
DataKitchen's TestGen and Microsoft's Purview complement each other: Purview serves as the governance and catalog "source of truth," while TestGen is the deep,…
-
Blog There Is No Microsoft Data Quality Tool in 2026. There Are Nineteen.
You went looking for the Microsoft data quality tool. There isn't one. There are nineteen features across five products, twelve of them manual, and they don't talk to…
- Answer
What is the best data quality and data observability tool on Azure?
On Azure, pick a tool that profiles Synapse, Azure SQL, Fabric, and Blob files, generates tests, and monitors freshness and volume in your tenant.
- Answer
What data quality dashboards should a SQL Server and Power BI team build?
The six types of data quality dashboard mapped onto a SQL Server and Power BI stack: who each is for, what it is built from, and which ones TestGen fills in for you.
Why teams choose TestGen
-
You Don't Have Time to Write Tests. TestGen Does It Automatically.
You're buried in customer requests. You have no time to write tests, let alone innovate. TestGen scans your data and generates the tests and anomaly detectors for you. No coding, no massive YAML configuration.
-
A catalog shows you the data. Tests tell you if it’s right.
TestGen profiles every column, writes the tests, and monitors freshness, volume, and schema, so the assets Purview catalogs come with a quality score.
See how TestGen works →
-
Runs inside your database
Tests run as read-only SQL where the data sits. Nothing is copied out, and nothing is written back.
-
Apache 2.0, core engine free
Every detector, every characteristic, and every test type but the 12 Commercial Pharma types. In the open source.
-
Self-hosted
Your data never leaves your environment.
-
Bootstrapped since 2013
Profitable, independent, and not pivoting next quarter.
Open Source with Reasonable Enterprise Pricing
Open source TestGen is free for one user, one project, and one database connection. Enterprise is a flat $100 per month per user and database connection, with unlimited tables and data volume. No per-table tax. A year of enterprise data quality costs roughly one data engineer's monthly salary.
See pricingTestGen versus the competition
Head-to-head comparisons with every major data quality and observability tool. Search for the one you are evaluating.
No vendors match. .
- Bristol Myers Squibb
- Eisai
- Catholic Relief Services
- Progeny Health
- X4
- KOA
- CNH
Questions
Does TestGen replace Microsoft Purview?
No. Purview is the catalog and governance system of record: search, lineage, glossaries, sensitivity labels and access control. TestGen is the testing engine beside it, generating and running the data quality tests.
What is the difference in how rules are created?
Both tools profile data and run checks. In Purview each data quality rule is configured by hand. TestGen generates tests from a profile of 55 column characteristics, so thousands of tests exist before anyone writes a rule.
Does TestGen work with on-prem SQL Server and non-Microsoft databases?
Yes. TestGen runs in your own environment and tests SQL Server, Azure SQL, Synapse, Fabric, Snowflake, Databricks, PostgreSQL and more.
How does the cost compare?
Our comparison page estimates Purview at $60K–$150K a year for 10 users and 1,000 tables. TestGen Enterprise is $12,000 to $36,000 a year at the same size, and the open source edition is free.
Can I use TestGen and Purview on the same data?
Yes. Purview stays the catalog and system of record; TestGen tests the tables it describes and gives each a quality score.
Which data quality dimensions does TestGen score?
Accuracy, completeness, consistency, timeliness, uniqueness and validity. Every check maps to one of them, and scores roll up per table, table group and dimension.
Does TestGen change anything in my database?
No. TestGen runs read-only SELECT queries to profile data, run tests and collect monitor metrics. It never runs INSERT, UPDATE, DELETE, CREATE, ALTER or DROP, and it stores only computed results (statistics, scores and test outcomes) in its own application database.
Free book
Not ready to try it out now? Read the free book.
The DataOps Way to Data Quality and Data Observability covers why data quality keeps failing and what to do about it: culture, testing strategy, monitoring, and the seven deadly sins of AI-generated pipelines.
- 620 pages
- 63 chapters
- Updated September 2026
- By Chris Bergh, Chip Bloche, and Gil Benghiat