Open source data quality for Microsoft Fabric
Microsoft Fabric data quality, without writing tests.
TestGen tests the data in your Fabric Lakehouses and Warehouses. It connects through the SQL analytics endpoint, profiles every table, and generates thousands of tests. Read-only. Free and open source.
Install TestGen
Pick your platform. Free and open source.
# TestGen install for Mac, Linux.
# Download the latest installer.
$ curl -o dk-installer.py \
'https://raw.githubusercontent.com/DataKitchen/data-observability-installer/main/dk-installer.py'
# Run the install command.
$ python3 dk-installer.py tg install
# Full docs: docs.datakitchen.io/testgen/get-started/install-on-mac-linux/- Double-click
dk-installer.exein your Downloads folder. - If Windows shows a protection warning, choose More info, then Run anyway.
- Pick TestGen, then Docker or pip. The installer seeds demo data, opens the UI, and prints your login.
# Full docs: docs.datakitchen.io/testgen/get-started/install-on-windows/
- 55data profiling column characteristics
- 32data hygiene detectors
- 34test types auto-generated from the profile
- 19business rule and custom SQL test types
Connect TestGen to Microsoft Fabric
TestGen reads your Lakehouse and Warehouse tables through the Fabric SQL analytics endpoint, which is read-only by design. It stores only results: statistics, scores, and test outcomes.
- What gets tested
- Tables in your Fabric Lakehouses and Warehouses, all stored in OneLake.
- Sign-in
- Entra ID service principal or Managed Identity.
- Access
- Contributor (or higher) on the workspace. No item-level grants needed.
- Coming from SQL Server or Synapse?
- TestGen tests SQL Server and Synapse too, so you can compare before and after a migration.
Ask your Fabric admin for three things
- An Entra ID service principal (or a Managed Identity) for TestGen. SQL logins don’t work on the Fabric SQL analytics endpoint.
- Contributor (or higher) on the Fabric workspace that holds your Lakehouse or Warehouse.
- The tenant setting Service principals can call Fabric public APIs (formerly “Service principals can use Fabric APIs”) turned on in the Fabric admin portal.
Workspace-level access is enough; no item-level permissions. Details in the access requirements.
Microsoft Fabric data problems TestGen catches
What goes wrong in Lakehouses and Warehouses before it shows up in a Power BI report. Each card lists the TestGen checks that find it.
-
A pipeline or Dataflow reloaded
The same rows landed twice in a Lakehouse table.
-
A shortcut or feed went stale
A table that should update daily stopped.
-
Columns changed type between loads
File-based loads where a column flips between numbers and text.
-
Warehouse totals don’t match the source
Fact tables that drift from the Lakehouse they were built from.
-
Nulls that break the report
Missing values in the columns a semantic model aggregates.
-
The numbers moved for no reason
Averages and distributions that shift between loads.
What TestGen does on Microsoft Fabric
Everything below is in the open source release, and all of it works on Fabric. Enterprise adds team access, security, Commercial Pharma tests, and support.
-
Auto-Generated Tests (documentation, opens in a new tab)
Point TestGen at a Fabric Lakehouse or Warehouse and it writes the tests, then runs them through the SQL analytics endpoint.
-
Data Profiling (documentation, opens in a new tab)
55 characteristics per column for every Lakehouse and Warehouse table, including columns that flip type between file loads.
-
Hygiene Detection (documentation, opens in a new tab)
Finds what pipelines and Dataflows leave behind: duplicate reloads, odd blanks, mixed types and placeholder values.
-
Anomaly Detection (documentation, opens in a new tab)
Freshness, volume, schema and custom-metric monitors flag a stale shortcut, a Delta table whose schema changed, or a load that came in short.
-
Data Catalog (documentation, opens in a new tab)
Profiles, hygiene issues, test results, PII flags and Critical Data Elements for your Fabric tables, next to whatever Purview catalogs.
-
Quality Scoring (documentation, opens in a new tab)
Scorecards by dimension and Critical Data Element for the tables your Power BI semantic models read from.
-
MCP Server (the TestGen MCP cheat sheet)
Ask Claude, Copilot or Cursor what failed in your Lakehouse and see the rows behind it, through 112 MCP tools that run in your environment.
-
Open Source Core, UI Included
Unlimited tables and a full UI. The core data profiling, testing and monitoring engine is free under Apache 2.0, for one Fabric connection. Enterprise adds more connections, team access, security and support.
From DataKitchen on Microsoft Fabric
- Answer
How do you test data quality in a Microsoft Fabric Warehouse?
Test a Fabric Warehouse by connecting over its SQL connection string with Entra ID, running data profiling on every table, generating tests, and monitoring freshness…
- Answer
How do you set up an Entra service principal so a tool can read your Fabric workspace?
Microsoft's steps to register an Entra app, create a secret, enable service principals in Fabric tenant settings, and add the principal to a workspace.
- Answer
How do you validate a SQL Server to Microsoft Fabric migration?
Profile SQL Server before the move, run comparison tests on the Fabric warehouse or lakehouse after it, and sign off when every failure matches a planned change.
- Answer
What is the best data quality and data observability tool for Microsoft Fabric and OneLake?
The best Fabric data quality tool profiles every lakehouse table, generates tests, monitors freshness, volume, and schema, and runs as SQL in your tenant.
- Answer
What data quality dashboards should a SQL Server and Power BI team build?
The six types of data quality dashboard mapped onto a SQL Server and Power BI stack: who each is for, what it is built from, and which ones TestGen fills in for you.
-
Blog Testing Every Layer of the Medallion Architecture, Now with OneLake at the Center
DataOps TestGen 5.92.1 connects to Microsoft OneLake through the Fabric SQL analytics endpoint. Here is what Microsoft's current medallion guidance says, why…
Why teams choose TestGen
-
You Don't Have Time to Write Tests. TestGen Does It Automatically.
You're buried in customer requests. You have no time to write tests, let alone innovate. TestGen scans your data and generates the tests and anomaly detectors for you. No coding, no massive YAML configuration.
-
Fabric moves your data. It doesn’t test it for you.
TestGen profiles every Lakehouse and Warehouse table, writes the tests, and monitors freshness, volume, and schema between pipeline runs.
See how TestGen works →
-
Runs inside Fabric
Tests run as read-only SQL where the data sits. Nothing is copied out, and nothing is written back.
-
Apache 2.0, core engine free
Every detector, every characteristic, and every test type but the 12 Commercial Pharma types. In the open source.
-
Self-hosted
Your data never leaves your environment.
-
Bootstrapped since 2013
Profitable, independent, and not pivoting next quarter.
Open Source with Reasonable Enterprise Pricing
Open source TestGen is free for one user, one project, and one database connection. Enterprise is a flat $100 per month per user and database connection, with unlimited tables and data volume. No per-table tax. A year of enterprise data quality costs roughly one data engineer's monthly salary.
See pricingTestGen versus the competition
Head-to-head comparisons with every major data quality and observability tool. Search for the one you are evaluating.
No vendors match. .
- Bristol Myers Squibb
- Eisai
- Catholic Relief Services
- Progeny Health
- X4
- KOA
- CNH
Questions
What does TestGen test in Microsoft Fabric?
The tables in your Fabric Lakehouses and Warehouses, all stored in OneLake, read through the Fabric SQL analytics endpoint. TestGen tests the data; it does not test pipelines, notebooks or Power BI reports directly.
Can TestGen use a SQL login on Fabric?
No. The Fabric SQL analytics endpoint does not accept SQL logins. TestGen signs in with an Entra ID service principal or a Managed Identity.
What access does TestGen need in Fabric?
Contributor (or higher) on the Fabric workspace, and, for a service principal, the tenant setting “Service principals can call Fabric public APIs” (formerly “Service principals can use Fabric APIs”) turned on in the Fabric admin portal. See how to set up the service principal. Workspace-level access is enough; no item-level permissions are needed.
Does TestGen change anything in my database?
No. TestGen runs read-only SELECT queries to profile data, run tests and collect monitor metrics. It never runs INSERT, UPDATE, DELETE, CREATE, ALTER or DROP, and it stores only computed results (statistics, scores and test outcomes) in its own application database.
Does TestGen replace Microsoft Purview data quality in Fabric?
No, it works beside it. Purview catalogs and governs; TestGen generates and runs the tests. See TestGen beside Purview.
Can TestGen test my Power BI semantic models?
Not directly. TestGen tests the Lakehouse and Warehouse tables your semantic models read from, which is where most report errors start.
Is TestGen free?
The open source edition is free under Apache 2.0, for one user, one project and one connection, with every detector and test type except the Commercial Pharma tests. Enterprise is $100 per user per month plus $100 per database connection per month, with no per-table fees. See pricing.
Free book
Not ready to try it out now? Read the free book.
The DataOps Way to Data Quality and Data Observability covers why data quality keeps failing and what to do about it: culture, testing strategy, monitoring, and the seven deadly sins of AI-generated pipelines.
- 620 pages
- 63 chapters
- Updated September 2026
- By Chris Bergh, Chip Bloche, and Gil Benghiat