How do you profile, test and monitor data quality in SQL Server with TestGen?

DataOps TestGen connects to Microsoft SQL Server with a read-only SQL login, profiles every column on 55 characteristics, generates tests that run as T-SQL inside your database, and monitors freshness, volume, and schema over time. Results land on data quality dashboards and scorecards. TestGen is Apache 2.0 and free for one connection.

Key points

  • TestGen connects to Microsoft SQL Server with a username and password and needs only SELECT on the target tables; information_schema.columns, sys.objects, and sys.partitions are readable by default, per TestGen's Access Requirements page in October 2026.
  • Microsoft's SSIS Data Profiling Task works only on data stored in SQL Server, skips varchar(max) and nvarchar(max) in its text profiles, and writes XML that you read in a separate viewer.
  • Microsoft discontinued Data Quality Services and Master Data Services in SQL Server 2025 and supports them only in SQL Server 2022 and earlier, per Microsoft Learn.
  • SQL Server's default case-insensitive collation counts values that differ only in case as one distinct value, and every collation treats values that differ only in trailing spaces as one, per TestGen's migration docs.
  • Open-source TestGen covers one user, one database connection, and one project with unlimited tables; Enterprise is $100 per user per month plus $100 per database connection per month.

This page is for SQL Server you run yourself, on premises or on an Azure VM. Azure SQL Database has its own page, and if you’re moving off SQL Server, start with the migration answer.

Connect

Create a SQL login for TestGen. Per Access Requirements, checked October 2026, that’s the only method listed for SQL Server. Windows authentication isn’t.

The grants are short:

PermissionWhy TestGen needs it
SELECT on tables in the target schemaProfile, run tests, collect monitor metrics
SELECT on information_schema.columnsDiscover tables, columns, and types
SELECT on sys.objects, sys.partitionsApproximate row counts

The last two are readable by default. You don’t need VIEW DEFINITION. You can add the login to db_datareader, but that reads every user table and view in the database, so grant SELECT on the schemas you want tested. If TestGen runs in Docker on the same machine as SQL Server, use host.docker.internal as the host.

Profile

TestGen profiles 55 column characteristics and runs 32 hygiene detectors. Every column lands in one of four general types (Alpha, Numeric, Datetime, Boolean), and the data type suggestion catches a varchar column full of dates or amounts.

Two SQL Server habits to know. The default case-insensitive collation counts ACME and acme as one distinct value, and any collation treats trailing-space variants as one (migration docs). And on a busy server, start small. The hotel reservation case study, on a 1.2 TB SQL Server database, says to set TestGen to one or two threads and raise the count once runs are stable.

Test and observe

Generation picks from 34 test types and sets thresholds from the profile. You add the rules only you know from 19 business-rule and custom SQL types. Every test runs as T-SQL inside SQL Server. Schedule runs in TestGen, or trigger them from any scheduler through the REST API.

Monitors learn freshness, volume, schema, and custom metric patterns and flag what breaks them. If SSIS loads the warehouse, DataOps Observability’s SSIS agent shows the package run next to the table it loaded.

Dashboard

Each table group gets a scorecard on the Quality Dashboard. Score Explorer builds a dashboard around your critical data elements, and the issue report exports as PDF for the team that owns the source.

Walkthrough

DataKitchen’s docs have no SQL Server quickstart, so these are the general steps from Manage Connections. The Azure SQL and Synapse quickstart is the closest worked example.

  1. Select Connections, click Add Connection, and choose SQL Server as the data source.
  2. Enter the host, port, database, and the TestGen login, then click Test Connection.
  3. Create a table group: a schema plus include and exclude masks.
  4. Click Verify Access, leave Run profiling on, and turn on Generate and schedule tests and Configure monitors.
  5. Review hygiene issues, then open the Quality Dashboard.
TestGen Quality Dashboard with one scorecard showing a total score, a CDE score, and scores by quality dimension

The screenshot is TestGen’s Quality Dashboard, the scorecard view you reach in step 5. It is not from a SQL Server database.

TestGen vs native SQL Server

ConstraintsSSIS Data Profiling TaskDataOps TestGen
What it doesRejects bad rows at write timeEight profiles, written to XML55-characteristic profile, 32 hygiene detectors, generated tests
Generates testsNoNo34 test types
Monitors over timeNoNoFreshness, volume, schema, custom metrics
SourcesThat tableSQL Server only; skips varchar(max) in text profiles14 databases plus files

Sources: Data Profiling Task, checked October 2026. Data Quality Services and Master Data Services are discontinued in SQL Server 2025. Purview runs data quality scans on on-premises SQL Server but doesn’t profile it, per Microsoft’s supported sources. DataKitchen’s nineteen-features post maps the rest of the Microsoft stack.

Keep your constraints. They stop impossible rows. Pick TestGen when the problem is plausible-but-wrong data that no constraint describes, across more tables than anyone will hand-write tests for.

The core profiling, testing, and monitoring engine is free and open source. Enterprise adds team access, security, Commercial Pharma tests, source-data controls for AI, and support. Start with the install, and see which databases TestGen supports and the Azure overview.

Documentation: Connect your Data, Access Requirements, Manage Connections, Monitor Tables, Run Profiling and Tests, Verify a Data Migration, Observability SSIS agent.

Frequently asked questions

Short answers to the follow-up questions people ask next.

Does TestGen support Windows authentication for SQL Server?

No. As of October 2026, TestGen's Access Requirements page lists username and password as the only authentication method for Microsoft SQL Server, so you create a SQL login for TestGen. Azure SQL Database is different: it also accepts a managed identity or an Entra ID service principal.

Does TestGen need db_datareader or VIEW DEFINITION on SQL Server?

Neither is required. TestGen needs SELECT on the tables it profiles and tests, and reads information_schema.columns, sys.objects, and sys.partitions, which a database user can read by default. Adding the login to db_datareader works, but that role reads every user table and view in the database, so grant SELECT on the schemas you want tested instead.

Does any data leave my environment?

No. TestGen runs profiling and tests as T-SQL inside your SQL Server, through a read-only login. TestGen's own database stores results and metadata: aggregates, statistics, test outcomes, and, for display, each column's most frequent values and minimum and maximum text values. TestGen runs on a server you choose, on premises or on an Azure VM, so nothing goes to a DataKitchen service.

Can TestGen replace the SSIS Data Profiling Task or Data Quality Services?

It replaces the profiling and adds tests. The SSIS Data Profiling Task computes eight profiles and writes XML; TestGen profiles 55 column characteristics, flags 32 hygiene issues, and generates tests from 34 test types. Microsoft discontinued Data Quality Services in SQL Server 2025. TestGen doesn't do DQS-style cleansing, matching, or survivorship, so that part needs another tool.

Can TestGen check a SQL Server to Snowflake migration?

Yes. TestGen's Table Comparison tests are generated from the SQL Server profile and run on Snowflake after the move. Per TestGen's migration docs, hash tests on text columns need UTF-8 collations, which means SQL Server 2019 or later. SQL Server's case-insensitive default collation and trailing-space handling can also show as failures on a correct copy.

Try it on your own data

DataOps TestGen is free and open source. Profile a table and generate its tests this afternoon.