How do you profile, test and monitor data quality in Azure SQL Database with TestGen?

DataOps TestGen connects to Azure SQL Database with a read-only contained user, a managed identity, or an Entra ID service principal, runs profiling, data tests, and monitors as SQL inside the database, and publishes scorecards on a data quality dashboard. You add a firewall rule for the TestGen host. It's Apache 2.0 and free for one connection.

Key points

  • DataOps TestGen authenticates to Azure SQL Database with username and password, an Azure managed identity, or an Entra ID service principal (since TestGen 5.92.1, August 27, 2026).
  • TestGen needs only SELECT on the target tables in Azure SQL Database; every database user can already read the information_schema views and sys.partitions, per DataKitchen's Access Requirements page in October 2026.
  • Azure SQL Database's server-level firewall blocks all access to a new logical server until you add a rule, so the TestGen host's outbound IP needs a server-level or database-level firewall rule.
  • Microsoft Purview data quality supports Azure SQL Database, but its freshness rule does not apply to Azure SQL tables, per Microsoft Learn's troubleshooting page in 2026.
  • TestGen's Verify a Data Migration docs say its hash tests miss text changes on SQL Server versions without UTF-8 collations; Azure SQL Database supports UTF-8 collations, and its default case-insensitive collation still merges values that differ only in case.

This page is the Azure SQL Database setup. Profiling works the same as on SQL Server, so the SQL Server page covers the types; for the wider Azure choice, see the Azure answer.

One DataKitchen customer runs a care management application on about 1,000 tables across Azure SQL databases and managed instances. Clinicians type into it from 8am to 8pm. Birth dates get transposed and newborn weights get entered wrong.

Their first lesson was about load, not data. Profiling five tables of under a million rows each against the live database slowed the application for 20 minutes. Profiling moved to a replicated copy on an overnight schedule. Do that from day one.

Connect: a contained user and a firewall rule

Access Requirements asks for SELECT on the target tables and nothing more. Every database user can already read the information_schema views and sys.partitions. Authentication is a username and password, an Azure managed identity, or an Entra ID service principal.

Use a contained user. It authenticates at the database, with no login in master:

CREATE USER testgen WITH PASSWORD = 'choose-a-strong-password';
ALTER ROLE db_datareader ADD MEMBER testgen;

For an Entra identity, create the user FROM EXTERNAL PROVIDER instead and add it to the same role.

Then the firewall. A new logical server blocks all access to its public endpoint until you add a rule. Add the TestGen host’s outbound IP as a database-level rule with sp_set_database_firewall_rule, or a server-level rule in the portal. Skip Allow Azure services and resources to access this server. Microsoft warns it admits every Azure service, other customers’ included.

Profile and test

Profiling captures 55 characteristics per column and flags 32 kinds of hygiene issues, such as non-standard blank values, data type mismatches, and potential PII. TestGen generates 34 test types from that profile, and you configure 19 business-rule and custom SQL types yourself. Every test runs as SQL inside Azure SQL Database. Schedule it in TestGen, or call the REST API from your CI or from Azure Data Factory.

One collation note from Verify a Data Migration. The default case-insensitive collation counts values that differ only in case as one distinct value, so Distinct Count comparisons against a case-sensitive copy fail on correct data. The same page says the hash tests miss text changes on SQL Server versions without UTF-8 collations. Azure SQL Database has them.

Observe and score

Monitors track freshness, volume, and schema, and alerts go to email, Slack, Microsoft Teams, or a webhook. Each table group gets a scorecard with a total score and a critical data element (CDE) score. Every hygiene issue has a PDF Issue Report with sample rows and the SQL to reproduce it.

Walkthrough from the quickstart

  1. Select Connections, click Add Connection, and pick Azure SQL Database.
  2. Enter the server name as Host, the database, and the contained user. Click Test Connection.
  3. Create a table group for the schema, then click Verify Access.
  4. Run profiling and review the hygiene issues.
  5. Generate tests, run them, and open the Quality Dashboard.
TestGen Quality Dashboard with a scorecard showing a total score, a CDE score, profiling and testing sub-scores, and scores for accuracy, completeness, consistency, timeliness, uniqueness, and validity

That’s the Quality Dashboard on TestGen’s demo data, not an Azure database: one scorecard with total and CDE scores and a score per quality dimension.

TestGen vs Purview on Azure SQL Database

DataOps TestGenMicrosoft Purview data quality
Azure SQL DatabaseProfiling, tests, monitorsProfiling and data quality scans
FreshnessFreshness monitor learns the update scheduleFreshness rule not available for Azure SQL tables
AuthenticationContained user, managed identity, or service principalManaged identity only
Tests34 types generated from the profileHand-built and AI-suggested rules

Purview sources: supported sources, overview, and troubleshooting, checked October 21, 2026. Purview is the better fit when governance owns the budget and wants scores rolled up to data products. TestGen is the better fit when the data team owns the problem. DataKitchen’s August 2026 post maps the rest of Microsoft’s options, and the Purview comparison estimates Purview at $60,000 to $150,000 a year for 10 users and 1,000 tables (DataKitchen’s estimate).

Pricing

The core profiling, testing, and monitoring engine is free and open source. Enterprise adds team access, security, Commercial Pharma tests, source-data controls for AI, and support. Open source covers one user, one database connection, one project, and unlimited tables. Enterprise is $100 per user per month plus $100 per database connection per month. Install open source TestGen. Migrating between databases? See the migration answer and the migration post.

Documentation: Azure SQL and Synapse quickstart, Access Requirements, Manage Connections, Data Profiling, Monitor Tables, Run Profiling and Tests, Verify a Data Migration

Frequently asked questions

Short answers to the follow-up questions people ask next.

Does any data leave my environment?

No. TestGen runs its profiling, tests, and monitors as read-only SQL inside Azure SQL Database. It stores results and metadata in its own application database: statistics, test outcomes, hygiene findings, and the most frequent values in each profiled column. It never runs INSERT, UPDATE, DELETE, CREATE, ALTER, or DROP against your database.

Should TestGen use a contained user or a server login on Azure SQL Database?

A contained user. Microsoft Learn describes contained users as authenticating at the user database with no login in master, and recommends pairing them with database-level firewall rules. Create one with CREATE USER and a password, add it to db_datareader, and TestGen has the SELECT access it needs and nothing else.

What firewall rule does TestGen need for Azure SQL Database?

One that allows the TestGen host's outbound IP address on TCP port 1433. Microsoft Learn says a new logical server's firewall blocks all access to its public endpoint until you add a rule. Avoid the Allow Azure services setting for this: Microsoft warns it admits every Azure service, including services in other customers' subscriptions.

Can TestGen use a managed identity or service principal with Azure SQL Database?

Yes. As of October 2026, TestGen connects to Azure SQL Database with a username and password, an Azure managed identity, or an Entra ID service principal. Service principal support arrived in TestGen 5.92.1 on August 27, 2026. Create the identity as a contained user in the database with FROM EXTERNAL PROVIDER and grant it read access.

Should I use Microsoft Purview or TestGen on Azure SQL Database?

Use Purview if you want a Microsoft catalog with data quality scores rolled up to data products. It supports Azure SQL Database, but its freshness rule doesn't apply to Azure SQL tables, per Microsoft Learn. Use TestGen if you want freshness, volume, and schema monitors plus 34 test types generated from a profile of 55 column characteristics.

Try it on your own data

DataOps TestGen is free and open source. Profile a table and generate its tests this afternoon.