DataOps TestGen Technical Overview

Introduction

Sixty-two generated tests ran green on a 300,000-row sample of NYC taxi rides. The sample held fares as low as negative $367 and 17,477 trips with zero passengers. Every test passed.

That is the right result, and it’s the whole point of this paper. Tests generated from your data learn what your data already looks like, so impossibility reads as normal. On the full month of 2,964,624 rides, seven business rules that an AI assistant wrote from the profile, working through TestGen’s MCP server, found 171,627 trips with zero or null passengers and 37,448 negative fares. Profiling took 1 minute 48 seconds. Running all 69 tests took 6 seconds.

So you need both halves. Generated tests cover the broad, boring, necessary checks that nobody has time to write. Configured tests carry the rules that live in somebody’s head. DataOps TestGen does the first half for you and gives you a fast way to do the second.

Most data teams have neither. A March 2020 Gartner survey found data professionals spend 56% of their time on operational execution and only 22% on innovation that delivers value. In a 2021 survey of 600 data engineers by DataKitchen and data.world, 50% named too much time finding and fixing errors as a source of burnout. Tests go unwritten, and customers find the errors.

DataOps TestGen profiles every column, flags hygiene problems, generates a test suite that fits your actual data, runs it inside your database, and watches your tables between runs. It’s free and open source under the Apache 2.0 license, and it runs entirely inside your network.

The Problem DataOps TestGen Solves

Production data errors are expensive in three ways. Customers find them first, so trust drops. They take time to trace, so engineers spend the week on re-work. And wrong numbers in reports and models turn into wrong business decisions, some of them with compliance risk attached.

The obvious fix is more tests. It doesn’t happen, for two reasons.

You don’t have time. A data engineer looks after hundreds of pipelines, dozens of sources, and a backlog that never shrinks. Learning every dataset well enough to write good tests for it is a project nobody schedules.

You don’t have the business context. The rule that matters (“this specialty pharmacy reports units, not scripts”) sits with an analyst or a data steward, not with the person who owns the pipeline. Even when tests exist, they need adjusting when new data arrives, and nobody owns that either.

So you need a standard set of tests created automatically from the data, a way to refine them as the data changes, and a way to configure business rules with the people who know them. That’s what TestGen is for.

How TestGen Works: Profile, Generate, Run, Refine

Manual testing is slow and error-prone, and it doesn’t scale past the tables somebody remembered. TestGen runs the whole loop automatically, before, during, and after data production.

Before production	During production	After production
Profile each column (55 characteristics)	Run generated tests	Review results, dispositions, and history
Detect 32 kinds of data hygiene issue	Run configured business rule and custom SQL tests	Refine thresholds and lock the tests you tuned
Generate tests from the profile	Run table monitors on their own schedule	Re-profile and regenerate when the data changes
Optionally configure business rule tests	Trigger runs from your orchestrator through the REST API	Share issue reports and notifications

TestGen organizes the work in projects, connections, table groups, and test suites. A table group is the set of tables you profile and test together, and it can carry its own test suites, monitors, and schedules.

Data Profiling

Data profiling is a periodic x-ray of every column in a table group. A profiling run captures 55 characteristics per column and keeps past profiles, so you can compare one run with the next. The results populate the Data Catalog, drive hygiene detection, become the baseline for test generation, configure Freshness monitors, and feed quality scores.

Profiling queries each column of every table separately. That makes it more expensive than a test run, and it doesn’t belong in the production validation step. The documentation recommends profiling every two weeks to once a month, during off-hours. Since TestGen 5.134.0 you can pause a long run and resume it later from where it stopped.

Sampling is optional and narrower than it used to be. It applies only to pattern and character analysis and to the list of frequent values. Record counts, distinct values, minimums, maximums, averages, and percentiles always cover the whole table, so tests generated from a sampled profile still hold on the full data.

One rule makes profiling far more useful: profile homogeneous data. The documentation’s example is a table that sells both pencils and cars. Its price range runs from two cents to $100,000, and a range that wide catches nothing for either product. Point a table group at a view per category, or at the trailing 12 months, and the generated ranges get tight enough to mean something.

The 55 characteristics, grouped. Names follow the profiling characteristics reference.

Group	Profiling characteristics
Names, types, and position (7)	Column name, Column type, General type, Functional data type, Functional table type, Table name, Position
Averages and spread (4)	Average value, Average length, Average embedded spaces, Standard deviation value
Minimum and maximum (9)	Minimum value, Minimum value Gr0, Maximum value, Minimum date, Maximum date, Minimum length, Maximum length, Minimum text, Maximum text
Percentiles (3)	Percentile 25, Percentile 50, Percentile 75
Counts (10)	Record count, Value count, Numeric count, Boolean True count, Embedded space count, Lead space count, Quoted value count, Includes digit count, Filled value count, Fractional sum
Blanks and zeros (3)	Null value count, Zero length count, Zero value count
Dates (10)	Date count, Date days present, Date weeks present, Date months present, Future date count, Within 1mo date count, Within 1yr date count, Before 1yr date count, Before 5yr date count, Before 20yr date count
Distinct values (4)	Distinct value count, Distinct standard value count, Distinct pattern count, Distinct value hash
Frequencies and patterns (3)	Top frequent values, Top patterns, Standard pattern match
Classification (2)	Data type suggestion, PII flag

Data Hygiene Detectors

Once profiling finishes, 32 data hygiene detectors read the results. They report where a column’s contents don’t match its declared type, its name, or the rest of the schema: a text column that’s mostly dates, “N/A” posing as a blank, a likely Social Security number in a column called notes. You get these findings before you run a single test.

Each finding carries a likelihood. Definite indicates a data problem, Likely typically does, and Possible is a speculative finding that’s often a problem and sometimes benign. Review the Definite and Likely findings first, fix the types or add patching steps, then profile again.

Data type and format (14). Suggested data type · Character column with mostly numeric values · Character column with mostly date values · Character column with numbers and units · Delimited data embedded in column · Leading spaces · Quoted values · Non-printing characters · Non-alpha name or address · Non-alpha prefixed name · Inconsistent casing · Invalid USA zip code format · Invalid USA ZIP-3 format · Personally identifiable information

Missing and incomplete data (5). No column values present · Non-standard blank values · Small percentage of missing values found · Potential duplicate values · Similar values match when standardized

Value and pattern consistency (8). Pattern inconsistency within column · Pattern inconsistency across tables · Multiple data types per column name (major) · Multiple data types per column name (minor) · Unexpected boolean values · Variant codings for same values · Small percentage of divergent values · Unexpected numeric values

Unexpected content and dates (5). Unexpected column contains emails · Unexpected column contains US states · Unlikely dates out of typical range · No table dates within one year · No table dates within six months

Generated Tests

After profiling, TestGen can generate a test suite from the latest profile. It picks the test types whose criteria match each column and sets thresholds and parameters from what it found. A table with no email column never gets an email format test. The 33 test types in the Standard generation set work this way.

The goal is to cast a wide net for data problems that nobody predicted. It’s the same idea as a burglar alarm with a sensor on every entrance, because the burglar only needs one window. Your job in refining the tests is to keep them sensitive to real problems while cutting the false positives that aren’t worth a follow-up.

That takes some effort. It’s far less effort than writing a blanket of tests by hand, which in practice never gets finished. And when you refine a test, its result history and the column’s profile are one click away.

A generated test is only as good as its baseline. Tests generated from a profile pass on the data they came from, by definition, which is exactly why the taxi tests passed on negative fares. So generate from a baseline you trust, or generate from an older profile and evaluate the tests on today’s data.

Regenerate every few weeks, or when your data changes shape. Regeneration refreshes the unlocked generated tests, adding, removing, and retuning them. Lock a test after you tune it and regeneration leaves it alone. Tests you created by hand are never touched. The point isn’t a test suite that changes every week. It’s a stable set of rules that fits the data you have.

The 34 generated test types, by dimension, as the test-type reference groups them:

Dimension	Generated test types (Standard set)
Accuracy (5)	New Shift, Outliers Above, Outliers Below, Variability Increase, Variability Decrease
Completeness (7)	Required Entry, Required No Placeholder, Percent Missing, Daily Records, Weekly Records, Monthly Records, Volume
Consistency (1)	Average Shift
Timeliness (5)	Recency, Date Count, Past Dates, Future Year, Table Freshness
Uniqueness (3)	Unique Values, Percent Unique, Duplicate Rows
Validity (13)	Minimum Date, Minimum Value, Alpha Truncation, Decimal Truncation, Value Count, Constant Match, Value Match, Pattern Match, Email Format, Street Address, US State, Valid US Zip, Valid US Zip-3

A few examples of how the criteria read. Unique Values is generated when every value in a column is distinct, in tables over 500 rows. Recency is generated for a date column whose latest value is within 62 days of the run. Outliers Above is generated for continuous measures with more than 30 distinct values, and fails when more than 5% of values sit two standard deviations above the baseline mean.

Business Rule and Custom SQL Tests

Nineteen test types are configured by hand, because a profile can’t infer intent. You supply the reference table, the tolerance, or the condition, on the Test Definitions page.

Kind	Configured test types
Reconciliation against another table (6)	Reference Match, Aggregate Balance, Aggregate Balance Percent, Aggregate Balance Range, Aggregate Minimum, Distribution Shift
Timeframes (2)	Timeframe Match, Timeframe No Drops
Row counts (2)	Row Count, Row Range
Bounds and lists (4)	Maximum Value, Maximum Date, Text Length Range, Value Match All
Trend and schema (2)	Metric Trend, Schema Conformance
Custom SQL (3)	Custom Condition, SQL Metric, Custom Test

The three custom SQL tests cover the rules no built-in type checks. Custom Condition is a per-row condition on one table, such as “each order ships on or after its order date”, and it’s the fastest of the three. SQL Metric runs a query that returns one number and checks it against a minimum, a maximum, or both. Custom Test runs any query across one or more tables and fails if it returns rows, such as orders whose total doesn’t match the sum of their line items.

Custom tests report in the same results, scores, and notifications as every generated test. Enterprise can also upload a test type that packages a rule for reuse across columns and test suites, and a test type uploaded with selection criteria is generated like a built-in one.

Business rules are where the rules in somebody else’s head go. They’re also the tests you can share with data stewards, because the logic is already written and the steward only fills in the parameters.

Table Comparison Tests for Migrations

TestGen 5.134.0 added a Table Comparison generation set of 32 test types. They answer one question: does this copy match its source? You profile the source, generate the comparison tests from that profile, transfer them to the copy with a mapping file for any renamed tables and columns, and run them there. The migration workflow works across different connections and across database and file sources.

The 32 tests compare row and column counts, column types, null, zero, and distinct counts, minimums, maximums, sums, standard deviations, date ranges, text lengths, leading spaces, letter case, non-printing characters, frequent values and patterns, and hashes of the values themselves, including keyed hashes for tables with a key column. The hash tests need the table group’s Profile for table comparison setting.

Commercial Pharma Tests

Enterprise adds 12 Commercial Pharma tests for the identifiers in claims, prescribing, and clinical trial data: Valid NPI, Valid DEA, Valid NDC, Valid NDC Product, Valid NDC Any Form, Valid ICD-10 Diagnosis, Valid ICD-9 Diagnosis, Valid ICD Diagnosis, Valid ICD-10 Procedure (PCS), Valid HCPCS J-Code, Valid CMS POS, and Valid NCT. They have their own generation set, and on a column where one applies it replaces the general Pattern Match test.

They check form, and that has limits. Eleven of the 12 check form only, plus a check digit where the spec defines one. Only Valid CMS POS checks membership in a published code set. A transposed NPI is still ten digits and passes, which is why a join between two feeds needs its own test.

Reviewing Results and Sharing Issues

Every test run lists its failures and warnings with the measure, the threshold, and the history of that test across runs, so you can tell a new failure from one that has flickered for six weeks. From a result you can open the column’s profile, query the source for sample rows that currently fail, download a PDF issue report with the summary, sample data, and SQL, or export results to Excel.

You record a disposition on each result: Confirm, Dismiss, or Mute. Dismissed results stop counting against the quality score. Flags and timestamped notes stay on the test definition, so the context survives into the next run.

Summary of Profiling, Hygiene Detection, and Tests

Capability	Count
Profiling characteristics per column	55
Data hygiene detectors	32
Test types generated from the profile	34
Business rule and custom SQL test types, configured by hand	19
Table Comparison test types (Table Comparison set)	32
Commercial Pharma test types (Enterprise)	12
Table monitor types (freshness, volume, schema, metric)	4

Counts are from TestGen’s own test definitions at release 5.134.0, October 2026. The documentation describes Volume and Metric Trend with the table monitors; TestGen counts Volume among the generated tests and Metric Trend and Schema Conformance among the configured ones.

Tests by Data Quality Dimension

How do you assess data quality? Most teams use six dimensions: accuracy, completeness, consistency, timeliness, validity, and uniqueness. TestGen files every test type and almost every hygiene detector under one of them, so you can compare your results against these definitions of the data quality dimensions:

1. Accuracy: does the value reflect reality?
2. Completeness: are the rows and values you expect actually there?
3. Consistency: does the data match its baseline, related tables, and earlier periods?
4. Timeliness: does the data arrive on schedule and cover the expected dates?
5. Validity: does each value follow its format, range, list, or business rule?
6. Uniqueness: is each record or value present only where it should be?

Dimension	Generated tests	Configured tests	Table Comparison tests	Hygiene detectors
Accuracy	5	3	0	1
Completeness	7	2	0	3
Consistency	1	6	32	6
Timeliness	5	0	0	2
Uniqueness	3	0	0	2
Validity	13	8	0	17
Total	34	19	32	31

The 12 Commercial Pharma tests are all Validity tests. One hygiene detector, Suggested data type, carries no dimension in the documentation, which is why that column totals 31 of 32. Quality scores can be broken down by the same six dimensions.

The table makes one thing obvious: generated tests lean hard on validity, completeness, and timeliness, because those are what a profile can see. Consistency across tables is mostly configured, or comes from Table Comparison. That’s the gap between syntax and meaning that generated tests can’t close on their own.

Table Monitors

Tests check content against rules. Monitors watch how a table behaves over time and learn what normal looks like, so nobody has to set the threshold by hand. There are four types. Precision about which part is machine learning matters here, because the industry is loose with the word.

Monitor	Watches	How the threshold is set	How it’s created
Freshness	Whether the table updated on its usual schedule: late, earlier than expected, or later than expected	Percentile distribution of gaps between updates, with the schedule inferred from history	Generated from profiling, which picks the columns to fingerprint
Volume	Unexpected jumps or drops in row count	SARIMAX time-series model, after about 30 runs of training	Generated for every table in the table group
Schema	Tables and columns added or dropped, and column changes	No model: every change is an anomaly	Generated for the table group
Metric	A number you define as a SQL aggregate, such as a null count or an average	SARIMAX time-series model, as with Volume	Created by hand, several per table if you like

Monitors run on their own schedule, separate from test suites. The documentation recommends running them at about twice the rate your tables change, so a table updated daily gets checked every 12 hours. A sensitivity setting widens or narrows the band: for Volume and Metric it runs from ±3.0 standard deviations (Low) to ±2.0 (High), and for Freshness from the 99th to the 80th percentile of historical gaps. You can exclude weekends and holiday calendars from training, and override any monitor with static thresholds or a historical calculation.

Monitor results don’t count toward quality scores. They answer an operational question (did the pipeline run?) rather than a content question.

Quality Scores and Scorecards

A quality score is the share of data points that are clean. TestGen computes a profiling score from hygiene findings and a testing score from test failures, and the overall score is the product of the two. Both have to be good for the total to be good. Scores refresh after every profiling or test run, and dismissing a finding removes it from the score.

Scoring is weighted, because not every column deserves equal weight. Weights come from the semantic types profiling infers:

Table type	Weight	Column type	Weight
Entity (master, dimension)	10×	ID, surrogate key, unique ID	3×
Domain (reference, lookup)	5×	Foreign key	2.5×
Bridge (linking tables)	5×	Email, phone, person name, entity name	2×
Summary (aggregations)	1.5×	Address, date, measurement, code	1.5×
Transaction (fact tables)	1×	Boolean, category	1×
		Flag, attribute	0.75×
		Description, constant, sequence	0.5×

PII columns get an extra multiplier on top. Several issues on one column combine probabilistically instead of adding up, so overlapping findings don’t inflate the damage. A project can turn weighting off, and then every record counts equally.

A scorecard is a saved slice of those scores. Build one in the Score Explorer by filtering on any catalog metadata. It can show a total score, a CDE score computed only from Critical Data Elements, and a breakdown by category: quality dimension, impact dimension (conformance, regularity, reliability, usability), table group, business domain, data source, transform level, and more. The CDE score is usually the number an executive wants, because it leaves out the columns nobody reads.

Data Catalog, PII, and Critical Data Elements

The Data Catalog is where profiling output becomes something you can browse. For every table and column it shows the overall, profiling, and testing scores, hygiene findings, test failures from the latest run, profiling statistics with history across runs, and a preview of up to 100 distinct sample rows pulled from the source on demand. It can also write a CREATE TABLE script that uses the suggested data types.

You annotate the catalog with descriptions and nine metadata tags: data source, source system, source process, business domain, stakeholder group, transform level, aggregation level, data product, and data classification. Tags inherit from table group to table to column, and you can edit them in bulk, import them from CSV, or export them to Excel.

Three flags matter more than the rest.

PII is detected during profiling by matching column names and value patterns against known categories, such as Social Security numbers, card numbers, email addresses, and phone numbers, ranked high, moderate, or low risk. A flagged column is masked for any user whose role lacks PII access.

Critical Data Element (CDE) marks the data that matters. Profiling can suggest CDEs automatically, you can set them by hand or by CSV, and a scorecard can score CDE columns alone.

Excluded Data Element (XDE) removes noise columns, such as audit timestamps, from profiling and test generation.

Notifications and Scheduling

Notifications fire after profiling runs, test runs, monitor anomalies, and quality score drops. They go to email, Slack, Microsoft Teams, or any HTTPS endpoint that accepts JSON. That last one is how a failure opens a Jira work item: a Jira automation rule receives the webhook and creates the ticket. Test run rules can fire always, on failures, on failures and warnings, or only on failures the previous run didn’t have.

Schedules run profiling, tests, and monitors on preset intervals or a cron expression, in the timezone you choose. The recommended rhythm is profiling every two weeks to monthly, tests daily to weekly, and monitors at about twice the rate your data changes. When a pipeline should decide when a test runs, call the REST API instead.

Programmatic Access: REST API and MCP Server

Two programmatic interfaces sit beside the web UI. They share one port, separate from the UI’s.

The REST API lets anything that can make an HTTP request drive TestGen. It triggers profiling and test runs and returns their results. It onboards a dataset end to end: table group, test suite, monitoring, profiling, generation, and the first run. It imports and exports test definitions so you can promote them between environments, schedules runs with cron, and reads hygiene findings, PII flags, quality scores, and monitor time series. That’s what makes TestGen usable from an orchestrator: call it at the point in the pipeline where the check belongs, and stop the pipeline on a failure before bad data moves downstream. Airflow, Azure Data Factory, Dagster, a shell step: anything with HTTP. The full endpoint reference is also served by every running instance.

The MCP server gives AI assistants the same reach through the Model Context Protocol. It works with Claude, Claude Code, Cursor, Copilot, OpenAI Codex, and other MCP clients. Ask “which tests failed last night?” or “show me the bad rows” and the assistant picks the tool. As of October 2026 the supported tools page lists 6 prompts and 112 tools in eight groups:

Tool group	Tools
Investigate quality issues	27
Manage TestGen configuration	27
Browse profiling results	17
Browse monitor health and events	16
Trigger profiling, tests, and test generation	11
Discover what TestGen knows about	7
Track data quality scores	6
Data contracts (contracts arrive in November 2026)	1
Total	112

Three details matter in a security review.

1. It runs inside your network and needs no inbound connection from outside it.
2. It acts as the signed-in user. Every call runs with that person’s TestGen role, so an assistant sees only the projects the role allows and can’t do anything the user can’t.
3. Source data protection can keep data values out of everything the MCP server returns, for governance policies that let TestGen read production data but forbid those values from reaching a model. Counts, structure, and configuration stay visible. The Enterprise System Settings page can switch it on without a restart.

Both interfaces authenticate with a bearer token. An OAuth 2.1 client works in both editions: a one-hour access token, refreshed from a refresh token valid for 30 days. Modern MCP clients register and sign in through the browser on their own. A personal access token is Enterprise, carries an expiry an administrator caps, and is how a single sign-on user connects an AI client.

Architecture and Deployment

TestGen is a self-hosted application that runs inside your network. DataKitchen hosts no part of it.

Component	Role
Core engine	Profiling, test generation, test execution, monitoring, and quality scoring
Application database (PostgreSQL)	Configuration, profiling results, test definitions, results, monitor history, schedules, and scores. Never your data
Scheduler	Queues profiling, generation, test, and monitor jobs
Web UI	Every task in the product, in a browser
REST API and MCP server	Programmatic and AI-assistant access, on one port separate from the UI
Email server (optional)	Your SMTP server, for email notifications
OpenID Connect provider (optional, Enterprise)	Single sign-on

Install options. The dk-installer program installs TestGen on Mac, Linux, or Windows in 5 to 10 minutes. The default is a Docker Compose application with two containers: the engine, which also serves the UI, API, MCP server, and scheduler, and PostgreSQL. A pip install with an embedded PostgreSQL suits evaluation. Either edition can use an external PostgreSQL server you back up and upgrade on your own schedule. Enterprise adds Helm charts for Kubernetes, the recommended production deployment. TestGen runs as a single application pod and does not scale horizontally.

There is no command-line interface. The 2023 edition of this paper listed CLI commands; the web UI, the REST API, and the MCP server replaced them.

TestGen also sends test results to DataOps Observability, which places them alongside the rest of your Data Journey, from data source to the report your customer reads.

Connectors: Databases and Files

As of release 5.134.0, TestGen connects to 14 databases. For each one, the access requirements are read-only: schema metadata plus SELECT on the tables.

Database	Database	Database
Amazon Aurora PostgreSQL	Databricks SQL	Microsoft SQL Server
Amazon Redshift	DuckDB	Oracle Database (12c and later)
Azure SQL Database	Google BigQuery	PostgreSQL
Azure Synapse Analytics	Microsoft OneLake (Microsoft Fabric)	Salesforce Data 360
	SAP HANA	Snowflake

Files. Since 5.134.0, TestGen profiles, tests, and monitors CSV, Parquet, and JSON files on a local path, in Amazon S3, or in Azure Blob Storage, with no database in front. A query engine inside TestGen reads them, which means file and DuckDB runs use TestGen’s own memory and disk; the documentation has a sizing guide.

External tables. TestGen also tests Apache Iceberg tables and Parquet, Avro, ORC, CSV, and JSON files that a warehouse exposes as external tables, through Amazon Redshift Spectrum or Snowflake external tables.

Security: What Leaves Your Environment

A security review asks one question first: what leaves our environment? Nothing goes to DataKitchen.

For a database source, every profiling, testing, and monitoring query runs as a SELECT inside your database. The database does the compute and returns aggregates: row counts, column statistics, pass and fail outcomes. TestGen never issues INSERT, UPDATE, DELETE, CREATE, ALTER, or DROP against your data. Failing rows are queried on demand when someone opens a result, and Data Preview pulls at most 100 distinct rows the same way.

DuckDB and file sources are the one place data touches TestGen’s disk. A query engine inside TestGen reads them, CSV and JSON files from S3 or Azure Blob are copied for the run, and a run short of memory spills to disk. Those temporary copies are deleted when the run ends.

The application database holds configuration and results, never your data. The one outbound signal is optional anonymous usage telemetry, which carries no data content and switches off with TG_ANALYTICS=no.

Area	Detail
Hosting	Self-hosted only. No DataKitchen-hosted option and no access to your environment
Network	No inbound connections from outside your network; inside it, one port for the UI and one for the API and MCP server
Containers	Alpine Linux, dedicated non-root user, no privileged mode; SBOM and provenance attestations on builds
Credentials at rest	AES-256-CBC with a PBKDF2-derived key, for database passwords and keys
Passwords and sessions	bcrypt hashing; HMAC-SHA256 signed session tokens; no stored passwords under SSO
Transport	HTTPS for the UI, API, and MCP server in both editions, or TLS terminated at your proxy
Release scanning	SAST and secret detection on every code change; Docker Scout on every release
Penetration testing	Independent, about every two years, against OWASP ASVS
Source code	Core engine open source under Apache 2.0, so your security team can read it

Access Control and Editions

Enterprise governs access with five project roles:

Role	Can do	PII
Admin	Everything in the project	Visible
Data Quality	Manage table groups and test suites, run profiling and tests, disposition results, edit catalog tags	Visible
Analyst	Disposition results and edit catalog tags; read-only elsewhere	Redacted
Business	Read-only access to everything	Redacted
Catalog	Read-only access to the Data Catalog	Redacted

Roles are assigned per project, so one person can hold different access in different projects, and a system administrator manages projects and users across the instance. Single sign-on works with any OpenID Connect provider, such as Okta, Microsoft Entra ID, Auth0, or Ping Identity. Users are created on first login.

The core profiling, testing, and monitoring engine is free and open source. Enterprise adds team access, security, Commercial Pharma tests, source-data controls for AI, and support.

In practice, open source is the full engine with the UI, the REST API, and the MCP server, limited to one user, one connection, and one project, with unlimited tables and data volume. Enterprise removes those limits and adds SSO, the five roles and PII masking, personal access tokens, uploaded test types, custom branding, Kubernetes deployment, and support from DataKitchen. Enterprise costs $100 per user per month plus $100 per database connection per month. See pricing and the edition comparison.

What DataOps TestGen Does Not Do

A technical overview that only lists strengths is a brochure. These are the boundaries.

No issue ticketing. TestGen detects findings and records what you decided about them: dispositions, flags, notes, and a result history that answers whether a failure is new or has been flaky for six weeks. Assignment, escalation, and closure belong in Jira or ServiceNow, and a second queue nobody reads helps no one. A webhook notification can open a Jira work item for you, and findings also move out through the API, the MCP server, email, Slack, Teams, or DataOps Observability.

No column lineage. When data breaks, the useful question is usually which process ran wrong, not which column feeds which column. A column graph tells you what’s downstream; it rarely tells you what happened at 2am. Process lineage lives in DataOps Observability.

No audit log yet. Today you see the current test definition and who may change it, not a per-edit history with author and timestamp. Audit logs arrive in TestGen in November 2026.

No root-cause explanation. TestGen can show you that failures cluster in one table and started on one date. Why they started lives in your pipeline code, your tickets, and your change history, which TestGen never sees. Its job is to be the best possible input to whoever, or whatever, has that context, which is what the MCP server is for.

Example DataOps TestGen Use Cases

Data Production Testing

In daily production you load, transform, predict, and report with a stack of different tools. The step most architectures forget is testing the data during production. Trigger a TestGen test suite from your orchestrator right after the load, and halt the run on a failure.

Black Box ETL Testing

You inherit a legacy ETL process built years ago. The developers are gone and the documentation is thin or missing, so nobody can say what the transformation should produce. You can still test it.

Profile and test both ends, then compare the two sets of results. Generated tests on each end catch the day either side changes shape. Where a step is meant to copy a table unchanged, Table Comparison tests generated from the source check the copy against it, row counts and sums and value hashes included. You don’t need to know what the ETL does to see when its behavior changes.

Understanding Distributed Data Mart Quality

Databases pile up across an organization, each holding a dataset somebody depends on. You get asked to set one standard across all of them and report comparable quality numbers. TestGen applies the same profiling, hygiene detection, generated tests, and scoring to every connection, so the results are comparable from one database to the next. Open source covers one connection; Enterprise covers as many as you connect.

Coordinating Test Definitions with Data Stewards

You know the pipeline. The data steward knows what the numbers mean. Most test programs stall in the gap between the two.

TestGen gives the steward a way in without a database login of their own. Generate a first-pass suite, export the test definitions to Excel for review, and let the people who know the business mark what’s wrong. Changes come back through the UI, or as a JSON file of test definitions you import into the suite. In Enterprise, the Analyst role lets a steward disposition results and edit catalog tags while PII stays redacted.

Test Raw Data Sources on Arrival

Data engineers often take feeds from dozens, sometimes hundreds, of providers. “Hoping for the best” isn’t a manufacturing strategy. Catch errors as early as you can: test each provider’s data when it lands, before integration starts.

Fast feedback holds the provider responsible, and it keeps you from holding the bag for an error you didn’t make. Generated tests and hygiene detection screen for the problems nobody anticipated in a feed. Since 5.134.0 the landing zone can be the files themselves, in S3, Azure Blob, or a local folder, tested with no load step first. The issue report gives you something specific to send back.

Multi-Layered Database Structures

Many teams organize a warehouse or lake in layers: L1 for raw source data, L2 for cleaned and integrated data, L3 for the tables people report from. Medallion architecture calls the same idea bronze, silver, and gold. Each layer can lose or mangle data on the way through, and each layer needs its own tests.

TestGen tests every layer in place. Point a table group at each layer, tag it with its transform level, and score the layers side by side. Aggregate Balance and Reference Match tests check that what left L1 arrived intact in L2, and that L3 totals still tie out to the detail under them.

Testing Inside a Data Journey

Your data passes through many tools: ETL tools, orchestrators, Azure Data Factory, Databricks, notebooks, dashboards, buckets, servers, and custom scripts. We call that multi-tool path a Data Journey. Observing a Data Journey means tracking every level of the stack, data, tools, servers, code, and tests, with start times, durations, test results, and infrastructure events, so you know whether everything ran on time and without errors, and which part didn’t.

DataOps Observability provides the Data Journey, expectations, alerts, and analysis. DataOps TestGen provides in-database test results. DataOps Automation runs Python and containerized tests. Both send results to Observability.

Why DataOps TestGen

The goal is simple: find the errors in your data before your customer does. Generate the broad checks automatically, run them inside the database during production, and give engineers room to grow from generated tests to fill-in-the-blank business rules that need context.

What keeps that sustainable is mostly what TestGen doesn’t ask of you. There’s no test code to write for the first 80% of coverage, and no YAML. Test definitions document themselves, with the threshold, the baseline, and the query on every result. Thresholds refine over time instead of being guessed once. Queries run in your database as SELECTs, so the performance cost lands where you can see it and schedule it. And because the engine is open source, you can read every query it runs.

DataKitchen Tools for Testing Every Step of a Data Journey

Errors don’t only happen in data. They happen anywhere between the source and the report. While TestGen covers the data in your database, every step needs a test:

1. Test raw data.
2. Test integrated data.
3. Test what your tools produce from the data: reports and models.
4. Test the whole orchestrated process, from source to value, across every tool and data store.
5. Test in development as well as production.

Task	DataKitchen tool
Data profiling and hygiene detection	DataOps TestGen: 55 characteristics, 32 detectors
Tests generated from the data profile	DataOps TestGen: 34 test types, plus 32 Table Comparison test types
Configurable business rule and custom SQL tests, including complex SQL	DataOps TestGen: 19 test types, plus 12 Commercial Pharma tests in Enterprise
Table freshness, volume, schema, and metric monitoring	DataOps TestGen: 4 monitor types
Python and containerized tests	DataOps Automation
Data Journey process monitoring, alerting, and analytics	DataOps Observability

Additional Resources

• What is TestGen?: the documentation’s overview, and the source for this edition
• Install DataOps TestGen: free and open source, running on a laptop in minutes
• DataOps TestGen product page
• DataOps Observability: Principles and Ideas
• DataOps Observability Technical Product Overview
• DataOps Observability and Automation software

Related Reading

• DataOps TestGen White Paper: the problem TestGen was built for, told as “the mystery box full of data errors”
• A Guide to DataOps Tests: the four places to test, and where generated tests fit
• Data Quality: The DataOps Way: the method around the tooling
• I Wrote Data Quality Tests for 3 Million NYC Taxi Rides in Five Minutes: generated tests, business rules, and MCP on a public dataset
• New in TestGen: Prove Your Data Migration Worked: Table Comparison tests in practice
• Scaling Data Reliability: The Definitive Guide to Test Coverage: how much testing is enough
• Test Coverage Beats Data Lineage: why a static blueprint is not a smoke detector
• The Seven Deadly Sins of Data Quality: the habits that leave databases untested

---

FAQ