What is the best data quality and data observability tool on Google Cloud?

The best data quality and data observability tool on Google Cloud profiles every BigQuery table, generates data tests from the profile, monitors freshness, volume, and schema, and publishes results on a data quality dashboard, running its SQL as query jobs in your own project so no rows leave. Check Cloud Storage file support separately: not every tool reads GCS directly.

Key points

  • Google renamed Dataplex Universal Catalog to Knowledge Catalog on April 10, 2026; its auto data quality runs rules on BigQuery and Iceberg REST Catalog tables, with up to 1,000 rules per scan.
  • Knowledge Catalog data profiling can recommend data quality rules, but anomaly scanning is a separate data profile scan, per Google's auto data quality overview in October 2026.
  • DataOps TestGen connects to Google BigQuery with a service account that holds roles/bigquery.dataViewer on the dataset and roles/bigquery.jobUser on the project.
  • As of TestGen 5.134.0 in October 2026, TestGen reads CSV, Parquet, and JSON files from a local path, Amazon S3, or Azure Blob Storage, not from Google Cloud Storage.
  • Open-source TestGen covers one user, one database connection, and one project with unlimited tables; Enterprise is $100 per user per month plus $100 per database connection per month.

This page is the Google Cloud overview. The BigQuery answer and its TestGen walkthrough cover service account setup and query cost in detail.

What Google Cloud gives you natively

Dataplex Universal Catalog became Knowledge Catalog on April 10, 2026. Its auto data quality runs rules on BigQuery and Iceberg REST Catalog tables. Each rule maps to one of seven dimensions, from freshness to uniqueness, and data profiling can recommend rules for you. It’s serverless and already in your console.

The limits, per Google’s overview checked in October 2026: up to 1,000 rules per scan, and column-level scores only through the API. Anomaly scanning isn’t part of auto data quality at all. Google points you to the separate data profile scan for that.

Tools worth considering on Google Cloud

ToolProfilingGenerated data testsObservability monitorsData quality dashboardsRuns in-platformOpen sourcePrice basis
DataOps TestGen55 column characteristics34 test typesFreshness, volume, schema, custom metricsScorecards per table groupSQL jobs inside BigQueryApache 2.0Free for one connection; $100 per user and $100 per connection per month
Knowledge Catalog (Dataplex)Data profile scansRule recommendations from profilingAnomalies via data profile scanData quality scoresServerless in Google CloudNoData compute units
Monte CarloYesPartialML-based; BigQuery freshness, volume, schemaYesSaaSNoCredits; pricing on request
SodaYesPartialMetric monitoring and anomaly detectionYesSelf-hosted runner optionSoda Core, Elastic License 2.0Free plan; Team $750 per month

Sources, checked October 2026: Monte Carlo integrations, which list BigQuery with field-level lineage; Soda data sources, which list GCP BigQuery and Google CloudSQL; and Soda pricing. “Partial” ratings come from DataKitchen’s Monte Carlo and Soda comparisons, reviewed June 2026.

Cloud Storage files

TestGen doesn’t read GCS. Its file connections, added in TestGen 5.134.0, cover a local path, Amazon S3, and Azure Blob Storage. If your vendor drops land in a GCS bucket, test them after they load into BigQuery, or copy them to a location TestGen reads. Better to learn that here than on day two.

Who should pick what

Pick Knowledge Catalog if your governance team already lives in it and wants rules next to the catalog. Pick Monte Carlo if you want a managed service with ML anomaly detection and BigQuery lineage, and the budget fits: DataKitchen’s comparison estimates $120,000 to $250,000 a year for 10 users and 1,000 tables.

Pick TestGen if you want the tests written for you. It added BigQuery in October 2025, profiles every column, and generates tests from the profile, all as query jobs in your project. The core profiling, testing, and monitoring engine is free and open source. Enterprise adds team access, security, Commercial Pharma tests, source-data controls for AI, and support.

See also: which databases TestGen supports and install TestGen.

Documentation: Connect your Data, Access Requirements, Manage Connections, Monitor Tables. DataKitchen’s docs have no BigQuery quickstart as of October 2026.

Frequently asked questions

Short answers to the follow-up questions people ask next.

Can I use TestGen alongside Knowledge Catalog (Dataplex) data quality checks?

Yes. Knowledge Catalog, formerly Dataplex Universal Catalog, runs rules you define or accept from its profiling recommendations on BigQuery and Iceberg REST Catalog tables. TestGen profiles the same BigQuery datasets on 55 column characteristics and generates tests from 34 test types. Keep the Knowledge Catalog rules your governance team owns and let TestGen cover the tables no one wrote rules for.

What does a data quality tool on Google Cloud cost for a small team?

Open-source TestGen is free for one user, one BigQuery connection, and one project, with unlimited tables. TestGen Enterprise is $100 per user per month plus $100 per connection per month. TestGen's queries run as BigQuery jobs in your project, so they show up on your BigQuery bill. Knowledge Catalog data quality scans bill in data compute units, per Google's October 2026 docs.

Does any data leave my environment?

No. TestGen runs profiling and tests as SQL query jobs inside your BigQuery project, under a service account you control. TestGen's database stores results and metadata: aggregates, statistics, test outcomes, and the most frequent values found in profiling. TestGen itself runs on a machine you choose, such as a Compute Engine VM in your own Google Cloud project.

Can TestGen test files in Google Cloud Storage?

No, not directly. As of TestGen 5.134.0, released October 14, 2026, file connections read CSV, Parquet, and JSON from a local path, Amazon S3, or Azure Blob Storage. Google Cloud Storage is not on that list. Load the files into BigQuery and test the table, or copy them to a location TestGen reads.

Can TestGen run on Google Cloud?

Yes. TestGen installs on a Linux machine you control, so a Compute Engine VM works. DataKitchen's docs size an Enterprise install at 4 CPUs, 16 GB of memory, and 100 GB of disk, and name c4-standard-4 on Google Compute Engine as an example. TestGen's own metadata database can run on a managed PostgreSQL service such as Google Cloud SQL.

Try it on your own data

DataOps TestGen is free and open source. Profile a table and generate its tests this afternoon.