What is the best data quality and data observability tool on AWS?

The best data quality and data observability tool on AWS profiles every table, generates data tests from the profile, monitors freshness, volume, and schema, and publishes results on a data quality dashboard, all from inside your own VPC so no rows leave. It should cover Redshift, Aurora PostgreSQL, and S3 files with one set of tests and one dashboard.

By Chris Bergh Published October 21, 2026

Key points

AWS Glue Data Quality is built on the open-source Deequ library, recommends rules for tables in the Glue Data Catalog, adds ML anomaly detection, and bills $0.44 per DPU-hour with a 2-DPU minimum, per AWS's pricing page in October 2026.
As of TestGen 5.134.0 in October 2026, DataOps TestGen connects to Amazon Redshift, Redshift Spectrum external tables, Amazon Aurora PostgreSQL, and CSV, Parquet, and JSON files in Amazon S3.
TestGen runs profiling and tests as SQL inside Redshift and Aurora, and reads S3 files with an embedded DuckDB engine on your own TestGen host, which needs only s3:ListBucket and s3:GetObject.
Open-source TestGen covers one user, one database connection, and one project with unlimited tables; Enterprise is $100 per user per month plus $100 per database connection per month.
Monte Carlo lists Amazon Redshift, Amazon S3, AWS Glue, and Amazon Athena integrations; DataKitchen's comparison page estimates it at $120,000 to $250,000 a year for 10 users and 1,000 tables (estimate reviewed June 2026).

This page is the AWS overview. The Redshift answer and its TestGen walkthrough go deeper on the warehouse, and Aurora PostgreSQL and S3 files have their own pages.

What AWS gives you natively

AWS Glue Data Quality is built on the open-source Deequ library. It’s serverless. You write rules in DQDL, or ask it to recommend rules from a table’s statistics, and it scores each run and flags anomalies with ML. That’s a real head start.

The limit is scope. Rules hang off the Glue Data Catalog or a Glue ETL job, and rule recommendations work only for catalog tables, not ETL jobs. Pricing is $0.44 per DPU-hour with a 2-DPU, one-minute minimum (AWS pricing, checked October 2026). Deequ itself runs on Spark, so you bring a cluster.

Tools worth considering on AWS

Tool	Profiling	Generated data tests	Observability monitors	Data quality dashboards	Runs in-platform	Open source	Price basis
DataOps TestGen	55 column characteristics	34 test types	Freshness, volume, schema, custom metrics	Scorecards per table group	SQL inside Redshift and Aurora; S3 files on your TestGen host	Apache 2.0	Free for one connection; $100 per user and $100 per connection per month
AWS Glue Data Quality	Statistics and analyzers	Rule recommendations, catalog tables only	ML anomaly detection	Data quality score	Serverless, in your AWS account	No (built on Deequ)	$0.44 per DPU-hour
Deequ	Yes, on Spark	Constraint suggestions	Anomaly detection on metrics	None	Your Spark cluster	Apache 2.0	$0 software
Monte Carlo	Yes	Partial	ML-based, Redshift, S3, Glue, Athena	Yes	SaaS	No	Credits, paid per monitor (pricing)
Soda	Yes	Partial	Metric monitoring and anomaly detection	Yes	Self-hosted runner option	Soda Core, Elastic License 2.0	Free plan; Team $750 per month

Sources, checked October 2026: Monte Carlo integrations, Soda Redshift connection and Soda pricing. “Partial” ratings come from DataKitchen’s Monte Carlo and Soda comparisons, reviewed June 2026.

Who should pick what

Pick Glue Data Quality if your pipelines already run as Glue jobs and you want no server at all. Pick Monte Carlo if you want a managed service with ML anomaly detection and lineage from Redshift query logs, and the budget fits: DataKitchen’s comparison estimates $120,000 to $250,000 a year for 10 users and 1,000 tables. Both do things TestGen doesn’t.

Pick TestGen if you want the tests written for you. It profiles Redshift, Redshift Spectrum external tables, Aurora PostgreSQL, and S3 files, then generates tests from the profile. The SQL runs in your database. The core profiling, testing, and monitoring engine is free and open source. Enterprise adds team access, security, Commercial Pharma tests, source-data controls for AI, and support.

Deequ and Glue both profile well. The gap is the step after: a profile tells you what’s in a column, and somebody still has to turn it into tests. Usually nobody does.

Documentation: Redshift quickstart, Connect your Data, Access Requirements, Manage Connections, Monitor Tables.

Frequently asked questions

Short answers to the follow-up questions people ask next.

Can I use TestGen alongside AWS Glue Data Quality?

Yes. Glue Data Quality evaluates DQDL rules on Glue Data Catalog tables and inside Glue ETL jobs, so it fits checks written into a Glue pipeline. TestGen profiles Redshift, Aurora PostgreSQL, and S3 files and generates tests from 34 test types. Keep Glue rules where a Glue job already runs and let TestGen cover the tables no one wrote rules for.

What does a data quality tool on AWS cost for a small team?

Open-source TestGen is free for one user, one database connection, and one project, with unlimited tables. Covering Redshift and an S3 bucket takes two connections, which is TestGen Enterprise at $100 per user per month plus $100 per connection per month. AWS Glue Data Quality bills $0.44 per DPU-hour with a 2-DPU minimum, per AWS's October 2026 pricing page.

Does any data leave my environment?

No. TestGen runs profiling and tests as SQL inside Redshift or Aurora PostgreSQL, and reads S3 files with an embedded DuckDB engine on your own TestGen host. TestGen's database stores results and metadata: aggregates, statistics, test outcomes, and the most frequent values and minimum and maximum text values that profiling keeps for display. A file run can keep a temporary copy of the data on the TestGen host's disk until the run ends.

Can TestGen run inside my AWS VPC?

Yes. TestGen installs on a machine you control, so you run it on an EC2 instance inside your VPC. DataKitchen's docs size an Enterprise install at 4 CPUs, 16 GB of memory, and 100 GB of disk, such as a t3.xlarge. For S3 access through an EC2 instance role on Docker, set the instance metadata hop limit to 2.

Does TestGen support Amazon Athena?

No. As of TestGen 5.134.0 in October 2026, Athena is not one of the 14 supported databases. For data in S3, TestGen reads CSV, Parquet, and JSON files directly through an Amazon S3 connection, and it tests Parquet and other formats that Redshift Spectrum exposes as external tables.

Try it on your own data

DataOps TestGen is free and open source. Profile a table and generate its tests this afternoon.

Install Open Source TestGen Request a Demo