Answers
Answers to typical questions from data teams.
62 answers · Updated
For data teams
Getting started
- Data quality testing or data observability: which does a small data team need first? Start with data observability on every table, then add data quality tests to the tables where a wrong value costs money. Read the answer
- How do you cut analytics cycle time? You cut analytics cycle time by removing the waits, not by coding faster. In DataKitchen and Eckerson's 2019 survey of 300 data professionals, 78% needed days or longer to get a development environment and 76% needed days or longer to deploy a change. Read the answer
- How do you reduce data firefighting without hiring more engineers? Measure data firefighting, then remove its causes. Tag every ticket that came from something breaking, review incidents monthly for the causes that recur, and put tripwires between pipeline steps so bad data stops before anyone reads it. Read the answer
- How should a two-person data team start data quality testing? Start on the couple dozen tables where a wrong number reaches someone who can escalate it. Read the answer
- What should go in the handover when your only data engineer leaves? A handover from your only data engineer should capture everything that lives only in their head, then prove it by running a week without them. Read the answer
- Which data observability software balances depth with low overhead for a startup? The lowest-overhead way for a startup to monitor its data is a free tool, such as open-source DataOps TestGen, that puts freshness, volume, and schema monitors on every table and generates baseline tests from a profile. Read the answer
Testing
- How do you check vendor CSV, Excel, and Parquet files before they break your reports? Check vendor files where they land, before the load. Profile each delivery, compare its columns and row count with the last one, and test required fields, codes, and ranges. Read the answer
- How do you test data before a BI or analytics release? Test an analytics release by running the changed pipeline in development or QA on data pulled from production, then comparing its outputs with the current version's. Read the answer
- How do you test data pipelines when you don't have a data engineer? Use a tool that generates tests, so you don't have to write test code. Read the answer
- How do you validate a database migration with a small team? Profile the source before the move and keep it as your baseline. Write down every renamed table and column, run tests built from the source profile against the target, and list each change you meant to make. Read the answer
- What is shift-left data quality testing? Shift-left data quality testing moves checks earlier on the data production line, toward the source, so you catch a bad record at entry, not in a dashboard. Read the answer
- When a data quality check fails, should you stop the pipeline, quarantine the data, retry, or publish? DataKitchen's rule: stop the pipeline when wrong data would be used, can't be pulled back, and users can live with yesterday's numbers. Read the answer
- When are dbt tests enough, and when does a small team need more? dbt tests are enough when your problems live inside dbt models you own and you can hand-write a test for every column that matters. Read the answer
- Which data quality checks should a small data team set up first? Set up table checks first: freshness, volume, and schema on every critical table. Then add column checks on keys and important fields: not null, unique, valid values, sane ranges and dates. Read the answer
- Why do two dashboards show different numbers for the same metric? Two dashboards can both be working and still disagree, because each computes the metric its own way. Read the answer
Observability
- How do you evaluate data observability for a multi-tenant SaaS product? Evaluate data observability for a multi-tenant SaaS product on four things: the pricing unit, how a new tenant gets onboarded, whether one test suite can be reused across identical tenant schemas, and where tenant data goes. Read the answer
- How do you reduce false-positive data alerts without missing real problems? Reduce false-positive data alerts by fixing why they fire, not by loosening thresholds. Use baselines that learn weekends and month-end from 90 days of history, as DataKitchen's Guide to DataOps Tests recommends. Read the answer
- Which data observability software best protects customer-facing dashboards from bad data? You protect a customer-facing dashboard by stopping bad data before it reaches the tables the dashboard reads, not by alerting after it lands. Read the answer
- Which data observability software works best alongside Datadog and your existing monitoring stack? Yes. Data observability runs alongside Datadog because the two watch different things: Datadog watches servers and logs, and data observability watches whether tables arrived on time, complete, and correct. Read the answer
- Why does data observability miss freshness failures and schema drift? Data observability misses freshness failures and schema drift for reasons around the monitor, not the math. Read the answer
- Why is end-to-end data observability hard across tools and pipelines? End-to-end data observability is hard because every tool in your stack reports only on itself. Read the answer
AI
- Can AI write your data quality tests? Yes. An AI model reading a real data profile can write useful business-rule tests in minutes. Read the answer
- How do you put analyst-written SQL and Python into production without rebuilding it? Keep the code and add what production code has around it: tests on inputs and outputs, version control, monitoring, a schedule on your shared data platform, and a review step for changes. Read the answer
Tools
- How do you evaluate a data quality tool on your own data in two weeks? Evaluate a data quality tool by planting failures in your own tables and counting what it catches. Read the answer
- What are the best data quality testing tools in 2026? The best data quality testing tool writes most of your tests, catches breakage before a user does, and runs inside your existing workflow. Read the answer
- What are the best DataOps automation tools in 2026? The best DataOps automation tool covers the five capabilities in Gartner's 2024 Market Guide to DataOps: pipeline orchestration, observability, test automation, deployment automation, and environment management. Read the answer
- What are the best free and open-source data quality and observability tools in 2026? The main free options in 2026 are DataOps TestGen and DataOps Observability (both from DataKitchen), Soda Core, Elementary, GX Core, dbt tests, and DQOps. Read the answer
- What are the best open-source data profiling tools? The main open-source data profiling tools are YData-Profiling for rich single-table reports, Deequ for Spark scale, DataCleaner and OpenRefine for interactive cleanup, Great Expectations for bootstrapping expectations, and DataKitchen's DataOps TestGen, which profiles 55 column characteristics. Read the answer
About TestGen
- Can a one-to-three person data team run TestGen without a QA engineer? Yes. DataOps TestGen installs with Docker, and a DataKitchen post from May 2026 puts it at about 15 minutes from docker compose up to a first quality score. Read the answer
- Does TestGen work with AI coding assistants like Claude, Cursor, and Copilot? Yes. DataOps TestGen ships an open-source Model Context Protocol server with 96 tools, as of its June 30, 2026 release, so any MCP client can profile data, generate and run tests, read scorecards, and triage failures. Read the answer
- How does TestGen verify that a cloud data migration worked? DataOps TestGen profiles the source database, generates Table Comparison tests from that profile, and runs them on the cloud target, comparing counts, lengths, sums, patterns, and value hashes column by column. Read the answer
- How is TestGen different from dbt tests or Great Expectations? dbt tests and Great Expectations' GX Core have you write every check yourself, in YAML, SQL, or Python. Read the answer
- Is DataOps TestGen free? Yes. Open-source DataOps TestGen is free under the Apache 2.0 license for one user, one database connection, and one project, with unlimited tables, as of TestGen 5.134.0 in October 2026. Read the answer
- TestGen or Great Expectations: which should I use? Use DataOps TestGen if you want tests generated from a profile of your data, with a UI and monitors, at $0 for one user. Read the answer
- What's the difference between TestGen open source and Enterprise? TestGen open source and Enterprise run the same engine, so profiling, generated tests, hygiene checks, monitors, and the MCP server are in both, as of TestGen 5.134.0 in October 2026. Read the answer
- Which databases does DataOps TestGen work with? As of TestGen 5.134.0 in October 2026, DataOps TestGen works with 14 databases, including Snowflake, Databricks SQL, BigQuery, Redshift, SQL Server, Oracle, SAP HANA, PostgreSQL, Microsoft OneLake, Salesforce Data 360, and DuckDB. Read the answer
Costs
- How do you show a CFO what a three-person data team is worth? Show the CFO four numbers, not a philosophy: errors that reached a customer, cycle time from request to usable result, work delivered, and what the people using your data say. Read the answer
- How much does data quality and data observability software cost a small team in 2026? A small team can pay $0 for open-source data quality and observability software. DataKitchen TestGen Enterprise lists at $100 per user and $100 per database connection per month, so four users on two databases pay $7,200 a year. Read the answer
- How much engineering time does a data quality tool take to run each week? DataKitchen hasn't published a weekly-hours figure for running a data quality tool, so measure your own: alert triage, threshold tuning, incident review, upgrades, credential rotation, and compute. Read the answer
For commercial pharma
These answers also live on DataKitchen for Pharma, alongside the launch calendar, the data-source reference, and the pharma FAQ.
Pharma data quality
- How do you count unique patients across specialty pharmacies and hub feeds without double-counting? You usually can't get one exact unique-patient count across specialty pharmacies, because each pharmacy de-identifies patients its own way. Read the answer
- How do you find duplicate and mismatched HCP records across Veeva, IQVIA, and the warehouse? Test the identifiers before you match on them, then test the match. Validate every NPI and DEA by its check digit, check that each feed's prescriber keys resolve to exactly one master record, and surface unmatched records for stewardship instead of dropping them. Read the answer
- How do you validate Open Payments (Sunshine Act) data before you submit it to CMS? Test the file against the reporting rule, not against its own history. Before the March 31 deadline, check every physician record for NPI, license state, and specialty, every covered product for an NDC or device identifier, every payment date, and the de minimis logic against that year's CMS thresholds. Read the answer
- Is an NBRx decline real, or caused by late, incomplete, or restated data? Treat an NBRx decline as a data problem until five checks pass, in order: the file arrived once and on time, it is complete against its control count or history, overlapping periods match your last kept delivery, the payer and specialty mix did not jump, and the report used the current alignment version. Read the answer
- What should we test on every IQVIA, Symphony, specialty pharmacy, and 867 feed before it loads? Test every delivery for five things before it reaches a report: that it arrived on time and only once, that it is complete against the vendor's control count or its own history, that no column was renamed, that units did not change, and that past periods did not move. Read the answer
- What should we validate in Veeva CRM data before it feeds field reports? Validate the relationships, not the formats. Veeva CRM activity arrives daily and well structured, so its failures sit at the joins: calls logged against accounts missing from the account master, prescriber keys that resolve to zero or two master records, and activity and performance computed on different alignment versions. Read the answer
- Why did last quarter's pharma sales numbers change, and was it performance or the data? Last quarter's pharma sales numbers usually change because the vendor restated them. Syndicated sales data is part actual, part projection, and vendors rerun recent months when better inputs arrive. Read the answer
- Why do generic data quality tools miss bad NPI, NDC, DEA, and HCPCS codes? Because generic tools check shape, not the rules each code is built on. A transposed NPI is still ten digits, so a pattern check passes it; only the Luhn check digit catches it. Read the answer
- Why don't our specialty pharmacy, 867, and sales numbers match? They rarely match because each feed counts a different thing: different date definitions, different weeks, units versus packages, and NDCs at different levels. Read the answer
Pharma platform
- Should a small pharma build its commercial data warehouse on Snowflake or Databricks? Either works, so pick the one your team already runs. DataKitchen builds commercial pharma platforms on both and has published no evidence that one is cheaper or more accurate for this work. Read the answer
- What is AI data modernization for commercial pharma, and is our data ready for it? AI data modernization is building the system an AI-first data team works in: agents on top of your existing warehouse, with a real repo, real tests, and a deploy path. Read the answer
- What should a regulated pharma team look for in data observability and DataOps tools? Look for tests that run inside your warehouse so rows never leave, a recorded history of every test result and disposition, lineage from vendor feed to report, version-controlled promotion from development to production, and a deployment you control. Read the answer
- Which data vendors and platforms does a small pharma commercial team actually use? A small pharma commercial stack draws from six layers: syndicated prescription data (IQVIA, ICON/Symphony), specialty pharmacy and distributor feeds, Veeva CRM, a prescriber master such as Veeva Network, a Snowflake or Databricks warehouse, and Tableau or Power BI. Read the answer
Pharma operations
- Does 21 CFR Part 11 or GxP apply to our commercial pharma data? Mostly no. 21 CFR Part 11 covers electronic records that an FDA regulation requires you to keep, plus records you submit to FDA. Read the answer
- How do you add a second brand or indication without rebuilding the commercial data platform? You add a second brand without a rebuild when the first sits on conformed physician, payer, product, time, and territory dimensions, so the new brand adds rows and fact tables, not a schema. Read the answer
- How do you replace a commercial pharma data vendor without breaking reporting? Take the work over feed by feed, not on a cutover date. Inventory the feeds and the reports people read, confirm you own the code, credentials, and cloud account, put tests on each feed before you change it, then run old and new side by side until the numbers reconcile. Read the answer
- How do you test territory alignment and incentive compensation data before payout? Test alignment as a relationship, not a column. Before the comp run, confirm every prescriber resolves to one master record, every zip and territory exists in the current hierarchy, territory totals still tie to the national total, and activity and sales use the same alignment version. Read the answer
- Our launch is three months away and the data platform is behind. What do we prioritize? DataKitchen recommends building only the weekly reports the field and leadership read in launch week: launch and forecast tracking, field activity and performance, incentive comp, and specialty pharmacy dispense for a specialty brand. Read the answer
- Should an emerging pharma build, hire, bring in a dedicated team, or use open source for commercial data? Pick the model that leaves you owning the code and feed knowledge when people leave. Read the answer
- What commercial data do we need in place 12 months before our first launch? Twelve months out, you need your core feeds landing in your own cloud, your prescriber master and alignment decided, and a first tested star schema in front of analysts. Read the answer
- What should a commercial data vendor file specification and acceptance agreement include? A commercial data vendor file specification should state the grain, the delivery cadence and window, a control record with row counts and amount totals, event and calendar definitions, units and identifier formats, a restatement and correction policy, notice before any layout change, the tests a file must pass to be accepted, and how fast a rejected file gets fixed. Read the answer
- What should a small or midsize pharma expect to spend on commercial data and analytics? Expect most of the bill to be people, not cloud or software. Six fully loaded commercial data FTEs in Boston or the Bay Area run $1.5 million to $2 million a year. Read the answer
- What should our commercial data partner deliver every month? Every month your commercial data partner should show feeds delivered on schedule, defects opened, closed, and still open, change requests and how fast they shipped, cost against the agreed rate, documentation brought current, and proof that the code and tests still live in your repository. Read the answer
No answer matches that search. Try a shorter word, or ask us directly.
Don't see your question?
Ask the people who build DataOps TestGen and run data platforms for a living.