Most of the data behind your launch came from someone else. IQVIA, ICON/Symphony, a dozen specialty pharmacies, your 3PL, your patient hub. You did not create it, you cannot control its schedule, and when it is wrong it rarely errors.
The post on the commercial data most likely to be wrong catalogues the ways it fails. The useful observation is that five of them repeat across every feed. So you run the same five checks on every delivery, then add the check each feed type needs.
Five checks on every delivery
| Failure | What you see | Test |
|---|---|---|
| Late, missing, or loaded twice | Nothing. The pipeline is green. | Freshness monitor; max load date is today; one delivery per expected period |
| Partial file | 26,000 rows where 40,000 belong, no error | Row count against the vendor control record, or volume bounds learned from the feed’s history |
| Renamed column | RX_CNT becomes RxCount; the field fills with nulls | Schema drift test |
| Unit or scale change | Units become packages, dollars become cents; one product jumps 10x | Metric trend against that product’s own history |
| History moved | Last quarter’s actuals change inside a clean file | Diff overlapping periods against the snapshot you kept |
The first four are extract-to-load checks. They run in seconds and should be exact where the vendor gives you a control count. Sum the amount column on both sides at fixed precision, so two engines don’t disagree over floating point.
The fifth is the one most teams skip. Third-party sales data is part actual, part projection. The vendor reruns the projection when better inputs arrive, and your actuals change underneath you. The restatement answer covers how to split that test between the restated months and the stable tail.
Use learned bounds, not fixed thresholds. A hardcoded “expect at least 50,000 rows” goes stale as the data grows, then fires false alarms or gets commented out. One rare-disease biotech’s warehouse shipped with 150 checks and had about 20 still enabled, because the rest fired on the same upstream problems every morning.
What each feed type adds
The data sources reference lists 37 commercial feeds with the failure characteristic of each. These are the ones you asked about.
| Feed | How it arrives | Its own failure | Add this check |
|---|---|---|---|
| IQVIA, ICON/Symphony prescription and prescriber data | Weekly and monthly, often wide, periods across columns | Restatement on refresh; payer or specialty mix shifts from a mapping change | Normalize to long format first, then Distribution Shift on the mix |
| Medical and pharmacy claims, such as Komodo Health | Weekly or monthly, large, lagged by adjudication | Malformed NPI, NDC, or ICD values that fail the join silently | Identifier validation before the join (why pattern checks miss them) |
| Specialty pharmacy dispense and status | One file per pharmacy, each with its own format and calendar | Ship, fill, and dispense dates mean different things; one patient under three tokens | Map to one event date and one calendar before comparing (867 and specialty answer) |
| 3PL and distribution, EDI 867 and 852 | Weekly, EDI, positional formats | Units switch to packages, price arrives in cents | Metric Trend per product against its own history |
| Patient services and hub | Daily or weekly from the hub vendor | Milestone dates out of sequence, such as ship before enrollment | Custom Condition on milestone order |
| Copay and adjudication | Weekly from the copay vendor | Redemptions that never reconcile to a dispense | Reference Match from redemption to dispense |
Two of the hardest failures are not in any single feed. A prescriber who appears in two sources under two records, and a territory realignment that moves a rep’s numbers, both pass every check above. Those are relationship tests, covered in finding duplicate HCP records and testing alignment before the comp run.
Where the checks run
Run arrival checks on the raw stage, before anything reaches a mart. A two-stage FITT architecture keeps raw data exactly as it arrived, so a bad delivery fails somewhere the field never sees, and the snapshot you need for the history diff is already there.
In DataOps TestGen, freshness, volume, and schema monitors run on every table in the table group, with volume bounds learned from each table’s history, so you do not hand-write them per feed. Reference Match, Aggregate Balance, Distribution Shift, and Custom Condition cover the relational and feed-specific checks, as the testing post lays out. If your vendors drop files to S3 or a folder rather than a database, checking vendor files before they load shows the setup.
The test that matters most is the one you can answer in a minute: did every row from last night’s delivery land? If you can’t say, find out before the field does.