How do you find duplicate and mismatched HCP records across Veeva, IQVIA, and the warehouse?

Test the identifiers before you match on them, then test the match. Validate every NPI and DEA by its check digit, check that each feed's prescriber keys resolve to exactly one master record, and surface unmatched records for stewardship instead of dropping them. A pattern check alone passes transposed NPIs that silently fail the join.

Key points

  • A transposed NPI is ten digits, all numeric, and passes a pattern check every time. Only the Luhn check digit, computed with the 80840 prefix, catches it, and even Luhn misses a swapped adjacent 0 and 9.
  • When two prescriber sources disagree, the targeting list gets a ghost and alignment double-counts, and nobody notices until the incentive comp run.
  • Syndicated provider IDs are stable within a vendor and meaningless across vendors, so the master has to link on NPI, ME, DEA, and vendor IDs together.
  • In the 2024 CMS Open Payments file, 13,890 practitioner records lacked an NPI, license state, or specialty, and 7,729 of them were nurse practitioners and physician assistants.
  • Non-matched records quietly excluded from the analytic master are how a launch loses prescribers that no report ever mentions.

Picture the ticket. A rep writes in on Monday because their territory is short about 40 TRx. You trace it to one prescriber whose NPI in the specialty pharmacy feed doesn’t match the NPI in Veeva. Ten digits, all numeric. Two of them are transposed, and the join has been dropping that prescriber since March. The pattern test on that column was green the entire time.

That is the mismatch half of the problem. The duplicate half runs the other way. IQVIA says Dr. Smith wrote 30 scripts. Your MDM says there are two Dr. Smiths. Now the targeting list has a ghost and your alignment double-counts, and nobody notices until the incentive comp run.

Both come from the same fact: no single source holds the whole prescriber. The record is assembled from syndicated data, specialty pharmacy, claims, and CRM, and each source is keyed its own way.

Test the identifier before you trust the match

A matching rule is only as good as the keys it matches on. So the first tests run on the landing table, before anything joins.

An NPI’s tenth digit is a Luhn check digit computed over the first nine, with the 80840 prefix the spec requires folded into the sum. Here is the arithmetic for CMS’s published example NPI, 1234567893, and an illustrative transposition of it with the fourth and fifth digits swapped:

Step12345678931235467893 (illustrative: 4 and 5 transposed)
Double every other digit of the first nine, starting with the first (subtract 9 if over 9)2, 2, 6, 4, 1, 6, 5, 8, 92, 2, 6, 5, 8, 6, 5, 8, 9
Sum4351
Add 24 for the 80840 prefix6775
Check digit needed to reach the next multiple of 1035
Check digit in the value33
ResultPassFail

A regex sees ten digits in both columns and passes both.

Luhn has one blind spot: swap an adjacent 0 and 9 and the sum doesn’t change, so that transposition passes the check digit too.

DEA numbers carry their own checksum: the seventh digit has to equal the first, third, and fifth digits added together, plus twice the sum of the second, fourth, and sixth, mod ten. As of 27 August 2026, DataOps TestGen Enterprise generates Valid NPI and Valid DEA from your profiling run, along with ten other Commercial Pharma tests. Generation needs the column name and the profiled values to agree, so a column named code_3 gets nothing. Add that one by hand.

Then test the relationships

Every individual value can be valid while the relationship between them is broken. That is the failure column-level checks never see. Four tests cover most of it:

What you checkWhat it catchesTestGen test
Every prescriber key in each feed exists in the masterA prescriber in one source and absent from the master, so their activity never reaches a reportReference Match
Each master key appears onceTwo records for one prescriber after a multi-source mergeDuplicate Rows, Unique Values
Every target and call plan entry resolves to a current master recordTargets pointing at prescribers who no longer existReference Match
CRM activity lands on an account in the account masterCalls that disappear from the roll-upReference Match

Watch the vendor IDs too. A syndicated provider ID is stable within a vendor and meaningless across vendors, so it can’t be the master key. ME numbers show up in some sources and not others. The analytic master we build links prescribers on NPI, ME, DEA, and vendor provider IDs together, and keeps prescribers the MDM never mastered instead of excluding them.

Expect the gaps you didn’t design for

Master data built around physicians breaks on everyone else. When we tested the 2024 CMS Open Payments file, 13,890 physician and practitioner records were missing an NPI, a license state, or a specialty. 7,729 of them were nurse practitioners and physician assistants, who only became covered recipients in 2021. Manufacturers assemble that file from expense systems, CRM, and vendor feeds, which is the same plumbing your HCP master runs on.

The Massachusetts slice of the same file spelled Boston three ways, and its 740 distinct recipient cities collapsed to 445 after stripping punctuation and spaces. Don’t match prescribers on name and address. Match on identifiers you have tested.

What the tests won’t tell you

None of this replaces Veeva Network or whatever you use for HCP mastering. A Luhn-valid NPI could have been issued; it isn’t proof that it was, and the tests never call NPPES. MDM tells you what it could match. Identifier and relationship tests tell you what arrived broken and what fell out of the match, on every load, before the comp run finds it for you.

Frequently asked questions

Short answers to the follow-up questions people ask next.

Why does a valid-looking NPI still break the join to Veeva?

Because a pattern check counts to ten and stops. An NPI's tenth digit is a Luhn check digit computed over the first nine plus the 80840 prefix, and a transposition of two adjacent digits breaks it. A regex passes the transposed value, the join drops the prescriber, and every downstream number comes in low with nothing erroring.

Does a passing NPI check prove the prescriber is real?

No. A Luhn-valid NPI is a number that could have been issued, not proof that one was. The Commercial Pharma tests in TestGen Enterprise check format and check digits and never call NPPES, so they cannot say whether the provider is active or whether the NPI belongs to the prescriber the feed claims. A failure is definite; a pass narrows the possibilities.

Does this replace Veeva Network or our MDM?

No. Veeva Network or your MDM tells you what it could match. Identifier tests tell you what arrived broken, per column and per feed, on the landing table, before a match rate has the chance to hide it. The two answer different questions, and you need both on a commercial platform.

Why are nurse practitioners and PAs missing from our HCP master?

Because most manufacturers built HCP master data around physicians. Nurse practitioners and physician assistants became Open Payments covered recipients in 2021, and in the 2024 CMS file 7,729 of the 13,890 practitioner records missing an NPI, license state, or specialty were NPs and PAs. That is a master data problem before it is a compliance problem.

What happens to prescribers the MDM never matched?

Often they quietly disappear. Non-matched records get excluded rather than surfaced for stewardship, which is how a launch loses prescribers that no report ever mentions. An analytic master that links every identifier available, NPI, ME, DEA, and vendor provider IDs, keeps the unmastered prescribers visible and lets you test them like any other row.

Talk to the people who test this data every week

DataKitchen builds and runs commercial data platforms for small and midsize pharma, with the tests in place from day one.