Universes are not supporting documentation. They are the population a CMS program audit samples from — which is why a defect in the file distorts every test run against it.
The Sampling Frame
What CMS receives, and why defective universes distort the audit
A program audit does not evaluate your operation directly. It evaluates record sets, and CMS draws its samples from them. The universes you submit are not supporting documentation. They are the population the entire audit is sampled from.
That distinction matters more than it first appears. If the universe is defective, the sample CMS pulls from it is defective too, even if your actual operations are sound. A missing case, a miscalculated date, or a record that landed in the wrong table can trigger a finding without reflecting anything about how the plan actually operates. CMS has a specific classification for exactly this failure mode: Invalid Data Submission, cited when a universe is inaccurate or incomplete and CMS cannot determine compliance from it. An IDS finding says the file failed, not that the underlying process did, and it lands on the audit record regardless of how well that process actually performs.
The argument for scrubbing before submission
Universes are not a formality that documents your work. They are the sampling frame the rest of the audit is built on, and a defective one puts every other test CMS runs on unstable ground.
So where do these defects actually come from?
Five Defect Types
The five ways universes fail
Universe defects are not random, and they rarely announce themselves. A file can look complete, format correctly, and still carry problems that only surface once CMS starts sampling from it. Five categories capture the recurring patterns behind most universe defects.
Defect inspector — select a type
Originates in extract mapping, format conversion, or manual data entry
Originates in source system timestamp logic or manual case tracking
Originates in disconnected source systems for determinations, appeals, and effectuations
Originates in extract filter criteria that exclude cases incorrectly
Originates in source systems updated at different times or by different teams
Field names and values are illustrative, drawn to show each defect’s mechanism rather than any specific CMS record layout. Confirm every check against the current protocol layout for the cycle you are submitting in.
Structural defects. Field order, data types, date formats, and required fields left blank. These are the most mechanical failures and also the most preventable, since they can be caught by comparing the file against the current record layout before anyone reviews a single case.
Logical defects. A decision letter dated as mailed before the decision was actually made, timeliness computed from the wrong start point, or dates that don’t sequence logically against each other. These defects can exist in a file that is structurally perfect, since structure checks confirm a field is a valid date, not that the date makes sense in context.
Reconciliation defects. Record counts that do not tie across related tables. An organization determination that does not connect to its reconsideration record, or an appeal count that does not match the underlying determination count, signals a break in how the extract logic handles related cases, and it is far cheaper to catch before submission than after.
Completeness defects. Cases excluded by an extract filter that should not have applied. This is often invisible in the file itself, since the problem is what is missing, not what is present, and it typically only surfaces when someone checks the extract logic against the actual case population it was supposed to capture.
Consistency defects. The same case represented differently across related submissions, such as a status or outcome that does not match between an ODAG table and a related grievance or appeal record. These defects point to source systems that are not synchronized, or extract processes that pull the same case at different points in time.
Knowing what can go wrong is only useful if there is time to fix it. That is where timelines are most often squeezed.
The Production Window
The production window is the real constraint
The deadline that matters is not the audit timeline as a whole. It is the roughly 15 business days between the engagement letter and universe submission. That is the window in which the extract has to be produced, scrubbed against every defect category above, and cleared for submission.
Roughly 15 business days, start to submission
Segment widths are illustrative. The fixed quantity is the window itself — whatever the scrub flags has to be corrected and re-checked inside it.
Built over months. Policies, evidence, oversight controls. Does not extend the window.
Lives entirely inside the window. If the extract can’t be produced and scrubbed in time, readiness doesn’t help. Two different problems.
Treating audit readiness and extract production as one problem is where timelines quietly fail. The window absorbs production, scrubbing, and correction — not just submission.
Weeks of general audit readiness do not help inside that window if the extract itself cannot be produced and scrubbed in time. Readiness and extract production are two different problems, and treating them as one is where timelines quietly fail.
How to protect the window
Run practice pulls outside of audit season, not during it. A practice pull surfaces structural and logical defects while there is still time to fix the extract logic itself, rather than discovering them for the first time under a 15-business-day clock.
Even a plan that has protected this window internally is not fully in control of it. Part of the universe usually belongs to someone else.
Delegated Data
Getting delegate data in before the scrub
If any part of intake, determinations, appeals, effectuations, or pharmacy claims processing is delegated, that data has to enter the process before scrubbing starts, not alongside it.
The scope here is wider than intake, determinations, and appeals alone. CDAG’s own structure requires a separate universe table for coverage determination exception requests (CDER), distinct from standard coverage determinations, and a separate table for Part D effectuations of overturned decisions. None of that scope shifts with delegation. Whatever is delegated, the plan remains responsible for the accuracy and compliance of that data, and for the member experience behind it, once it lands in the universe.
Delegate data arrives before the scrub, not alongside it
Standardized submission template
One format across every delegate type, so the scrub can actually evaluate the data
Delegates submit in their own formats and date conventions, so their data cannot drop straight into a scrub built for the plan’s own systems. Standardizing the template removes a category of defect before it forms.
Delegates typically submit in their own formats and date conventions, which means that data cannot be dropped directly into a scrub built for the plan’s own systems. Standardizing a submission template across every delegate type, so all of that data arrives in a format the scrub can actually evaluate, removes a category of defect before it has a chance to form.
This also means delegate data needs its own lead time ahead of the plan’s internal checks. A scrub that starts before every delegate’s data has arrived and been standardized is scrubbing an incomplete universe, regardless of how thorough the checks themselves are.
With delegate data accounted for, it would be reasonable to assume a scrub built once can be reused every cycle. It cannot.
Protocol Drift
Why the check set needs versioning too
A scrub is only as good as the layout it checks against, and CMS protocol layouts change between audit cycles.
Same file, two layouts
This is not a failure of the scrub. It is a failure to update what the scrub is checking against. Field names here are illustrative — the point is the mismatch, not these specific fields.
Every audit cycle should start with a confirmation that the check set matches the current protocol layout, not an assumption that last cycle’s checks still apply.
A check set built against last year’s protocol will pass a file that this year’s protocol rejects. That is not a failure of the scrub itself. It is a failure to update what the scrub is checking against. Every audit cycle should start with a confirmation that the check set matches the current protocol layout, not an assumption that last cycle’s checks still apply.
At this point, it would be fair to assume a thorough, current, well-versioned scrub is enough on its own. It is not, and being honest about where it stops matters.
The Automation Boundary
What automated checks do not catch
Automated checks are well suited to structure, logic, and reconciliation. They can confirm a date is formatted correctly, that a decision letter wasn’t dated as mailed before the decision was made, or that record counts tie across tables.
Where the scrub stops and a reviewer starts
What the scrub catches
Data quality rules, checkable against the layout and against other tables
What needs a human reviewer
Judgment calls about business rules, not data quality rules
Automated scrubbing reduces the number of things a human reviewer has to check by hand. It does not remove the need for that review.
What they cannot confirm is whether the extract logic matches how the plan actually defines a case. That is a judgment call about business rules, not a data quality rule, and it requires a human reviewer who understands how the plan classifies and tracks cases to confirm the extract reflects that definition. Completeness defects, in particular, tend to live exactly in this gap, since a filter can be technically well formed and still exclude cases it should not.
Automated scrubbing reduces the number of things a human reviewer has to check by hand. It does not remove the need for that review.
Put together, this is what a defensible universe actually requires: five defect types checked, a fixed window respected, delegate data accounted for, a current check set, and a human review where automation stops. The table below is what that looks like in practice.
Pre-Submission Checklist
What to check before you submit
Defect · origin · pre-submission check
| Defect | Where it originates | What to check before submission |
|---|---|---|
| Structural | Extract mapping, format conversion, or manual data entry | Field order, data types, date formats, and required fields against the current record layout |
| Logical | Source system timestamp logic or manual case tracking | Date sequencing, mailed-before-decided errors, and the start point used for timeliness calculations |
| Reconciliation | Disconnected source systems for determinations, appeals, and effectuations | Record counts and case identifiers tying across related tables |
| Completeness | Extract filter criteria that exclude cases incorrectly | The extract’s inclusion logic against the actual case population it should capture |
| Consistency | Source systems updated at different times or by different teams | The same case’s status and outcome across every table where it appears |
Each row pairs a defect type with the place it tends to originate, so the check lands where the defect is actually introduced rather than where it eventually shows up.
Where This Leaves Plans
Where this leaves plans heading into an audit
Every defect category above is preventable, and none of them requires new operational capability to fix. What they require is a scrub that is comprehensive, current with the latest protocol layout, and run early enough inside the 15-business-day window to leave time for correction.
Inovaare’s Universe Management System has been built around exactly that constraint. It generates the universe, scrubs it against the current CMS record layout, and aggregates it across the required review period, catching structural, logical, and reconciliation defects before submission, so the human review that only a plan’s own team can do is spent confirming case definitions and completeness, not chasing formatting errors.
Reported universe prep time
Reported by plans using the Universe Management System; actual timelines depend on the number of universes in scope, source-system readiness, and how much of the data is delegated. Bar lengths are drawn to the stated durations.
Plans using it have reported reducing universe prep from six to eight weeks to under one week, depending on the number of universes in scope, source-system readiness, and how much of the data is delegated, and without relying on IT to build or maintain the extract. That leaves room inside the roughly 15-business-day window instead of running against it. The same discipline extends to every universe a program audit touches, not just ODAG and CDAG.
Scrub ODAG and CDAG universes before they reach CMS
If your team is heading into a program audit, see how the Universe Management System generates, scrubs and clears universes against the current CMS record layout.
Sources: CMS 2020 and 2021 Program Audit Process Overview; CMS Audit Submission Checklist; CMS CDAG Protocol and Data Request. All CMS-specific timelines, table structures, and finding classifications cited above are drawn directly from these CMS documents.