Quality improvement in community SUD systems often fails for a simple reason: the numbers are not dependable enough to manage. Teams spend hours debating whether a rate is “real,” providers distrust comparisons, and commissioners end up using lagging crisis indicators because they feel safer than fragile engagement metrics. If an outcomes framework is going to drive decisions, the system needs explicit data integrity rules that make measures repeatable, explainable, and auditable. In practice, that means building measurement workflows that sit alongside delivery: intake, clinical care, care coordination, peers, and follow-up.
Two starting points help prevent drift. First, keep your measurement design aligned with the system’s intended operating model for improvement by using the Outcomes, quality measures, and continuous improvement tag as the anchor taxonomy for what “counts” as performance. Second, ensure your measures reflect real-world delivery across programs by cross-walking them to community-based SUD service models, so the same measure can be interpreted consistently across outpatient, mobile, and peer-enabled pathways.
Why data integrity is a system design issue, not a reporting task
In high-volume SUD systems, data breaks in predictable places: duplicated records across referral channels, missing discharges, inconsistent coding of levels of care, and follow-up work that happens outside the EHR (especially for peers, outreach, and re-engagement). Data integrity is therefore a governance and workflow problem. If staff do not have a practical way to correct errors, the dataset becomes a permanent record of operational friction rather than patient experience.
Most funders and oversight bodies expect measurement to be reproducible and defensible. State authorities administering block grant and state general-fund programs typically require consistent definitions over time (so trends are meaningful) and evidence of data quality checks (so performance claims can be verified). Medicaid managed care and county purchasing teams increasingly expect auditable measure specifications, including clear denominators, exclusions, and documented data sources, because they are accountable for network performance and value-based payment integrity.
Build the “denominator story” before you build the dashboard
Dashboards collapse when the denominator is unstable. The question is not “How many people had follow-up?” but “Follow-up among whom, counted how, and when do they enter and leave the eligible population?” Denominator discipline requires explicit entry rules (eligibility start), exit rules (discharge/transfer/inactivity), and exclusion rules (episodes that should not be compared).
- Entry rule: first completed clinical assessment, first billable encounter, or first verified outreach contact—choose one and apply it consistently.
- Exit rule: documented discharge, 60/90 days of inactivity, transfer to another provider, incarceration, or death—again, define it and operationalize it.
- Episode logic: whether you treat re-entry as a new episode or continuation is a policy decision with performance implications.
When these rules are missing, systems inadvertently punish providers serving people with unstable housing, justice involvement, and higher mobility—because the denominator churn creates “failure” that is actually a tracking problem. Denominator discipline is also the bridge between equity and measurement: without stable rules, stratification becomes noise.
Operational example 1: A two-stage validation gate for referrals, intakes, and “ghost” cases
What happens in day-to-day delivery
Referral streams (ED, hotline, court, outreach) feed a shared queue. Intake coordinators apply a first-stage validation gate: deduplicate by name/DOB, confirm contact method, and record a single “episode start” timestamp. A second-stage gate happens at first clinical assessment: the clinician confirms the presenting need, assigns the service pathway, and selects an episode status that becomes the denominator anchor. A data steward runs a weekly exception report for records with missing episode start, multiple starts, or no assessment within a set timeframe.
Why the practice exists (failure mode it addresses)
SUD systems routinely accumulate “ghost cases”: referrals that never convert, duplicate records created under different identifiers, or intake attempts logged as treatment engagement. Without a validation gate, engagement and access metrics become inflated or distorted, and teams can’t tell whether a delay is a service capacity problem or a tracking artifact.
What goes wrong if it is absent
Dashboards show rising “intake volume” while clinics report empty slots, or the opposite—staff feel overwhelmed but access metrics look stable. Commissioners respond by adding reporting pressure, which increases documentation burden and creates more inconsistent data entry. Providers serving higher-acuity populations appear to underperform because their referrals generate more duplicate records and incomplete conversions.
What observable outcome it produces
Systems can report a credible conversion funnel (referral → contacted → assessed → engaged) with a clear audit trail. Weekly exceptions shrink over time, and the gap between operational reality and dashboard signals narrows. Leaders can set defensible thresholds for “time to assessment” because the denominator is stable and consistently defined.
Operational example 2: Claims–EHR reconciliation for MAT continuity measures
What happens in day-to-day delivery
A measurement analyst runs a monthly reconciliation between EHR medication orders, pharmacy claims (where available), and visit/encounter data. The system maintains a “MAT continuity ledger” that flags: missing prescriber NPI, gaps in days covered beyond a threshold, and mismatched medication start dates. Care coordinators receive a worklist for high-risk gaps and document resolution steps (contact pharmacy, reschedule prescriber visit, confirm incarceration/hospitalization status, update transfer information).
Why the practice exists (failure mode it addresses)
MAT access and retention measures often break because the EHR shows an order while claims show no fill, or because a fill occurs after a missed follow-up that never gets recorded. Reconciliation is designed to prevent false reassurance and ensure continuity measures reflect actual medication access, not just documentation of intent.
What goes wrong if it is absent
Systems assume continuity is improving based on orders and visit counts, while overdose risk rises due to unrecognized medication gaps. Providers get surprised by avoidable ED utilization because the system did not detect missed doses or failed pharmacy handoffs. Contract discussions become adversarial because neither side trusts the data.
What observable outcome it produces
Medication continuity indicators become actionable: the system can quantify “gap drivers” (missed follow-up, pharmacy barriers, coverage transitions) and show improvement tied to specific fixes. Audit readiness improves because the system can explain discrepancies and demonstrate that measures are based on reconciled sources.
Operational example 3: Defensible activity tracking for peer and outreach work without inflating “contacts made”
What happens in day-to-day delivery
Peers document work in a structured activity note with three required fields: purpose (engagement, harm reduction, care navigation), channel (in-person, phone, street outreach), and linkage status (scheduled, attended, declined, unreachable). Supervisors review a weekly sample for completeness and alignment to role boundaries. The data team converts activity into outcome-linked indicators (e.g., “contact leading to attended appointment within 7 days”) rather than raw counts.
Why the practice exists (failure mode it addresses)
Peer work is often measured as volume because it is easy. That creates perverse incentives and makes peer integration look like a marketing layer rather than operational infrastructure. Structured tracking exists to connect peer activity to measurable pathway outcomes while protecting peers from being pushed into clinical documentation patterns that don’t fit the role.
What goes wrong if it is absent
Peer programs become vulnerable in budget cycles because leaders can’t show impact beyond stories. Alternatively, peer teams get pressured to maximize contacts, which leads to superficial interactions and burnout. Systems can’t distinguish meaningful re-engagement work from administrative attempts, and quality improvement efforts target the wrong problem.
What observable outcome it produces
Peer impact becomes visible in a way commissioners can defend: improved appointment show rates, faster linkage after outreach, and fewer “lost-to-follow-up” episodes. Quality reviews can trace performance back to workflow adherence (documentation completeness, escalation use) rather than relying on anecdotes or inflated activity metrics.
Governance controls that keep measurement stable over time
High-performing systems treat measurement definitions as controlled documents. They maintain a measure dictionary (specification, denominator/exclusions, data sources, update cadence), use change control (when definitions change, trend lines are annotated), and run routine data quality checks (missingness, outliers, duplicate episodes). They also separate “performance management” from “data cleaning”: providers should not have to litigate data issues inside contract meetings—those issues belong in a defined reconciliation process with timelines and responsibilities.
Finally, dashboards should be built to support improvement rhythms. A monthly performance pack that includes an exceptions appendix (what was excluded and why) is often more useful than a real-time dashboard that hides data instability. When the system can explain its numbers, it can credibly act on them.