Visit-level quality measurement reveals missed opportunities and overuse in preventive oral health care

Abstract

Background.

Traditional annual dental quality measures fail to capture real-time care delivery and can mask critical gaps in patient-specific treatment. The authors developed and implemented visit-level adaptations of established caries quality measures to enable actionable, point-of-care assessment.

Methods.

The authors conducted a retrospective analysis of 1.09 million dental visits across 400,000 patients (2021–2023) at 4 US institutions using axiUm (Henry Schein Corp) or Epic (Epic Systems) electronic health records. The authors transformed 2 annual measures involving caries risk assessment and risk-based preventive treatment into visit-level metrics using a 2 × 2 matrix framework: action due and performed, due and not performed, not due and performed, not due and not performed. Risk-stratified recall visit intervals determined when actions were due.

Results.

Visit-level performance consistently underperformed annual metrics. Annual caries risk assessment completion ranged from 60% through 99% across sites; however, visit-level adherence when actions were due dropped, ranging from 41% through 72%. Results of preventive treatment showed greater disparities; only from 28% through 35% of indicated treatments were delivered when due, and from 7% through 32% of visits included treatment when not indicated. Sites with the highest annual performance paradoxically had the most overtreatment.

Conclusions.

Visit-level quality measurement exposes substantial undertreatment and inappropriate overtreatment invisible in annual reporting. This granular approach revealed that high-performing institutions according to traditional metrics may be delivering unnecessary care and missing critical prevention opportunities. These findings provide an empirical case study of how misaligned incentives and flawed metrics could systematically drive misuse of resources. Implementation of real-time, visit-based quality measures in clinical workflows could fundamentally transform electronic health records from documentation tools into active instruments for delivering appropriate, value-based, person-centered care.

Practical Implications.

Oral health care systems should consider visit-level quality measurement as a means to provide actionable, real-time feedback that can improve clinical decision making at the point of care and prevent waste associated with volume-based care.

Quality measures are essential tools for evaluating health care delivery effectiveness and translating scientific advancements into improved patient outcomes.^1,2 Despite the proliferation of measures, there has not been commensurate effort to assess how well they align with the intended measurement objectives. Present measures predominantly emphasize feasibility, prioritizing use of more easily available data and providing limited insights into care improvement.¹ It has also been suggested that quality measurement be integrated with the care delivery process rather than existing as a parallel, separate enterprise performed intermittently and retrospectively.³ Such seamless integration is now possible through the ubiquitous use of electronic health record (EHR) systems in health care delivery.⁴ EHRs are essential for providing a comprehensive view of the patient’s clinical journey, enable better tracking of health outcomes, ensure better adherence to clinical guidelines, and help identify care gaps. EHRs also facilitate the implementation of quality measures, helping health care providers evaluate and improve their practices using evidence-based standards and patient outcomes.

There are predominantly 2 distinct approaches for assessing the quality of care provided to patients: periodic and visit-based. Periodic (eg, annual, quarterly, or monthly) measures typically assess the cumulative performance on various quality indicators, such as the percentage of patients receiving preventive services within the defined period. These measures are often used for broad performance evaluation, strategic planning, and long-term quality improvement. They are instrumental in identifying outliers and trends over time, guiding policy decisions, and allocating resources effectively. Visit-level measures,⁵ conversely, focus on the quality of care delivered per patient visit in real time. These measures evaluate specific actions or interventions performed during that visit, such as appropriate intervention for symptoms presented or timely follow-up on a test result.⁶ Visit-level measures are more immediate and actionable, providing direct feedback to the care team members on specific clinical actions. They are used to enhance patient care in real-time, adjust clinical practices quickly, address gaps, and customize care at the point of service. This customization of care is a key tenet of person-centered care.⁷

Caries is a preventable chronic health condition,⁸ and, although the etiologic factors have been well described for decades, reducing the number of teeth lost due to caries remains a public health challenge.^9,10 Failure to prevent caries or intervene at an early stage can escalate into more complex and costly rehabilitative treatments.^11,12 Prevention and timely management of caries is therefore beneficial. In response, various caries risk assessment (CRA) and prediction tools have been developed on the basis of findings from either cross-sectional studies, in which various caries-related factors are identified using statistical models that identify the risk factors, or longitudinal studies, in which factors are correlated to new caries development over a period of years.^13,14 These tools consider multiple data points, including medical history, lifestyle, dietary habits, fluoride exposure, and the state of oral health. By means of assessing these factors, CRA tools can help dental care professionals identify people at elevated risk of developing caries, enabling targeted preventive measures and personalized care plans to reduce this risk and promote oral health. Beyond risk assessment, various caries preventative protocols and materials have also been developed and widely instituted, including fluoridation,^15–17 pit and fissure sealants,¹⁸ and antimicrobials (eg, chlorhexidine and xylitol).¹⁹ All of these notwithstanding, the United States has the third highest global case load of caries in permanent teeth, and by far the highest expenditure related to oral disease.²⁰

The US health care system is plagued by a crisis of value; billions are spent on services that provide little benefit, and high-value care is often missed.²¹ Quality measurement systems designed to address this frequently fail and may even exacerbate the problem by means of rewarding volume over patient-centered appropriateness.²² In our study, we introduced a novel visit-level measurement framework using the microcosm of oral health care, a sector with more than $160 billion in annual expenditure,²³ as a scalable method for differentiating value from potential waste at the point of care.

By means of juxtaposing quality measurement from retrospective annual reports with visit-based person-centered assessments, we illustrate how well-designed quality measures could act as active instruments for delivering appropriate, value-based care across all health care fields.

METHODS

Periodic (annual) and visit-based electronic measures regarding caries management were developed and implemented across 4 dental institutions, comprising 2 dental schools and 2 large multisite dental groups. All participating sites have a history of established research collaboration. Three of the institutions use the axiUm EHR system (Henry Schein Corp) and 1 uses the Epic EHR system (Epic Systems). These institutions serve a diverse patient population through a range of care settings, including private practices, specialty clinics, and teaching clinics. We received institutional review board approval from all collaborating institutions before conducting our study; the single institutional review board number is HSC-DB-14–1051, from The University of Texas Health Science Center at Houston Human Subjects Committee.

Measure definition

Table 1 provides definitions of the electronic measures (e-measures).

Table 1.

Description of caries management electronic measures.

MEASURE	ANNUAL	VISIT LEVEL
CRA *	Proportion of patients (any age) who received a CRA within the reporting year	Proportion of visits when CRA was performed when due (due time varies according to previous caries risk level and age group)
Caries Preventative Treatment (Including Sealants)	Percentage of patients (any age) at elevated risk of developing caries who received a preventive treatment on or after the date of the CRA during the reporting year	Percentage of visits of patients (any age) at elevated risk of developing caries who received a preventive treatment within a risk-appropriate duration, after determination of elevated caries risk level

^*CRA: Caries risk assessment.

Measure development

The EHR-based measures were developed previously, and we followed the same process used by those authors.^24–26 Briefly, a structured query language script was developed to enable extraction of structured documentation from the relevant location within the EHR across participating institutions. A common data model was developed for extracting data from EHRs at the visit level for all eligible patients. For adaptation of use within the Epic EHR system, we developed SAS code using SAS software, Version 9.3 (SAS Institute) to extract data after assessing the data availability and feasibility for each measure. The process was iterative and included meeting with clinical experts and informaticists to understand the workflow and data storage and retrospective chart reviews to determine when code revisions were needed. The final data were configured to map to the same procedures and diagnoses as other participating institutions and structured to the same standard format. The measure scores were computed for both annual and visit-level approaches.

Measure validation

The validation of the e-measures followed a 3-step process^27,28

1. The structured query language queries were used to identify eligible patients for each of the measures.

2. Estimation of sample size for manual chart reviews: for a calculated measure score given by the queried population, with a margin of error of .05, a standard significance level of .05, and a normal curve with a 2-tailed cutoff of 1.96 (z = 1.96), we estimated the sample size required for manual review. These manual reviews were the reference standard.

3. Validation scores: 2 reviewers from each site, with experience in the use of their site’s EHR system, conducted manual chart reviews using the predefined criteria for the numerator and the denominator as a guide.

Measure definitions

Annual measures were reported as proportion of patients (numerator, denominator). For visit-based measures, we categorized each visit into 1 of 4 options in a matrix: 1 was a clinical action or intervention due (yes, no) and 2 was the action performed (yes, no) (Table 2).

Table 2.

Visit-based measures matrix.

	ACTION DUE	ACTION NOT DUE
Action Performed	no./percentage	no./percentage
Action Not Performed	no./percentage	no./percentage

We then tabulated the number of visits in each cell. To calculate an overall measure score from the matrix above, we included in the denominator all cases that fall into the category where the action was due. For the measure numerator, we included only those cases for which the action was due and was performed. Visits when the action was not due to be performed were excluded from the primary measure but were included in secondary analysis.

Annual reporting measures focus on meeting criteria within a predefined calendar period, but visit-level measures accommodate integrating factors of previous risk levels and age grouping. For example, extreme-risk patients were due for a CRA in 3 months and high-risk patients were due for a CRA in 6 months. Table 3 outlines all recall visit periods used in our calculations, factoring in age. A CRA was considered due at the index visit for patients without a prior assessment. All due dates allowed for a 1-month grace period; for example, CRA completed within 4 months for an extreme-risk patient was still counted as success.

Table 3.

Expected duration of recall visit.

PREVIOUS RISK LEVEL	EXPECTED DURATION OF RECALL VISITS, MO
	< 14 Y	≥ 14 Y
Caries Risk Assessment
Low	12	24
Moderate	6	12
High	3	6
Extreme	3	3
Caries Preventative Treatment
Low	Not applicable	Not applicable
Moderate	6	6
High	3	6
Extreme	3	3

Statistical analysis

The finalized data extraction script was executed to extract data for 3 calendar years (2021–2023) at all sites. Descriptive statistics were calculated for each measure score. All analyses were conducted using Stata, Release 17 (StataCorp) and R, Version 4.4.0 (The R Project for Statistical Computing) software.^29,30 Lastly, to synthesize and visualize the clinical information, we developed dashboards aggregating each site’s performance over time. The dashboards (for annual measures) displayed the frequency and percentage of all caries performance measures as a time series chart over the 3-year study period. Dashboards for visit-specific measures displayed relative performance on what actions or interventions were due vs performed at the index visit.

RESULTS

Data on 2 dental quality measures, CRA and caries preventative treatments, were analyzed across participating sites for the years 2021, 2022, and 2023. For site 4, data were available for 2021 and 2022 only. Data included information from 1.09 million visits and almost 400,000 patients. Performance for each measure was reported both annually and per visit. Visit-level performance was analyzed using a 2 × 2 matrix categorizing visits on the basis of whether a clinical action was due and whether it was performed.

Sites 1 and 3 had consistently high CRA performance for both annual and visit-level measures, although visit-level performance rates were consistently marginally lower than annual. Site 2 had annual performance rates from 60.2% through 63.1% and visit-level performance rates ranged from 56.7% through 60.8%, consistently remaining a few percentage points below the annual figures. This was also the case for site 4.

Figure 1 provides visit-level performance vs annual measure performance as defined above. At site 1, CRA was performed in more than 99% of visits when they were due, but they also had a substantial number of CRAs performed even when not due. Site 3 also had CRAs completed in 96% of visits when due, alongside a considerable volume of CRA performed when not due. In contrast, site 2 performed CRAs in approximately 58% of visits when due, indicating missed opportunities; CRAs were also performed when not due, but to a lesser extent. Site 4 had a CRA performance rate of approximately 43%, signifying most visits when an indicated CRA was not performed, with fewer instances of CRAs being performed when not due.

Figure 1.

Annual and visit-level measure performance.

Regarding preventive treatments, site 1 performed preventive interventions in approximately 86% of visits when they were due and had a number of preventive interventions performed when not due. Site 4 had a performance rate of approximately 70% when due, with a considerable number of preventive interventions performed when not indicated. Sites 2 and 3 had lower preventive intervention performance rates when due, approximately 54% and 58%, respectively, indicating that in a substantial portion of visits when an intervention was indicated, it was not performed; both sites also recorded preventive interventions performed when not due.

Figure 2 illustrates the proportion of total visits falling into each cell of a 2 × 2 matrix for each site, providing a more nuanced view of visit-level performance across all possible action-status combinations. When all 4 categories are considered (actions due and performed, due and not performed, not due and performed, not due and not performed), overall performance declined. For instance, although site 1 achieved a 99% CRA completion rate when only visits for which the action was due were analyzed, the combined rate for visits at which CRA was appropriately performed when due and appropriately omitted when not due was only 72%. This drop in performance when evaluated against the full matrix was observed consistently across all sites.

Figure 2.

Visit-level action completion according to measure and site. Because risk-based preventive treatments are only considered due when caries risk has been assessed, the denominator for this metric is limited to visits when a caries risk assessment (CRA) was completed. Sites with lower CRA completion (sites 1 and 3) will therefore have smaller denominators for this component.

DISCUSSION

We explored 2 distinct approaches to quality measurement using the management of caries as a data-rich case study for a universal challenge in health care. We found that although traditional annual quality metrics suggested high performance, a more granular visit-level analysis uncovered missed opportunities for essential care and delivery of services not indicated in risk-based guidelines. This paradox in which high-performing systems according to annual standards may simultaneously exhibit substantial undertreatment and overtreatment is not a problem unique to dentistry. It is an artifact of a measurement paradigm common across medicine that is ill-suited for the complexities of person-centered care. Our results highlight that this measurement approach can be replaced with a scalable visit-level framework that provides the real-time clinical intelligence necessary to advance the broader goals of health care reform.

Quality improvement efforts that incorporate periodic quality measures have been found to help promote use of care.³¹ However, volume does not represent value. Some patients fail to receive appropriate care and others receive care that will not benefit them, encouraging the inappropriate usage of care.^32,33 As health systems respond to the challenges of potential waste and abuse, visit-level measures present the opportunity to implement patient-centered approaches that will drive suitable decision making for patients, rather than just across-the-board policy changes that decrease or increase usage, more or less arbitrarily. A patient-centered performance management system would help clinicians and patients make individualized decisions about optimal care for common clinical situations. With adoption of highly developed EHR systems in the dental setting, we have an opportunity and the capability to deliver individualized care at a level of sophistication that can encourage thoughtful, evidence-based, clinical decision making.³⁴

As highlighted in the Methods section, a CRA was considered due at the index visit for patients without a recorded prior CRA in the lookback window (ie, 2021–2023). Because of high patient turnover or historical CRAs that may have been documented before 2021, this rule could be responsible for the relatively high proportion of CRA encounters classified as due at sites 1 and 3, an effect analogous to left-censoring.

Although annual metrics aggregate performance over 1 year, they obscure timing and appropriateness at the point of care. Visit-level metrics directly assess whether guideline-recommended actions are taken when due on the basis of each patient’s risk profile and visit context. This makes them more sensitive to missed opportunities and potential overuse. As we found in our study, institutions with high annual CRA rates also had high rates of unnecessary assessments when not indicated, an inefficiency invisible to annual summaries.

The action not due but performed category warrants careful interpretation. Although it may flag potential overtreatment, it could also represent appropriate clinical discretion, shared decision making, or precautionary care on the basis of a patient’s circumstances not captured in guidelines. Distinguishing true overtreatment,³⁵ driven by means of misaligned incentives³⁶ or defensive medicine, from appropriate personalized care requires contextual analysis beyond what structured data can provide.

Most public reports of annual measures often presume that patients can effectively interpret quality metrics. It has been reported, however, that even patients with education beyond high school and decision makers have difficulty understanding basic statistics, so it is unsurprising that many view these reports as not helpful.³⁷ Rather than choosing among providers of a specific service, it is conceivable that patients are searching for an answer to a more fundamental question: “Is the suggested treatment or procedure the most suitable choice for me?”³⁸ As such, for patients and providers to have a meaningful discussion about the use of a clinical service, it is essential that they understand the value of care for the specific patient, and that the provider is not incentivized to recommend the service regardless of the patient’s characteristics, which is a common unintended consequence of annual measures.³⁹

Recall visit interval guidelines vary across countries and are often shaped according to insurance policies rather than evidence. Traditionally, examinations every 6 months have been routine, regardless of patient risk. Expert bodies now recommend risk-based intervals⁴⁰ although the evi dence is limited.⁴¹ In a 2020 Cochrane review, researchers found no difference in caries outcomes between 6-, 12-, 24-month recall visit intervals and risk-based recall visits over 4 years.⁴² Shorter intervals may allow earlier intervention but risk overtreatment; longer intervals may reduce unnecessary care but delay diagnosis. Visit-level quality measures can help strike the right balance by means of aligning recall visit decisions with patient outcomes.

In a forward-looking article published in The New England Journal of Medicine more than a decade ago,⁴³ McGlynn and colleagues proposed several initiatives for reimagining quality measures. These included the development of an integrated patient dashboard, which would present the key dimensions and components of each patient’s health status. They also recommended the identification of optimal guidelines and their translation into a checklist of essential steps, prioritizing interventions most likely to enhance a patient’s health status. In addition, they suggested the implementation of enterprise information systems functionality to support a flexible and adaptable measurement approach. We adhered closely to this framework during the formulation and implementation of these visit-based dental quality measures.

Our study has some limitations. First, we relied on retrospective extraction of routine EHR data from 4 large organizations, so documentation errors, missing fields, and site-specific customizations may have misrepresented whether a preventive action was due or performed. Second, the due rules are anchored to clinical guidelines and must be updated as recommendations evolve. Third, structured data do not always capture provider rationale or patient preference, so actions recorded when not due may reflect appropriate clinical judgment, and missed actions may have been consciously deferred. In addition, we did not account for the hierarchical nesting of patients within providers and providers within institutions. As such, the correlation among observations at the provider or clinic level was not statistically modeled. This should be addressed in future research using multilevel modeling. This mixed-effects framework would also help identify the best site-specific improvement implementation targets. Finally, we examined process measures only; it remains untested whether improving visit-level performance translates into lower caries incidence, reduced cost, or better patient experience. These weaknesses are offset by strengths. Our analyses draw on an exceptionally large real-world data set of more than 1 million dental visits across 4 geographically and organizationally diverse sites, and we used a common data model and previously validated e-measure algorithms that run successfully in both axiUm and Epic EHR environments. By means of converting annual measures into the intuitive due vs done visit-level matrix, we provide clinicians with actionable, chairside feedback that annual reporting cannot deliver, illustrating the practical feasibility of real-time, person-centered quality monitoring.

Quality measures are not goals in and of themselves; they serve as initial indicators of quality improvement. Enhanced capacity to report on performance will not, by itself, add value or result in substantial improvements in clinical care. For measures to have the intended effect, they must be integrated with other related measures and then tethered to potential implementation strategies in an improvement bundle.⁴⁴ Future efforts to advance visit-level quality measures will critically depend on evidence of their cost-effectiveness through robust return on investment analysis results that capture long-term benefits and on fundamentally reforming incentive structures to align with person-centered, value-based care principles.

CONCLUSIONS

Visit-level dental quality measures expose what annual metrics hide, that even high-performing institutions miss from 28% through 72% of indicated preventive care and deliver unnecessary treatment in up to 32% of visits. Visit-level metrics create the foundation for evidence-based recall visit intervals, value-based payment models, and truly person-centered care. Our work shows that real-time quality assessment is not only technically feasible but also essential for dentistry’s future. What was aspirational is now achievable.

ABBREVIATION KEY

CRA

Caries risk assessment

EHR

Electronic health record

E-measure

Electronic measure