Quantifying Dental Office–Originating Adverse Events: The Dental Practice Study Methods

Abstract

Background:

Preventable medical errors in hospital settings are the third leading cause of deaths in the United States. However, less is known about harm that occurs in patients in outpatient settings, where the majority of care is delivered. We do not know the likelihood that a patient sitting in a dentist chair will experience harm. Additionally, we do not know if patients of certain race, age, sex, or socioeconomic status disproportionately experience iatrogenic harm.

Methods:

We initiated the Dental Practice Study (DPS) with the aim of determining the frequency and types of adverse events (AEs) that occur in dentistry on the basis of retrospective chart audit. This article discusses the 6-month pilot phase of the DPS during which we explored the feasibility and efficiency of our multistaged review process to detect AEs.

Results:

At sites 1, 2, and 3, respectively, 2 reviewers abstracted 21, 11, and 23 probable AEs, respectively, from the 100 patient charts audited per site. At site 2, a third reviewer audited the same 100 charts and found only 1 additional probable AE. Of the total 56 probable AEs (from 300 charts), the expert panel confirmed 9 AE cases. This equals 3 AEs per 100 patients per year. Patients who experienced an AE tended to be male and older and to have undergone more procedures within the study year.

Conclusions:

This article presents an overview of the DPS. It describes the methods used and summarizes the results of its pilot phase. To minimize threats to dental patient safety, a starting point is to understand their basic epidemiology, both in terms of their frequency and the extent to which they affect different populations.

Nonmaleficence, or the principle to first do no harm, is one of the cardinal rules for health care providers.¹ Nevertheless, preventable medical errors in hospital settings are the third leading cause of deaths in the United States and account for a projected 251,000 mortalities and many more morbidities annually.² However, less is known about harm that occurs in patients in ambulatory care or outpatient settings, where the majority of care is delivered.³ As such, although the vast majority of health care delivery takes place in these ambulatory care centers, efforts to improve safety have mostly focused on the inpatient setting.³

Dental care is predominantly delivered in ambulatory care settings. Until recently, there has been little to no knowledge about the potential dangers of dental care in the United States, where approximately 196,000 dentists⁴ provide dental care every year to 83% of all US children (aged 2–17 years) and 62% of all US adults (aged ≥18 years).⁴ Using a national incident reporting system database, a UK study discussed that more than 2000 patient safety incidents were reported by dentists during 1 calendar year. Many of those incidents/errors did not result in patient harm, but the authors suggested that the numbers may be grossly underestimated because of the perceived risk of damage to professional reputation and livelihood if dentists report patient safety incidents.⁵ A 2010 Finnish study also reported on an Internet-based questionnaire that requested practicing dentists to respond to questions on any patient safety incidents that had occurred during the preceding year. A total of 1041 dentists responded (response proportion 54%), and almost one-third reported that at least 1 patient safety incident occurred at their practice in the preceding 12 months.⁶ In the United States, we are now beginning to understand that harm associated with dental treatment can be significant⁶—aspiration of dental devices,⁷ orofacial necrotizing fasciitis,⁸ intracerebral hematoma,⁹ human immunodeficiency virus/hepatitis infection transmission,¹⁰ major allergic reaction, and death have all been reported in the literature.¹¹ The inherent risk of dental care is not surprising, given that dentists, like physicians, routinely perform highly technical procedures in complex environments, work in teams, and use a multitude of materials, devices, and tools.¹² Concerns about patient safety, however, should not only be evoked by these infrequently reported major adverse event (AE)/egregious error cases; we should also be equally concerned about the more frequent so-called mundane, system-driven AEs. As 1 commentator puts it, “…making the field of patient safety all about death has risks. Just as most deaths do not involve medical error, most medical errors do not produce death—but they can still produce substantial morbidity, costs, suffering, and distress. Drawing attention only to death as the focus of patient safety efforts risks drawing resources away from many settings of care—including almost all nonhospital environments—where death is not the most relevant outcome.”^13,14 At the moment, we do not yet know the likelihood a patient sitting in a dentist chair will experience harm. In objective terms, we also do not know if certain dental treatments are more dangerous than others. Additionally, we do not know if patients of certain race, age, sex, or socioeconomic status disproportionately experience harm in association with dental care.

Most other fields of human endeavor, including construction, agriculture, and aviation, have embraced a “businesslike” systematic approach to safety that anticipates and mitigates everyday risks.¹⁵ Health care should inculcate such standards as patients tend to place a significant level of trust in their providers and clinics.¹⁵ To minimize threats to patient safety, a starting point is to understand their basic epidemiology, both in terms of their frequency and the extent to which they affect different populations.

The Harvard Medical Practice Study (HMPS)¹⁶ was not the first study to examine AEs in health care settings, but it established the standard by which AEs are measured.^17–19 It also laid the ground-work for discussions on patient safety in several countries, and the results offered one of the first large sample estimates of AEs in the health services research literature.²⁰ Its methods were based on the 1977 California medical insurance feasibility study.²¹ The HMPS method for identifying AEs relied on a 2-staged chart review process. The first stage was carried out by nurses to screen patient records that are likely to include an AE. Culled charts were subsequently reviewed in more detail by physicians to confirm the presence of AEs. Although some criticisms emerged for this process (including variations in physicians’ assessments), it has endured to become the orthodoxy for research on AEs.

Building on the conceptual framework of the HMPS,¹⁶ we initiated the Dental Practice Study (DPS) with the aim of determining the frequency and types of AEs that occur in dentistry on the basis of retrospective chart audit. As additional objectives, we planned to (a) explore the distribution of dental procedures associated with these AEs; (b) assess for disparities (age, sex, insurance type, and race/ethnicity) in the frequency of AEs; and (c) quantify the increase in the dental office–originating AEs that occur when medical records of the same patients are concurrently audited. This article discusses the 6-month pilot phase of the DPS during which we explored the feasibility and efficiency of our multistaged review process to detect AEs.

METHODS

The specific goals of the 6-month pilot phase of the DPS were to examine the sensitivity of the 2-reviewer data abstraction process to detect probable AE, to develop and review performance of an automated electronic health record (EHR) data extraction script, and to calibrate expert panel members for AE determination. Permission to carry out the study was obtained from each participating institution’s institutional review board.

Definition of Terms

Adverse events have been variably defined in different studies, but a common denominator is that there is some degree of injury to the patient, observed after a health care intervention. For the present study, AE was defined as: physical harm associated with dental treatment within a time frame relevant to the clinical scenario. Using this definition, suspected progression of a disease process will not be considered an AE. For example, the development of a periapical abscess soon after the placement of a deep dental restoration on a carious tooth will not be an AE. Some of our other guidelines can be found in Appendix A (http://links.lww.com/JPS/A128).

AE Type

Adverse events experienced in dentistry do not fit exactly in the various classification structures developed in medicine by the Institute for Healthcare Improvement (IHI), the National Coordinating Council for Medication Error Reporting and Prevention, and the National Quality Forum. By classifying AEs, we can succinctly summarize information, standardize language, and allow for proper comparisons. Consequently, we repurposed those previous classification schemes, leveraging each one’s unique advantages to develop a 12-category classification for dental office–originating AEs.²² The 12 categories classify harm resulting from AEs as allergy, toxicity or foreign body response; aspiration or ingestion of foreign body; infection; wrong site, wrong patient, or wrong procedure errors; bleeding; pain; hard tissue injury; soft tissue injury; nerve injury; other orofacial harm; other systemic harm; or other harm.

AE Severity

In order to describe the impact of an AE on the patient, as with many related studies in medicine, we adopted the IHI severity scale. Because dental and medical patient safety experts are ideologically alike in how they consider AE severity, we only slightly modified the IHI outline for AE severity classification to apply to dentistry. Our modified schema is depicted in Figure 1.

FIGURE 1.

Decision tree for AE severity assessment.

Study Sites

The DPS is being conducted at 2 dental schools and 1 multispecialty large group practice:

• Harvard School of Dental Medicine (HSDM): At HSDM, patients obtain dental care at the Harvard Dental Center. Patients receive care in the fields of general dentistry, periodontics, prosthodontics, implant dentistry, endodontics, orthodontics, oral surgery, and dental hygiene. The HSDM’s predoctoral class size is 35 students per year; its residency programs have an enrollment of approximately 100 students; its private faculty group practice operates similarly to a private practice setting. The axiUm EHR (Exan, Vancouver, British Columbia, Canada) and a standardized dental diagnostic terminology^23–25 (DDS) were both implemented in 2009. Together, the EHR and diagnostic terminology are used as the basis for conducting research, improving patient care, patient safety, and quality improvement efforts.

• University of Texas Houston School of Dentistry (UTHealth): The UTHealth patient care programs operate a faculty practice, as well as advanced education and predoctoral clinics. UTDentists, the school’s faculty practice, has practitioners in general practice, dental hygiene, endodontics, imaging, oral pathology, oral surgery, orthodontics, pediatric dentistry, periodontics, and prosthodontics. In addition to the on-site clinics, UTHealth has affiliations with Houston-area hospitals, school districts throughout the Greater Houston Area, and a wide variety of clinics, community organizations, and long-term health care centers. As the only dental school in southeast Texas, UTHealth is a primary source of oral health care for low-income families and for patients with special needs and/or medical comorbidities. UTHealth also uses a standardized diagnostic terminology and the axiUm EHR for documentation of patient care.

• HealthPartners (HP): HealthPartners is the largest consumer-governed nonprofit health care organization in the United States, providing care, insurance coverage, research, and education to improve health and well-being in partnership with its members, patients, and community. The organization includes a multispecialty group practice of more than 1700 physicians and 70 dentists, 7 hospitals, 47 primary care clinics, 22 urgent care locations, 22 dental clinics, and numerous specialty practices in Minnesota and western Wisconsin. HealthPartners has an integrated dental (GSD Groups EHR, Exan, Vancouver) and medical (Epic, Verona, WI) EHR system that allows auditing of both charts for most of their patients; as such, at this site, chart reviewers were able to additionally fully review the medical charts of the dental patients for probable AEs.

Study Process

An overview of our proposed study process is represented in Figure 2. All study sites use an EHR for documentation of patient care; consequently, we were able to reduce the burden on chart reviewers by automating the abstraction of much of the covariate information (age, sex, race/ethnicity, insurance information, treatment, etc). A generic SQL (structured query language) script was developed to generate output in a standard format at the sites. The development of the script followed an iterative process, in which feasibility and data availability were explored at each stage and modifications made as appropriate. As each site had a different workflow, minor site-level configuration changes had to be made to the script before executing it.

FIGURE 2.

Simplified overview of study process.

This data extraction script was also used to randomize the order of the charts in order to enable random chart selection. The randomly selected patient charts and prepopulated information were subsequently imported directly into REDCap, a secure, Web-based application designed to support data capture for research studies.²⁶ After data import, at each site, 2 reviewers independently reviewed all the randomly selected patient charts and logged their assessments using the corresponding patient study ID into REDCap. Site chart reviewers audited charts to verify the accuracy of the prepopulated covariate information and searched for probable AEs within the study year. To determine what incident could be a probable AE, reviewers needed to answer 2 questions: (1) Was there some form of physical harm to the patient? (2) Was the harm associated with a dental intervention? If yes was answered to both questions, the incident was considered a probable AE. If the reviewer was unsure, the incident was still considered a probable AE. If the reviewer could definitively answer no to either/both questions, the incident was not included as a probable AE. A sample output of our data entry template can be seen in Appendix B (http://links.lww.com/JPS/A129). Reviews were completed in cycles of 100; after completion of a batch of reviews (100 charts), site-level data were synthesized (eg, to remove duplicate probable AEs, when the same incident was caught by both reviewers) and scrubbed to ensure that no Protected Health Information were inadvertently included, and feedback provided to reviewers if more information on any specific case was needed. The list of probable AEs was then forwarded to the expert panel (discussed below). Panel members independently reviewed the list of probable AEs (with allied information) and made a determination of which ones were actual AEs, based on the predetermined definition and guidelines. After independent assessments, the panelists met to discuss cases in which their individual assessments varied and come to a consensus. For a probable AE to be considered an AE, full consensus was required. Adverse events were then to be classified and assigned a severity level by the panel. This process was followed during the 6-month pilot (test run) phase of the study.

During the pilot phase of the DPS, we randomly selected and reviewed 100 patient charts per study site using the 2014 calendar year as the study period. Additionally, at one of the study sites, a trained third reviewer was asked to independently review the same charts (as the 2 regular chart reviewers) and document the presence/absence of probable AEs. We hoped that this third review would help us determine if having 2 independent reviewers was sufficiently sensitive for detecting most/all probable AEs.

Training and Calibration

Chart Reviewers

There were 2 chart auditors per site, including 4 dentists and 2 nurses. All have extensive experience with chart reviews and are familiar with each site’s EHR customizations and clinical workflow. At the project’s kickoff, all reviewers underwent a comprehensive training session that detailed the goals of the project, the specifics of each data point to be abstracted, and guidelines to assist in decision making. This was followed by regular weekly calibration sessions using test cases. We continued this process (for 3 months) until all the reviewers were comfortable and had a general understanding of probable AEs. To standardize the review process, a written manual, including definitions, was developed, discussed, and approved by all reviewers before the start of the pilot phase of the study. During this training period, each chart reviewer independently audited more than 50 test cases.

AE Panel Calibration

Our expert panel consists of 5 dentists. E.K. is an internationally known dental patient safety expert, an oral surgeon, and a university professor. Panelist 1 has also done wide-ranging work in the field of dental patient safety and is a practicing general dentist and also a university professor. D.B.R. is an orofacial pain expert, a seasoned researcher, and a regional director of the National Dental Practice–Based Research Network. Panelist 2 is a practicing general dentist and a university professor. D.W. is an assistant dental director with extensive experience in quality-related dental research. Calibration was done utilizing the probable AE cases obtained from the pilot phase of the project. Panel members were encouraged to make decisions on AE determination by using the established guidelines (Appendix A [http://links.lww.com/JPS/A128]) and their clinical experience/judgment. Consensus panel meeting sessions were facilitated by M.W., a university professor who is versed in dental patient safety and adept at guiding conversations toward achieving consensus. Four of the 5 panel members participated in the review of the pilot cases.

Statistical Considerations

For the main (postpilot) phase of the DPS, our sampling fraction will be 1 in every 24 patient charts. This assumes an AE proportion of 2.5% (SE, 0.005), which is based on the assumption that AE occurrence will be lower in outpatient settings than what is reported in hospitals. Our primary measure is to estimate the proportion of dental care–seeking individuals who experienced an AE within the year of study. We will secondarily estimate (1) the proportion of AE types according to the classification scheme described previously, (2) the incidence of AEs normalized by the number of visits the patient had in the previous year, (3) which procedures are most often associated with AEs, and (4) the rate of occurrence of more significant harm (severity scale categories E2, G2, H, I, shown in Fig. 1). It was also determined that if oversampling of certain racial/ethnic groups was needed we would make weighted corrections to estimates via linear transformation that maps the proportion back to the original population.

Beyond obtaining estimates, we will consider the respective associations between the several independent variables representing priority populations with (1) rate of AEs, defined as “percentage of individuals who experienced an AE within the year of study” and (2) the mean or median (as appropriate) number of AEs per individual. We will secondarily assess the respective associations between the independent variables and (1) the rate of more significant harm, classified as significant harm, an intervention required to sustain life, or death (severity scale categories E2/G2, H, I, shown in Fig. 1), and (2) rate of AEs/more significant harm normalized by the number of visits the person had in the previous year.

The occurrence of at least 1 AE is our primary outcome of interest. It is a binary response-dependent variable where either an AE has occurred or it has not within the calendar year. The proceeding categorical data analysis will include the following tests:

Age, race, and insurance status (categorical).

In order to determine whether there are statistically significant differences in the proportion/yr (incidence rate) of AEs within age categories (quartiles) or race categories, a χ² test of association will be utilized. The χ² test is 1-tailed and will be conducted at a significance level of α = 0.05 as is the standard. P < 0.05 on this test would yield evidence that there is a significant difference within the rate of AEs among different age or race categories, respectively. To compare the number of AEs, we will use the analysis of variance or the Kruskal-Wallis test, as appropriate.

Sex and Hispanic ethnicity (binary).

In order to determine whether there are statistically significant differences in the proportion/year (incidence rate) of AEs by sex group and Hispanic ethnicity, respectively, a 2-sample z test of proportions will be the method used. This test will be 2-tailed and conducted at a significance level of α = 0.05 as is the standard. P < 0.05 on this test would yield evidence that there is a significant difference in the rate of AEs by sex or ethnicity, respectively. The χ² test and the 2-sample z test for proportions are statistically equivalent when both the dependent and independent variables are binary. To compare the number of AEs, we will use the t test or the Mann-Whitney U test, as appropriate. Finally, we will conduct exploratory multivariate logistic regression analysis, with AE occurrence as the dependent variable and the remaining variables (age, race, ethnicity, insurance status, and sex) as independent variables.

RESULTS

Performance of the Automated EHR Data Extraction Script

During the pilot phase, chart reviewers verified that the prepopulated information extracted by the SQL script was complete and accurate. The result of this verification exercise is shown in Table 1. Manual review was considered the criterion standard. At all sites, the script did not perform perfectly in extracting race, insurance, and ethnicity information. Site 3, additionally, had some inaccuracies in counting the number of dental procedures each patient underwent within the study year.

TABLE 1.

Concordance Between Script and Manual Review

	Site 1	Site 2	Site 3
Race	74%	49%	93%
Ethnicity	85%	46%	100%
Insurance type	75%	72%	95%
Count of completed procedures	100%	100%	83%

Our race/ethnicity categorization is consistent with the revised National Institutes of Health standards and contains 5 minimum categories for race: American Indian or Alaska Native, Asian, black or African American, Native Hawaiian or Other Pacific Islander, and white. There were also 2 categories for ethnicity: “Hispanic or Latino” and “not Hispanic or Latino.” There were hundreds of insurance types, but we recoded the information as private insurance (eg, employer-sponsored insurance), government-sponsored insurance (eg, CHIP, Medicaid), and out-of-pocket payment. The count of completed procedures was organized by specialty (CDT categories)—diagnostic procedures, preventive procedures, restorative procedures, endodontic procedures, periodontics procedures, and so on. For this count, the script had perfect concordance with manual review except for site 3 (Table 1).

AE Estimate/Sensitivity of the 2-Reviewer Chart Audit Process

In the study year, 18,970, 10,088, and 83,131 patients were seen at the study sites. At sites 1, 2, and 3, reviewers abstracted 21 (1 of which was discovered by both reviewers), 11 (2 of which were discovered by both reviewers), and 23 (10 of which were discovered by both reviewers) probable AEs, respectively, from the 100 patient charts audited per site. At site 2, a third reviewer audited the same 100 charts and found only 1 additional probable AE. Of the total 56 probable AEs (from 300 charts), the panel confirmed 9 AE cases. This equals 3 AEs per 100 patients per year. As shown in Table 2, patients with an AE tended to be male and older and to have undergone more procedures in the study year. Table 3 itemizes the list of AEs observed from the pilot study.

TABLE 2.

Comparison of Patient Characteristics

	Patients With AE	All Patients
M:F	2:1	2:3
Average age, y	45	41
Average no. procedures	6.3/person per year	4.7/person per year
Average no. procedures (nonexam)	3.6/person per year	2.8/person per year

TABLE 3.

List of Observed AEs

	AE Observed	Dental Procedure
1.	Severe tachycardia and light-headedness—procedure had to be terminated and postponed	Dental filling
2.	Persistent traumatic ulcer	Lower partial denture
3.	Development of cold sores after traumatic dental procedure	Dental implant surgery
4.	Persistent bleeding (for days)	Dental extraction
5.	Oral soft tissue laceration from loose wires	Orthodontic procedure
6.	Inability to swallow	Dental anesthesia
7.	Peri-implantitis accompanied by severe bone damage	Dental implant surgery
8.	Chronic trauma to tongue from margin of dental restoration	Dental filling
9.	Inadvertent trauma to soft tissue remote from surgical site	Dental extraction

Panel Calibration

The distribution of the panel members’ assessments can be seen in Figure 3. Of the 56 probable AEs identified by the chart reviewers and forwarded to panelists for independent assessments, 25 had straightforward agreements, of which all agreed 2 were AEs, and 23 were not. Of the 31 for which they did not reach initial consensus, 24 had a majority agreement (3:1) and 7 were split equally (2:2) among the 4 participating panel members.

FIGURE 3.

Initial agreement among panelists.

DISCUSSION

DPS Methods

The World Health Organization prescribes a process for research to identify solutions for enhancing patient safety and reducing harm to patients (Fig. 4). The process is divided into a number of stages that describe the actions required and may also be used to assess the stage of development of a country/practice in the area of patient safety.²⁷ The first step—measuring harm—refers to “counting how many patients are harmed or killed each year and from which types of AEs.” This exploration of the nature and scale of harm to patients serves to draw attention to harm caused by health care systems. Dentistry is yet to achieve this first step as there have been no published studies that report on the rates of AE occurrence associated with dental procedures in the United States. Without such evidence, we cannot begin to understand causes, identify solutions, and translate increasing knowledge into safer care. The present study hopes to start filling this knowledge gap and activate the sequence of events that lead to improved and safer care.

FIGURE 4.

The research cycle: measuring harm. Adapted from the World Health Organization (WHO). World Alliance for Patient Safety. Research for Patient Safety: Better Knowledge for Safer Care. Geneva: WHO; 2008.

The various methods that have been used for estimating the frequency of health care–related AEs include prospective cohort studies, malpractice claims analysis, retrospective chart reviews, global trigger tools, and reporting systems. Michel and colleagues,²⁸ in a 2004 comparative analysis, discussed 3 of the most popular methods—prospective (data collected during hospital stay), retrospective (data collected after discharge), and cross-sectional methods (data gathered for a specific day). They concluded that the retrospective method is more appropriate for estimating rates of AEs, and the prospective method should be preferred for describing root causes and consequences of AEs. Other researchers have compared retrospective chart audits with incident reporting systems and also establish that reporting systems grossly underestimate both the scale and severity of AE occurrence.²⁸ Retrospective review, perhaps because it relies on the written history of patients’ experiences and provides a longitudinal view not available through any other method, is therefore often considered the best method for identifying AEs. Because of the alignment of the strengths of retrospective review with our project goals, we elected to utilize chart reviews for the accomplishment of our project goals.

Retrospective review, nonetheless, has its criticisms; documentation in patient records may be incomplete, allowing some AEs to escape notice; it is frequently labor-intensive and time consuming, and it is often difficult to disentangle the contribution of health care intervention from the underlying disease processes. Thus, even with a carefully structured review process, there is substantial variation in the judgments of reviewers in studies that have used this method.²⁹ Although it is impossible to fully mitigate these challenges, in the DPS, our process was carefully tailored to attenuate their effect.

First, we use 2 reviewers (instead of one) to do the initial screening of all selected patient charts in order to minimize the likelihood of missing a probable AE. Along the same lines, instead of giving the chart reviewers a list of specific criteria to look for (as with the HMPS and most related studies), we broadened the scope of what should be considered a probable AE by instructing reviewers to ask the 2 screening questions referenced in METHODS. Additionally, as one of the participating sites allows us to concurrently review the medical charts of selected patients, we hope to be able to catch AEs that might be missed because the patients presented to the emergency room/medical outpatient center rather than to a dental center.

Second, in order to mollify the significant variations in judgment regarding what might/might not be an AE, we decided to use a panel instead of 2 clinicians for AE determination. Attaining consensus by committee is a well-documented and effective method that hinges on the fact that collective decision making by experts is better than individual decision making.³⁰ Panel selection was designed to include participants representative of various dental specialties and practice settings. To ensure thoroughness, the postpilot phase of the project will also include 2 stages of panel review. The first will involve independent reviews of probable AEs to make AE determination, and the second involves committee deliberations to discuss cases in which the independent assessments vary. Decision making at both stages will be guided by the guidelines (Appendix A [http://links.lww.com/JPS/A128]), to ensure consistency. These guidelines, however, will be amended and improved upon as the panel encounters cases that validly challenge their basis. A summary of how our methods compare with the HMPS methods is shown in Table 4.

TABLE 4.

Comparison of the HMPS and DPS Methods

	HMPS	DPS
Study setting	Hospital	Outpatient dental clinics
Site selection	Stratified random sampling within 1 state	Convenience sampling involving 3 states
Chart selection	Random sampling with oversampling and undersampling for certain groups	Simple random sampling with oversampling for certain groups
	Sampling fraction: 1 in 85	Sampling fraction: 1 in 24
Study process	Retrospective chart reviews	Retrospective chart reviews
• Design	2 Nurses	2 Nurses/dentists
• First-stage review	1 Physician	Expert panel
• Second-stage review	Manual review of paper-based treatment notes	Review of EHR treatment notes (with some automation of data extraction)
• Method of data collection
AE definition	Unintended injury due to medical intervention leading to death, disability, or prolonged hospital stay	Physical harm associated with dental treatment within a time frame relevant to the clinical scenario
Screening criteria for reviewers	18 Screening criteria21	2 Screening criteria

Pilot Run

As shown in Table 1, the computer script developed to extract data from the EHR initially had challenges in pulling out the requisite information. This was largely due to vastly different documentation practices among sites and also among individual practitioners within the same site and some chart reviewer training issues. Computer queries function best when information is consistently entered into designated EHR fields. Based on the initial run of the script across the sites, and drawing from our success in using similar computer scripts to abstract demographic EHR information in relation to other studies with very high PPV and sensitivity, we were able to modify the script to obtain complete information. Although there was not a lot of overlap in the probable AE cases identified by both reviewers at each of the sites, the decision to use 2 reviewers was not to assay for degree of calibration; rather, it was to ensure that no probable AEs are missed. And as this pilot run also suggested, the incremental value of using an additional third reviewer is negligible. Previous AE studies showed poor to moderate interrater agreement for the determination of AEs, but as shown in Figure 3, at least 3 of the 4 panel members who participated in the pilot phase had initial agreement for 49 of the 56 probable AE cases.

The rate of AEs observed in this preliminary run is similar to those reported in medicine,³¹ but the severity was generally less, as anticipated. Using our classification and severity schemes, we determined that most of the AEs were associated with “soft tissue injury” and temporary. Pilot studies, however, do not provide a meaningful effect size estimate for planning subsequent studies because of the imprecision inherent in data from small samples.³² As such, these initial estimates are neither representative nor generalizable, and we refrain from overanalyzing our findings. The purpose of conducting this pilot study was to examine the feasibility of the approach that is intended to be used in the larger-scale study.

Dentistry, as with other health care specialties, has been afforded the privilege and obligation of self-government.³³ This implicitly comes with the responsibility of vigilantly maintaining high practice standards and fostering a culture that promotes the safety of patients. Understanding the frequency and nature of AEs will assist in prioritizing prevention strategies and research efforts aimed at improving dental care. Such knowledge will also inform the design of patient safety programs and the formulation of safety-related performance indicators and guidelines. Dental care spending represents 4% of overall health care expenditures. Out-of-pocket spending for dental services accounts for 40% of dental spending, and private health insurance accounts for 47% of dental spending.^34,35 These all mean that minimizing the wastage that results from errors and AEs has potential significant implications for cost savings, especially for patients.³⁵

CONCLUSIONS

This article presents an overview of the DPS. It describes the methods used and summarizes the results of its pilot phase. To minimize threats to patient safety, a starting point is to understand their basic epidemiology, both in terms of their frequency and the extent to which they affect different populations.