TREATMENT PLANNING IN DENTISTRY USING AN ELECTRONIC HEALTH RECORD: IMPLICATIONS FOR UNDERGRADUATE EDUCATION

BACKGROUND/ LITERATURE REVIEW

Treatment planning (Figure 1) is the process of formulating a rational sequence of treatment steps designed to eliminate disease and restore efficient, comfortable, esthetic and masticatory function to a patient (1).

Figure 1.

Using information obtained from the patient history, clinical examination and diagnostic tests, the clinician discriminates between pertinent and non-pertinent information in order to arrive at diagnoses. This ability to discriminate is one of the key differences between experts and beginners (2) and highlights the importance of proper and targeted education in taking novices through competence towards expertise.

When developing a treatment plan, the provider should follow a general phasing or sequencing format designed to solve the patient’s dental problems in a way that first manages the patient’s emergent concerns (e.g. pain and infection). The next step is disease (e.g., caries and periodontal disease) removal and tooth restoration, then tooth replacement and reconstruction. Once these priorities have been met, esthetic and cosmetic concerns are addressed and lastly, preventive and maintenance measures are included in a comprehensive treatment plan. This approach enables division of the treatment into several phases. Any given phase may contain several individual procedures, some of which may be sequenced in a specific order(3) .

Most dental educators and accreditors consider treatment planning an essential element for both student education, and provision of dental services (4). In spite of its importance, however, treatment planning has received little attention in the dental literature and there appears to be no consistent format being followed in the teaching and development of treatment plans within dental school curricula (5, 6). In one study on using standardized patients to assess presentation of a dental treatment plan, the investigators reported that 94% of dental students obtained a complete set of information from the patients, but the same students were not so successful in identifying the patient’s goals for treatment - only 81% of students were successful (7). A 1984 survey of 70 American and Canadian dental schools reported that most dental schools offer preclinical treatment planning information and also develop clinical treatment plans but that there were no curricular guidelines devoted to the principles of dental treatment planning (8). In a more recent study (5)conducted on treatment planning in dentistry in 2001, the investigators sent out questionnaires (on questions relating to treatment plan preparation, process and outcomes) to the clinical administrators of 54 U.S. dental schools with instructions for the questionnaires to be forwarded to the faculty member most familiar with the details of treatment planning in each school. Results from 47 schools revealed that schools typically screen patients prior to assignment, and then expect the student-clinician to complete the planning process. Additionally, the comprehensive plan was found to be most strongly influenced by patient needs and requests rather than students’ curricular requirements in the overwhelming majority of schools.

Despite the contributions of these earlier studies, no investigation, to our knowledge, has been carried out to explore the subject of treatment planning since the advent of electronic health record (EHR) systems use in dentistry. Traditionally, provision of clinical care in the dental setting had consisted of the clinician, the patient, and a paper record. EHRs have been shown to increase safety through evidence-based decision support, quality management, and outcomes reporting (9–11). As treatment planning has become embedded in the prevailing technology, students think through patient presentations, arrive at diagnoses, and plan treatments using EHRs. Thus, although the principles of treatment planning remain, the clinical environment is changing and dental education should be cognizant of and keep pace with these changes.

Technology has revolutionized medical and dental care delivery in many ways with a direct influence on how patients are cared for. It, however, represents a fundamental shift in the demands of the care environment – a shift which can impact the efficiency and effectiveness of clinical personnel (12) , hence, our foray into the subject.

In this paper, we approach the topic of treatment planning in the context of EHRs while also looking critically at the unexplored process of, and link between, selecting a diagnosis and planning a treatment within the EHR framework. We do this by evaluating and reporting on how dental students from two U.S. schools performed when asked to complete diagnosis and treatment planning tasks in an EHR. Our aim did not include comparing results between students from the participating schools.

MATERIALS AND METHODS

Participant recruitment and consent

Permission to carry out the research was obtained from each participating institution’s designated IRB (Institutional Review Board). An invitation was e-mailed out to eligible third and fourth year clinical students at the Harvard School of Dental Medicine (HSDM) and University of California, School of Dentistry, San Francisco (UCSF). All the students recruited and included in this study were actively involved in provision of dental care to patients, had been using an EHR (axiUm, Exan, Vancouver, BC, Canada) for at least six months, and had all completed at least two years of training at the time of enrollment including specific training in treatment planning using an EHR. Students who indicated interest in participating gave either an electronic or paper-based consent after receiving comprehensive information describing the research protocol.

The treatment planning process in axiUm

axiUm ® (Exan Corporation., Vancouver, BC, Canada) is a dental institution management software used by the majority of dental schools in the United States. The EHR module in axiUm is the electronic equivalent of the patient's chart, allowing for all clinical information, including treatments and forms to be stored in one paperless database. The axiUm EHR includes a treatment plan module developed by dental educators and thought leaders and follows the treatment planning philosophy of Stefanac et al .

Within the UCSF and HSDM dental students’ clinics, treatment planning is carried out using the EHR. There are two distinct ways to plan a treatment in axiUm. The first involves the ‘treatment planning’ module. Here, students create a comprehensive treatment plan by stepping through the module following a standard process in which a student (a) enters the patient’s problems/complaints; (b) selects the appropriate diagnoses from a comprehensive list; (c) enters the treatment objectives, which represent the intent or rationale for the final treatment plan, usually expressed as short statements and articulate clear goals from both the student’s and patient’s perspective; and (d) enters a detailed plan for each of the selected diagnosis (Figure 2). Following treatment planning, the student obtains instructor approval and patient consent before beginning treatment.

Figure 2.

Treatment planning can also be done more quickly for returning patients or patients with very few uncomplicated complaints through the ‘Chart Add’ function in axiUm. Using this ‘chart add’ function, a diagnosis can be selected from a list, and the corresponding treatment is selected and added immediately. In this study, we tested participants’ ability to properly select diagnoses and plan for treatment using both the ‘treatment planning module’ and the ‘chart add’ function in axiUm using two patient based cases (clinical scenarios).

The clinical scenarios and data collection

Two clinical scenarios were developed by our research workgroup to present the participants with typical patient cases seen in everyday dental practice. The tasks to be completed ranged from emergency palliative care (basic) to dental rehabilitation using fixed partial dentures (relatively complicated). Scenario 1 was a relatively straightforward case with a single diagnosis and students were asked to use the ‘chart add’ function (Appendix 1). Scenario 2 was more complicated and required that multiple diagnoses and procedures be entered into the EHR using the ‘treatment planning’ module (Appendix 2). The original patient scenarios were drafted by BT and reviewed by EK and JW for clinical content. Requisite changes were incorporated and the revised cases were reviewed by MW for usability and clarity. The second revision was then presented to the entire research team and approved by members for use.

A team of cognitive scientists from the University of Texas at Houston administered the exercises. Each participating student was provided with the two test patients (clinical scenarios) in axiUm and was required to select the appropriate diagnoses and make a comprehensive treatment plan in axiUm. Participants were also asked to ‘think-aloud’ while completing the task so as to gain understanding of their thought and reasoning processes during problem-solving and decision making. No time limit was given for completion of the exercises. Each student completed the tasks individually under standardized conditions, with an investigator unobtrusively in attendance. Every student’s interaction with the EHR was recorded while s/he attempted to complete treatment planning for the clinical scenarios. A video recording of the computer screen of the participants was made using the Morae Recorder Version 3.2 (Techsmith). Recordings were subsequently downloaded for evaluation of the students’ entries.

Evaluation of diagnosis selection and treatment plans

The assessment of students’ entries into the EHR was performed by three independent and calibrated authors, who are dentists and members of the dental faculty. The dentists were trained in use of the diagnostic terms, have practical knowledge of standard dental practice, including the American dental treatment codes (Code on Dental Procedures & Nomenclature) (13), are experienced users of the EHR and are actively involved in student teaching and evaluation – two are clinical departmental heads and the last is a professor in a clinical dental department. Each evaluator reviewed and graded the completed exercises for each study participant by reviewing screenshots from the downloaded video recording of every student’s interaction with the EHR. The evaluators independently assessed every entry (diagnosis/treatment/treatment plan sequence) as correct, incorrect or missing. The evaluators scored diagnosis entry, diagnosis-treatment pairing, and sequencing of treatment. Every correctly-entered diagnosis was given a score of 2, and every correctly-entered diagnosis-treatment pair or sequencing was given a score of 1. No points (0) were given for incorrect, incorrectly paired, incorrectly sequenced or missing entries. The maximum obtainable score (cumulative) for the first and second scenarios were 6 and 46 points, respectively. Scores were then converted to percent correct and grouped by scenario, diagnostic category, and school. Subsequently, the evaluators met to adjudicate their assessments when complete agreement was not obtained, resulting in consensus scores for each participant. This consensus was used for the final marking scheme. The individual observers’ ratings were compared with the adjudicated scoring system to determine the degree of agreement (Kappa). The results are reported as means (with standard deviations) and level of significance was set at 0.05. All analyses were performed using SAS, Version 9.2 (SAS Institute, Cary, NC).

RESULTS

The overall Kappa was 0.53 and this indicates moderate agreement. For the UCSF data, the Kappa was 0.54. For HSDM, the Kappa was 0.52. For UCSF, each evaluator’s Kappa with the adjudicated results ranged from 0.47 to 0.66. For Harvard, each evaluator’s Kappa with the adjudicated results ranged from 0.41 to 0.86.

A total of twenty-five students participated in the study, twelve from HSDM and thirteen from UCSF. Composite results and results from each school (by diagnostic category), blinded as School 1 and 2, are presented in Table 1. Overall, the participants earned 48.2% of the possible points available. Participants at School 2 earned a mean (SD) of 54.3% (12%) vs. participants at School 1, who earned a mean (SD) of 41.9% (13.2%). For Scenario 1, the participants (combined HSDM and UCSF) earned 51.4% of the possible points available. In situations whereby respondents selected the correct diagnosis, the correct treatment was also selected 93.4% of the time (57/61). However, when the respondent did not select the correct diagnosis, the correct treatment was selected only 81.4% of the time (48/59). This difference was statistically significant (p=0.045)

Table 1.

	Overall scores		School 1		School 2
	Mean	SD	Mean	SD	Mean	SD
Total	48.2	13.8	41.9	13.2	54.3	12.0
Task 1	51.4	32.9	47.2	30.8	55.6	35.8
Task 2
Periodontitis	34.1	14.0	31.1	11.9	37.1	15.7
Gingivitis	68.3	27.0	65.0	32.1	71.7	21.7
Partial Edentulism	73.5	25.6	65.9	26.3	81.1	23.4
Caries Risk	23.8	18.4	15.5	13.2	32.1	19.6
Primary Caries	67.5	26.3	58.3	24.8	76.7	25.4

Overall, the participants earned 41.9% of the potential points. At School 2, the mean (SD) was 37.5% (19.5%) vs. 46.4% (17.4%) at School 1 (p=0.2494). For Treatment, the overall percent correct was 59.8%. At School 2, the mean (SD) was 64.3% (8.1%) vs. the participants at School 1, which scored a mean (SD) of 55.4% (5.4%). For Phasing and Sequencing, participants earned 41.7% of the total possible points overall. For School 2, the mean (SD) was 55.2% (11.9%) vs. School 1, which had a mean (SD) of 28.1% (21.5%).

Very little pertinent information was obtained with respect to participants’ thought processes from the ‘think-aloud’ approach.

DISCUSSION

We report on how twenty-five students from two American dental schools performed with regard to selecting the most appropriate diagnosis, treatment and sequencing of treatment in a dental EHR system. In many cases, participants scored less than half of the maximum points attainable (Table 1). Students generally fared better selecting the appropriate treatment(s) compared to choosing the correct diagnoses but performed least favorably when organizing the sequence of their treatment plans (Table 2).

Table 2.

	Overall scores		School 1		School 2
	Mean	SD	Mean	SD	Mean	SD
Diagnosis	41.9	18.6	46.4	17.4	37.5	19.6
Treatment	59.8	8.1	55.4	5.4	64.3	8.1
Phase and Sequence	41.7	21.9	28.1	21.5	55.2	11.9

Although there is no previous study with which we may compare our results, the participants’ performance highlights the need to appraise the current process by which treatment planning is taught in U.S. dental schools and also to consider the impact of technology on the fundamental skills of diagnosis and treatment planning within the modern educational setting.

As noted earlier, technology has influenced medical and dental care delivery in many ways with a direct effect on how patients are cared for. It promises to reduce the need for paper patient entries, to allow for the transfer of patient records among providers and to assist with clinical decision support and compliance with standards of care guidelines (14). It, however, does represent a fundamental shift in the demands of the care environment – a shift which can impact the efficiency and effectiveness of clinical personnel (12).

The skill of treatment planning has heretofore been taught in the context of paper records in which student diagnosticians write out their findings and treatments in a free-text format. EHRs, conversely are most powerful in the context of structured data (15) , thus, there is a greater emphasis on the entry of coded values for diagnoses and procedures. Within axiUm, students are now expected to select the most appropriate diagnosis from a pre-populated list and then choose the most appropriate treatment code to pair with each of the diagnoses. The need to select a diagnosis from a pre-populated list and also to pair a specific diagnosis to its appropriate treatment is a marked shift in paradigm, a shift that should be acknowledged in the content of the training provided to students.

Looking more closely at our results, students generally scored higher in Scenario 1, with a mean score of 51.4%, as compared to Scenario 2, with a mean score of 34.1%. There are a number of potential explanations for this observation. First and perhaps most obvious is that the disparity in performance may be simply because Scenario 1 was a relatively more straightforward, single diagnosis case of pericoronits which required only emergency palliative treatment. This explanation is particularly plausible because when a patient presents with multiple complaints and/or has many significant findings on examination, it is more difficult for students to arrive at accurate diagnoses and formulate an appropriate treatment plan, as they will need to quantitatively sift through a lot of information to do this. Fewer complaints present a less stiff challenge and students generally do better in such situations.

Considering the role of technology, another possible reason for the better scores of students in Scenario 1 is that they were required to use the quicker and less challenging ‘chart add’ function in axiUm. A third reason may be that students generally perform better when working with some diagnostic-treatment categories than others.

The ‘chart add’ function in axiUm, as explained earlier, is a simpler way of adding a diagnosis and planning treatment in axiUm and as a consequence, students find it easier using ‘chart add’ as opposed to when utilizing the treatment planning module. It is important to note that although ‘chart add’ is quicker and easier to use, it is not always the better way for students to select a diagnosis and plan a treatment within axiUm. This is because students differ from competent or expert dentists not only in their limited abilities to quantitatively sift through data but also the qualitative aspects of their reasoning processes (16, 17). When clinicians with experience are diagnosing routine cases, they usually use knowledge structures and reasoning processes that are developed by continuous exposure to patients (18, 19). This type of reasoning (described as a ‘forward reasoning process’) allows them to obtain a precise diagnosis more rapidly and as such, the ‘chart add’ function in axiUm is a useful tool to help maximize efficiency in this group of people. It is however not always as useful as the ‘treatment planning’ module when the provider is a student– in which case, the student steps through a sequential process of entering the requisite information, arriving at a diagnosis, and planning treatment for each of the diagnoses and in so doing, completes the same task as the expert, but in a slower and stepwise manner with a minimal chance of omitting salient points in the patient complaint and clinical findings that have a bearing on the appropriateness and quality of the treatment plan.

Thirdly, even though students performed better overall in Scenario 1, when the results from Scenario 2 were divided into the different diagnosis categories, students across the board earned the most points for the ‘Gingivitis’, ‘Partial Edentulism’, and ‘Caries’ diagnoses. This raises the possibility that students may be better at selecting some diagnoses (and planning their subsequent treatments) than others. This disparity in students’ ability to choose some diagnoses over the others could be due to the way in which the diagnoses are categorized in axiUm; and the familiarity of the students with some diagnoses or how to find them within axiUm – an effect directly attributable to the technology in use.

Additionally, students generally fared better selecting the appropriate treatment(s) compared with choosing the correct diagnoses, raising the question of how important it is to select the right diagnoses since clinicians may still provide the right care, given an incorrect diagnosis. Studies have however shown that improving diagnostic abilities has positive effects on treatment decisions, and treatment outcomes (20). This assertion is further corroborated by the present study. In situations whereby respondents selected the correct diagnosis, the correct treatment was also selected 93.4% of the time (57/61). However, when the respondent did not select the correct diagnosis, the correct treatment was selected only 81.4% of the time (48/59). This difference was statistically significant (p=0.045).

Furthermore, the lack of a holistic view in approaching the clinical scenarios was a recurring theme in the treatment plans drawn up by the participants. Preventive and maintenance measures such as caries risk assessment, oral hygiene instructions, nutritional counseling and re-evaluation of periodontal status were frequently missing from the students’ plans. Therefore, another salient point to emphasize from this work is that prevention is part of a complete treatment plan, as treatment should improve the trajectory of oral health rather than simply react to existing problems (21, 22).

Finally, it is also important to add that the performance of students at carrying out these tasks may have been influenced by the interface to the ‘technology’ (EHR). Some of our future work will attempt to evaluate the impact of the design of the EHR on the ability of students to successfully carry out treatment planning related tasks.

Our study has a number of limitations which may influence the interpretation of its findings. First, the body of information presented here was obtained from twenty five participants from two dental schools and they were recruited by convenience sampling. There is, therefore, the possibility of selection bias, in which case the results will not be generalizable to the entire dental student body in the United States (23). Secondly, although the investigator who administered the test scenario cases was just in quiet attendance and no time constraint was given, students’ performance may have been negatively influenced relative to their typical performance. This is because students are generally more familiar with selecting diagnoses and planning treatments unobserved. Third is the fact that we were unable to gather information through the ‘think-aloud’ process and this limited our ability to better understand the reasoning processes of the students as they worked through the EHR. Although we observed a statistical difference in the overall performance of students between the two schools, this work involves only a limited number of participants and so, we are not comfortable speculating that this difference is as a result of the difference in teaching approach employed by both schools or the differing years of experience of axiUm use in both schools. Finally, agreement among the evaluators was only moderate (overall kappa, 0.53), and as this could reflect genuine differences in professional judgment, it is important not to make overreaching conclusions from the results.

Despite these limitations, this work contributes as the first investigation to quantitatively report on student treatment planning performance in dentistry in the context of EHRs. Our results highlight the importance of explicitly including technology considerations within diagnosis and treatment planning pedagogy and evaluation. Findings from such can help identify educational gaps. Our results indicate, for instance, that students at School 1 could benefit from additional attention to treatment phasing and sequencing, whereas students at School 2 might benefit from further instruction in diagnosis selection (Table 2).

Given the growing integration of technology into clinical care, its impact on student education and on students’ development of the requisite expertise merits consideration. Future efforts should build upon this work by including a larger number of schools, using randomly selected samples and performing additional qualitative analyses of the data.

Appendix 1

Scenario 1: Please read the following and carry out the tasks listed toward the bottom of the page

African-American female, age (23 yrs)

Clinical Vignette:

CC: I think my wisdom tooth is just erupting and it is quite painful

Medical History: Patient is pregnant - first trimester; Peptic Ulcer Disease, last episode was about 7 Years ago; no allergies

Dental history: Previous dental visits for fillings; Previous extraction (about 5 years ago) after failed RCT on the same tooth; extraction was atraumatic and post-op period was uneventful: Previous history of brief orthodontic treatment (about 10 years ago), cannot remember the details

Meds: None

Supplements: Multivitamins

Family History. Father has bronchial asthma; one of her elder siblings had all last molars extracted

Social History: ETOH - one glass occasionally

EO: Body temperature (measured at the armpit) 100°F; Submandibular lymph nodes on the right side are enlarged and slightly tender. Otherwise, EOE is wnl

IO: Slight pain on month opening, Mild odor from the mouth; Oral hygiene is good. Soft tissues look normal but gingiva appears swollen and inflamed in the region of #32, mild plaque accumulation in the same area. Shallow and mildly tender traumatic ulcer on the lateral aspect of the tongue adjacent to the grossly carious #30

Missing teeth: #1, 13, 16

Partially erupted: #17, 32

Decay. #30 grossly carious

Composite: #3 MO

Overbite (vertical overbite): About 25%

Perio exam: Probing depths are wnl; Unable to properly examine the inflamed area on the posterior aspect of the mandible on the right side due to local tenderness

Tasks

For this patient with pericoronitis, please do the following:

Using “chart add”, please do the following tasks, putting them in as completed procedures:

• Periapical Xray or #32 including the appropriate diagnostic code

• Emergency palliative treatment on #32s including the appropriate diagnostic code

Appendix 2

Scenario 2: Please read the following and carry out the tasks listed towards the bottom of the page

Caucasian male, age (57 yrs)

Clinical Vignette:

CC: “ I want to replace my missing tooth”

Just got a role as a talk show host at work and is concerned that the space left behind by the missing tooth will show when he smiles.

Medical History: Had chickenpox as a child, and underwent an appendicectomy about 12 years ago, no post-op complications

Dental history: Previous extraction of mobile tooth over 20 years ago. Tooth became very mobile after impact during a fight. Has had previous fillings. Stopped regular dental check-ups about 2 years ago due to busy work schedule

Meds: Aspirin 75mg/day, Diazepam 5mg/day (occasionally, to combat insomnia)

Supplements: None

Family History: No significant family history

Social History: ETOH - 2 glasses/day; does not take tobacco in any form

EOB: Normal

IO: Oral hygiene is fair, mild accumulation of plaque and calculus. Patient has a midline palatal torus

Missing teeth: #1, 12, 16

Decayed: # 18 occlusal with decay to the DEJ

Filled: #14, 19 amalgam

Periapical Xray: Coronal radiolucency on #18, extending to the DEJ.

Perio exam: Generalized chronic marginal gingivitis.

Probing depth: #18 mesial surface – 5mm

• #19 distal surface – 5mm

• Otherwise, wnl

Tasks

Please prepare a treatment plan using the “treatment module”, including the appropriate diagnostic codes and terms:

• Caries Risk Assessment and appropriate treatment

• Prophylaxis

• Limited Scaling and root planning (deep cleaning around #18, 19)

• Restoration #18

• Fixed partial denture (bridge) to replace #12 with abutments #11 and #13

Appendix 3 The correct responses

Problem	Diagnoses	Treatment
TASK 1
Partially Impacted third molar	Pericoronitis	Intra-oral 1st PA film
Partially Impacted third molar	Pericoronitis	Emergency Palliative or Short visit problem focused
TASK 2
Periodontitis	Localized Moderate Chronic Periodontitis	Scaling/root planing 1–3 teeth
		Oral Hygiene Instructions
		Reevaluation perio charting (= 1 month eval)
Gingivitis	Plaque Induced Gingival Disease with local contributing factors	Prophy - adult
Partially edentulous	Partial Anodontia, acquired	Crown - porcelain to high noble
		Crown - porcelain to high noble
		Pontic-porc fused to high noble metal
		Bridge Drawing Bar
Caries	Caries Risk High	Nutritional counseling
		Caries risk assessment
		Caries prevention packet
		Topical fluoride Varnish
Caries	Primary Caries at DEJ	Composite 1 surface etching/ bonding
		Amalgam - 1 surface (remove Tx points)