Lessons Learned from OpenNotes Learning Mode and Subsequent Implementation across a Pediatric Health System

Abstract

Background  The 21st Century Cures Act has accelerated adoption of OpenNotes, providing new opportunities for patient and family engagement in their care. However, these regulations present new challenges, particularly for pediatric health systems aiming to improve information sharing while minimizing risks associated with adolescent confidentiality and safety.

Objective  Describe lessons learned preparing for OpenNotes across a pediatric health system during a 4-month trial period (referred to as “Learning Mode”) in which clinical notes were not shared by default but decision support was present describing the upcoming change and physicians could request feedback on complex cases from a multidisciplinary team.

Methods  During Learning Mode (December 3, 2020–March 9, 2021), implementation included (1) educational text at the top of commonly used note types indicating that notes would soon be shared and providing guidance, (2) a new confidential note type, and (3) a mechanism for physicians to elicit feedback from a multidisciplinary OpenNotes working group for complex cases with questions related to OpenNotes. The working group reviewed lessons learned from this period, as well as implementation of OpenNotes from March 10, 2021 to June 30, 2021.

Results  During Learning Mode, 779 confidential notes were written across the system. The working group provided feedback on 14 complex cases and also reviewed 7 randomly selected confidential notes. The proportion of physician notes shared with patients increased from 1.3% to 88.4% after default sharing of notes to the patient portal. Key lessons learned included (1) sensitive information was often present in autopopulated elements, differential diagnoses, and supervising physician note attestations; and (2) incorrect reasons were often selected by clinicians for withholding notes but this accuracy improved with new designs.

Conclusion  While OpenNotes provides an unprecedented opportunity to engage pediatric patients and their families, targeted education and electronic health record designs are needed to mitigate potential harms of inappropriate disclosures.

Background and Significance

With the enactment of the 21st Century Cures Act, patients are positioned to have unprecedented access to their clinical data in electronic form, especially for clinical notes written by health care providers. 1 The OpenNotes initiative now lists over 50 million patients as having access to their notes, and this count has likely increased since the Cures Act compliance deadline in April 2021. 2 OpenNotes has the potential to engage pediatric patients and families in a novel way that improves understanding, trust, adherence to treatment recommendations, and ultimately outcomes. 3 Adolescent patients largely understand clinical concepts relevant to their health, and report high satisfaction scores in regard to note information when given access. 4 The Society of Adolescent Health and Medicine and the North American Society for Pediatric and Adolescent Gynecology support improved access for adolescents to their own health information. 5 6 Note sharing can also improve family members' abilities to assist in the care process, ensure continuity between providers, identify errors, and promote advocacy. 7 8 9 10 OpenNotes may be especially important for engaging people from some racial and ethnic minority groups, as well as less educated patients. 11

Pediatric health systems face special challenges in the implementation of OpenNotes. For example, Bourgeois et al describe ethical issues unique to pediatrics such as an adolescent patient disclosing information about sexual activity that they want withheld from their parents, disclosures or tests indicating nonpaternity, and concerns that parental decisions may be harming children. 3 The Information Blocking rules provide explicit exceptions to note sharing such as privacy and preventing harm but these exceptions have narrow definitions. The use of privacy exceptions heavily depend on state laws for adolescent consent to treatment, and these vary substantially across the country. 12 Early reports of OpenNotes in pediatrics suggest many preimplementation concerns from physicians do not materialize after implementation, similar to what has been observed for adults. 13 However, most published studies in pediatrics have focused on limited implementations, primarily in ambulatory settings, with relatively low note reading rates. 14 The scope of the 21st Century Cures Act rules extend beyond well-studied areas into sharing of inpatient and emergency department notes to patient portals with little to no delay. Further clarity is required on practical challenges associated with note sharing implementations in these settings, strategies to educate providers on appropriate documentation techniques, and electronic health record (EHR) design approaches to encourage best practices where education alone may not suffice. 15

At Children's Healthcare of Atlanta (CHOA), educational materials and EHR interventions were prepared in anticipation of the original compliance date for OpenNotes in November 2020. The Office of the National Coordinator (ONC) for Health Information Technology delayed the compliance date to April 2021 due to the novel coronavirus disease 2019 (COVID-19) pandemic. OpenNotes Learning Mode was then implemented at CHOA from December 2020 to March 2021, during which time notes were not shared by default to the patient portal, but the key EHR interventions were retained including just-in-time education at the top of commonly used note templates describing OpenNotes, confidential note types (note types not released to the patient portal), and easy access to links for feedback from a multidisciplinary OpenNotes team for challenging situations ( Fig. 1 ).

Fig. 1.

A written reminder (denoted here by box with asterisk) was added to certain note templates to remind providers about OpenNotes and note sharing to the patient portal, with links to guidance documents and to request an audit for feedback from the working group.

Objectives

In this best practice paper, we aimed to develop a just-in-time education system to help providers adapt to the OpenNotes requirements in the 21st Century Cures Act which involves unique challenges for pediatric health systems. We describe lessons learned in the development and implementation of such a system, as well as early results postimplementation of OpenNotes to inform EHR design and educational efforts.

Methods

Setting

This study was performed at an urban pediatric health system in the Southeastern United States that contains three freestanding children's hospitals with hybrid academic and community models, as well as comprehensive outpatient pediatric primary and specialty care. The EHR (Epic Systems, Verona, Wisconsin, United States) is integrated across the system with all clinical documentation performed directly in the EHR. State law dictates that parents of adolescents provide consent for care and that the parent has primary access to health information until the age of 18 years; exceptions where an adolescent may consent to their own medical treatment absent parental knowledge and/or consent include for pregnancy, pregnancy prevention, treatment of sexually transmitted infections, or substance abuse. 16 17 18 19 The health system's patient portal policy allows parents and legal guardians to obtain full proxy access to their child's medical records, with adolescents aged 13 to 17 years also able to obtain their own patient portal account with the parent's consent. However, technical constraints currently do not allow confidential notes to be shared with the adolescent without also sharing them with the parent/legal guardian. In addition, certain sensitive diagnoses and laboratory results that may be restricted from disclosure to parents/legal guardians by state law have been suppressed automatically from all patient and proxy portal accounts. For patients aged 12 years and under, only the parent/legal guardian may obtain a patient portal account.

The institution's OpenNotes working group included stakeholders from clinical, legal, compliance, technical, and administrative teams. This interdisciplinary group met twice per week to address operational concerns associated with the 21st Century Cures Act and information blocking. The team had initially planned a go live of OpenNotes for November 2020 corresponding with the original compliance date from the ONC. On notification of a delay in enforcement, the working group decided to adjust the implementation to incorporate OpenNotes Learning Mode in December 2020 to provide clinicians with an opportunity to become more familiar with OpenNotes and gain practice with EHR modifications and policy changes in advance of the federal reporting requirements deadline in April 2021. The institution subsequently went live with OpenNotes in March 2021, including sharing by default for most physician notes and notes written by certain other types of providers (e.g., excluding nursing and social work notes, among others). In addition, notes written by the Child Advocacy team (i.e., child abuse pediatricians), as well as notes with a department specialty of “Research,” were also excluded.

Learning Mode

The intent of Learning Mode was to encourage learning at the individual level (i.e., practicing documentation as if OpenNotes was live, using confidential notes when appropriate, adjusting note templates as needed, and obtaining feedback) and at the system level (identification of common themes and documentation errors, iterative EHR design, and educational efforts). Learning Mode consisted of a set of technical and workflow changes within the EHR, feedback for difficult situations identified by physicians, and auditing by the working group of randomly selected notes that the author deemed should be withheld from the patient portal. Of note, during the Learning Mode period, the “Share with Patient” button causing notes to be visible in the patient portal was not selected by default, although physicians could manually click this button for their note to cross to the patient portal.

Educational Text

Educational text was added to the 300 most commonly used note templates by providers including a written reminder that beginning in Spring 2021, most clinical notes would become visible to the patient/family in the patient portal shortly after signing ( Fig. 1 ). It stated that certain sensitive information should be documented in a separate confidential note type and contained a link to a more comprehensive guidance document describing appropriate use of the Preventing Harm and Privacy exceptions, as well as instances when providers may seek verbal consent to have a confidential conversation with a minor patient from the patient's parent/legal guardian. It also included a link that providers could use to tag the note and request an audit by the working group for feedback. The educational text was intentionally designed to require active removal prior to signing the note to increase its visibility and encourage providers to document as if OpenNotes was live.

Confidential Note Type

A new confidential note type was created that did not include a button option of “Share with Patient.” Confidential notes were not visible to the patient or parent/legal guardian through the patient portal but were visible internally to other clinical providers. Consistent with prior studies, 20 the confidential note type was intended to be used by providers to document sensitive details that fall within defined exceptions for information sharing with the patient or their legal guardian, while nonsensitive portions of the encounter were to be documented separately within a normal note type to be shared with the patient or their legal guardian. When selected, the confidential note type presented default text including instructions at the top that would disappear automatically on signing the note, as well as a field for the provider to select justification for withholding the documented information from the patient portal ( Fig. 2 ). In discussion with legal and compliance teams, justification was allowed for the Preventing Harm exception, the Privacy exception, or if the legal guardian provided verbal consent for a confidential conversation with the patient. Once signed, this confidential note type was visible to clinical users, with all confidential notes for a patient easily found by clinical users through a filter within the EHR notes and chart review activities. Such confidential notes did not cross to the patient portal or to any response from the health information management department to a request for medical records, thereby retaining the confidentiality of adolescent medical records allowable under state or federal law.

Fig. 2.

A pick list was iteratively designed to specify rationale for withholding a note. (1) The initial list design displayed all three reasons, with verbal consent at the bottom. (2) In the second design, the verbal consent reason was moved to the top of the list. (3) The third design utilized a cascading pick list, with verbal consent and “Other reason” visible at first; if “Other reason” was selected, the Privacy and Preventing Harm exceptions then became visible to select (see arrow).

Feedback

The working group continued to meet on a weekly cadence throughout Learning Mode and after the institution went live for OpenNotes in March 2021 to review operational issues and made iterative improvements to the technical design. The group also reviewed confidential note feedback requests and questions from providers; one of the authors took notes on the discussions which were then formulated as feedback and circulated among the working group and to the requesting provider. If all feedback requests were completed at the time of a working group meeting, then the working group reviewed a randomly selected subset of signed confidential notes for design purposes and to validate whether the technical and workflow changes implemented were being used effectively by providers.

OpenNotes Go Live

Beginning in March 10, 2021, the “Share with Patient” button was selected by default for all physician notes and certain other provider types except when the author had a specialty of Child Advocacy and certain other predetermined departments. Other elements of Learning Mode were left the same or with adjustments as described in Lessons Learned below. There was no specific educational outreach to patients or families regarding this new functionality.

OpenNotes Usage

All eligible signed provider notes (excluding nonproviders, e.g., social work and nursing) for patients of any age were extracted through queries of the EHR database. The frequency of note sharing, note reading by families, and use of confidential notes prior to Learning Mode (July 1, 2020–December 2, 2020), during Learning Mode (December 3, 2020–March 9, 2021), and after OpenNotes went live (March 10, 2021–June 30, 2021) was compared. Notes were assigned to a group based on the note date of service.

Results

The total number of provider notes signed, shared, and read by families are presented over time in Fig. 3 . In the 5-month period prior to Learning Mode, a total of 531,344 provider notes were signed of which 2,158 (0.4%) were shared, yielding an average of 13.9 notes shared per day. Of these, 598 notes were read by families (27.7% of shared notes and 0.1% of all notes), yielding 3.9 notes per day read by families across the system. During Learning Mode, 4,337 (1.3%) of 336,900 provider notes were shared or 44.7 notes shared per day. However, only 505 of these notes were read by families (11.6% of shared notes and 0.1% of all notes), yielding 5.4 notes read per day across the system. Since OpenNotes went live, 395,172 (88.4%) of 447,244 provider notes were shared of which 26,671 were read by families (6.7% of shared notes and 6.0% of all notes), yielding 236.0 notes read per day across the system. During all three periods, 11% of the time that a note was signed, there was an associated active patient portal account. It was not possible in this implementation to determine the identity of the actual reader of the note, such as the patient or their caregiver.

Fig. 3.

Run chart of provider notes, notes shared, and notes read by the patient/family over time, with data aggregated per week.

There were a total of 2,589 confidential notes written, 779 (8.0 per day) during Learning Mode and 1,810 (16.0 per day) since OpenNotes went live. Of these, 304 (11.7%) were written for patients younger than 13 years, 1,643 (63%) were written for patients between 13 and 17 years of age, 581 (22%) were written for patients ≥18 years old, and 66 notes were written for patients without a documented age. Confidential notes were written across 31 different specialties ( Fig. 4 ), with the highest volume coming from the following four specialties: (1) pediatrics ( n  = 1,723), (2) pediatric emergency medicine ( n  = 320), (3) adolescent medicine ( n  = 116), and (4) inpatient psychiatry ( n  = 102), with other specialties having lower confidential note counts. The working group received 14 requests from clinicians for feedback on confidential notes, and also reviewed a random subset of seven confidential notes to evaluate how clinicians were using the confidential note type and to refine the initial technical design based on institutional usage patterns.

Fig. 4.

Confidential note distribution by provider specialty.

Discussion and Lessons Learned

Through the implementation and review of OpenNotes Learning Mode, several key themes were identified which informed iterative improvements to the EHR technical design and clinical workflows. Broadly, the working group's review of confidential notes showed that many providers were correctly documenting nonsensitive details in their main notes which are released to parents or legal guardians and patients with patient portal accounts, while documenting sensitive details, such as a pediatric HEADSS history, 21 within a separate confidential note. However, there remained certain scenarios where patterns of errors in documentation were identified.

Sensitive Information in Shared Notes

Autopopulated Data Elements

First, note templates often incorporate autopopulated elements from elsewhere in the EHR such as laboratory results, problem lists, and diagnoses. In several notes, sensitive data, such as pregnancy or sexually transmitted infection testing, were automatically populated and visible in the main note instead of within a separate confidential note ( Fig. 5 ). Providers may not be aware of or thoroughly review all of the results that may autopopulate within note templates, and this could lead to sensitive information being inappropriately disclosed to the parent/legal guardian. Providers and departments should be encouraged to review their note templates to modify or remove autopopulated elements as necessary. These note template reviews have the added benefit of reducing or eliminating so-called “note bloat” and should also be coordinated with institutional billing teams when necessary to ensure an appropriate balance between retention of information required for billing purposes and the need for confidentiality.

Fig. 5.

Autopopulated data elements in note templates, such as laboratory results or problem list sections, can often include sensitive data.

In addition, each patient note should be reviewed and any sensitive autopopulated data removed prior to note signing and release to the patient portal. This is necessary because, even after streamlining note templates, there are some laboratory tests which may infrequently return sensitive information (e.g., a synovial fluid culture that grows Neisseria gonorrhoeae ) that may be prohibited to put into nonconfidential note types per state law.

Differential Diagnoses

Second, trainee (i.e., fellow physician, resident physician, and medical student) notes often included very detailed and lengthy differential diagnosis lists. While these are reasonable for education purposes, some relatively improbable differential diagnoses may cause undue concern to the patient or their family. Prior research has shown that parental confusion and anxiety are concerns associated with the sharing of clinical notes for pediatric patients, though in other cases, patient and parent sentiment regarding shared notes and laboratory results has been shown to be largely positive after implementation. 22 23 24 To mitigate risks, these differential diagnosis note sections also should be reviewed more thoroughly and shortened, if clinically appropriate, prior to note signing. It is particularly difficult at an institutional level to educate trainees on these documentation best practices due to high turnover rates during rotations, especially for subspecialty trainees who spend most of their time outside of pediatric settings.

Furthermore, in the initial implementation, resident physician notes did not always require cosignature by a supervising physician and could be automatically released to the patient portal without supervising physician review. All resident physician notes were updated to require cosignature by a supervising physician prior to release to the patient portal.

Supervising Physician Note Attestations and Cosignatures

Third, while many notes that were reviewed had sensitive history and examination details partitioned into a separate confidential note, some primary notes still included sensitive details within the supervising physician note attestation or addendum. The implementation of educational text did not appear in attestations since the original author, often a trainee, had to delete this educational text prior to signing their note. A technical solution to provide this guidance during the attestation process was not identified and, therefore, this risk was highlighted in additional education for supervising physicians. Additionally, requiring a separate supervising physician attestation for the confidential note encourages and may remind supervising physicians to avoid including confidential information in the main note attestation.

Incorrect Reasons for Withholding Notes

Representation of the Exceptions Affects Accuracy

Across both the review of feedback requests, as well as randomly selected notes, providers were frequently selecting an incorrect reason for withholding a confidential note from release. The Preventing Harm and Privacy exceptions have narrow definitions and use cases under state and federal law, and these exceptions were being inappropriately selected in many scenarios. After identifying this trend during Learning Mode, the pick list for exception rationale within the confidential note template was modified ( Fig. 2 ), reordering such that “legal guardian verbally consents to confidential conversation with patient” was at the top of the list, as there was suspicion that it may have been missed visually by providers as it was previously at the bottom of the list. However, even with this initial change, providers continued to sometimes incorrectly select the Preventing Harm or Privacy Exceptions during Learning Mode. To further improve this workflow, a cascading pick list was subsequently created with only two initial options: (1) a verbal consent for a confidential conversation, and (2) “Other Reason”; if a provider selected the “Other Reason,” the Privacy and Preventing Harm exception choices would then become visible. The text for each pick list option was also modified to include a short educational instruction for when each exception is applicable (in addition to the preexisting link to the tip sheet to explain these exceptions in more detail). After this second change, the working group perceived an improvement in the appropriateness of the reason a confidential note was written in their audits and in their clinical practice.

Deletion of Reason for Withholding Note

In some notes that were reviewed, the provider had manually deleted the pick list requesting the reason for withholding the confidential note which impairs proper documentation for compliance purposes. To encourage providers to include their reason for withholding the note, the text “Required” was added to the beginning of this pick list. In current state within the Epic EHR, there exists functionality that if the “Share with Patient” button is deselected and the note signed, the system can provide a hard stop requiring justification. However, for confidential note types without a “Share with Patient” button, this functionality does not exist. This functionality should ideally be developed and offered directly by EHR vendors for confidential note types.

Exceptionally Sensitive Encounters

Finally, a small subset of notes that were reviewed illustrated rare cases of exceptionally difficult encounters to document in the context of OpenNotes. In one such scenario, a note feedback request was received from a provider for an encounter where there was a high suspicion of abuse or human trafficking, for which release of any details of the encounter, such as notes, results, or problem lists, may lead to harm to the patient. In these scenarios, the working group felt that the best application of the Preventing Harm exception was to exclude the entire clinical encounter and associated information from release to the patient portal. The working group then created an ability for providers to file a data element in a note to exclude an entire encounter from the patient portal. A clinician could enter an autotext (.EXCLUDEENCOUNTER) into a note; signing that note with a discrete reason for exclusion would file the data element which was referenced in filters preventing the encounter from displaying in the patient portal. However, this feature was enabled only within emergency department or urgent care settings as appointment or hospital details may have been previously available within the patient portal in other, nonemergent care settings. Retroactive exclusion of encounter details in those other nonemergent settings may lead to deletion of data that was previously in the patient portal which may increase risks of patient harm.

Limitations

This analysis is focused on a single institution and these results were not compared with other organizations, and as such, they may not be generalizable due to differences in culture and workflow. Additionally, this approach was tailored to state law and unique internal policies, and legal variances in note sharing requirements at the state level may require additional consideration. The lessons described in this study are based principally on the discussion and impressions of this OpenNotes working group. A sufficient number of confidential notes were not reviewed to make a quantitative estimate of the impact of these changes on documentation practices. Nonconfidential notes, except for a subset from encounters in which an associated confidential note was also written by the same provider, were not reviewed to determine if they contained confidential information, as this was outside the scope of our work. The scope of the educational text intervention may also have been limited, as it was only added to 300 commonly used note templates. For a subset of notes, which were neither shared nor of the confidential note type, the reason the note was not shared was unable to be identified. Additionally, there was no explicit maintenance plan for these interventions after the conclusion of Learning Mode. Design patterns were specific to current regulations, and will likely need to be modified as regulation change.

Conclusion

Through OpenNotes Learning Mode, a just-in-time education system built into the EHR with technical modifications was found to help providers adapt to the new changes mandated by the 21st Century Cures Act. Nonetheless, iterative design changes and educational efforts have been required to improve compliance, reduce inappropriate disclosures, and adhere to both the letter and the spirit of the law to improve patient and family engagement through OpenNotes.

Changing to default sharing of notes led to a clear increase in the number of notes shared and read by patients and families. While the proportion of shared notes read by families decreased as more notes were shared, it has subsequently trended upward since OpenNotes went live in this system in March 2021. As more families become familiar with the Cures Act and patient portals, the demand and consumption of OpenNotes are likely to increase. This trend is likely to lead to increased patient and family engagement but may also uncover new challenges. Future work is needed to determine the impact of OpenNotes on documentation practices, particularly for sensitive topics such as sexual histories, as well as the impact of EHR design changes and specific educational interventions on compliance, patient and family engagement, adherence to treatment recommendations, and patient outcomes.

Clinical Relevance Statement

Implementing a just-in-time education system within the EHR can encourage documentation practices that comply with the information blocking rules in the 21st Century Cures Act. Several common pitfalls in confidential documentation practices can be avoided through careful user interface and workflow design. Balancing confidentiality and transparency in documentation is especially challenging in pediatric health care settings where complex legal and ethical challenges are common.

Multiple Choice Questions

1. Which of the following actions was described as a strategy to lower the risk of accidental disclosure of confidential information in trainee notes?

   1. Restricting all trainee notes from being shared to the patient portal.

   2. Displaying pop-up guidance when keywords containing sensitive topics are typed.

   3. Requiring that supervising physicians cosign trainee notes prior to release.

   4. Relying primarily on educational modules during trainee onboarding to encourage usage of confidential notes.

   Correct Answer: The correct answer is option c. Requiring cosignature on all trainee notes by supervising physicians allows trainees to continue participating in the documentation process, enables supervising physicians to provide an additional layer of oversight to avoid accidental disclosures of confidential information, and provides transparency for patients to review these notes through the patient portal. Detection of sensitive keywords was not utilized in this study, and relying solely on onboarding processes for training is unlikely to be effective due to its passive and one-time nature.

2. Which of the following has been shown to be an effective mechanism to collect the rationale for why a user created a confidential note?

   1. Free-text fields within the note.

   2. Cascading pick list with educational prompts.

   3. Random auditing of signed notes at regular intervals.

   4. E-mail-based surveys requesting rationale for withholding information from the primary note.

   Correct Answer: The correct answer is option b. A cascading pick list with educational prompts allows for more common selections to be prioritized and provides just-in-time guidance for clinicians. Free-text fields can be difficult to parse during compliance reviews, and random auditing of signed notes is a retrospective rather than prospective process to encourage good documentation. E-mail-based surveys are impractical as they would require users to leave the EHR.