Actionable Recommendations in the Bright Futures Child Health Supervision Guidelines

Summary

Background

With the growing use of electronic health record systems, there is a demand for an electronic version of the leading American pediatric preventive care guideline, Bright Futures. As computer implementation requires actionable recommendations, it is important to assess to what degree Bright Futures meets criteria for actionability.

Objectives

We aimed to 1) determine the number of actionable recommendations in the current edition of Bright Futures and 2) to recommend a specific format for representing an important class of guidelines in a way that better facilitates computer implementation.

Methods

We consolidated all action statements in Bright Futures into recommendations. We then used two dimensions (decidability and executability) in the Guideline Implementability Appraisal v 2.0 (GLIA) to determine the actionability of the recommendations. Decidability means the recommendation states precisely under what conditions to perform those actions. Executability means actions are stated specifically, unambiguously and in sufficient detail. The results were presented in a figure titled Service Interval Diagram (SID), describing actionable recommendations, age intervals during which they are applicable, and how frequently they should occur in that interval.

Results

We consolidated 2161 action items into 245 recommendations and identified 52 that were actionable (21%). Almost exclusively, these recommendations addressed screening, such as newborn metabolic screening, or child safety, such as car seat use. A limited number (n=13) of recommendations for other areas of anticipatory guidance were also actionable. No recommendations on child discipline, family function or mental health met our criteria for actionability. The SID representing these recommendations is presented in a figure.

Conclusion

Only a portion of the Bright Futures Guidelines meets criteria for actionability. Substantial work lies ahead to develop most recommendations for anticipatory guidance into a computer implementable format.

1. Introduction

The American Academy of Pediatrics’ (AAP) Bright Futures: Guidelines for Health Supervision of Infants, Children and Adolescents, 3rd edition (Bright Futures) is the most widely accepted guideline for pediatric preventive health care in the U.S. The guideline was first published in 1994 and is funded by the Department of Health and Human Services, Maternal and Child Health Bureau [1]. Bright Futures was developed by AAP members in multidisciplinary expert panels and an evidence panel, and reviewed both within and outside the AAP. Bright Futures consists of a multitude of health supervision recommendations for children from birth through 21 years of age. With the growing use of electronic health record (EHR) systems among pediatricians, there is a demand for an electronic version of Bright Futures and the AAP is supportive of such development. Although there are known barriers to computer implementation of guidelines to overcome [2–4], an electronic version of Bright Futures presents an opportunity to support clinician decision making.

Our group has experience with pediatric computerized clinical decision support from the CHICA (Child Health Improvement through Computer Automation) system used in our pediatric clinics since 2004 [5–7]. One requirement for computer implementation of a guideline is actionable recommendations [8]. Actionable recommendation statements are both decidable (i.e., every condition is described clearly enough so that reasonable practitioners would agree on the clinical circumstances for which the recommendation should be applied) and executable (i.e., the recommended action [what to do] is stated specifically and unambiguously).

Whether a guideline is actionable or not can be assessed through a formal and validated guideline assessment tool such as the GuideLine Implementability Appraisal v 2.0 (GLIA) [9]. GLIA was developed by Shiffman et al in 2005, and identifies obstacles to the implementation of guidelines [10]. The instrument has been previously used for a variety of recommendations with multiple publications attesting to its usefulness [11–14] and its ability to assess implementability [15]. The most recent version of GLIA consists of one global dimension (assessing the guideline as a whole) and eight dimensions to assess factors intrinsic to the guideline that may affect successful implementation. The eight dimensions are executability, decidability, validity, flexibility, effect on process of care, measurability, novelty/innovation and computability. Decidability and executability are considered the most critical criteria for overall computer implementation by the instrument developer. The computability criteria itself is both setting and electronic health record specific and comes into plays only when the decidability and executability criteria are met.

An advantage of computer implementation of guideline recommendations is that it can generate clinical decision support using a child’s age and gender and by tracking what services have already been delivered to the child [6]. The current version of Bright Futures is not organized to easily implement this capability in a computer [1]. Instead, recommendations are listed according to what should happen at each of the 21 pre-defined health supervision visits. This organization is familiar to most pediatricians through the “periodicity schedule” published by the AAP [16]. A problem with this format is that it assumes that the child will be seen for health supervision at each age and that previous visits have been completed. However, children may miss or delay visits. A child scheduled for a well-child visit may present with an illness or concern that takes up most of the visit time, leaving some health supervision topics unaddressed. If a clinician is not able to address a recommendation at the 4 months visit, he or she has to consider whether to do so at a later visit. Moreover, if an issue has been addressed at one visit, it may not be necessary to address it at subsequent visits. For all these reasons, it would therefore be of great value if recommendations in an electronic version of Bright Futures were based on the age at which the child presents, not based on the specific predefined health supervision visit the child is due for. For example, if a clinician was shown what services had already been delivered, and which ones had not yet been delivered but were age-appropriate at the given visit, an electronic version of Bright Futures could support the clinician at any health supervision encounter. These challenges apply to a broader class of guidelines that recommend specific actions at particular ages, times, or stages of disease, and which recommend those actions take place at certain intervals.

2. Objectives

An electronic version of the recommendations in Bright Futures is desired. In this study we aimed to 1) determine the number of actionable recommendations in the current edition of Bright Futures and 2) represent actionable Bright Futures recommendations in a format that would show the age range over which the recommendation is applicable regardless of whether it corresponds to a standard health supervision visit.

3. Methods

3.1 Consolidation of Bright Futures Recommendations

An average of more than 100 action statements (actions) are listed for each of the 21 health supervision visits described in Bright Futures. These actions are organized in separate sections; 415 actions under “health supervision,” 319 actions under “physical examination,” 208 actions under “screening” (universal and selective), 1219 actions under “anticipatory guidance,” for a total of 2161 actions [1]. Many actions are repeated across health supervision visits or address different aspects of the same recommendation. There are, for example, many specific instructions presented as actions under the overall recommendation to use a car seat.

To reduce the number of redundant action statements, repeated actions were grouped, and smaller steps were grouped into the larger actions. We assessed each of the 21 health supervision visits in Bright Futures and grouped all remaining actions into discrete and often broad recommendations such as newborn screening, assessment of maternal health, assessment of school performance, assessment of sleep, physical activity, nutrition, and car seat/booster seat /safety belt use (Supplement Appendix). We excluded 1) items listed under physical examination, and 2) immunizations. Physical exam was considered a separate part of the encounter unlikely to be considered optional by clinicians. Immunizations guidelines are already specified by the Centers for Disease Control and Prevention (CDC) and the AAP and are continuously revised. The schedules are not incorporated in the printed version of Bright Futures.

3.2 Application of two GLIA dimensions to Bright Futures

In this study, we defined an actionable recommendation as one that met the two GLIA dimensions decidability and executability, both highly desirable criteria for useful decision support [9]. All GLIA criteria are valuable for improving guidelines, but it is impossible to create computer implementable decision rules if the guideline statements are vague on the decidability and executability criteria.

Moreover, if these two criteria are not met, assessment of the remaining criteria becomes very difficult. For example, validity (including quality of evidence) cannot be assessed without knowing what is recommended (executability) for whom and when (decidability), and the same is true for assessing effect on process of care, measurability (adherence and identification of outcomes), and novelty. Also, the flexibility criteria actually require that the decidability criteria are met before one can assess whether the guideline also allows for alternative interpretation, i.e. is flexible). Lastly, as described, the computability criteria require knowledge about the specific setting and electronic health record used for the implementation. For these reasons, we decided that the assessment of actionability (as defined by the GLIA criteria decidability and executability) was the most important first step in the development of an electronic version of Bright Futures.

As each health supervision visit in Bright Futures includes several sections about the visit context, priorities, health supervision and anticipatory guidance, we assessed all sections of the guideline text to determine decidability and executability. For example, one item at the 4 month health supervision visit is to “remind parents about proper car safety seat use and the importance of putting the infant in a rear seat of the vehicle.” For this example, we determined the decidability criteria met, because Bright Futures makes it clear parents of all 4 months old children need this recommendation. However, if analyzed by just reading this section separately, that action is not described in enough detail to meet the executability criteria. Specifically, what is meant by “proper?” However, looking across all sections under the 4 month visit, there are very detailed descriptions of “proper” use such as not placing the car seat in front seat, age and weight for rear facing, etc. The recommendation on car seat use at the 4 month health supervision visit was, therefore, considered executable in our analysis. If a recommendation that spanned more than one visit (for example car seat use) was described in enough detail to be decidable and executable at least once in Bright Futures, we concluded that recommendation was decidable and executable, and thus actionable, overall.

Two authors (SMF, JLS) determined the decidability and executability of recommendations from each Bright Futures chapter independently. Because Bright Futures is organized by child age partly addressing the decidability criteria, both authors elected to assess this criterion first and eliminate recommendations that were not decidable. Only if the decidability criterion was met, did we assess the executability criterion. For each recommendation, any disagreement was resolved through discussion until a consensus of whether the actionability criteria (decidable AND executable) was met.

3.3 Design of Service Interval Diagram (Figure 1) for Actionable Recommendations

Fig. 1.

Service Interval Diagram

To represent actionable recommendations in a format that would show which recommendation applied to a child at a given age regardless of whether it corresponded to a standard health supervision visit, we developed an alternative representation we call a Service Interval Diagram (SID). (► Figure 1) This age based format has similarity to the CDC immunization schedules already well known by clinicians. We first introduced our version of this concept in the appendix of the AAP guideline for Health Supervision for Children with Down Syndrome [17]. The SID spans health maintenance recommendations for all ages from birth to 21 years, but represents the appropriate time for delivering services as a continuous interval. The SID was generated based on 1) the Bright Futures’ recommendations at a given visit and 2) the pediatric expertise from the two authors with clinical experience (SMF, SMD). As an example, in Bright Futures, selective tuberculosis screening is listed at the 1 month, 6 months, 12 months and 18 months visits, and annually from the 2 year visit. It is not listed at the 2 months, 4 months, 9 months and 15 months visit. However, the two authors with clinical expertise together concluded that the recommendation could be justified at all these ages, if it had not been addressed prior. In the SID, we therefore made a modification and extended the bar representing selective tuberculosis screening from 1 month to 21 years. In the SID, we indicated recommendations listed at a specific visit in Bright Futures and author generated age interval modifications separately. To illustrate what age interval modifications we made, the SID uses a bar to represent recommendations listed at a specific visit in Bright Futures, and hash marks represent age interval modifications made by the authors. This first version of the SID was developed with the typical health supervision visit in mind. Recommendations assuming age appropriate development, such as for example solid food introduction, were not modified for children with known developmental delay.

4. Results

By consolidating recommendations that were repeated across visits or consisted of many smaller actions, we reduced the total number of recommendations from 2161 to 245. The extracted recommendations are listed in the ► Supplementary Appendix. Of the resulting 245 recommendations we determined 52 (21%) to be actionable. These were included in the SID (► Figure 1). Almost exclusively, these recommendations addressed screening, such as newborn screening or hearing screening, or child safety, such as car seat use or poison prevention. A limited number (n=13) of recommendations for other areas of anticipatory guidance were also actionable. The actionable recommendations are described in more detail below.

4.1 Actionable Recommendations for Screening

In Bright Futures, there is a separate section addressing recommendations for universal and selective screening. For each health supervision visit, these recommendations are separated from the rest of the text in a table under the heading “Screening.” In our analysis, 100% of the recommendations (n=24) in these tables clearly met decidability and executability criteria, i.e. were considered actionable. The Bright Futures table defines whether the screening was universal or selective and for which age it was applicable. In addition, the table describes the risk assessments (as applicable), actions recommended for universal screening as well as selective screening when the risk assessment is positive. We determined all recommended universal and selective screening in Bright Futures to be actionable by our criteria. (► Figure 1)

4.2 Actionable Recommendations Concerning Injury Prevention

Recommendations on injury prevention are described in the section for anticipatory guidance under one or several separate subheadings, depending on the chapter. We concluded that 43% of these listed recommendations were actionable (n=17). There were, for example, decidable and executable recommendations for car seat/booster seat/safety belt use, safe sleep practices, including “back to sleep,” helmet use, and fire safety (► Figure 1). There were, however, many recommendations addressing injury prevention that were not actionable in their current form. The majority were recommendations for older children. For example, recommendations such as “learning to manage conflict nonviolently,” “avoiding risky situations,” “avoiding violent people,” “avoiding situations in which drugs and alcohol are readily available” were not described in enough detail for the provider to know specifically what to do (be executable).

4.3 Other Actionable Recommendations

Besides screening and injury prevention, other areas such as nutrition and lifestyle guidance, had only a limited number of actionable recommendations (nutrition = 21%, lifestyle guidance = 37%) (► Figure 1). We identified six actionable recommendations addressing nutrition. There were also seven actionable recommendations for lifestyle guidance such as sleep routine, toilet training, hygiene, oral health and limiting TV time. By contrast, we found no recommendations on discipline, family function and mental health to be actionable. Most often, these recommendations failed both the decidability and the executability criteria. In other words, it was not clear who should be screened for these types of issues, or what a provider should do if he or she identified a parent or child who needed further work up.

4.4 The Service Interval Diagram

Our SID lists all the 52 Bright Futures recommendations we determined to be actionable (► Figure 1). These recommendations are divided into four areas; screening interventions (n=22), injury prevention (n=17), nutrition (n=6) and lifestyle guidance (n=7). For the purpose of developing a format based on the age at which a child presents, not the visit, we made age interval modifications (as described under SID design in the Methods section) to 31 of the 52 actionable recommendations. For the areas of nutrition guidance and lifestyle guidance, it was necessary to make such age interval modifications to all recommendations but two (drug prevention and solid food introduction) (► Figure 1). The number of actionable recommendations at a given age ranges from 10 – 17 (strictly based on the content in Bright Futures) and 16–28 (if the age interval modifications are included).

5. Discussion

In this study we estimate the number of actionable recommendations in the Bright Futures Guideline. By merging duplicate recommendations at various visits and retaining only recommendations that are described in a decidable and executable way at least once in the overall guideline, we conclude that the number of actionable recommendations is limited (52/245, 21% of recommendations). Recommendations in areas believed to be important by many pediatricians and researchers today, such as discipline, family function and mental health, are not presented in an actionable way. Furthermore, the importance of specific recommendations for parent education has been documented [18–19], but the majority of parent education recommendations in Bright Futures are not specific enough to be actionable.

The complexities and vagaries of multiple AAP guidelines have been previously described. [20] Although clinicians likely already recognize that Bright Futures includes many actions under each recommendation, a consolidation to broader recommendations has not been previously described. More importantly, an electronic version of Bright Futures is desired in today’s era of electronic medical records and our study is the first to systematically assess the suitability of the Bright Futures guidelines for computer implementation.

Substantial modifications to Bright Futures will be required before it can be effectively implemented in computerized clinical decision support. Some recommendations in the current version of the guideline would become actionable by addressing the decidability criteria and better guide the clinicians in how to select the patients at risk. Others recommendations, including those addressing sensitive topics such as child discipline and mental health, also needs to be more executable. For many of these recommendations it is not clear what to actually do, for example whether to test the patient and if so with what instrument, or whether to refer etc.

Additionally, the Bright Futures chapters, listing health supervision visit action statements from age 1 week to 21 years, span 287 pages (page 289–575). [1] Yet, as we consolidated the action statements, those pages cover only 245 discrete recommendations (actionable and non-actionable). This is a consequence of the guidelines being organized by visit, which requires repetition of each recommendation for every visit where applicable. Moreover, because the same topic may be described differently at different visits, there is a risk of inconsistency across the document. We found the inconsistency among the descriptions of the same recommendation in different parts of Bright Futures vary in degree from slight to extreme. This inconsistency is possibly explained by how Bright Futures was authored; a given group of authors was responsible for writing recommendations for particular ages rather than writing particular recommendation topics across all age groups. Age based recommendations as well as standardization of wording for each recommendation would be useful for both a future electronic version and for interpretation by users of the Bright Futures books.

Lastly, it should also be recognized that the question of actionability may apply differently to implementation by people versus computers. However, at a minimum a statement must be decidable and executable by a human in order to be adapted to a computer. For example, car seat use at the 4 month visit has specific details that can be described about how the car seat should be used. These may be helpful to a human user, but it may not be immediately obvious how this would be implemented in a computer system. A free text description might be shown to a clinician about the topics to cover. A clinician might check off boxes for each item that was discussed such as “rear facing”, “age restrictions”, etc. This approach starts with a recommendation that is executable for the human clinician and can be adapted for implementation in any number of computer decision support scenarios.

Our study has limitations. Both the consolidation of Bright Futures into discrete recommendations and the application of the GLIA instrument required some judgment calls after author discussion. In addition, decidability and executability are highly desirable for implementation in a computerized clinical decision support system, but even such actionable recommendations are not necessarily recommendations supported by evidence and applying the validity dimension of GLIA (quality of evidence) was beyond the scope of this study. However, we believe a systematic approach to the guidelines, using GLIA, and independent assessments by two authors make the application as objective as possible. It is conceivable that consolidation of redundant recommendations impacted our actionability assessments, but because we considered a recommendation actionable overall if it was actionable anywhere in the guideline, we think we biased in the opposite direction.

Finally, although Bright Futures represents perhaps the most important collection of guidelines in pediatrics, it also represents a class of guidelines for which the SID format may be valuable. These are guidelines that recommend specific actions at particular ages, times, or stages of disease, and which recommend those actions take place at certain intervals. So our intent is not only to evaluate the Bright Futures guidelines in particular, but to also recommend a specific strategy for representing this important class of guidelines that better facilitates computer implementation.

6. Conclusion

Out of 2161 actions in the Bright Futures Guidelines, only 52 (21%) meet criteria for actionability according to the decidability and executability criteria in GLIA. With the increased use of electronic medical records, an electronic version of Bright Futures would be complementary but would require a systematic assessment of suitability for computer implementation. The computer implementable format of the anticipatory guidance in Bright Futures should require discrete recommendations (actionable and non-actionable), age based topics for consistency and completely standardized wording.