Practice Analysis and Determining the Knowledge and Skills Expected of a Pediatric Rheumatologist

Abstract

Objective

The scope of clinical practice of pediatric rheumatology has been difficult to define. The lack of definition prevents an accurate understanding of the knowledge and skills required of practicing pediatric rheumatologists. A practice analysis process was used with the goal of establishing a precise definition of clinical pediatric rheumatology practice. The definition of practice will improve training and the creation of relevant certification examinations.

Methods

A practice analysis approach used meetings with a representative panel of pediatric rheumatologists to create a practice analysis document (PAD) and a test content outline (TCO). Panel experience, entrustable professional activities, and the current TCO were used to guide the process. Surveys were administered to fellowship program directors (PDs) and a broader group of practicing pediatric rheumatologists to revise and validate the content of the documents.

Results

A PAD was created, including 14 categories of conditions diagnosed or managed by pediatric rheumatologists and eight domains of practice, with the tasks, knowledge, and skills required to perform these tasks. The survey of PDs (n = 10) indicated that the PAD content is important and useful. A TCO was created and consists of 18 domains used to define content areas to be assessed on certifying examinations. The survey of practicing pediatric rheumatologists (n = 127) indicated that the TCO domains are relevant.

Conclusion

A practice analysis process produced valuable resources for defining the clinical practice of pediatric rheumatology. The PAD and TCO can be used to develop more specific training curricula and to create relevant certification examinations.

INTRODUCTION

The scope of clinical practice of pediatric rheumatology is broad and includes multisystem diseases, inflammatory diseases affecting specific organs or anatomical sites (eg, arthritis, uveitis, autoimmune encephalitis), autoinflammatory conditions, and musculoskeletal pain syndromes. In addition, the continuing discovery of novel inflammatory diseases and advances in our understanding of immunopathology is resulting in an expansion in the number and variety of patients referred to pediatric rheumatologists for consultation and management. As a result, it has remained difficult to precisely define the breadth and the limits of pediatric rheumatology clinical practice (1, 2). The lack of a precise definition prevents an accurate understanding of the knowledge and skills required and expected of practicing pediatric rheumatologists.

SIGNIFICANCE & INNOVATIONS.

• A practice analysis process was used to define the knowledge and skills required for safe and effective pediatric rheumatology practice.

• Defining the practice of pediatric rheumatology is essential to effectively train pediatric rheumatologists and to develop certification methods that assess the critical knowledge and skills required for practice.

• The product generated by the practice analysis process will provide guidance to fellowship training programs and will be very useful to certifying boards.

The lack of a consensus definition of clinical practice limits the ability of pediatric rheumatology fellowship training programs to develop curricula that will prepare individuals adequately for independent practice. Although programs have benchmarks that were developed by the Accreditation Council for Graduate Medical Education (ACGME) (3) and the American Board of Pediatrics (ABP) (4), these benchmarks are too focused on programmatic requirements or are too generic. Thus, their use in ensuring that individual trainees receive the education necessary to practice pediatric rheumatology remains limited (5, 6). Training programs have historically developed curricula and experiences independently and without guidance or consensus regarding the specific knowledge and skills that are needed to ensure the competence of individuals completing the training program.

Additionally, a clearer definition of the scope of practice of pediatric rheumatology is essential as the ABP evaluates candidates for certification. Currently, the ABP uses a board certification examination and confirmation of the successful completion of a fellowship training program to certify candidates. Defining the scope of practice more specifically and completely will be helpful in ensuring that board certification exams assess the knowledge required to competently practice pediatric rheumatology.

This issue of practice definition is not unique to pediatric rheumatology. Other subspecialties have recognized a similar need to specify their scope of practice. As a result, the ABP has developed a process for analyzing the practice of pediatric subspecialties (7). This process results in a comprehensive description of the professional responsibilities and activities of a subspecialist and identifies the knowledge and skills required to carry out these responsibilities and activities. Defining the scope of practice more specifically and completely will be helpful in ensuring that board certification exams assess the knowledge required to competently practice pediatric rheumatology (7). Practice analysis of several pediatric subspecialties has been previously described (8, 9, 10). The purpose of this manuscript is to delineate the process used to analyze the practice of pediatric rheumatology and to describe the products that resulted from this analysis. The intent is for the ABP to reference these products to create certifying examinations. In addition, fellowship training programs may use these products to guide the creation of appropriate curricula.

MATERIALS AND METHODS

An approach previously used for other pediatric subspecialty practice analyses was used (7, 8, 9, 10). A series of meetings was held with a representative panel of pediatric rheumatologists to create two products: a practice analysis document (PAD) and an updated pediatric rheumatology test content outline (TCO). The PAD includes a list of rheumatic diseases, clinical conditions associated with rheumatic diseases, and musculoskeletal conditions commonly evaluated by or managed by pediatric rheumatologists; a list of tasks that are performed by pediatric rheumatologists as they evaluate and manage the diseases and conditions on the generated list; and the knowledge and skills needed to safely and effectively perform the identified tasks. The updated pediatric rheumatology TCO contains an organized list of the topics that are assessed on the ABP pediatric rheumatology examinations and the percentage of questions that are related to each topic. For each document, a survey was administered to a broader group of practicing pediatric rheumatologists to validate the content within the document and make revisions using feedback received.

Panel recruitment

The ABP emailed a survey to 335 certified pediatric rheumatologists to solicit interest in participating on the practice analysis panel. Of these, 30 (9%) responded. Referencing the ABP Pediatric Physicians Workforce Data Book (11), the ABP used survey responses to select 14 individuals who represented the practice setting, sex, race and ethnicity, geographic location, and age demographic of the pediatric rheumatology community (11). Two individuals were selected because they held positions as pediatric rheumatology fellowship program directors (PDs). Also, by design, two incumbent members of the ABP Pediatric Rheumatology Subboard (12), the group responsible for writing and reviewing certification examination questions, were selected.

The mean age of panel members was 49 years (SD 8.0 years, range 35‐60 years), and panel members had an average of 12 years of practice experience (SD 7 years, range 2‐25 years). Eight members self‐identified as White (57%), three as Asian (22%), one each as Hispanic (7%), and Middle Eastern (7%), and one preferred not to answer (7%).

Panel activities

The panel met virtually for a series of 10 meetings held over an 8‐month period. At the initial meeting, the practice analysis process was reviewed. The PAD was drafted in meetings 2 through 5. Panel experience, the pediatric rheumatology entrustable professional activities, and the current TCO were used to guide the discussions. Other educational resources, including textbooks, journals, and clinical guidelines, were consulted when needed. Consensus was reached by majority opinion. Once the draft PAD was completed, a survey was developed for PDs, who were asked to provide importance scores (not at all important [1], moderately important [2], extremely important [3]) on each of the rheumatic conditions and PAD domains, along with open‐ended feedback on any condition, task, knowledge, or skill which was missing, or which should be removed. Additional survey questions included: 1) Overall, how would you rate the usefulness of the PAD for PDs? and 2) Overall, how would you rate the usefulness of the PAD for developing a fellowship training curriculum? Optional response scores for these questions were 1) not at all useful, 2) somewhat useful, 3) moderately useful, and 4) extremely useful. Likert scales were used in these surveys to be consistent with the survey method used in a previously published practice analysis (10). The PAD was then finalized during the sixth meeting using information from the PD survey to guide revisions. The panel used the subsequent three meetings to create a TCO. Using the PAD and the current TCO, the panel created an updated draft TCO that comprised 14 content domains (similar to domains in the PAD in which areas of similar content are grouped together) based on the PAD list of rheumatic diseases, rheumatic disease associations, and musculoskeletal conditions. Four nonclinical content domains were included in the TCO to address generic topics relevant to practicing pediatric rheumatologists. A similar survey to the one used for the PAD was distributed to all board‐certified pediatric rheumatologist diplomates. The survey asked diplomates to provide a relevance score (not at all [1], slightly [2], moderately [3], or very relevant [4]) for each specific topic and to provide open‐ended feedback on any content area which was missing, or which should be removed. Diplomates were also asked to recommend the percentage of total questions (weights) related to each high‐level content domain that should be included on pediatric rheumatology certification examinations. At the 10th and final panel meeting, the TCO survey feedback was considered by the panel to make final content changes and to determine the final content domain weights. The draft was then presented to the ABP Pediatric Rheumatology Subboard for minor editing and approval. The process and the methods for the practice analysis are outlined in Figure 1.

Figure 1.

Process and methods of practice analysis. EPA, entrustable professional activities.

RESULTS

PAD document and survey of PDs

The PAD includes the following: 1) a list of 14 areas of rheumatic diseases, rheumatic disease associations, and musculoskeletal conditions that pediatric rheumatologists diagnose and/or manage in practice (noninflammatory musculoskeletal pain, juvenile arthritis and associated disorders, system lupus erythematosus [SLE] and SLE‐related organ involvement, other rheumatic and inflammatory disorders, idiopathic inflammatory myositis, vasculitis and related disorders, sclerodermas and related disorders, autoinflammatory disorders, primary immunodeficiencies and other disorders associated with inflammatory autoimmune manifestations, arthritis related to infection and associated disorders, skeletal lesions and neoplasms that mimic rheumatic disorders, bone and connective tissue disorders, musculoskeletal manifestations of other chronic disorders, and dermatologic disorders and mimics of rheumatic disorders) and 2) eight domains, or areas of practice, with related tasks, knowledge, and skills (diagnosis of rheumatic diseases, rheumatic disease associations, and musculoskeletal conditions; management of rheumatic disorders, rheumatic disease associations, and musculoskeletal associations; communication and care coordination; fiscal responsibility and management; public health and health systems; advocacy for the pediatric rheumatology community; scholarly activities; and behavioral and mental health). Within each domain, the PAD lists all the related tasks performed by a pediatric rheumatologist, and the knowledge and skills required to perform those tasks (see Appendix for complete PAD).

The PAD survey was distributed to 42 PDs and 10 responded (24%) using the Likert importance scale previously described. Overall, the PDs indicated that the content of the PAD was important with an average rating across all the conditions and domains of 2.78 and a range of 2.3 to 3. The PDs also indicated that the PAD is useful for the community of pediatric rheumatology training directors (average rating of 3.33) and that the PAD will be useful in developing fellowship training curricula (average rating of 3.33). The open‐ended feedback provided in the survey was taken into consideration by the panel when making final revisions to the PAD.

TCO document and TCO survey

Using the PAD and the current TCO, the panel created an updated TCO, which comprised 18 content domains, including the PAD rheumatic disease and related condition list and the nonclinical domains. Within each content domain, subdomains (and in some cases, subsubdomains) are used to define more specific content areas to be assessed on certifying examinations (for example, within the content domain of other rheumatic and inflammatory disorders, there is a subdomain of Overlap Syndromes with a subsubdomain of Mixed Connective Tissue Disease). One important distinction between the PAD and the TCO is that the TCO outlines only the knowledge and skills from the PAD which are important to test and are testable using a multiple‐choice item format. A second distinction is that the TCO defines four universal tasks that pediatric rheumatologists perform as they use the knowledge and skills within the clinical content domains (domains 3‐16) that describe rheumatic diseases and related conditions. These tasks include pathophysiology, epidemiology and risk assessment, diagnosis, and management and treatment. A survey regarding the TCO was distributed to all board‐certified pediatric rheumatologists with email addresses on file with the ABP. In total, 455 pediatric rheumatologists were emailed the survey invitation and 127 (28%) responded. The survey data indicated that all 18 content domains were relevant to pediatric rheumatology practice, with a mean score of 3.4 and range of 2.7 to 4. In addition, the specific content areas were also found to be relevant with a mean score of 3.4 and range of 2.3 to 4 (n = 156). The panel considered the scores, the open‐ended feedback, and the recommended weights for each domain from the survey when creating the final document. The ABP Pediatric Rheumatology Subboard (n = 13) approved the document with minor edits and revisions (Table 1).

Table 1.

Final test content outline domains and weights

Content domains	Weight, %
Anatomy, genetics, and physiology	6
Drug therapy	5
JIA and associated disorders	17
SLE and SLE‐related organ involvement	14
Vasculitis and related disorders	9
IIM and associated disorders	8
Sclerodermas and related disorders	6
Autoinflammatory disorders	5
Other rheumatic and inflammatory disorders	4
Noninflammatory musculoskeletal pain	4
Musculoskeletal conditions related to infection	3
Dermatologic disorders and mimics of rheumatic disorders	3
Musculoskeletal manifestations of other chronic disorders	3
Bone and connective tissue disorders	3
Primary immunodeficiencies	2
Skeletal lesions and neoplasms	2
Communication, care coordination, and psychological support	2
Core knowledge in scholarly activities	4

Abbreviations: IIM, idiopathic inflammatory myositis; JIA, juvenile idiopathic arthritis; SLE, systemic lupus erythematosus.

DISCUSSION

The recognition of pediatric rheumatology as a distinct pediatric subspecialty is relatively recent. Rheumatic diseases specific to childhood were not identified until the 1800s, and arthritis distinct from adult forms was first described by Still in 1897 (13). The care of children with rheumatic diseases over the following 80 years was provided largely by general pediatricians, other pediatric subspecialists, or by physicians trained in internal medicine. Nonaccredited pediatric rheumatology fellowship training programs began in several institutions in the 1960s, with the first textbook dedicated to pediatric rheumatology published in 1982. In the early 1990s, the subspecialty was first recognized and accredited by the ACGME, and in 1992, the first pediatric rheumatology board certification examination was administered (5). Since the inception of the certification program through 2021, 510 pediatric rheumatologists have been certified (11), and the number of pediatric rheumatology fellowship training programs has grown to 38 as of 2023. However, despite this growth, there remains a considerable workforce shortage with six states lacking a pediatric rheumatologist and six others with only one.

Throughout this evolution, defining the scope of practice of pediatric rheumatology has been challenging. As noted by Sir Thomas Barlow, in the late 1800s “The fundamental difficulty in discussing rheumatology consists in defining what we mean by it” (14). One of the difficulties is that the recognition and understanding of childhood immunologic and inflammatory conditions has been expanding rapidly. Over the past two decades, there have been great advances in the understanding of immunology and genetics and in the development of novel effective biologic therapies. Consequently, clinicians can consider, diagnose, and manage an increasing number of diseases and conditions that were previously unknown or poorly understood. Pediatric rheumatologists are now frequently asked to assist with management of patients with hyperinflammation (for example, in systemic inflammatory response syndrome or macrophage activation syndrome) in patients undergoing novel treatments, such as after chimeric antigen receptor T cell therapy, and in the treatment of novel syndromes with immunologic underpinnings, such as multisystem inflammatory syndrome in children. Additionally, organ‐specific inflammatory diseases are being continually discovered, including several autoimmune neurologic diseases (anti‐N‐methyl‐d‐aspartate receptor encephalitis, myelin oligodendrocyte glycoprotein antibody disease), and rheumatologists are increasingly consulted to help manage these conditions.

With the expansion of recently recognized inflammatory and immunologic conditions and diseases, there are an increasing number of children that rheumatologists now provide care for. Periodically defining this broad and rapidly changing scope of pediatric rheumatology practice is necessary for at least two reasons. First, training programs need guidance to ensure that they provide the experiences and resources required to best prepare trainees for the clinical practice of pediatric rheumatology. Second, professional organizations charged with certifying that physicians and other providers are competent to practice pediatric rheumatology need to know that their methods of evaluation are assessing the knowledge and skills deemed essential to this practice.

Our process of practice analysis has resulted in a PAD, which establishes the knowledge and skills needed to accomplish the tasks essential to the practice of pediatric rheumatology. This process has now been used successfully by several pediatric subspecialties to create similar documents (8, 9, 10). We acknowledge that the process has some limitations. For example, the Likert scales used in our surveys are subjective and may not accurately distinguish the elements of practice that are truly essential. Nonetheless, the scores we received from our surveys indicate that the content of both the PAD and the TCO were important and relevant to the broad community of practicing pediatric rheumatologists. The PAD document lists the conditions and diseases that a pediatric rheumatologist is expected to diagnose and/or manage. It is anticipated that this definition will prove to be useful to fellowship training programs as suggested by the results of our poll of PDs. It is also anticipated that accrediting bodies and certification boards will find the definition most helpful in establishing the expectations of both training programs and individuals planning to seek board certification.

Changes to the PAD are anticipated because novel diseases and conditions will continue to emerge, the ability to deliver care in varying ways has increased (eg, ultrasonography, telemedicine), and pediatric rheumatologists will need to continue to expand their knowledge and skills. Ultrasonography is being increasingly used, and its application is likely to expand further, potentially leading to the expectation that all pediatric rheumatologists will acquire the skills to perform ultrasound. Telemedicine will also likely be increasingly used, and this method of delivering care requires specific skills that rheumatologists may need to develop. Future practice analyses will determine whether there will be an expectation that pediatric rheumatologists possess these skills and will also determine the knowledge expectations that will continue to emerge. The ABP currently plans to conduct a practice analysis every 5 to 6 years to ensure that changes in practice are reflected in the PAD and the TCO such that the knowledge and skills that are assessed on board certification examinations remain those that are most essential to the practice of pediatric rheumatology.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted. Robert C. Brucia had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design

Brucia.

Acquisition of data

Brucia, Hayward, Brunner, Lopez‐Pena, Shenoi, Soybilgic, Nocton.

Analysis and intepretation of data

Brucia, Hayward, Brunner, Lopez‐Pena, Shenoi, Soybilgic, Notcon.

Supporting information

Disclosure form:

Appendix S1: Supporting Information