Quality Measures for Pediatric Orthopaedic Surgery: A Systematic Review

Abstract

Background:

Quality measures provide a way to assess healthcare delivery and to identify areas for improvement that can inform patient care delivery. When operationalized by a hospital or a payer, quality measures can also be tied to physician or hospital reimbursement. Prior work on quality measures in orthopaedic surgery have identified substantial gaps in measurement portfolios and have highlighted areas for future measure development. This study aims to identify the portfolio of quality measures in pediatric orthopaedic surgery.

Methods:

We used methodology of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and reviewed PubMed/MEDLINE and EMBASE, the American Academy of Orthopaedic Surgery, National Quality Forum, Agency for Healthcare Research and Quality, and the National Quality Measures Clearinghouse for quality measures and candidate quality measures. Quality measure and candidate quality measures were categorized as structure, process, or outcome. Measures were also classified into one of the six National Quality Strategy priorities (safety, effective, patient-centered, timely, efficient, and equitable).

Results:

A review of PubMed/EMBASE returned 1,640 potential quality measures and articles. A review of AAOS, NQF, and AHRQ databases found 80 potential quality measures. After screening we found a total of 18 quality measures and candidate quality measures specifically for pediatric orthopaedic surgery. Quality measures addressed conditions such as supracondylar humerus fractures, developmental dysplasia of the hip, and osteochondritis dissecans. There were 10 process measures, 8 outcome measure, and 0 structure measures. When we categorized by National Quality Strategy priorities and found 50% (9/18) were effective clinical care, 44% (8/18) were person and care-giver centered experience and outcomes, 6% (1/18) were efficient use of resources.

Conclusions:

There are few quality measures and candidate quality measures to assess pediatric orthopaedic surgery. Of the quality measure available, process measures are relatively over-represented. Pediatric orthopaedic surgeons can lead the development of outcome (e.g. patient reported outcomes after surgery) and structure measures (e.g. subspecialty training certification) to assess quality of care in pediatric orthopaedic surgery.

INTRODUCTION

Pediatric musculoskeletal conditions are common and constitute 6–8% of visits to pediatricians.^1,2 Similar to other common conditions within medicine there should be guidelines and policies to ensure safety and quality. Quality measures are a way for payers and healthcare systems to operationalize those guidelines and policies; therefore, pediatric orthopaedic surgeon should be aware of the current quality measures.

Quality measures allow for assessment of health care delivery by examining the structure (capacity to deliver care), the process (the administration of care), and outcome (the result of care).³ These measures can be used to allocate resources, assess physician performance, and structure reimbursement models. For example, Centers for Medicaid and Medicare Services (CMS) Hospital Value-Based Purchasing (VBP) Program incentivizes payments based on the quality of care provided. According to CMS, a quality measure should focus on a high impact area that matters to patients and providers, address a gap or variation in care, should be evidence based, feasible to incorporate, valid, and reliable.^4,5 The National Strategy for Quality Improvement in Health Care (NQS) has destined six domains for how to improve quality care 1) safe 2) effective 3) patient-centered 4) timely 5) efficient and 6) equitable.⁶

One previous study assessed quality indicators in pediatric orthopaedics and found mortality and post-operative complications were the most frequently reported indicators.⁷ Though mortality is frequently reported, it is not an ideal quality measure as it does not accurately measure the quality of care or other aspects of value based care (i.e., safety and cost effectiveness).^8–10 Additionally, mortality is a rare event and has little opportunity for improvement or discriminating between high and low performers. Traditional quality measures, like mortality, are insufficient indicators of quality of care because they do not account for the uniqueness of pediatric orthopaedics and the potential complications, like growth plate arrest. Mortality is also a poor quality measure because it is a rare event in pediatric orthopaedics, therefore it has little opportunity for improvement or discriminating between high and low performers.

The purpose of this study is to systematically review current and candidate quality measures for pediatric orthopaedic surgery. We aim to assess these measures across the Donabedian domains of structure, process, and outcome and across the six NQS priorities (safety, effective, patient-centered, timely, efficient, and equitable). We hypothesize that there will be few quality measures and candidate quality measures for pediatric orthopaedic surgery. Additionally, we hypothesize quality measures will relatively over represent process measures and that there will be a lack of structure and outcome measures.

MATERIALS & METHODS

We used the Preferred Reporting items for Systematic Reviews and Meta-Analyses Statement (PRISMA) methodology to perform our systematic review.¹¹ PubMed/MEDLINE and EMBASE were searched January, 2021 using words such as “quality,” “measure,” “improvement,” and “guideline” were included with terms specific to pediatric orthopaedic surgery (Appendix).

Two members of the research team independently reviewed each of the identified studies after initial screening was performed. Conflicts between reviewers were resolved by a board-certified fellowship trained orthopaedic surgeon with expertise in quality measures (RNK). We included quality measures and candidate quality measures related to the operative or non-operative care of pediatric patients with orthopaedic conditions or injuries.

The American Academy of Orthopaedic Surgery (AAOS), and Pediatric Orthopaedic Society of North America (POSNA) websites were searched for candidate measures relevant to pediatric orthopaedic surgery. Clinical practice guidelines were included as candidate quality measures if they were developed in accordance with the Institute of Medicine (IOM) criteria for development of clinical practice guidelines, founded on clinical evidence including at least one Level 1 study, and carried the highest level of recommendation by the organization (Table 1).^12–14 Therefore, we include clinical practice guidelines with a “strong”, “moderate”, or “consensus” recommendation. We excluded appropriate use criteria as they do not meet the definition of quality measure as defined by the IOM. Quality measures and candidate quality measures that did not specifically address the pediatric population (e.g. the musculoskeletal infection) or did not state level of evidence were excluded. Individual patient reported outcomes measurement tools (e.g PROMIS) and quality-of-life questionnaires are insufficient for measuring performance and are not actionable and were excluded.¹⁵

Table 1:

Search terms and websites used to search for candidate quality measures

Source	Organization	Details/Content
MEDLINE/PubMed	United States National Library of Medicine	Database of published biomedical and life sciences literature
EMBASE	Elsevier	Database of published biomedical and pharmacological literature
American Academy of Orthopaedic Surgeons (AAOS)-Clinical Practice Guidelines (CPG)	US professional orthopaedic society	Evidence-based guidelines for current orthopaedic diagnostic, treatment, and postoperative procedures
National Quality Forum (NQF)- Quality Positioning System (QPS)	US non-profit organization that promotes healthcare quality through measurement and public reporting	Database of quality measures endorsed by NQF
Agency for Healthcare Research and Quality (AHRQ)- National Quality Measures Clearinghouse (NQMC)	Agency within the US Department of Health and Human Services	Database of quality measures endorsed by US governmental agencies and other private groups
Pediatric Orthopaedic Society of North America (POSNA)	North American professional organization for pediatric orthopedics

We also manually reviewed the National Quality Forum (NQF) and Agency for Healthcare Research and Quality (AHRQ) to identify additional quality measures relevant to pediatric orthopaedic surgery. Due to the heterogeneous organizational style of each site, each was independently searched and screened by each reviewer to ensure the accurate identification of quality measures.

Quality measure and candidate quality measures were subsequently categorized as structure, process, and outcome as described by Donabedian.³ Additional classification into the six NQS priorities (safety, effective, patient-centered, timely, efficient, and equitable) was also performed. Similar methodology has been used previously to identify and review quality measures in other orthopaedic specialties.^{12,13,16–18}

RESULTS

Our initial search of PubMed/EMBASE returned 1,640 quality measures and articles. After removal of duplicated and irrelevant articles identified through title and abstract screening, 15 were included for full text review. Of these, we did not find any new quality measures that were not already identified in either AAOS, NQF, or AHRQ database.

After review of AAOS, NQF, and AHRQ databases we found 80 potential measures (Figure 1). NQF measures that were not currently endorsed were excluded. We found a total of 18 quality measures and candidate quality measures specifically for pediatric orthopaedic surgery. Of these measures none were categorized as structure, 10 (56%) were categorized as process measures (e.g. flexible intramedullary nailing to treat children age five to eleven years diagnosed with diaphyseal femur fractures), and 8 (44%) as outcome measures (e.g. functional status change or patients with general orthopaedic impairments). When we categorized by National Quality Strategy priorities we found 50% (9/18) were effective clinical care, 44% (8/18) were person and care-giver centered experience and outcomes, 6% (1/18) were efficient use of resources.

Figure 1:

PRISMA Flow Diagram

*Consider, if feasible to do so, reporting the number of records identified from each database or register searched (rather than the total number across all databases/registers).

**If automation tools were used, indicate how many records were excluded by a human and how many were excluded by automation tools.

AAOS Clinical Practice Guidelines

We analyzed the AAOS Clinical Practice Guidelines for pediatric supracondylar fractures (2011), pediatric diaphyseal femur fractures (2020), detection of nonoperative management of pediatric developmental dysplasia of the hip in infants up to six months of age (2014), diagnosis and treatment of osteochondritis dissecans (2010), management of surgical site infections (2018), and management of anterior cruciate ligament injuries (2014). We included all candidate quality measure that were pediatric orthopaedic specific and held strong or moderate strength recommendation or held consensus recommendation. We found 10 candidate quality measures (Table 2). All measures were process measures. According to the NQS priorities 9 measures addressed effective clinical care while 1 measure addressed efficient use of resources.

Table 2:

Pediatric Orthopaedic Surgery-Specific Candidate Quality Measures and Domains

Candidate Quality Measures in Pediatric Orthopaedic Surgery	Source	Strength	Donabedian Domain	NQS priorities
Treatment of Pediatric Supracondylar Humerus Fractures: Nonsurgical immobilization of the injured limb for patients with acute (Gartland type 1) or non-displaced pediatric supracondylar fractures of the humerus or posterior fat pad sign	AAOS	Moderate	Process	Effective clinical care
Treatment of Pediatric Supracondylar Humerus Fractures: Closed reduction with pin fixation for patients with displaced (Gartland type II and III, and displaced flexion) pediatric supracondylar fractures of the humerus	AAOS	Moderate	Process	Effective clinical care
Treatment of Pediatric Supracondylar Humerus Fractures: In the absence of reliable evidence, the opinion of the work group is that emergent closed reduction of displaced pediatric supracondylar humerus fractures be performed in patients with deceased perfusion of the hand.	AAOS	Consensus	Process	Effective clinical care
Treatment of Pediatric Supracondylar Humerus Fractures: In the absence of reliable evidence, the opinion of the work groups is that open exploration of the antecubital fossa be performed in patient who have absent wrist pulses and are under perfused after reduction and pinning of displaced pediatric supracondylar humerus fractures	AAOS	Consensus	Process	Effective clinical care
Treatment of Pediatric Supracondylar Humerus Fractures: Strong evidence supports that children younger than thirty-six months with a diaphyseal femur fracture be evaluated for child abuse	AAOS	Strong	Process	Effective clinical care
Treatment of Pediatric Supracondylar Humerus Fractures: Moderate evidence supports early spica casting or traction with delayed spica casting for children age six months to five years with a diaphyseal femur fracture with less than a 2 cm of shortening	AAOS	Moderate	Process	Effective clinical care
Treatment of Pediatric Supracondylar Humerus Fractures: Strong evidence supports the use of flexible intramedullary nailing to treat children age five to eleven years diagnosed with diaphyseal femur fractures	AAOS	Strong	Process	Effective clinical care
Detection and Nonoperative Management of Pediatric Developmental Dysplasia of the Hip in Infants up to Six Months of Age Moderate evidence supports not performing universal ultrasound screening of newborn infants	AAOS	Moderate	Process	Efficiency and cost reduction
Detection and Nonoperative Management of Pediatric Developmental Dysplasia of the Hip in Infants up to Six Months of Age Moderate evidence supports performing an imaging study before 6 months of age in infants with one or more of the following risk factors: breech presentation, family history, or history of clinical instability	AAOS	Moderate	Process	Effective clinical care
The Diagnosis and Treatment of Osteochondritis Dissecans: In the absence of reliable evidence, it is the opinion of the work group that symptomatic skeletally immature patients with salvageable unstable or displaced OCD lesions be offered the option of surgery	AAOS	Consensus	Process	Effective clinical care

NQF QPS

We analyzed the NQF database and identified eight quality measures that are currently endorsed by NQF and applicable to pediatric orthopaedic surgery (Table 3). All measures only partially included pediatric aged patients (quality measures were aimed at patients 14 years and older). All quality measures were outcome measures. According to the NQS priorities all eight measures addressed person and care-giver centered experience and outcomes.

Table 3:

Pediatric Orthopaedic Surgery-Specific Quality Measures and Domains

Quality Measures in Pediatric Orthopaedic Surgery	Source	Endorsement	Donabedian Domain	NQS priorities
Functional status change for patients with elbow, wrist and hand impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes
Functional status change for patients with foot and ankle impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes
Functional status change for patients with general orthopaedic impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes
Functional status change for patients with hip impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes
Functional status change for patients with knee impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes
Functional Status Change for Patients with Low Back Impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes
Functional Status Change for Patients with Neck Impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes
Functional status change for patients with Shoulder impairments (patients 14 and over)	NQF	Endorsed	Outcome	Person and care-giver centered experience and outcomes

DISCUSSION

We found 18 quality measures applicable to pediatric orthopaedic surgery. Ten measures were candidate quality measure from the AAOS CPG, while the remaining 8 were from the NQF database. Most of the measures (56%) were process measures, while the remaining 44% of measures were outcome measures. We did not identify any structural measures that were specifically for pediatric orthopaedic surgery. According to the NQS priorities nine measures addressed effective clinical care, eight measures addressed person and care-giver centered experience and outcomes, and one measure addressed efficient use of resources.

Quality measures are becoming more necessary as government and payment agencies shift focus to value-based care. This was further illuminated in the 2011 US Department of Health and Human Service report National Strategy for Quality Improvement in Heath Care.¹⁹ As adult healthcare fields accumulate quality measures, pediatric healthcare fields are coming along as well. AHRQ published pediatric quality measures, however, none of the quality measures were specifically for pediatric orthopaedic surgery.

We found fewer quality measures applicable to pediatric orthopaedic surgery compared to other fields of orthopaedic surgery.^{12,13,16–18,20} In similar systematic reviews there were 134 measures identified in the field of hand surgery, 116 measures in arthroplasty, 74 measures in spine surgery, and 116 in orthopaedic sports medicine. Similar to other fields of orthopaedic surgery, quality measures in pediatric orthopaedic surgery are unequally represented, with process measures being the most common. For example, 98% of quality measures identified in hand surgery and 80% of the quality measures identified in spine surgery were process measures.^13,18 Process measures may be the most common because they are the easiest to implement compared to outcome and structure measures.

Although process measures were the most common measure identified in our review, we identified 10 outcome measures that are specifically for pediatric orthopaedic surgery. All of the outcome measures identified were broadly applicable to pediatric orthopaedics (i.e. functional status change for patients with knee impairments). Additionally, all of the outcome measures identified were for patients 14 years of age and older, therefore these measures were only applicable to a subset of the pediatric population (age 14–18). Outcome measures are often considered the most important measure because working to achieve outcome measures facilitates the development of process and structural measures. For example, aiming to meet the outcome measure of decreased post-operative infections would provide incentive to deliver pre-operative antibiotics within an hour before surgery.

Patient reported outcomes measures (PROM) and instruments, such as the PROMIS and SRS, that serve as outcome assessment tools are not quality measures. Although these instruments are being used to assess outcomes from the patient, their use as quality measures requires further research. For example, PROMs use as a quality measure for total joint arthroplasty required first development of a process measure for collection, and now as an outcome measurement with thresholds for expected improvement after surgery. Pediatric orthopaedic surgery PROMs have not gone through the necessary steps to become quality measures.

There are multiple limitations to our study. First, our screening method was constructed to include only articles with the highest levels of evidence. Therefore we may have missed potential quality measures which currently are not supported by high level evidence. Additionally, our search terms were designed to be comprehensive, however, we may have missed relevant articles in our search. In order to mitigate this risk we manually reviewed multiple databases. Furthermore, we only included peer reviewed articles that were published in the English language, therefore we may have missed quality measures that were published in other languages.

In summary, our study has determined that only a handful of quality measures are specifically for pediatric orthopaedic surgery. The quality measures that are available are not equally distributed amongst the Donabedian framework of structure, process, and outcome measures and process measures are relatively over-represented. This study illuminates the need to develop more quality measures for pediatric orthopaedic surgery. This is especially important to assess and improve the delivery of high-value care. Additionally, quality measure development will be necessary as payment models shift to value-based reimbursement models. Pediatric orthopaedic surgeons are in optimal positions to help lead in the development of quality measures for pediatric orthopaedic surgery.

Appendix:

Search Terms

(“Orthopedic Procedures”[Mesh] OR “Bone Diseases/surgery”[Mesh] OR ((surgery[tw] OR surgical[tw] OR operative[tw]) AND (orthoped*[tw] OR orthopaed*[tw])) OR “hand/surgery”[mesh] OR “hand surgery”[tw] OR “physis” [tw] OR “epiphysis” [tw] OR “Osteogenesis imperfecta”[tw] OR “osteopetrosis”[tw] OR “skeletal dysplasia”[tw] OR “achondroplasia”[tw] OR “hypochondroplasia”[tw] OR “pseudoachondroplasia”[tw] OR “neurofibromatosis”[tw] OR “Rett syndrome”[tw] OR “mucopolysaccharidosis”[tw] OR “arthrogryposis”[tw] OR “musculoskeletal infection”[tw] OR “osteomyelitis”[tw] OR “septic arthritis”[tw] OR “benign bone tumor”[tw] OR “benign bone tumors”[tw] OR “osteoid osteoma”[tw] OR “osteoblastoma”[tw] OR “osteochondroma”[tw] OR “enchondroma”[tw] OR “chondroblastoma”[tw] OR “chondromyxoid fibroma”[tw] OR “nonossifying fibroma”[tw] OR “fibrous cortical defect”[tw] OR “fibrous dysplasia”[tw] OR “osteofibrous dysplasia”[tw] OR “unicameral bone cysts”[tw] OR “aneurysmal bone cysts”[tw] OR “malignant bone tumor”[tw] OR “malignant bone tumors”[tw] OR “osteosarcoma”[tw] OR “Ewing sarcoma”[tw] OR “rhadomyosarcoma”[tw] OR “synovial sarcoma”[tw] OR “malignant peripheral nerve sheath tumor”[tw] OR “cerebral palsy”[tw] OR “gait”[tw] OR “myelomeningocele”[tw] OR “spina bifida”[tw] OR “muscular dystrophy”[tw] OR “duchenne muscular dystrophy”[tw] OR “becker muscular dystrophy”[tw] OR “spinal muscular atrophy”[tw] OR “hereditary motor sensory neuropathies”[tw] OR “spine deformity”[tw] OR “scoliosis”[tw] OR “idiopathic scoliosis”[tw] OR “congenital scoliosis”[tw] OR “kyphosis”[tw] OR “scheuermann’s disease”[tw] OR “scheuermann disease”[tw] OR “spondylolysis”[tw] OR “spondylolisthesis”[tw] OR “torticollis”[tw] OR “brachial plexus palsy”[tw] OR “polydactyly”[tw] OR “syndactyly”[tw] OR “trigger finger”[tw] OR “trigger thumb”[tw] OR “fracture”[tw] OR “supracondylar humerus fracture”[tw] OR “femur fracture”[tw] OR “both bone forearm fracture”[tw] OR “dislocation”[tw] OR “synostosis”[tw] OR “pseudarthrosis”[tw] OR “compartment syndrome”[tw] OR “developmental hip dysplasia”[tw] OR “legg calve perthes”[tw] OR “slipped capital femoral epiphysis”[tw] OR “rotational variation”[tw] OR “intoeing”[tw] OR “outtoeing”[tw] OR “genu varum”[tw] OR “genu valgrum”[tw] OR “toe walking”[tw] OR “lemb length discrepancy”[tw] OR “accessory navicular”[tw] OR “clubfoot”[tw] OR “congenital talipes equinovarus”[tw] OR “congenital vertical talus”[tw] OR “curly toe”[tw] OR “sever apophysitis”[tw] OR “tarsal coalition”[tw] OR “avascular necrosis”[tw] OR “discoid meniscus”[tw] OR “meniscus injury”[tw] OR “osteochondritis dissecans”[tw] OR “osgood schlatter syndrome”[tw] OR “anterior knee pain”[tw] OR “anterior cruciate ligament injury” [tw] OR “Reconstructive Surgical Procedures”[mesh]) AND (child*[tw] OR infan*[tw] OR pediatr*[tw] OR paediatr*[tw] OR adolescent[tw]) AND (“quality improvement” [mesh] OR “quality improvement” [tw] OR “quality control” [tw] OR “quality measurement” [tw] OR “quality” [ti] OR “quality assessment” [tw] OR “quality measure” [tw] OR “quality metric” [tw] OR “quality indicator” [tw] OR “appropriate use” [tw] OR “practice guideline” [tw] OR “guideline” [ti] OR “outcome assessment” [tw] OR “performance measure” [tw] OR “performance metric” [tw] OR “performance indicator” [tw] OR Guideline [ptyp] OR Practice Guideline [ptyp]) AND English AND (“quality”[title] OR “Guideline”[title])