Abstract
Objective:
To determine the ease of use, accuracy, consistency, reliability, and reproducibility for rating lower limb conditions when transitioning to the AMA Guides to the Evaluation of Permanent Impairment (AMA Guides) Sixth Edition 2024 developed using the RAND/UCLA modified Delphi Appropriateness Method compared with the AMA Guides Sixth Edition 2008.
Methods:
Three physician experts and four premedical students completed two rounds of impairment ratings using methods from the AMA Guides Sixth Edition 2008 versus the AMA Guides Sixth Edition 2024. Impairment values and completion times for each method were compared across groups.
Results:
For experts, the average time to complete an impairment rating was 4.1 minutes using the AMA Guides Sixth Edition 2024 compared with 16.7 minutes using the AMA Guides Sixth Edition 2008, maintaining 100% accuracy and reliability for both methods. Premedical students averaged 5.3 minutes with the AMA Guides Sixth Edition 2024 and 24.0 minutes with the AMA Guides Sixth Edition 2008, showing increased accuracy, consistency, reliability, and reproducibility with AMA Guides 2024.
Conclusion:
The AMA Guides Sixth Edition 2024 represents a notable advancement in impairment evaluation, offering a more efficient, accurate, and reliable system for lower limb impairment assessments. This update not only benefits healthcare providers and patients but also sets a new standard in occupational health by preserving the accuracy, consistency, reliability, and reproducibility of impairment ratings while streamlining the evaluation process.
The American Medical Association's (AMA) Guides to the Evaluation of Permanent Impairment (AMA Guides) are the most frequently used standard in the United States and internationally for rating loss of function, often described as impairment, following an injury or illness.1,2 In many United States legal systems, an independent medical examination (IME) is necessary to determine whether a compensable injury has occurred.3 Within the claims process, the independent medical evaluator is relied upon to offer a professional opinion that establishes that a permanent medical condition exists. The evaluator must assess the severity and extent of the condition, often by determining the impairment rating based on objective and reproducible findings determined by the clinical history, physical examination, and relevant clinical tests and studies. The impairment value often serves as a baseline for the administrative and legal systems to provide a monetary award.1,2
The AMA Guides Sixth Edition 2008 has faced several criticisms, including a steep learning curve, the need for extensive evaluator training to provide an accurate impairment value, inconsistency and ambiguity in definitions, poor reliability and reproducibility, and lack of content validity.4-8
Quality IME reports are invaluable to all parties involved and stem from thorough evaluations by physicians with expertise in clinical medicine and medicolegal assessment.9 The evaluator must remain impartial and provide sound conclusions. A crucial aspect of the process involves collecting accurate and relevant data specific to the case (called specific individual elements [SIEs]) and apply this information using current science and evidence-based medicine.
Given the importance of thorough IME reports, the laborious and time-consuming process of completing musculoskeletal (MSK) impairment ratings using the AMA Guides Sixth Edition 2008 is not ideal. Evaluators must use multiple tables to determine modifiers, adding notable complexity and time to the process. By contrast, the AMA Guides Sixth Edition 2024 simplifies the process by providing all criteria (SIEs) for an impairment rating in a single diagnosis-based impairment (DBI) table.
In June 2019, the AMA established the AMA Guides Editorial Panel (Guides Panel) to periodically review and update the AMA Guides, aiming to incorporate the latest advancements in medical diagnosis, treatment methods, patient outcomes, functional loss measures, and impairment ratings. In response to feedback from various stakeholders, the Guides Panel formed an MSK subcommittee in August 2022 to review the chapters on the upper limb, lower limb, and spine and pelvis. The combined efforts of the Guides Panel and the MSK subcommittee have led to notable updates, including the use of the RAND/UCLA modified Delphi Appropriateness Method, regular public comment periods, and the introduction of a new five-step method for performing impairment evaluations.10 The 2024 edition also features enhanced DBI tables, which integrate SIEs from clinical history, physical examination, and relevant clinical studies, ensuring that the AMA Guides remain current and reflective of modern medical practices.
Notably, three of the authors actively participated in the RAND/UCLA Modified Delphi process, which played a crucial role in refining the methodology for the updated edition.
This article (1) details the development of an updated stepwise impairment rating method for lower limb conditions that was designed using the RAND/UCLA Appropriateness Method (RAM) Delphi before the study's design and implementation; (2) explores the enhanced DBI tables for the AMA Guides Sixth Edition 2024, designed to standardize the impairment rating process for the lower limb; and (3) assesses the effect of these updates on the ease of use, accuracy, consistency, reliability, and reproducibility (both interrater and intrarater) of the AMA Guides Sixth Edition 2024 for lower limb conditions.
Methods
This study was designed and reported in accordance with the 15 Guidelines for Reporting Reliability and Agreement Studies.11
Updating the AMA Guides
The process to update the AMA Guides involved the appointment of author-editors for the MSK chapters and the establishment of a lower limb subcommittee by the 17 members of the Guides Panel.12 The lower limb subcommittee developed content based on the Sixth Edition 2008, updating it to reflect current diagnoses, treatments, and outcomes that were not available or common in 2008. Committed to using the most robust and current scientific knowledge, the subcommittee adopted the RAM with a modified Delphi process to evaluate existing scientific research on impairment evaluations.13
With the base content established, the subcommittee provided materials for review by a diverse group of stakeholders, including administrative law judges, administrators, attorneys, chiropractors, compensation claims judges, disability evaluators, emergency medicine specialists, forensic medicine experts, neurologists, occupational and orthopaedic surgeons, pain medicine specialists, physical medicine and rehabilitation experts, preventive medicine professionals, psychiatrists, psychologists, rehabilitation specialists, and state bureau of workers' compensation systems. Using feedback from these reviewers and contributors, the subcommittee drafted updates to the chapter based on the RAM process and presented the materials during multiple public comment periods (Figure 1). Public comments, along with feedback from reviewers and contributors, were incorporated into the multiple Delphi method reviews to develop a final version of the chapter, which was then approved by the Guides Panel14–17 (Figure 2).
Figure 1.
A Photograph showing an example of the guides digital public comment.
Figure 2.
Flowchart showing an illustration of the Delphi method.
AMA Guides Sixth Edition 2024 Updated Methods
The initial update focused on defining the necessary steps for completing an impairment evaluation and determining a rating. Using the RAM, a five-step process was developed and presented to multiple stakeholders for feedback. Based on this additional input, a consensus Delphi process was implemented, as shown in Table 1. This enhancement simplified the evaluation process by converting the concepts of class, grade, and impairment value into a single diagnostic row that incorporates SIEs from the clinical history, physical examination, and relevant clinical studies.
Table 1.
The Final 2024 Five-Step Process for Lower Limb Impairment Rating
| Step 1. Confirm a clinically relevant diagnosis (DX) |
| Step 2. Confirm maximum medical improvement (MMI) |
| Step 3. Identify the relevant diagnosis-based impairment (DBI) table |
| Step 4. Determine the diagnostic row, class, grade, and impairment value |
| Step 5. Guidelines for report documentation |
Enhanced Diagnosis-Based Impairment Tables
The second update focused on enhancing the DBI tables for the three MSK chapters by incorporating SIEs from clinical history, physical examination, and relevant clinical studies. These SIEs provide objectively verifiable criteria for diagnosis and offer insights into outcomes and functional losses. For instance, physical examination SIEs include sensory loss, deep tendon reflexes, motor strength, limb atrophy, leg length discrepancy, and amputation. Relevant clinical studies encompass radiographs, magnetic resonance images, CT scans, sonography, electrodiagnostic testing, and laboratory studies.
To refine these updates, the lower limb subcommittee employed a five-round modified Delphi process (Figure 2). This process was built upon the diagnoses and impairment values established in the AMA Guides Sixth Edition 2008, facilitating a quantitative assessment that reflects consensus on the essential diagnoses and SIEs necessary for completing an impairment rating based on the diagnostic rows in the updated DBI tables.
Using the Enhanced Diagnosis-Based Impairment Tables
The enhanced DBI tables for the lower limb are designed to align seamlessly with the evaluator's method for conducting an impairment assessment. The process begins with the evaluator identifying SIEs from the clinical history, physical examinations, and relevant clinical studies, which collectively confirm an accurate diagnosis. This diagnosis directs the evaluator to the appropriate DBI table. For example, a right knee with a complete medial and lateral meniscectomy that presents with a decreased range of motion for 110° of flexion would be located in DBI Table 16-20 Knee Impairment, diagnostic row 16-20-09 class 2A, as shown in Table 2.
Table 2.
Example of Diagnostic Row From the AMA Guides 2024 Listing Specific Individual Elements
| 16-20-09 class 2A (22% LLI) CH (residual symptoms) with mechanism of injury consistent with DX |
| and PE (all of the following) |
| Consistent with CS |
| Flexion 100°-110° |
| and CS (all of the following) imaging or findings at surgery demonstrates |
| Complete medial meniscal tear or meniscectomy |
| Complete lateral meniscal tear or meniscectomy |
Determining the Effect of the Updated Method and Enhanced Diagnosis-Based Impairment Tables
The third focus of the lower limb subcommittee was to assess the effect of using the AMA Guides Sixth Edition 2024 compared with the AMA Guides Sixth Edition 2008. The 2008 edition includes illustrative case examples (vignettes) designed to help healthcare professionals assess impairment ratings across various diagnoses. These examples use clinical data, including patient history, functional assessment, physical examination, clinical studies, diagnosis, and corresponding impairment ratings. However, the vignettes often present simplified diagnoses with limited clinical context and examination findings, aligning with the 2008 DBI table criteria and, thereby, oversimplifying the real-world complexities of an actual impairment evaluation. As a result, the estimated time for completing evaluations using the 2008 methodology does not accurately reflect the comprehensive data gathering and interpretation required in actual practice.
To evaluate the differences between the 2008 and 2024 methodologies, the subcommittee randomly selected five examples from the AMA Guides Sixth 2008 for comparison. The comparison aimed to test the ease of use (time to complete the rating), accuracy (agreement with published impairment values), consistency (ability to obtain the same results), reliability (yielding the same results over time), and reproducibility (ability to duplicate results both intrarater and interrater). Three physician experts from the Guides Panel and four premedical students participated in this comparison. This sample size was determined based on the enhanced methodology's refinement through the RAND/UCLA Delphi process, which involved dozens of experts over 30 months. This structured, stepwise approach prioritizes expert consensus over large sample sizes, making a smaller, well-selected panel sufficient to validate reliability while maintaining methodological consistency.
According to Table 2-1, statement #8 in the AMA Guides Sixth Edition 2008, “The evaluating physician must use knowledge, skill, and ability generally accepted by the medical scientific community when evaluating an individual, to arrive at the correct impairment rating according to the Guides.” This statement underscores the knowledge gap between premedical students and fully qualified physicians. The physician authors of the AMA Guides Sixth Edition 2008 had already identified the necessary elements for the impairment evaluation. Involving premedical students rather than medical students or physicians was intended to enhance the understanding of the process among all stakeholders, including nonmedically trained individuals who may need to review impairment reports for administrative purposes. This approach aimed to improve comprehension of both the methodology and the rationale behind deriving an impairment value using the simplified five-step method in the 2024 edition.
Impairment Rating Evaluation Process
The three physician experts and four premedical students were provided with the AMA Guides 2008 steps, example data, and the AMA Guides Sixth Edition 2008 book (without the case examples). They first completed impairment ratings for each of the five examples using this material. Subsequently, they were given the AMA Guides 2024 steps and enhanced DBI tables to repeat the ratings for the same examples. Given the premedical students' limited familiarity with the impairment rating process, they received additional guidance and had the opportunity to ask questions about the steps before performing the evaluations.
Eight weeks later, the evaluators repeated the process, with the same examples randomly reordered for each participant. This interval was designed to minimize the influence of prior familiarity on assessments, thereby enhancing the accuracy of both intrarater and interrater reliability. In addition, this approach increased the number of data points for statistical analysis, improving the robustness of the study's findings.
Statistical Analyses
Data were tested for normality and did not meet requirements based on Skewness and Kurtosis (both > 1.0). Therefore, Wilcoxon rank-sum tests were used to assess differences between the AMA Guides Sixth Edition 2008 and the AMA Guides Sixth Edition 2024 ratings. Kappa statistics were calculated to evaluate the reliability of the AMA Guides ratings. An alpha level of 0.05 was used to determine statistical significance. All analyses were done using SAS 9.4 (SAS Institute).
Results
Time to Complete Rating (Ease of Use)
For the three physician experts, the time to complete the rating using the AMA Guides 2008 averaged 16.6 minutes. Accuracy and reliability remained 100% across both rounds, likely due to their familiarity with the impairment rating method. The time to complete the rating using the AMA Guides 2024 averaged 4.1 minutes (a 75.3% reduction), also with 100% accuracy and reliability. Cohen kappa, calculated using Microsoft Excel, was one for both interrater and intrarater accuracy and reliability.18,19
The average time for the four premedical students to complete both rounds for the five examples was 24.0 minutes for the AMA Guides 2008 and 5.3 minutes (a 77.9% reduction) for the AMA Guides 2024 (Table 3). The longer average time for the AMA Guides 2008 was primarily due to the need to locate modifiers in the lower extremity non–key factors tables for functional history adjustment, physical examination adjustment, and clinical studies adjustment and apply these values to the net adjustment formula. It is important to recognize that the premedical students were using simplified examples tailored to the 2008 methodology, reducing the need for students to identify “preferred” grade modifiers—mild, moderate, or severe—as outlined in the grade modifier tables. Such simplified examples do not reflect the complexity typically encountered in practicing evaluators completing actual impairment ratings. Despite those simplifications, the time costs were substantially great for the 2008 method compared with the 2024 method.
Table 3.
Average Time to Complete Impairment Ratings, AMA Guides 2008 Versus AMA Guides 2024
| Example | PMS1 | PMS2 | PMS3 | PMS4 | ||||
| 2008 | 2014 | 2008 | 2014 | 2008 | 2014 | 2008 | 2014 | |
| Round 1 | ||||||||
| 1 | 22 | 5 | 23 | 5 | 23 | 5 | 28 | 5 |
| 2 | 20 | 5 | 29 | 4 | 21 | 5 | 23 | 5 |
| 3 | 20 | 5 | 21 | 5 | 20 | 4 | 22 | 5 |
| 4 | 21 | 6 | 22 | 6 | 23 | 6 | 24 | 6 |
| 5 | 26 | 6 | 27 | 6 | 27 | 6 | 31 | 6 |
| Average time, mina | 21.8 | 5.4 | 24.4 | 5.2 | 22.8 | 5.2 | 25.6 | 5.4 |
| 23.7 | Average time in minutes 2008 for all PMSs | |||||||
| 5.3 | Average time in minutes 2014 for all PMSs | |||||||
| Round 2 | ||||||||
| 1 | 22 | 6 | 23 | 3 | 23 | 5 | 28 | 5 |
| 2 | 20 | 5 | 29 | 4 | 21 | 4 | 23 | 5 |
| 3 | 21 | 5 | 22 | 5 | 21 | 4 | 23 | 5 |
| 4 | 21 | 6 | 22 | 6 | 23 | 6 | 24 | 6 |
| 5 | 28 | 6 | 29 | 6 | 29 | 6 | 33 | 6 |
| Average time, minb | 22.4 | 5.6 | 25 | 4.8 | 23.4 | 5 | 26.2 | 5.4 |
PMS = premedical student
Round 1 combined average time (min) for all PMSs: Guides 2008, 23.7; Guides 2024, 5.3.
Round 2 combined average time (min) for all PMSs: Guides 2008, 24.3; Guides 2024, 5.2.
The time to complete the ratings for the physicians and the students was approximately the same for the second round, suggesting minimal learned benefit from previous exposure and familiarity.
For both the panel members and the students, a statistically significant difference was found in the time to complete the impairment ratings between the AMA Guides 2008 and AMA Guides 2024 (P < 0.0001), favoring the AMA Guides 2024 ratings. These statistically significant differences remained when stratifying analyses between panel members and students (P < 0.0001 for both).
Accuracy
Accuracy was defined as the ability to match the impairment values published in the Sixth Edition 2008 and the panel members' impairment values made using the Sixth Edition 2024. The accuracy data are listed in Table 4 for round 1 and round 2, conducted 8 weeks later. The value of 1 means impairment value agreement; 0, impairment value disagreement.
Table 4.
Accuracy of the AMA Guides 2008 vs the AMA Guides 2024
| Example | 2008 | 2008 | 2008 | 2008 | 2024 | 2024 | 2024 | 2024 |
| PMS1 | PMS2 | PMS3 | PMS4 | PMS1 | PMS2 | PMS3 | PMS4 | |
| Round 1 | ||||||||
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 2 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 |
| 3 | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 1 |
| 4 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 |
| 5 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 |
| Number correct | 3 | 3 | 3 | 1 | 5 | 5 | 5 | 5 |
| Round 2 | ||||||||
| 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 |
| 2 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 |
| 3 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 |
| 4 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 |
| 5 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 |
| Number correct | 3 | 2 | 2 | 1 | 5 | 5 | 5 | 5 |
PMS = premedical student
Round 1: For the AMA Guides 2008, three of four students rated three of five examples correctly, whereas one of four students rated one of five correctly. For the AMA Guides 2024, all four students rated all five examples correctly.
Round 2: For the AMA Guides 2008, one of four students rated three of five examples correctly, whereas two of four students rated two of five correctly, and one of four students rated one of five correctly. For the AMA Guides 2024, all four students rated all five examples correctly. The variability of the impairment values for the AMA Guides 2008 was related to the modifier tables listing mild, moderate, or severe rather than SIEs. The accuracy for the AMA Guides 2024 was related to the anatomical SIEs listed in the physical examination and relevant clinical studies for diagnostic row and linked impairment value in the DBI table.
The AMA Guides 2024 impairment ratings were statistically significantly more accurate (P < 0.0001) compared with the AMA Guides 2008 impairment ratings in both rounds.
Consistency, Reliability, and Reproducibility
Consistency refers specifically to the repeatability of results under the same conditions and is a component of reliability. Reliability is a broader measure that indicates the overall dependability of a measurement process, encompassing consistency and other aspects, including the absence of random errors and the ability to reproduce results across different evaluators and conditions. It is typically assessed through test-retest, interrater, and intrarater reliability. Reproducibility is the ability to achieve the same results across different observers, instruments, or protocols, indicating that the measurement process can be reliably duplicated.
Using the AMA Guides Sixth Edition 2008 and 2024, the physicians reported the correct impairment value for all five examples in both rounds. The data from the premedical students provides insight into the challenges of providing consistent, reliable, and reproducible impairment rating using the AMA Guides 2008 with less experienced evaluators, as seen in Table 5.
Table 5.
Consistency, Reliability, and Reproducibility of the AMA Guides 2008 Versus the AMA Guides 2024a
| 2008 Data | Round 1 | Round 2 | Round 1 | Round 2 | Round 1 | Round 2 | Round 1 | Round 2 |
| Example | PMS1 | PMS1 | PMS2 | PMS2 | PMS3 | PMS3 | PMS4 | PMS4 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 |
| 2 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 |
| 3 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
| 4 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 |
| 5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Overall number correct | 3 | 3 | 3 | 2 | 3 | 2 | 1 | 1 |
| Number correctly matched | 3/5 | 1/5 | 2/5 | 0/5 | ||||
PMS = premedical student
Data from 2024: 5/5 correct; 5/5 correctly matched for all assessments.
The differences between rounds 1 and 2 for the premedical students included the following:
One student matched the same three examples correctly and the same two examples incorrectly.
One student matched one example correctly and one example incorrectly in both rounds, and two examples correctly in round 1 but incorrectly in round 2.
One student matched two examples correctly in both rounds, two examples incorrectly in both rounds, and one example correctly in round 1 but incorrectly in round 2.
One student matched one example correctly in each round but not the same example, while incorrectly matching all other examples.
By contrast, using the 2024 method, all five students reported the correct impairment values for all five examples in both rounds. These data suggest that impairment ratings using the AMA Guides 2024 are more consistent, reliable, and reproducible than impairment ratings using the AMA Guides 2008. The AMA Guides 2008 Kappa statistic was 0.595, indicating that for these data, moderate to good reliability was observed for the AMA Guides 2008 compared with the Kappa statistic of 1.00 for the AMA Guides 2024 data, indicating perfect reliability (data not shown). These differences between these two calculated Kappa values are statistically significant (P < 0.01), indicating that the agreement for the AMA Guides 2024 is statistically significantly better than that for the AMA Guides 2008.
The AMA Guides 2008 primarily relied on generic, nonspecific grade modifiers for functional history, physical examination, and clinical studies, broadly categorized as mild, moderate, severe, or very severe. Evaluators were required to
(1) identify the correct DBI table,
(2) determine the appropriate grade modifiers for functional history, physical examination, and clinical studies,
(3) apply the net adjustment formula (mathematical explanation) to calculate the net adjustment value, and
(4) return to the original DBI table and adjust the impairment grade within a class accordingly to derive adjustment value for the final impairment value.
By contrast, the AMA Guides 2024 introduces DBI tables with structured diagnostic rows, integrating SIEs from the clinical history, physical examination, and relevant clinical studies. This streamlined approach eliminates the need for grade modifiers and net adjustment calculations. Instead, all relevant clinical information is applied within a single DBI table, where the appropriate diagnostic row directly determines the final impairment value.
Table 6 shows that differences between AMA Guides 2008 and 2024 provides a comparative analysis of these key differences, highlighting methodological advancements, improved rating precision, and enhanced impairment determination criteria in the 2024 edition of the AMA Guides.
Table 6.
Differences Between AMA Guides 2008 and 2024
| AMA Guides 2008 | AMA Guides 2024 |
| Identify DBI table | Identify DBI table |
| Obtain data for functional history, physical examination, and clinical studies | Obtain data (SIEs) for clinical history, physical examination, and relevant clinical studies |
| Apply the data to grade modifier tables for functional history, physical examination, and clinical studies | |
| Determine grade modifier value for functional history, physical examination, and clinical studies | |
| Apply grade modifier value to the net adjustment formula | |
| Obtain the net adjustment value | |
| Return to the original DBI table and locate the default impairment value for grade C | |
| Apply the net adjustment value to the final grade with the class | Apply the specific individual elements (SIEs) to the diagnostic row |
| Identify the impairment value based on the final grade | Identify the impairment value based on the correct diagnostic row |
DBI = diagnosis-based impairment
Discussion
This study confirms that the updated methods and enhanced DBI tables in the AMA Guides Sixth Edition 2024 for lower limb impairment evaluations markedly improve the standardized rating process for the ease of use, accuracy, consistency, reliability, and reproducibility (both interrater and intrarater). The AMA Guides 2024 are more user friendly and are particularly advantageous for implementation among inexperienced evaluators.
A modified Delphi method was used to develop an updated method and enhanced DBI tables for the lower limb chapter, while maintaining the AMA criteria of fair and equitable impairment ratings based on objectively verifiable anatomical or physiological findings. The subcommittee determined that requiring an evaluator to use all five steps listed in Table 1 resulted in improvement in the outcome measures of time to complete (ease of use), accuracy, consistency, reliability, and reproducibility.
The three physician experts completed impairment rating evaluations for five example diagnoses using the clinical data provided in the AMA Guides 2008. The outcome measures—accuracy, consistency, reliability, and reproducibility—were 100% for both the AMA Guides 2008 and the AMA Guides 2024 methods. These results were expected given the experts' familiarity and experience with the AMA Guides 2008 method. However, the completion time was still markedly shorter for the AMA Guides 2024, averaging 12.6 minutes less time (16.7 vs. 4.1 minutes, a decrease of 75.4%).
Concerns previously raised about the AMA Guides 2008 included complexity, lengthy completion times, lower accuracy (affected by the need to use multiple tables), and reduced reproducibility (lower intrarater and interrater performance). To measure the effect of these issues, premedical students were asked to rate the same examples as the experts. Their impairment ratings were evaluated using the same outcome measures as for the experts. As anticipated, the ease of use improved with the AMA Guides 2024, resulting in a time savings of 18.7 minutes (24.0 vs. 5.3 minutes) for the premedical students while improving accuracy, consistency, and reliability, with no apparent learning curve required.
These findings suggest that the AMA Guides 2024 methodology not only streamlines the impairment rating process but also enhances the quality and reliability of the evaluations, making it a superior choice for both experienced and novice evaluators. This implies that interpretation of reports by “consumers” (e.g., administrative law judges) would also be improved.
Feasibility of Implementing the Updated Approach to Impairment Ratings
The AMA Guides 2024 five-step approach provides a structured framework for determining the appropriateness and accuracy of impairment ratings, offering step-by-step documentation for easy review, understanding, and quality assurance oversight. Compared with the AMA Guides 2008, the AMA Guides 2024 approach reduces the need for extensive evaluator training and education by incorporating standard healthcare components such as clinical history, physical examination, and relevant clinical studies. The improved consistency and ease of use of the AMA Guides 2024 method are projected to result in cost-benefit savings for the workers' compensation system. However, appropriate jurisdictional or legislative acceptance will be required in many jurisdictions to realize these benefits.
Limitations
This study has several limitations. There is a limited body of research on impairment ratings, with much of the evidence base relying on consensus and expert opinions. Patients have widely varying conditions, impairments for the same conditions, medical histories, and other factors. Robust study designs evaluating impairment ratings, particularly for heterogeneous and complex patient presentations, have not been extensively implemented. Finally, this study used scenarios; however, large-scale assessments involving live patient evaluations would be important for further validation but are challenging due to the inherent variability in patients' responses to questions and tests across different examiners.
Future Research
Research is needed to further validate the effectiveness of the updated AMA Guides 2024 method and its application to other chapters in the AMA Guides involving different body parts. Comparative studies examining the effect of final impairment rating values between previous editions of the AMA Guides and the Sixth Edition 2024 method would help stakeholders understand potential financial effects. Additional research should consider the standardization of evaluation tools, legal and ethical considerations, training and education modules, global perspectives, and the effect on the international classification of functioning, disability, and health.20-22
Although the AMA Guides establish impairment values, the final determination of compensation or disability as a result of an injury or event rests with the adjudicating jurisdiction. It is important to distinguish that the Guides assess impairment, whereas disability determinations consider additional factors such as age, activities of daily living, education, occupation, geographic location, employer accommodations, social support, and community effect, all of which extend beyond impairment ratings.
Conclusion
The updates to the lower limb chapter of the AMA Guides aim to enhance the ease of use while markedly improving the accuracy, consistency, reliability, and reproducibility of impairment ratings. By employing a modified Delphi method, the revisions introduce quality measures and algorithms that offer a structured, evidence-based approach to impairment evaluations. This integration of expert consensus with a rigorous methodology addresses the limitations of previous editions, leading to more detailed and comprehensive evaluations.
Grounded in contemporary science, the AMA Guides Sixth Edition 2024 represents a notable contribution to the field of occupational health by providing a robust, evidence-backed decision-making framework. This approach ensures fair and equitable impairment ratings, reflecting the complexities of clinical practice. The integration of expert opinions with systematic methodologies not only enhances the theoretical soundness of the updates but also ensures their practical applicability, thereby increasing the credibility and utility of the AMA Guides.
Moreover, the structured framework of the AMA Guides 2024 reduces the need for extensive training and education for evaluators by incorporating familiar healthcare components such as clinical history, physical examination, and relevant clinical studies. This integration streamlines the evaluation process, making it more accessible and efficient for both novice and experienced evaluators. The improved consistency and user-friendliness of the AMA Guides 2024 are expected to result in notable cost savings for the workers' compensation system, although realizing these benefits will depend on appropriate jurisdictional or legislative acceptance.
In summary, the updates to the AMA Guides mark a substantial advancement in the standardization and accuracy of impairment ratings, promoting better outcomes for patients and providing more reliable data for healthcare providers and policymakers.
Acknowledgments
The authors thank Bubba Brown for assistance in editing and refining the manuscript before submission. They also acknowledge the efforts of the large number of research technicians, assistants, and other personnel from each research study group that made the collection of the data presented in this article possible. Special thanks to research assistance provided by Cynthia Perkins, BSDH, MLIS, Ascension through Christi Medical Library Wichita, Kansas, and the University of Alberta Library Services Edmonton, Alberta, Canada.
Footnotes
The study was designed and conducted by the authors, who were self-funded; the open access fee is being paid by the American Medical Association.
Dr. Melhorn and Dr. Martin are cochairs of the AMA Guides Editorial Panel for which they receive an administrative fee. Dr. Gelinas is an unpaid member of the Panel's advisory committee. Neither of the following authors nor any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this article: Dr. Hegmann and Dr. Thiese.
Ethical considerations and disclosures: This study was conducted with consideration for ethical standards of research. Because no patient was involvement, the RedCap IRB KUMC determined this study qualified for the designation as a quality improvement.
References
- 1.Kaplan SS: Correlation between the measures of impairment, according to the modified system of the American Medical Association, and function. J Bone Joint Surg Am 1999;81:438-439. [PubMed] [Google Scholar]
- 2.Ranavaya MI, Brigham CR: International use of the AMA Guides® to the Evaluation of Permanent Impairment. AMA Guides Newsl 2020;25:3-8. [Google Scholar]
- 3.Martin DW: Independent Medical Evaluation: A Practical Guide. Springer, 2018. [DOI] [PubMed] [Google Scholar]
- 4.Colledge A, Hunter B, Bunkall LD, Holmes EB: Impairment rating ambiguity in the United States: The Utah impairment Guides for calculating workers' compensation impairments. J Korean Med Sci 2009;24(suppl 2):S232-S241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Brigham C: AMA Guides 6th edition: New concepts, challenges, and opportunities. IAIABC J 2008;45:13-57. [Google Scholar]
- 6.Forst L, Friedman L, Chukwu A: Reliability of the AMA Guides to the Evaluation of Permanent Impairment. J Occup Environ Med 2010;52:1201-1203. [DOI] [PubMed] [Google Scholar]
- 7.Spieler EA, Barth PS, Burton JF, Himmelstein J, Rudolph L: Recommendations to guide revision of the Guides to the Evaluation of Permanent Impairment. JAMA 2000;283:519-523. [DOI] [PubMed] [Google Scholar]
- 8.Patel B, Buschbacher R, Crawford J: National variability in permanent partial impairment ratings. Am J Phys Med Rehabil 2003;82:302-306. [DOI] [PubMed] [Google Scholar]
- 9.Brigham C, Direnfeld LK, Feinberg S, Kertay L, Talmage JB: Independent medical evaluation best practices. AMA Guides Newsl 2017;22:3-18. [Google Scholar]
- 10.AMA Guides® to the Evaluation of Permanent Impairment: An Overview [AMA web site]. https://www.ama-assn.org/delivering-care/ama-guides/ama-guides-evaluation-permanent-impairment-overview. [Google Scholar]
- 11.Kottner J, Audigé L, Brorson S, et al. : Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 2011;64:96-106. [DOI] [PubMed] [Google Scholar]
- 12.AMA Guides Editorial Panel members [AMA website]. https://www.ama-assn.org/delivering-care/ama-guides/ama-guides-editorial-panel-members. [Google Scholar]
- 13.Fitch K, Bernstein SJ, Aguilar MD, et al. : The RAND/UCLA Appropriateness Method User's Manual. RAND, 2001. [Google Scholar]
- 14.AMA Guides Sixth Guides 2024 Public Comment Information [AMA website]. https://ama-guides.ama-assn.org/books/pages/public_comment [Google Scholar]
- 15.Diamond IR, Grant RC, Feldman BM, et al. : Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014;67:401-409. [DOI] [PubMed] [Google Scholar]
- 16.Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG: Guidance on Conducting and Reporting Delphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med 2017;31:684-706. [DOI] [PubMed] [Google Scholar]
- 17.Melhorn JM, Gelinas B, Martin DW, Hegmann KT, Thiese MS. Advancements in AMA Guides musculoskeletal impairment evaluations: Improved reliability and ease of use. J Occup Environ Med 2024;66:737-742. [DOI] [PubMed] [Google Scholar]
- 18.Parab S, Bhalerao S: Choosing statistical test. Int J Ayurveda Res 2010;1:187-191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.McHugh ML: Interrater reliability: The kappa statistic. Biochem Med (Zagreb) 2012;22:276-282. [PMC free article] [PubMed] [Google Scholar]
- 20.Institute of Medicine (US) and National Research Council (US) Committee to Review the Social Security Administration's Disability Decision Process Research, in Wunderlich GS, Rice DP, Amado NL, eds.: The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs (Ch 6: Improving the Disability Decision Process). National Academies Press, 2002. https://www.ncbi.nlm.nih.gov/books/NBK220741 [PubMed] [Google Scholar]
- 21.Angeloni S: Integrated disability management: An interdisciplinary and holistic approach. SAGE Open 2013;3. [Google Scholar]
- 22.Andrews G, Peters L, Guzman AM, Bird K: A comparison of two structured diagnostic interviews: CIDI and SCAN. Aust N Z J Psychiatry 1995;29:124-132. [DOI] [PubMed] [Google Scholar]