Unintended consequences of Mayo paraneoplastic evaluations

Abstract

Objective

To determine the proportion of true and false positives from paraneoplastic panels and effects on downstream testing/treatment.

Methods

Using a retrospective cohort study design, we identified 500 consecutive patients with Mayo paraneoplastic autoantibody testing and performed chart abstraction. Paraneoplastic presentation types were categorized into probable, possible, and other by consensus. True positives were defined as a positive antibody titer with no other explanation found in addition to one of the following: syndrome known to be associated with the antibody, clinical improvement with treatment, and new malignancy. Comparisons of diagnostic testing and treatments between false and true positives were performed. Multivariable logistic regression was used to evaluate associations between patient-level factors and true positives.

Results

The mean (SD) age of the population was 55.4 (17.1) years, and 55.4% were female, with 1.3 (1.2) years of follow-up. Of the 500 tests, 87 (17.4%, 95% confidence interval [CI] 14.1%–20.7%) were positive and 62 (71.3%, 95% CI 61.8%–80.8%) of these were false positives. Of those with a possible/other presentation (n = 369), 2 (0.5%, 95% CI 0.0%–1.0%) were true positives. CT of the chest (30.7% vs 11.8%, p ≤ 0.01) was performed more often in false positives than true negatives. Probable presentation type (odds ratio [OR] 57.9, 95% CI 12.5–268.0) and outpatient setting (OR 8.7, 95% CI 2.4–31.8) were associated with true-positive results.

Conclusion

Paraneoplastic tests result in a large proportion of false positives, particularly in those with clinical presentations that are not well established as paraneoplastic diseases. Future work should construct panels targeted to specific clinical presentations and ensure that tests are ordered in the appropriate clinical context.

Paraneoplastic neurologic syndromes are rare autoimmune conditions caused by the remote effects of malignancy.¹ Most are discovered by identifying a characteristic autoantibody.² These same autoantibodies can be found with similar clinical presentations but without malignancy.³ Some autoantibodies are associated with malignancy more often than others. Testing for these autoantibodies has revolutionized neurologic care by allowing characterization of previously unknown syndromes.^4–6 Furthermore, diagnosis may lead to earlier cancer diagnosis or immunomodulatory treatment.^1,7,8

While testing for paraneoplastic syndromes has improved patient care, current approaches may lead to unintended consequences. For example, the Mayo paraneoplastic panel consists of 15 antibodies with reflex testing of 6 additional antibodies. The high number of antibodies has the potential to increase sensitivity at the cost of specificity.⁹ Furthermore, the antibodies within this panel are each associated with specific neurologic presentations, some of which are localized to the CNS (e.g., anti-Ri) and others to the peripheral nervous system (e.g., acetylcholine receptor antibody).² Because these are combined into 1 panel, patients are tested for antibodies that are not associated with their specific clinical presentation. Finally, there is a potential for indication creep especially because testing panels are not based on the patient's clinical presentation.¹⁰

Given potential unintended consequences, we aimed to determine the proportion of true and false positives from the Mayo paraneoplastic panel using a retrospective cohort from a tertiary care center. We also investigated whether test characteristics differ by clinical presentation type and the associations of autoantibody testing with test/treatment use. Finally, we determined patient-level factors associated with true positives.

Methods

Population

We identified 500 consecutive patients at the University of Michigan Health System who had the Mayo serum paraneoplastic autoantibody evaluation (mayomedicallaboratories.com/test-catalog/2011/Overview/83380) sent in either the inpatient or outpatient setting from June 1, 2013, to March 26, 2014, and performed detailed medical chart abstraction (table 1). Specifically, we extracted the results for each of these panels, including the frequency of positive results for each antibody type. The charts were reviewed for patient demographics, ordering provider specialty (neurologist vs other), involvement of neurologic consultation service, smoking history, presence of weight loss, current and past cancer diagnoses, presenting neurologic symptoms and examination findings, time course, lumbar puncture results, imaging, biopsies, and immune or cancer treatments provided. Follow-up data were available for up to 4 years after the tests were performed. Of note, the Mayo paraneoplastic panel does not include all currently known encephalitis antibodies.

Table 1.

Antibodies included in the Mayo serum paraneoplastic autoantibody evaluation and paraneoplastic presentation types

Definition of paraneoplastic presentation types

Patient clinical presentations were categorized into probable paraneoplastic, possible paraneoplastic, and other by consensus of 3 neurologists (B.C.C., M.J.E., B.R.C.) starting with the clinical paraneoplastic presentations suggested by Graus et al.² and modifying to adequately reflect autoimmune presentations. Table 1 provides details.

Definition of paraneoplastic test result categories

We defined a true-positive result as an antibody titer above the normal range as provided by the Mayo Medical Laboratory with no other explanation found for the clinical presentation in addition to at least one of the following: the syndrome is known to be associated with the antibody, clinical improvement with immunosuppression or tumor treatment, or a new malignancy was found. If none of these criteria were met or another explanation was found, then a positive antibody titer was deemed a false-positive result. This definition was designed to err on the side of categorizing test results as true positives because it would classify monophasic illnesses that received treatment and incidentally identified tumors as true positives. Investigators were not blinded to the antibody result because one of the true-positive criteria was that the syndrome is known to be associated with the antibody. A true-negative result was defined as a negative antibody titer and either another explanation was found for the clinical presentation or no clinical improvement with treatment and no new malignancy was found. A false-negative result was defined as a negative antibody titer, no other explanation found, but clinical improvement with treatment or a new malignancy was found. To be considered a false-negative result, we did not require a syndrome highly compatible with a paraneoplastic disorder.

Measures of agreement

All cases with a positive autoantibody were reviewed independently by 2 neurologists (M.J.E. and B.R.C.). A κ coefficient was calculated on the basis of the categorization of paraneoplastic presentation types and test result categories. Cases with discrepancies were resolved by consensus (M.J.E., B.R.C., and B.C.C.).

Statistical analysis

Descriptive statistics were used to describe the demographic and clinical characteristics of the population. Confidence intervals (CIs) were calculated from the binomial distribution and the sample size of our population. Pearson χ² tests or Fisher exact tests (categorical variables) and t tests (continuous variables) were used to assess differences in the demographic and clinical variables between those with true-positive and those with false-positive results and between those with a paraneoplastic disease (true positives and false negatives) and those without (false positives and true negatives). Descriptive statistics were used to summarize the paraneoplastic test results stratified by clinical presentation type (probable, possible, other prespecified). Pearson χ² tests or Fisher exact tests were used to assess differences in the test and treatment use after paraneoplastic evaluation between those with true-positive and those with false-positive results and between those with false-positive and those with true-negative results.

Multivariable logistic regression was used to model the likelihood of a true-positive result as a function of clinical presentation type (probable vs possible/other), testing in the outpatient setting, age, sex, current or remote cancer, current or past smoking, weight loss, and time course (<6 months vs >6 months).

We reported the number of positive antibody results for each individual antibody and the antibody test result categories stratified by neurologist involvement (consulting or ordering the panel). We determined the number of false-positive and true-positive results for each specific presentation type as long as there were at least 3 positive antibody tests. We calculated the number of false-positive and true-positive results for each specific antibody type as long as there were at least 3 positive antibody tests.

All analyses were performed with R version 3.2.3 (R Foundation for Statistical Computing, Vienna, Austria).

Standard protocol approvals, registrations, and patient consents

The Institutional Review Board at the University of Michigan determined that this study was exempt.

Data availability

Anonymized data will be shared by request from any qualified investigator.

Results

Between June 1, 2013, and March 26, 2014, 500 serum paraneoplastic evaluations were sent to Mayo Medical Laboratory. In regard to categorizing paraneoplastic presentation types and test result categories, physician agreement was high (κ = 0.81 and 0.75, respectively). The mean (SD) age of the population was 55.4 (17.1) years; 55.4% were female (table 2). Remote cancer was reported in 17.4% with 8.2% having a current cancer. Patients had a mean (SD) of 1.3 (1.2) years of follow-up. Demographics were similar between those with true-positive, false-positive, true-negative, and false-negative results. No significant differences were observed between those with true-positive and false-positive results. Those with a paraneoplastic syndrome (true positives and false negatives) were older (62.0 vs 54.8 years, p ≤ 0.01), had longer follow-up (661.7 vs 471.6 days, p = 0.02), had more ataxia (23.7% vs 10.8%, p = 0.03), had more lumbar punctures performed (52.6% vs 5.8%, p ≤ 0.01), and more frequently had elevated CSF white blood cell counts (35.0% vs 13.2%, p = 0.02) compared with those without a paraneoplastic syndrome (false positives and true negatives).

Table 2.

Demographic and clinical variables in those with true-positive, false-positive, true-negative, and false-negative test results

Of the 500 panels, 87 (17.4%, 95% CI 14.1%–20.7%) revealed positive results (table 3). Of these, 15 (17.2%, 95% CI 9.3%–25.2%) had multiple antibodies found. On the basis of our test result categories, 25 of the positive cases were true positives (28.7%, 95% CI 19.2%–38.2%) and 62 were false positives (71.3%, 95% CI 61.8%–80.8%). Of the 413 negative panels, 400 (96.9%, 95% CI 95.2%–98.5%) were true negatives. Of all cases with a probable presentation type (n = 131), 17.6% (95% CI 11.0%–24.1%) were true positives and 3.1% (95% CI 0.1%–6.0%) were false positives. Therefore, in patients with probable presentations, positive tests were true positives 85.2% of the time and false positives 14.8% of the time. Of all cases with a possible or other presentation type (n = 369), 0.5% (95% CI 0.0%–1.0%) of results were true positives and 15.7% (95% CI 12.0%–19.4%) were false positives. Therefore, in patients with possible or other presentations, positive tests were true positives 3.3% of the time and false positives 96.7% of the time. Table 4 provides detailed diagnostic test characteristics.

Table 3.

Test results (true positive, false positive, true negative, and false negative) for different clinical presentations (probable, possible, and other)

Table 4.

Diagnostic test characteristics for detection of a paraneoplastic disease

The antibody that was most commonly found was the striated muscles antibody (n = 28, 5.6%), followed by the P/Q-type calcium channel antibody (n = 23, 4.6%), acetylcholine ganglionic receptor antibody (n = 15, 3.0%), K+ channel antibody (n = 12, 2.4%), N-type calcium channel antibody (n = 9, 1.8%), acetylcholine receptor antibody (n = 9, 1.8%), glutamic acid decarboxylase antibody (n = 5, 1.0%), and ANNA-1 antibody (n = 1, 0.2%) (figure 1A).

Figure 1. Paraneoplastic autoantibody test results.

Overall positive test results by (A) antibody and (B) breakdown of true-positive, false-positive, true-negative, and false-negative test results by neurologist involvement.

A neurologist was involved (as either primary treating physician or in consultation) in 429 (86%) of the paraneoplastic panels ordered (figure 1B). Of the 71 cases ordered without neurologist involvement, 0 were true positives, 8 were false positives, 61 were true negatives, and 2 were false negatives.

Commonly used diagnostic screening tests and treatments for malignancy and autoimmune conditions were compared between test result categories (table 5). PET scans (36.0% vs 3.2%, p ≤ 0.01), biopsies (16.0% vs 0%, p ≤ 0.01), IV immunoglobulin (28.0% vs 8.1%, p = 0.03), and steroids (52.0% vs 11.3%, p ≤ 0.01) were performed significantly more frequently in patients with true-positive results than in patients with false-positive results. CT of the chest (30.7% vs 11.8%, p ≤ 0.01) was performed significantly more frequently in patients with false positives than true negatives. CT of the abdomen/pelvis (p = 0.08), biopsies (p = 0.09), and plasmapheresis (p = 0.09) were performed more frequently in patients with false positives than true negatives, but the results were not statistically significant.

Table 5.

Comparing tests and treatments by test result (true negative and false negative)

The paraneoplastic panel was sent for myriad clinical syndromes with varying test characteristics (figure 2A). Lambert-Eaton myasthenic syndrome (6 of 6 positive results were true positives), myasthenia gravis (5 of 5), limbic encephalitis (4 of 6), subacute cerebellar degeneration (3 of 3), and encephalomyelitis (2 of 3) had a higher proportion of true positives than false positives and at least 3 positive results. In contrast, chronic sensorimotor polyneuropathy (2 of 13 positive results were true positives), myelopathy (0 of 8), generalized pain syndromes (0 of 8), gastrointestinal dysmotility (0 of 5), motor neuron disease (0 of 3), parkinsonism (0 of 3), CNS inflammatory disease (0 of 3), and acute/subacute sensory motor neuropathy (1 of 3) had a higher proportion of false positives than true positives and at least 3 positive results.

Figure 2. Breakdown of true-positive and false-positive results by clinical presentation and antibody type.

Number of true-positive and false-positive results by (A) clinical presentation and (B) antibody type. AChR = acetylcholine receptor; LEMS = Lambert-Eaton myasthenic syndrome.

The N-type calcium channel antibody was a false positive in 8 of 9 (89%) cases, acetylcholine receptor ganglionic neuronal antibody in 12 of 14 (86%), striated muscle antibody in 22 of 28 (79%), P/Q-type calcium channel antibody in 15 of 23 (65%), neuronal voltage-gated K+ channel antibody in 8 of 12 (64%), and acetylcholine receptor antibody in 4 of 9 (44%) (figure 2B). In contrast, the GAD65 antibody was a true positive in 5 of 5 cases and the ANNA-1 antibody in 1 of 1 (100%) case.

Multivariable logistic regression analyses showed that true-positive results were significantly associated with a probable presentation type (odds ratio [OR] 57.9, 95% CI 12.5–268.0) and testing in the outpatient setting (OR 8.7, 95% CI 2.4–31.8). Age (OR 1.02, 95% CI 0.99–1.05), sex (female, OR 0.9, 95% CI 0.3–2.3), current or remote cancer (OR 1.9, 95% CI 0.6–5.5), current or past smoking (OR 1.2, 95% CI 0.4–3.4), weight loss (OR 1.7, 95% CI 0.5–5.9), and time course (<6 months, OR 2.8, 95% CI 0.9–9.3) were not significantly associated with true-positive results.

Among our 500 cases, 64 had both serum and CSF paraneoplastic panels sent. Of these cases, 10 had a positive serum panel, but none had a positive CSF panel. Four of these 10 positive serum panels were determined to be true-positive results.

Discussion

In our retrospective cohort at a tertiary care center, the Mayo paraneoplastic panel was frequently ordered at an average of 1.68 panels per day. Neurologists play a large role in the use of this test; they were involved in 86% of the panels, including all with a true-positive result. Unintended consequences of panel testing included a high proportion of false positives compared to true positives (≈2.5 to 1) and use in clinical presentation types that are unlikely to be associated with paraneoplastic disorders (indication creep). Furthermore, 5 of the 6 antibodies that are most frequently positive are indicated primarily for specific peripheral nervous system disorders (N-type and P/Q-type calcium channel antibodies, acetylcholine receptor ganglionic neuronal antibody, striated muscle antibody, and acetylcholine receptor antibody), but because the panel is most often sent for CNS disorders, these peripheral antibodies are often sent for an inappropriate indication.¹¹ Therefore, these antibodies likely represent an opportunity to improve the test characteristics of paraneoplastic panel testing. Interventions to improve paraneoplastic test ordering include constructing panels that are specific to clinical presentations and providing clinical decision support to facilitate testing in appropriate populations.

A previous study investigating the test characteristics of the Mayo paraneoplastic panel revealed similar results.¹¹ Using a comparable definition of paraneoplastic disease, those investigators found that 15.9% of panels revealed a positive result with 78.4% of positive results representing false positives. In our population, 17.4% of panels revealed a positive result with 71.3% of positive results representing false positives. Other investigators have also found a high positive rate (12% and 14%).^12,13 Given the similar results across 2 populations, the evidence is mounting to suggest that improvements in our current approach to paraneoplastic testing are needed.

In contrast to previous work, our study is larger, formally tested measures of agreement for important judgment decisions, and stratified results by clinical presentation type. We found that all but 2 of the true-positive tests were found in those with presentation types that are well known to be associated with paraneoplastic diseases. Limiting testing to these patient populations would reduce the number of tests by 74% without significantly sacrificing sensitivity. Testing in certain clinical presentations such as myelopathy, generalized pain or migraine, and gastrointestinal dysmotility resulted in frequent false positives. Indication creep may be the result of multiple factors, including panels that are targeted for a broad range of paraneoplastic conditions rather than specific clinical presentations and research suggesting associations of antibodies with almost all neurologic conditions. Previous studies using Mayo Medical Laboratory results have shown that a vast array of clinical presentations can be seen in those with positive autoantibodies.^14–16 However, these investigations do not report a gold standard definition for paraneoplastic diseases; therefore, the test characteristics for different clinical presentation types are unclear. Furthermore, the high positive rates in control and neurologically asymptomatic patients with cancer suggest the potential for high false-positive rates such as demonstrated in this study. Development of paraneoplastic panels that are targeted to specific clinical presentations types is a viable strategy for mitigating these limitations. Panels for myasthenia gravis and Lambert-Eaton myasthenic syndrome already exist, but panels for encephalitis, subacute autonomic neuropathy, sensory neuronopathy, and stiff-person syndrome are examples of panels that are needed to replace current all-encompassing panels. While Mayo Medical Laboratory offers panels for encephalopathy, epilepsy, dementia, gastrointestinal dysmotility, and dysautonomia, the primary difference between panels is which antibodies are directly assessed and which ones are reflex testing only. New panels that are designed for specific clinical scenarios may reduce the risk of false-positive findings.

Unlike previous studies, we also evaluated for clinical predictors of true-positive results and for associations of false positives on downstream testing. We found that only 2 predictors were significantly associated with true-positive results. A probable presentation type had an OR approaching 60, whereas an outpatient presentation had an OR of ≈2. Therefore, the focus of clinicians should be on ordering these tests in the appropriate clinical presentations rather than other patient-level factors. We found that true-positive results are associated with many downstream diagnostic tests and treatments compared to false positives. Similarly, false positives are associated with many downstream diagnostic tests and treatments compared to true negatives, although the difference is only significant for chest CTs. These results suggest that physicians can distinguish true-positive results from false-positive results to some degree but that false-positive results likely lead to some downstream diagnostic cascades. These cascades can lead to more unneeded tests and treatments.^17,18 Future larger studies are needed to determine whether the increased testing and treatments lead to downstream harms.

The 6 autoantibodies that are most frequently positive all have potential reasons not to be included in a large panel for paraneoplastic disease. Striational muscle antibodies are the most frequent false-positive results in our population. Moreover, it is unclear which neurologic presentations are associated with this antibody other than myasthenia gravis, for which other antibodies have much better test characteristics. While positive striational muscle antibodies are most useful in indicating the potential for a thymoma in those with myasthenia gravis,¹⁵ patients with this condition are already routinely screened for this cancer.^19,20 Thus, 1 simple approach to limiting false positives may be to remove this antibody completely from paraneoplastic panels. P/Q- and N-type voltage-gated calcium channel antibodies are also frequent false positives. While these antibodies are well known to be associated with Lambert-Eaton myasthenic syndrome, associations with other neurologic presentations are much less clear. Positive rates that are comparable in populations with neurologic symptoms, healthy controls, and neurologically asymptomatic patients with cancer raise questions about their utility.¹⁶ The authors even suggest caution in interpreting low and medium titer results even though only 3% of the positive results they present are high titers. Likewise, ganglionic acetylcholine receptor antibodies are frequently false positives and have comparable positive rates in populations with neurologic symptoms, healthy controls, and neurologically asymptomatic patients with cancer.¹⁴ The neurologic presentation specificity (subacute autonomic neuropathy) also greatly declines as the titer decreases. Furthermore, voltage-gated potassium channel antibodies without antibodies to LGI1 and CAPR2 are not associated with autoimmune disease.²¹ Replacement of testing for voltage-gated potassium channel antibodies with testing for antibodies to LGI1 and CAPR2 would likely decrease the positive rate by half. Finally, muscle acetylcholine receptor antibodies are frequent false positives and have been shown to be associated only with myasthenia gravis.^22,23 Testing with this antibody should be limited to those with myasthenia gravis–like presentations and should include reflex testing to MUSK antibodies if negative. Whether paraneoplastic panel testing for patients with myasthenia gravis presentations occurs at other locations is unclear, but optimal care would be to focus on muscle acetylcholine receptor antibody and MUSK testing in these patients. Given the current evidence, the Mayo paraneoplastic panel could be greatly improved by removing the antibodies with the 6 highest positive rates and limiting testing of these antibodies to specific clinical presentations such as Lambert-Eaton myasthenic syndrome, subacute autonomic neuropathy, and myasthenia gravis. Striational muscle antibody testing has no clear current role, and voltage-gated potassium antibody testing should be replaced with more specific antibodies.

Limitations include the retrospective cohort design, which requires medical record chart abstraction and judgments as to the clinical presentation types and test result categories. We mitigated these potential issues by requiring 2 physicians to perform these assessments with high agreement and resolving differences through consensus. Investigators were not blinded to antibody results when determining test result categories because a syndrome known to be associated with the antibody was a criterion for true positives. Eliminating this criterion would have only increased the proportion of false positives. The small size of the current study limits our ability to determine clinical predictors of true positives and associations of false positives on downstream testing. However, despite this limitation, we found significant associations. Similarly, the small sample size limits definitive conclusions about the diagnostic test characteristics of the Mayo paraneoplastic panel, particularly for individual antibodies. The generalizability of these results to other practice settings is unclear and requires further study. The lack of a consensus gold standard definition of paraneoplastic disease is a potential issue; however, other investigators have used a similar definition and found comparable results. Likewise, no consensus exists on which clinical presentations are associated with paraneoplastic diseases. Despite this fact, we found only 2 true positives among patients we categorized as having possible or other presentation types. The short length of follow-up of this study, a mean of 1.3 years, may have led to an underestimate of cancer detection, which can occur several years later. Our definition of true positives was designed to err on the side of categorizing positive test results as true positives as opposed to false positives to provide the most generous assessment of the diagnostic characteristics of the Mayo paraneoplastic panel. Despite our definition, the proportion of false positives was high. Similarly, false negatives may include those with unknown autoimmune disorder, although the proportion in this category was low. The diagnostic characteristics of the individual autoantibodies vary greatly. For example, all positive ANNA-1 and GAD65 antibody results were true positives, although the numbers were small. We were also unable to determine the diagnostic characteristics of antibodies detected by cell-based assays because none of these antibodies were positive in our sample. Our results pertain to the Mayo serum paraneoplastic autoantibody evaluation. Other laboratories with paraneoplastic testing likely have different test characteristics based on the antibodies included in their panels. Furthermore, the Mayo paraneoplastic panel includes only a subset of all paraneoplastic antibodies, each with its own diagnostic test characteristic.

The Mayo paraneoplastic panel was ordered frequently, mostly with neurologist involvement, and had a high positive rate. Of the positive cases, the majority were false positives. Panel testing consisting of a large number of autoantibodies designed for many different clinical presentations has led to unintended consequences, including a high proportion of false positives, indication creep, and testing for antibodies that are not associated with each patient's specific clinical presentation. Interventions to improve current panels and aid physicians to order them in the appropriate clinical context are needed.

Glossary

CI

confidence interval

OR

odds ratio

Author contributions

Brian Callaghan, James Burke, Shih-Hon Li, and Matt Ebright were involved in the study design, interpretation of the statistical analysis, and critical revisions of the manuscript. Matt Ebright also wrote the manuscript. Ben Claytor and Anna Grisold were integrally involved in interpretation of the data and critical revisions of the manuscript. Evan Reynolds and Mousumi Banerjee were involved in performing and/or interpreting the statistical analysis and critical revisions of the manuscript.

Study funding

Dr. Callaghan is supported by an NIH K23 grant (NS079417) and a VA CSRD Merit (CX001504). Dr. Burke is supported by National Institute of Neurological Disorders and Stroke K08 NS082597 and R01 MD008879.

Disclosure

M. Ebright, S. Li, and E. Reynolds report no disclosures relevant to the manuscript. J. Burke has received compensation from AstraZeneca for his role on the adjudication committee of the Acute Stroke or Transient Ischaemic Attack Treated With Aspirin or Ticagrelor and Patient Outcomes (SOCRATES) trial. B. Claytor, A. Grisold, and M. Banerjee report no disclosures relevant to the manuscript. B. Callaghan consults for a Patient-Centered Outcomes Research Institute grant, consults for the immune tolerance network, and performs medical legal consultations. Go to Neurology.org/N for full disclosures.

Publication history

Received by Neurology April 10, 2018. Accepted in final form August 14, 2018.