Determination of accuracy of polycythemia vera diagnoses and use of the JAK2V617F test in the diagnostic scheme

Abstract

In 2005, three independent research groups described the presence of a specific mutation in the JAK2 gene, JAK2V617F, in patients with a Philadelphia chromosome-negative myeloproliferative neoplasm (MPN). The percentage of patients with the mutation varied according to specific disease with >98%of polycythemia vera (PV) patients having the mutation. In 2008, the World Health Organization issued new diagnostic criteria for PV including use of the JAK2V617F test as a major diagnostic criterion. The goal of the present study is to determine the accuracy of diagnosing PV in a community practice and reporting of PV to cancer registries, as well as assessing the integration of molecular testing into diagnostic paradigms. Using Geisinger Medical Center’s electronic medical records (EMR), patients with a PV diagnosis being seen by a hematologist/oncologist during 2004–2009 were identified. Records were reviewed by a single hematologist/oncologist to determine accuracy of the treating physician’s diagnosis and use of the molecular test for the JAK2V617F mutation. There was a diagnosis of PV from the treating physicians in 121 of the 204 evaluable patients (59 %) and another MPN in 21 (10 %). However, we confirmed a PV diagnosis in only 90 patients (44 %). Of the 90 confirmed PV patients, 64 were JAK2V617F-mutation positive while 24 were not tested. While JAK2V617F testing has made a major impact in facilitating the successful delineation of the type of polycythemia (PV versus secondary polycythemia) in patients evaluated in a large, community-based Hematology/Oncology practice, physician usage of other critical tests is inconsistent leading to errors in diagnosis. JAK2V617F mutation testing in combination with other diagnostic criteria may help reduce diagnostic errors.

Background

Polycythemia vera (PV) is one disease in a group of Philadelphia chromosome-negative myeloproliferative neoplasms (MPN) and is characterized by erythrocytosis, uncontrolled and autonomous hematopoiesis, and evolution to end-stage myelofibrosis or acute nonlymphocytic leukemia. In 1971, the Polycythemia Vera Study Group (PVSG) established clinical guidelines to differentiate PV from secondary polycythemia, stress erythrocytosis, and other causes of an elevated red blood count [1]. Originally intended as a means of confirming patient diagnosis for research study inclusion, these criteria became the standard used by physicians for routine patient evaluation [2]. In 2001, PV was included as a reportable cancer to state cancer registries. At that time, one major weakness identified was the lack of a single biomarker specific for PV. This weakness potentially impacted individual patient diagnosis and the accuracy of cancer registries.

In 2005, three independent groups of researchers described an acquired mutation in the JAK2 gene (JAK2V617F) that could be detected in the granulocytes of most patients with PV [3–5]. This discovery led several authors to propose new diagnostic algorithms for the evaluation of patients with suspected PV [6–8]. Serum erythropoietin (EPO) levels and testing for the JAK2V617F mutation were recommended as initial clinical tests when evaluating patients suspected of having PV. Patients who were both JAK2V617F-positive and had a suppressed serum EPO level were considered as definitely having PV and did not require further testing. A diagnosis of secondary polycythemia was confirmed in patients with neither abnormality. For patients having one of the two abnormalities, further testing was advised, including bone marrow histology and simultaneous red cell mass/plasma volume measurement.

In 2008, the World Health Organization (WHO) adopted new diagnostic criteria for PV and other MPNs, including essential thrombocythemia (ET) and primary myelofibrosis (PMF) [9, 10]. For a diagnosis of PV, patients must have erythrocytosis on multiple determinations (i.e., hemoglobin >18.5 g/dL), molecular evidence of clonality (such as shown by the presence of the JAK2V617F mutation), and one of three minor criteria: a low serum EPO level, an abnormal bone marrow histology exhibiting trilineage hyperplasia, or the presence of in vitro endogenous erythroid colony formation. Documentation of an elevated red cell mass was not required. However, many experts assert that red cell mass is crucial in differentiating PV from other MPNs [11–13]. While the WHO diagnostic criteria clearly separate PV from ET, and both from PMF, these lines of distinction are often blurred with some authors suggesting this group of JAK2V617F mutation-positive MPNs represent different phases of the same disease process [14].

A recent investigation into a cluster of PV cases highlighted these diagnostic issues [15]. Using the newly identified JAK2V617F mutation, diagnoses of PV cases from the state’s cancer registry and from active case findings were validated by an expert panel. Of the 62 enrolled cases, only 53 % of the PV cases reviewed were confirmed as having PV, 32 JAK2V617F-positive cases and 1 JAK2V617F-negative case. For the 29 cases where the PV diagnosis was not confirmed, 27 % were identified as having secondary polycythemia. These data suggest that improvements in diagnostic accuracy stemming from the addition of molecular testing would reduce the rate of error in diagnosis.

The primary objectives of the current study are to evaluate discrepancies between a patient’s recorded diagnosis and the diagnosis established by an expert review and to determine the use of the JAK2V617F mutation test. Additionally, the authors sought to assess the frequency of errors in the tumor registry encompassing both the inclusion of patients with a benign condition, secondary polycythemia, as well as misclassification between PV and other MPNs.

Methods

This is a retrospective study of electronic health records from patients seen in the outpatient clinics of the Geisinger Health System, by clinicians specializing in Hematology and/or Medical Oncology. Geisinger Health System’s electronic medical records from January 1, 2004 through December 31, 2009 were used for case identification. Patients who had at least one outpatient visit with a physician specializing in hematology or medical oncology and a billing code for PV (ICD9 238.4) were identified. Medical information, including serum EPO levels, leukocyte alkaline phosphatase scores, radiologic or other evidence of splenomegaly, and the results of JAK2V617F testing, was obtained for each patient. Institutional review board (IRB) approval for this project was granted by Geisinger’s IRB as study #2012-0144.

Patient charts were reviewed by a single physician (referred to as the reviewer) who has served on numerous expert panels for diagnosis confirmation of PV and other MPNs. The reviewer had access to physician progress notes and medical records. For each chart reviewed, the reviewer attempted to identify both the initial and final diagnosis as described by the treating physician in his/her progress notes. To determine the accuracy of physician diagnosis, the reviewer made a diagnosis based on previously described standards (the 1971 PVSG criteria and the 2008 WHO criteria, including the latter’s criteria for ET and PMF). When the available patient data favored a diagnosis, but did not meet all established criteria for that diagnosis, the reviewer based his diagnosis on the presence or absence of other findings typical in PVand related MPNs (including the presence or absence of panmyelosis and presence or lack of other reasons for secondary polycythemia). Additionally, the reviewer assessed the available clinical, laboratory, and radiographic data to ascertain if the physician’s diagnosis met criteria for the diagnosis of PV as described by either the PVSG or the 2008 WHO criteria (Table 1). Those cases that did not meet either set of criteria were further analyzed to determine if the patient had secondary polycythemia, probable PV, another MPN, or another diagnosis. Additionally, patient information was reviewed to determine if the patient was reported to the Pennsylvania Cancer Registry (PCR) or to Geisinger’s tumor registry.

Table 1.

Diagnosis criteria for PV for the Polycythemia Vera Study Group (PVSG) and the 2008 WHO

PVSGa	2008 WHOb
Major criteria
Elevated red cell mass	Hemoglobin >18.5 g/dL in men, 16.5 g/dL in women or other evidence of increased red cell volume*
Normal arterial O2 saturation	Presence of the JAK2 mutation or other functionally similar mutation such as JAK2 exon 12 deletion
Splenomegaly
Minor criteria**
Leukocytosis >12,000	Bone marrow biopsy showing hypercellularity for age with trilineage growth (panmyelosis) with prominent erythroid, granulocytic, and megakaryocytic proliferation
Thrombocytosis >400,000	Serum EPO level below the reference range for normal
Leukocyte alkaline phosphatase >100	Endogenous erythroid colony formation in vitro
Serum B12>900 or B12 binding capacity >2,200

^aFor the PVSG criteria, two of the minor criteria must be present in cases without splenomegaly

^bDiagnosis requires both major criteria and one minor criterion or the presence of the first major criterion together with two minor criteria

Statistical analysis was performed using either Chi-square tests or Fisher’s exact tests for categorical data, or McNemar’s test for paired categorical data, as appropriate. Weighted kappa statistics were used to assess the level of agreement between the tumor registry codes and the physicians’ diagnoses. The kappa value is measured on a scale of −1.0 to +1.0, where a value of 1 indicates perfect agreement, a value of 0 is exactly what would be expected by chance, and a negative value indicates less agreement than would be expected by chance [16]. All statistical analyses were performed using SAS 9.2 (SAS Institute Inc., Cary, NC).

Results

A total of 284 records were identified as having a billing code for a diagnosis of PV. Of the 284 records, three patients were excluded as they were seen only by pediatric hematology; none of these patients had PV. Additionally, three patients were excluded due to lack of physician documentation within the electronic medical record. One final patient was subsequently excluded since most of his care and evaluation was provided at another facility, and available records did not address the nature of the patient’s diagnostic evaluation. A total of 277 were assessed for inclusion in the analyses performed (Fig. 1).

Fig. 1.

Flow diagram of electronic health record eligibility

In determining accuracy of physician diagnosis, 204 of the 277 charts had sufficient data for the reviewer to make a diagnosis. Among the 204 fully evaluable patients, 143 patients (70 %) had a final diagnosis of an MPN, including 121 PV diagnoses, from the patient’s physician compared to only 124 patients who were diagnosed with an MPN by expert review of their chart (Table 2). The differences in physician diagnoses and reviewer diagnoses were mainly due to patients diagnosed as having PV by the physician but secondary polycythemia by the reviewer. The greatest discrepancy was seen in patients who did not undergo JAK2V617F testing (n=55). Of the patients not tested for the JAK2 mutation, the treating physician diagnosed 82 % as having an MPN compared to 58 % classified as having an MPN by the chart review (p value <0.001). The differences in diagnosis occurred mainly between 2004 (or before) and 2007, after which testing for the JAK2 mutation increased. For the 61 JAK2 mutation-negative patients, the reviewer confirmed only two cases of PV compared to eight physician-diagnosed PV cases (p value=0.04). For all evaluable patients (untested, JAK2V617F-positive, and JAK2V617F-negative), treating physicians were more likely than the reviewer to diagnose PV versus another MPN.

Table 2.

Differences between the physician’s final diagnosis and the reviewer diagnosis as shown by JAK2V617F test status

		JAK2V617F test status
		Not tested (n=55)a	Negative (n=61)	Positive (n=88)	Total (n=204)
PV diagnosis	Physician (%)	41 (75 %)	8 (13 %)	72 (82 %)	121 (59 %)
	Reviewer (%)	24 (44 %)	2 (3 %)	64 (73 %)	90 (44 %)
	p value	<0.001	0.04	0.011
Other MPN diagnosisb	Physician (%)	4 (7 %)	2 (2 %)	16 (18 %)	22 (11 %)
	Reviewer (%)	8 (15 %)	2 (3 %)	24 (27 %)	34 (17 %)
All MPN diagnoses	Physician (%)	45 (82 %)	10 (16 %)	88 (100 %)	143 (70 %)
	Reviewer (%)	32 (58 %)	4 (7 %)	88 (100 %)	124 (61 %)
	p value	<0.001	0.04	1
Non-MPN diagnosisc	Physician (%)	10 (18 %)	52 (85 %)	0 (0 %)	61 (30 %)
	Reviewer (%)	23 (42 %)	57 (93 %)	0 (0 %)	80 (39 %)

^aSample size is determined by physician diagnosis

^bMPNs include ET, PMF, and other cases where the exact MPN diagnosis was uncertain

^cIncludes secondary polycythemia, stress erythrocytosis, and normal

We also assessed the physician-diagnosed PV cases against the newer WHO 2008 guidelines and the PVSG guidelines (Table 3). Of the 121 patients with physician-diagnosed PV, 49 patients met the WHO criteria including 14 patients who met the standard based on abnormal bone marrow histology. Using the PVSG guidelines, only 21 of the 121 patients fulfilled the criteria. In the patients who failed to meet either PVSG or WHO criteria for the diagnosis of PV, physicians frequently either failed to obtain serum EPO levels or considered normal levels as indicative of PV. However, in several of these cases, the only serum EPO level on the chart was drawn years later, often after multiple phlebotomies. The relevance of these later results is debatable, since the published criteria referred to testing obtained on initial assessment.

Table 3.

Patients meeting diagnostic criteria for PV

		Patients meeting diagnostic criteria for PV
		PVSG	WHO	Both
JAK2V617F test status	Not tested	11	0	0
	Positive	10	48	9
	Negative	0	1	0

From the 277 records with a billing code for PV, 268 patient records had sufficient information to determine if each patient was correctly reported to a tumor registry (Table 4). Among these 268 patients, 146 were not registered in either Geisinger’s tumor registry or the Pennsylvania Cancer Registry. Of the 146 non-reported patients with a billing code for PV, 34 (23 %) were diagnosed by their physician as having PV, and 21 patients (14 %) were diagnosed with another MPN (Table 4). Seventy non-reported patients were diagnosed as having secondary polycythemia, and 21 were diagnosed with another, nonmalignant hematologic diagnosis. A total of 122 patients were reported to a tumor/cancer registry and were registered as PV (111 patients) or another MPN (11 patients). Seven of the 122 patients registered with an MPN were diagnosed by their physician as having secondary polycythemia. Statistical analysis revealed a weighted kappa of 0.54 (95 % CI 0.45–0.63, p value <0.001), indicating “moderate” agreement between the tumor registry code and the physician’s diagnosis.

Table 4.

Comparison between patient registration in a cancer registry and patient’s diagnosis (as determined by the patient’s treating physician)

Registry reporting statusa (n=268)	Physician’s diagnosisb
	Total sample size	PV	Other MPN	Secondary polycythemia	Other condition
Not in registry	146	34 (23 %)	21 (14 %)	70 (50 %)	21 (14 %)
In registry as PV	111	91 (82 %)	11 (10 %)	7 (6 %)	2 (2 %)
In registry as Other MPN	11	5 (45 %)	5 (45 %)	0	1 (9 %)

^aReported to Pennsylvania Cancer Registry or to the Geisinger Tumor Registry

^bPhysician’s diagnosis is determined from the chart progress notes

Discussion

In 2006, a new diagnostic paradigm for the assessment of patients with erythrocytosis was introduced. The WHO subsequently used this paradigm as a basis for their criteria for the diagnosis of PV [9]. The introduction and use of the JAK2V617F test has reduced one type of physician error, the diagnosis of PV in patients with secondary polycythemia. In the present study, JAK2V617F results frequently, but not universally, impacted the final diagnosis assigned by the patient’s hematologist/oncologist.

In the present study, only 49 of the 121 patients with physician-diagnosed PV met the 2008 WHO guidelines for PV. This is similar to a previous report which used criteria similar to the WHO guidelines. In that study, 18 PV patients of 38 participating, eligible cancer registry patients had a confirmed diagnosis of PV [15]. However, recent studies have recommended revision of the WHO guidelines, especially for diagnosis of patients with early-stage PV [11]. Silver et al. recommend including red cell mass determination for patients without increased hemoglobin or hematocrit. Conversely, some experts have suggested the reclassification of MPNs into a continuum of phases of the same disease [14].

The use of the JAK2V617F assay has been increasing since the introduction of the mutation as a diagnostic criterion. In the current study, physicians determined that several patients did not have PV based on the absence of the mutation. For several JAK2V617F-positive patients, the presence of the mutation resulted in a diagnostic change from suspected secondary polycythemia to PV. Physicians diagnosed 36 % of JAK2V617F-positive patients as having PV simply on the basis of molecular testing without either serum EPO or bone marrow testing. This study confirms that specialists in both medical oncology and hematology have adopted testing for this marker in their usual approach to patients with polycythemia and confirms the clinical utility of this assay. However, the physicians did not fully utilize established diagnostic criteria as promulgated by the WHO. At best, one can argue that a JAK2V617F-positive patient has a MPN, but physicians cannot distinguish PV from ET or another MPN solely on the basis of molecular testing [14]. Since the recognition of the JAK2V617F mutation, the American Society of Hematology has held an educational session focusing on these diseases at its annual meeting with the intent of educating physicians on appropriate diagnosing and treatment of PV and other MPNs. Additional attention has been focused on this family of diseases with the advent of a therapeutic agent that specifically targets the JAK–STAT pathway.

This study also identified discrepancies between the final diagnosis, as listed in the patient progress notes, and whether or not the same case was entered into an appropriate tumor registry. The discrepancies included registering PV patients who were ultimately diagnosed with secondary polycythemia, not registering patients with a final diagnosis of PV, and registering patients as PV cases who had another MPN. A source of registry errors was the observation that physicians who complete a work-up of a patient with erythrocytosis, and ultimately diagnose secondary polycythemia, do not update the problem list and ICD-9 billing code. While we did not specifically quantify this observation, it occurred in most of the cases where the physician’s final diagnosis was other than PV. This data identifies three flaws with the tumor registry—failure to capture all patients with PV, misclassification of patients with secondary polycythemia, and cases where PV was confused with other MPNs.

The current study has several limitations. The cohort from which the electronic health records were obtained was from one health care system and may not reflect practices at other health care systems. Additionally, cases were obtained via the ICD-9 code for PV and may not fully encompass patients seen only for ET or PMF. Lastly, the study period includes a time when diagnostic criteria for PV evolved to include molecular testing for clonality, specifically the JAK2V617F mutation. As molecular testing becomes part of standard practice and the new WHO diagnostic criteria are fully implemented, the errors described are expected to decline.

Conclusion

This study indicates several sources of error in both how physicians diagnose MPNs and how they are entered into tumor registries. While implementation of JAK2V617F mutation testing is occurring, physicians are not fully using the new WHO diagnostic paradigm. The results of this study indicate that epidemiologists studying MPNs, especially PV, should be aware of these sources of error and that attempting to ascertain the accuracy of patient data within any cohort being studied might help reduce error.