MedCV: An Interactive Visualization System for Patient Cohort Identification from Medical Claim Data

Abstract

Healthcare providers generate a medical claim after every patient visit. A medical claim consists of a list of medical codes describing the diagnosis and any treatment provided during the visit. Medical claims have been popular in medical research as a data source for retrospective cohort studies. This paper introduces a medical claim visualization system (MedCV) that supports cohort selection from medical claim data. MedCV was developed as part of a design study in collaboration with clinical researchers and statisticians. It helps a researcher to define inclusion rules for cohort selection by revealing relationships between medical codes and visualizing medical claims and patient timelines. Evaluation of our system through a user study indicates that MedCV enables domain experts to define high-quality inclusion rules in a time-efficient manner.

1. INTRODUCTION

Healthcare providers generate a medical claim after each patient visit as a list of medical codes describing the diagnosis and treatment. The two most common medical code vocabularies are the International Classification of Diseases (ICD) and Current Procedural Terminologies (CPT). Given the wide range of human diseases and the available treatments, the number of available medical codes is very large (in tens of thousands). Thus, it is challenging for humans to read and interpret medical claims.

Thanks to the ubiquity and standardized format of medical claim data, they have been popular in medical research as a data source for retrospective cohort studies of the association between factors such as demographics, diagnosis, and treatment and outcomes such as mortality and readmission [10, 12, 26, 29]. The first step in any study using medical claims is cohort identification [19, 27] or patient phenotyping [9, 25], which refer to identifying patients with a particular characteristic [3, 16]. Cohort identification requires specifying an inclusion criterion in a programmatic way that scans the medical claims and accurately selects the patients of interest. Writing an inclusion criterion by defining a set of codes corresponding to a particular diagnosis or treatment is daunting due to the vast number of medical codes.

The primary purpose of the proposed visualization software is to help medical researchers define inclusion criteria. In particular, we provide a visual interface that displays medical codes related to a query code. When a user enters a query code, the software shows other codes that either occur in similar types of claims or co-occur with the query code. To identify codes that occur in similar types of claims, we train word2vec [15] model on a medical claims database and use the cosine distance between the resulting code embeddings as the similarity measure. In order to identify codes that tend to co-occur with query code, we use the pairwise mutual information (PMI) metric [24]. In addition to finding the related codes, the software enables users to drill down and visualize claims and patient timelines containing a particular code or sets of codes.

Several data visualization tools have been proposed to help users gain insight into a large collection of electronic health records and medical claims, such as analyzing event sequences of multiple patients to understand associations between diagnoses, treatments, and outcomes [11, 17, 23, 28]. Composer is a representative visual tool [23] that helps orthopedic surgeons dynamically define treatment patterns from patient medical histories to analyze patient-reported outcomes. Rule-based queries were proposed to specify temporal constraints in cohort identification [11]. Medical event embedding was used for visual progression analysis by dividing the hospital visits into several stages [8]. However, we are unaware of prior research on visual analytics tools that support cohort identification from medical claims by exploiting similarity metrics between medical codes.

2. METHODOLOGY AND SYSTEM ARCHITECTURE

We use two metrics to define medical code similarity. The first relies on word2vec model [15] that represents codes as vectors, such that codes that occur in similar types of medical claims have similar representation [1, 2, 7]. The second uses PMI [24] that defines codes as similar if they tend to co-occur in claims. There is a mathematical relationship between PMI and word2vec similarity measures [22].

2.1. Word2vec method

Word2vec [15] has been used in medical informatics to represent medical words and codes as low dimensional vectors, or embeddings [1, 2, 5–7, 20]. In our application, word2vec model (skip-gram variant) treats every claim as a document and every code in a claim as a word. Let us denote a claim with K codes as N = {c₁, c₂, c_K}. The model calculates the log-likelihood of observing all codes in a claim given the remaining code in the claim c_t as

$$ℒ=\sum _{c_i\in C_{c_t}}\text{log\hspace{0.17em}}p\left(c_i\mid c_t\right),$$ (1)

where P(c_i|c_t) is the conditional probability of observing code c_i given c_t.

If U is a matrix of the scanned code embeddings and V is the matrix of the context code embeddings, where U, $V\in {ℝ}^{\left|V_{\mathit{\text{code}}}\right|\times f}$ and |V_code| is the total number of codes and f is the embedding dimension, then the conditional probability is defined using softmax function as

$$p\left(c_i\mid c_t\right)=\frac{e^{U_{c_t}\cdot V_{c_i}}}{{\sum }_{c_j\in \left|V_{\mathit{\text{code}}}\right|}{e}^{U_{c_t}\cdot V_{c_j}}},$$ (2)

where $U_{c_t}$ and $V_{c_i}$ are vectors of codes c_t and c_i, respectively. A stochastic gradient algorithm is used to maximize the objective function to learn vector representations of all the codes in the dataset. The cosine distance between the learned vector representations is a similarity score between two codes. The high cosine distance value means that two codes occur in similar types of claims, indicating that they have a similar meaning.

2.2. PMI method

PMI [24] measures if two codes tend to co-occur more often than by chance. The PMI between codes c_i and cj is defined as

$$\mathit{\text{PMI}}\left(c_i,c_j\right)=\mathit{\text{log}}\frac{P\left(c_i\mid c_j\right)}{P\left(c_i\right)},$$ (3)

where P(c_i | c_j) is the conditional probability of observing code c_i given code c_j in the same claim and P(c_i) is the marginal probability of seeing c_i in any claim. The probabilities can be calculated based on the counts of occurrence and co-occurrence of the codes. A high PMI value means that two codes co-occur together more frequently than by chance.

2.3. Visual system

To design a visual system that can help users to analyze the relationships between different medical codes and select appropriate codes to identify a cohort, we worked with physicians at Fox Chase Cancer Center for several months. We followed an iterative software development process [18] that required meeting domain experts to get their feedback about problem definition and design task analysis. Figure 1 illustrates the architecture of our MedCV system and Figure 2 shows the interface. MedCV has six components: A) Query view, B) Table view, C) Projection view, D) Selected code view, E) Claim view, and F) Patient view.

Figure 1:

The architecture of our proposed MedCV software

Figure 2:

A screenshot of MedCV with four views: (a) Query view: user writes a query, such as a patient treatment name or a medical code, (b): Table view: shows a ranked list of related medical codes based on two different metrics, (c) Projection view: displays medical code relationships in a 2D view responsive to user interaction, (d): Selected code view: keeps a record of medical codes selected by the user.

Query view:

Query view in Figure 2(a) allows user to enter a query and set code search parameters. As a query, a user can type a name of diagnosis or procedure or enter a CPT or ICD-9 medical code. A user can select which types of codes they are interested in exploring. Based on the user’s query, MedCV provides a set of codes related to the query code in both the Projection view and the Table view.

Projection view:

This view visualizes relationships between codes. From the learned word2vec representations of medical codes, our system finds the nearest neighbors of the query code. It provides a 2-dimensional t-SNE [14] projection of code vectors that allows users to explore the code relationships. The Projection view is shown in Figure 2(c). Different colors are used to group codes based on ICD-9 or CPT code hierarchy. When the mouse hovers over a circle, concise information with code description, frequency, and PMI value is shown for the corresponding code. This interface allows users to filter codes by clicking on the legend icon. The underlined code in the Projection view draws attention to neighboring codes with low PMI values (≤1) and high word2vec similarity. The underlined codes are interesting because word2vec neighbors with low PMI indicate codes that occur in the same types of claims but do not co-occur in the same claims.

Table view:

The Table view is an alternative way to inform users about the word2vec and PMI metrics, co-occurrence frequencies, and description of each neighboring code. There are two types of frequencies in the table: PF and CF are the times a code occurs in patients and claims, respectively; BCF is the number of times the query code and the listed code occur in the same claim. The table rows have the same color as the color of codes circled in the projection view. The table view allows sorting and filtering by code to further facilitate code selection.

The table view has a column “select” that provides three options to the user: “add”, “do not add”, and “explore”. If the user thinks the code is related to the query code and wants to add it as an inclusion criterion, the “add” is selected. After adding the code, it will be listed in the selected code view table. If the user thinks the code is unrelated to the given query, “not add” will be selected, which fades the row in the table view. If “explore” is selected, a new window opens that allows the user to observe representative claims containing this code.

Claim view:

The claim view shows fifty randomly selected claims in a t-SNE [14] projection that contains the code selected in the Table view, as shown in Figure 3. We use the vector of primary code in a claim to represent a claim as a vector for the t-SNE projection. The Claim view lets a user explore the claim distance in projected space and show some actual claim data with code descriptions.

Figure 3:

Claim view and patient view: Claim view shows 50 example claims for recently selected code by a user from Table view; Patient view shows the timeline for the selected claim from claim view. Here, a user selects claim #48 from the claim, and the Patient view shows the whole patient timeline of the selected claim #48 and highlights it in the blue circle. ICD-9 diagnosis and CPT codes are marked with the prefix “d_” and “h_”, respectively.

In addition, the user might be interested to know about a patient claim history surrounding a particular claim to understand the claim better. MedCV also shows a Patient view when the user clicks on a claim circle in the Claim view.

Patient view:

The Patient view shows a complete claim history of a selected patient, as seen in Figure 3. The x-axis shows time, and the y-axis shows a patient’s primary diagnosis. The selected claim is shown in blue in the figure. An expert user would focus on this point and look at the past and future claims to better understand the health condition of the patient and better reason about the selected claim.

Selected code view

Whenever the user decides to add a code from the Table view as an inclusion criterion for the given query, the code is listed in this view. After completing the user study, this list would be the final result for a given query or patient treatment.

3. DEMONSTRATION

A video demonstration of MedCV is available online¹ and below, we present a brief overview of the workflow of MedCV.

The user will interact with the above-mentioned views to discover inclusion criteria for a cohort. Figure 1 shows the workflow of different views. The user starts from the Query view and observes related codes in Projection and Table views. After analyzing code results, they can add codes as inclusion criteria in the Table view by selecting the “add” option from the “select” button. The selected code will appear in the Selected code view. However, if the user is interested in seeing some example claims for a code, they can go to the Claim view from the table by selecting the “explore” option. From the Claim view, the user can also observe the selected claim in the Patient view. After examining the Claim and Patient view, the user would return to the Table view to decide whether they are interested in adding the code or not. Finally, the Selected code view shows the final selected codes for cohort identification.

4. EVALUATION

To evaluate MedCV, we recorded subjective user experience [13] to understand if the users believe that it helps them to define inclusion rules for cohort identification tasks and if they are confident that their rules are good. We also measured how well the codes selected by users matched the gold standard.

4.1. Dataset

We used a publicly available synthetic dataset from DE-SynPUF² that is provided by the Centers for Medicare and Medicaid Services (CMS). This dataset contains claims for three types of visits (inpatient, outpatient, and carrier claims) corresponding to three years (2008–2010). The synthetic dataset is closely related to a real-life dataset, and previous research indicates that it retains essential properties about the actual patient population it has been derived from [4, 21]. We extracted 66 million claims from about 1 million synthetic patients. Each claim is a set of ICD-9 and CPT codes. There are a total of 14,838 unique ICD-9 diagnosis codes, 7,195 unique ICD-9 procedure codes, and 11,565 unique CPT codes.

4.2. Training process

We trained word2vec (skip-gram approach)³ and PMI⁴ on DE-SynPUF dataset. We used all the default parameters for the training (i.e. vector length = 100, learning rate = 0.01, epoch = 40, and number of negative samples = 5), other than the window size, which was set to a large number to include all codes in a claim as context.

4.3. User study

We recruited four medical experts to use our software and provide feedback. Three were medical experts from Fox Chase Cancer Center and one from Temple University Hospital. All participants had prior experience using software for patient data analysis. We prepared an introductory document and a video to explain the goals and tasks of our software to the users and demo its functionality. The training process took around 15 minutes.

To evaluate the ability of the participants to use our software for code selection, we asked them to find medical codes that identify patients receiving chemotherapy. In addition, we asked one of the participants to find the codes for excisional biopsy. For each treatment, we asked each participant to spend up to 15 minutes providing a list of CPT and ICD-9 medical codes that indicate the treatment. To evaluate the quality of the code selection, we compared the selection with the gold standard consisting of a list of medical codes manually extracted by clinical researchers during a multi-month effort, which was published in Appendix of [3].

At the end of each session, we asked the participants to fill out a survey to explain how they felt about their interaction with MedCV. In the survey, participants rated different aspects of MedCV on a scale from 1 to 10, such as effectiveness, flexibility, advantages, and future adoption.

4.4. Results

For the “chemotherapy” treatment, there are 21 codes listed in the gold standard from Appendix of [3]. Since four of those codes did not occur in over ten claims, we removed them from this list, resulting in 17 gold standard codes. All four participants identified at least ten codes, ranging from 11 to 16. None of the participants selected a code that was not on the list. This is an impressive result, considering that finding 17 gold standard codes required a multiple-month effort by two medical researchers.

For the “local excision of breast” treatment, the gold standard included 3 CPT codes. Our participant found all 3 CPT codes and selected two additional codes that were not in the gold standard. After consulting with the authors of [3], we concluded that those two additional codes are indeed related to biopsy or excision of breast (i.e., 19316 “suspension of breast”, 38525 “biopsy/removal lymph nodes”). Thus, we do not consider the selection to be incorrect.

We found from the participant survey that most participants thought the Table view was beneficial. Two of the participants found the Projection view less intuitive. All participants were positive about the effectiveness, advantages, and potential for future adoption of MedCV.

5. CONCLUSIONS

In this paper, we described a visual tool, MedCV, that aims to assist researchers and clinicians in defining the inclusion criteria for patient cohort identification from medical claim data. We demonstrated that visualizing codes, claims, and patient timelines positively impact users’ reasoning during the cohort identification task. For the evaluation of MedCV, we recruited four experts and performed a user study. The study revealed that the experts could rapidly find codes identifying two types of treatments. The experts also provided favorable feedback about the software. The results indicate that MedCV could become a valuable tool for retrospective cohort studies using medical claim data.

CCS CONCEPTS.

• Human-centered computing → Visualization toolkits.