Mapping Institution-Specific Study Descriptions to RadLex Playbook Entries

Abstract

The naming of imaging procedures is currently not standardized across institutions. As a result, it is a challenge to establish national registries, for instance, a national registry of dose to facilitate comparisons among different types of CT procedures. RSNA’s RadLex Playbook is an effort towards addressing this gap (by introducing a unique Playbook identifier called an RPID for each procedure), and the current research focuses on semi-automatically mapping institution-specific procedure descriptions to Playbook entries to assist with this standardization effort. We discuss an algorithm we have developed to facilitate the mapping process which first extracts RadLex codes from the procedure description and then uses the definition of an RPID to determine the most suitable RPID(s) for the extracted set of RadLex codes. We also developed a tool that has three modes of operations—a single procedure mapping mode that allows a user to map a single institution-specific procedure description to a Playbook entry, a bulk mode to process large number of descriptions, and an exploratory mode that assists a user to better understand how the selection of values for various Playbook attributes affects the resulting RPID. We validate our algorithms using 166 production CT procedure descriptions and discuss how the tool can be used by administrators to map institution-specific procedure descriptions to RPIDs.

Introduction

All institutions performing radiology imaging procedures maintain a list of procedures that a referring physician can select from when ordering a new imaging study either on paper or via a CPOE system. Each procedure is typically associated with a corresponding internal code and a description (referred to as the “procedure description”) according to a chargemaster list maintained by a radiology department. These codes and descriptions are created at an institutional level (as opposed to a regional or national level) resulting in a set of institution-specific procedures. As a result, similar exams performed at different institutions will have different codes and descriptions; for instance, one institution may label a CT abdomen/pelvis study without contrast as CT ABD/PEL WO while another may refer to the same study as CT ABDOMEN AND PELVIS W.O. CONT. This complicates interoperability, data sharing, and cross-institution data analytics efforts. A specific example of this is the American College of Radiology (ACR) Dose Index Registry (DIR) which is a data registry that allows imaging facilities to compare their CT dose metrics to regional and national values [1]. Due to variations in procedure naming (e.g., because of the use of synonyms, abbreviations, and acronyms), a standardized naming or coding scheme must be used before cross-institutional statistics can be computed.

To facilitate this effort by the ACR as well as support the other benefits of having a unified set of procedure descriptors and codes, the Radiological Society of North America (RSNA) has introduced the RadLex Playbook [2], a special component of the RadLex-controlled terminology [3]. Each Playbook entry includes (among other fields) an identifier (RPID) to uniquely identify each procedure, a short human readable description of the procedure and a set of RadLex identifiers (RIDs) that collectively define the RPID. The goal of the Playbook is to enable radiology sites to share information by linking the procedure with the RPID (e.g., in the DICOM header) instead of institution-specific exam codes and descriptions.

The Playbook initiative is a fairly recent effort and continues to expand; in its first release in November 2011, there were 342 CT procedure names [2] and the list is being routinely updated. The proposed mechanism to map institution-specific procedures to Playbook RPIDs is currently a manual process where a web form [4] or an Excel spreadsheet (that can be downloaded from [5]) is used to map each institution-specific procedure to a set of attributes (the mapping process is described in [6]). For instance, Fig. 1 shows a few selected attribute fields from this Excel spreadsheet that we have manually populated in the case of a CTA CHEST ABD/PEL procedure. Once this process has been completed for all procedures, the Excel file can be uploaded to RSNA which will then send back the file populated with corresponding RPIDs. However, this “mapping task is time consuming and difficult, requiring someone involved in CT scanning, either tech or radiologist, to perform” [7]. This could be even a greater challenge when multiple institutions are involved; for instance, a recent study across multiple institutions looking to send information to DIR mentions that the “exam mapping process via the RadLex Playbook to unify the protocol classification [required by ACR] has been the most challenging factor in the implementation process” [8]. The focus of the research presented herein is to automate this manual field mapping process and thereby automatically determine the most suitable RPID candidate(s) for a given institution-specific procedure description.

Fig. 1.

Few selected attribute fields (e.g., BODY_REGION, MODALITY_MODIFIER) from Playbook Excel template that have been populated in the case of an institution-specific procedure/exam description—CTA CHEST ABD/PEL

Methodology

Development of a Reference Knowledge Base

We first examined the various attributes that are included in the Excel template that are used to uniquely identify a procedure. For each of these attributes, we extracted the list of candidate values from the Excel template. None of the attributes had any common candidate values. There were four types of patient populations, 20 body regions, 34 modality modifiers, 13 procedure modifiers, 88 anatomy focuses, 3 lateralities, 53 reasons for exam, 6 techniques, and 7 values for contrast enhancement (resulting in a total of 228 candidate values across the attributes). For instance, for attribute LATERALITY, the three candidate values are BILATERAL, LEFT, and RIGHT.

The candidate values were then manually mapped to the corresponding RadLex terms (with the assistance of RadLex Browser [3]). For the above LATERALITY example, BILATERAL corresponds to RID5771, LEFT to RID5824, and RIGHT to RID5825 and so forth. We augmented this knowledge base so that common abbreviations, variations, and synonyms can be accounted for (e.g., LT is routinely used as an abbreviation for LEFT). Relevant regular expressions that can be used to refer to the concept were also included as part of the knowledge base (their use is discussed in “Algorithm Development”). Table 1 shows a representative list of regular expressions that were used to match the candidate concepts.

Table 1.

Representative list of regular expressions used to match candidate concepts

Playbook attribute	Concept	RadLex ID	Regular expression
BODY REGION	Lower extremity	RID2638	low(er)?\s+extrem(ity)?|extrem(ity)?\s+low(er)?|\ble\b
BODY REGION	Lumbar spine	RID34573	lumbar\s+spine|l\s+spine|lspine
BODY REGION	Chest	RID1243	cap
	Abdomen	RID56
	Pelvis	RID2507
MODALITY MODIFIER	Angiography	RID10371	angiogra(m|phy)|ang(io)?|avm|cta|cag
MODALITY MODIFIER	Colonography	RID35723	colonography|colongrphy|colonoscopy
ANATOMIC FOCUS	Retroperitoneum	RID431	retroperitone(al|um)|retro
ANATOMIC FOCUS	Kidney	RID205	kidney(s)?|renal|perirenal
LATERALITY	Left position	RID5824	left|\blt\b
REASON FOR EXAM	Postoperative	RID5729	postoperative|post(\s+|\-)?op(erative)?
TECHNIQUE	Thin section	RID36016	thin\s+section|th\s+sect
CONTRAST ENHANCEMENT	Imaging without IV contrast	RID28768	wo|oral\s+only|w0|w/0|w/o|no\s+iv|without\s+iv
CONTRAST ENHANCEMENT	Oral contrast	RID35817	oral\s+contrast|orl\s+con|oral\s+only

The regular expressions in Table 1 are used to extract the RadLex terms from the text in the institution-specific procedure descriptions; for instance, the expression

low(er)?\s + extrem(ity)?|extrem(ity)?\s + low(er)?|\ble\b

used to identify the various ways of referring to lower extremity would match “lower extremity,” “low extrem,” “lower extreme,” “low extremity,” “extremity lower,” “extremity low,” “extreme lower,” and the abbreviation “le.” Furthermore, where needed, we group multiple RadLex terms into a single regular expression so that for a given description, we can extract all the concepts. For instance, the abbreviation “cap” refers to “chest, abdomen, and pelvis” (see Table 1).

Defining an RPID

The PlayBook website [5] was used to generate a comma-separated values file containing a list of CT procedures (referred to as orderables). As of 12 December 2012, the file contains 1,136 CT procedure names. Along with several procedure description-related fields, the file contains a unique RPID for each procedure and a list of atomic RadLex terms that are necessary and sufficient to define an RPID. Each RPID is defined as a unique set of RadLex terms; for instance, RPID145, corresponding to RAD ORDER CT ABD PELVIS W IVCON, is defined by means of the following atomic RadLex terms: RID13060 (imaging procedure), RID10321 (CT), RID56 (abdomen), RID2507 (pelvis), and RID28769 (imaging with IV contrast).

Algorithm Development

We developed algorithms to parse an institution-specific procedure description to determine the atomic RadLex terms that are represented in the text description for the eight Playbook attributes (these are specified in the RPID mapping Excel spreadsheet—a portion of this is shown in Fig. 1)—body region, modality modifier, procedure modifier, anatomy focus, laterality, reason for exam, technique, and contrast enhancement. The goal is to use these atomic RadLex terms to determine the most suitable RPID candidate(s). Note that this RadLex term extraction is, by design, limited to those RadLex terms that are relevant for RPID identification, i.e., the 228 candidate values referenced in “Development of a Reference Knowledge Base”.

The first step involved preparing the input string for parsing (such as stop word removal and white space trimming). Using the reference knowledge base we developed (discussed in “Development of a Reference Knowledge Base”), we extract the RadLex terms for each of the eight attributes using regular expressions (if this information can be extracted from the text description—note that procedure descriptions are not always complete). To better understand the RadLex term (or concept) extraction process, let us consider the procedure description CT LOW EXTREM WO LT. In this case, the system first determines that the modality is CT (RID10321). The type of Playbook procedure we are interested in is radiology orderables (as opposed to protocols for instance) and therefore RID13060 (with description IMAGING PROCEDURE) is deemed as the corresponding procedure term. We then determine the body part(s) in the string by using regular expressions. For instance, in this case, the exemplary expression given in “Development of a Reference Knowledge Base” maps LOW EXTREM in the procedure description to RID2638 (with RadLex description LOWER EXTREMITY). Similarly, WO gets mapped to RID28768 (IMAGING WITHOUT IV CONTRAST) and LT gets mapped to laterality LEFT (RID5824). Every time a term gets mapped to one of the Playbook attributes, that concept gets added to a running list of concepts. In the given example, at the end of the concept extraction phase, this list contains {RID13060, RID10321, RID2638, RID28768, and RID5824}.

After all the atomic RadLex concepts (that are actually present) have been extracted from the procedure description, the next step is to determine the most suitable RPID for the extracted set of concepts. We do this by iteratively searching for the RPID(s) that contains the most number of atomic concepts in its definition matching (n−m) of the extracted concepts where n is the number of extracted concepts and m is the most recent iteration (m = 0 at the beginning). After each iteration, we determine the (n−m) combinations. For instance, in our example, we have five RadLex terms at the beginning. In this case, we find RPID25 with description CT LE LT WO IVCON as the matching term (at m = 0) since RPID25 in its definition has exactly the concepts that were extracted. The algorithm returns RPID25 in this case since the extracted RadLex terms are necessary and sufficient. In this example, if there was no direct match, the system would have looked at (5–1) concept combinations in the next iteration. In such a case, multiple RPIDs will be returned based on the longest possible match after accounting for the number of RIDs in the RPID. We require a minimum of three matching RadLex terms (based on having at least the procedure type, modality, and anatomy descriptors) for an RPID to be included in the matching results. There are two possible scenarios that need to be addressed:

1. The algorithm is able to extract less information from the description than is required for a direct match; for instance, if the procedure description was CT LT WO IVCON (without specifying LE), there will be only four RIDs instead of five. The system cannot find a unique RPID for the extracted RIDs, and therefore, would return RPID25 (CT LE LT WO IVCON) and RPID55 (CT UE LT WO IVCON) as the possible candidates based on the four RID match.

2. The algorithm extracts more information from the description than is strictly required for a match; for instance, if the procedure description was CT LE HIP WO RT, the system would extract six RIDs. However, Playbook does not contain a specific RPID that uniquely matches the extracted RIDs (more details on this provided in “Determining RPID(s) from Extracted RIDs”) since Playbook contains only a more generic description called CT LE HIP WO (RPID376) with only five RIDs. In this case, the algorithm first looks for a match with six RIDs and then looks for a five RID match. However, at this point, there are multiple matches—RPID376 (CT LE HIP WO IVCON), RPID874 (CT PELVIS LE HIP WO IVCON), RPID1086 (CT PELVIS LE HIP DUAL ENG CT WO IVCON), RPID1229 (CT PELVIS LE 3D IMAGE HIP WO IVCON), RPID1238 (CT SPINE PELVIS LE HIP WO IVCON), and RPID1484 (CT PELVIS LE L SPINE HIP WO IVCON). When presenting multiple matches, we order the results based on the number of RIDs. For this particular example, the algorithm first presents RPID376 since it has five RIDs in its definition whereas all others have more than five.

Data Extraction

To validate our algorithms, we extracted a list of currently used institution-specific procedure descriptions from the University of Chicago Medicine informatics systems. These descriptions contain no patient-related information, so this method development study was exempt from IRB review. There were 1,424 procedure descriptions altogether, out of which, 259 were for CT procedures. For the current study, we focused on the adult population only, and therefore, the list was narrowed down further to 166 procedure descriptions. Fifty of these procedures were used as the development set for this work, and the remaining 116 were used as the test set. The development set was used for developing the RID mapping algorithm, and thus, these cases were held out from the test set that was used for performance analyses.

Establishing Ground Truth

The authors manually determined the most appropriate RPID (i.e., the ground truth), if any, for the 166 CT procedure descriptions by comparing each institution-specific procedure description with the list of short descriptions from Playbook. For 39 procedures (34 %), there was an agreement that no unique RPID matching could be made for the procedure. The primary cause is due to the nonexistence of a corresponding RPID in Playbook (for the Playbook version used at the time of this work). For the remaining 127 procedures, all authors agreed on a unique RPID for each procedure: this includes the 50 procedures used in the development set and 77 procedures in the test set.

The authors also determined the most suitable set of RIDs for each procedure description from among the list of 228 candidate values available through Playbook. As described in “Development of a Reference Knowledge Base,” both the algorithm and evaluation are, by design, limited to these 228 candidate RIDs, as these are the only set that is relevant for defining RPIDs within our data set. As extraction of any other RID would have no impact on RPID selection, no attempt was made to map procedure descriptions to any other RID beyond these 228 either within the algorithm operation or evaluation. This RID extraction was performed to establish the performance of the mapping between procedure description and RIDs as the first step before mapping RIDs to RPIDs.

Results

Mapping Institution-Specific Procedure Descriptions to RIDs

The RID mapping accuracy for the 50 procedure descriptions in the development set is shown in Table 2.

Table 2.

RID mapping process results for the 50 descriptions in training set

#RIDs defining procedure	Number of procedure descriptions for which #RIDs (truth) = #RIDs (mapped) with exact RID match
3	8
4	16
5	15
6	10
7	0
8	1
Total	50 (100 %)

# Denotes “number of”

The RID mapping was 100 % accurate for this limited development set. This was not surprising since we are working with a constrained set of procedure descriptions and a small subset of RadLex terms, as opposed to, for instance, a more generic task such as extracting RadLex terms (from the full RadLex ontology) from free text radiology reports. The RPID mapping accuracy for this development set has been reported previously [9].

The accuracy of our intermediate RID mapping process for the 116 procedures in the test set is shown in Table 3, where the ground truth was defined as the RIDs manually extracted from the procedure description.

Table 3.

RID mapping process results for the 116 descriptions in test set

#RIDs extracted from procedure	#RIDs (truth) = #RIDs (mapped) with exact RID match	#RIDs (truth) > #RIDs (mapped)	#RIDs (truth) < #RIDs (mapped)	#RIDs (truth) = #RIDs (mapped) with inexact RID match
3	10	0	0	0
4	28	2	3	0
5	46	5	2	1
6	11	3	1	1
7	1	1	0	0
8	0	1	0	0
Total	96 (83 %)	12 (10 %)	6 (5 %)	2 (2 %)

Similarly, the RID mapping accuracy for the subset of 77 test procedures which had a matching RPID is shown in Table 4.

Table 4.

RID mapping process results for the 77 descriptions from the test set having a matching RPID

#RIDs defining RPID	#RIDs (truth) = #RIDs (mapped) with exact RPID match	#RIDs (truth) > #RIDs (mapped)	#RIDs (truth) < #RIDs (mapped)	#RIDs (truth) = #RIDs (mapped) with inexact RPID match
3	4	0	0	0
4	23	2	2	0
5	26	4	2	1
6	6	3	1	1
7	1	0	0	0
8	0	1	0	0
Total	60 (78 %)	10 (13 %)	5 (6 %)	2 (3 %)

For 78 % of the descriptions, we extracted exactly the RIDs that comprised the definition of the corresponding RPID. For 10 descriptions (13 %), we extracted fewer RIDs compared to the RIDs in RPID ground truth, with the maximum difference being two RIDs (for eight procedures, we extracted one fewer RID while two had two fewer RIDs).

For instance, institution-specific procedure CT DRAIN SUBDIA/SUBPHRN ABSCESS was mapped to RPID128 (with description CT ABD GUIDE SUB DIA DRAIN) which has six RIDs in Playbook—RID13060 (imaging procedure), RID10321 (CT), RID56 (abdomen), RID35732 (guidance), RID35778 (sub-diaphragm region), and RID10417 (drainage). However, our algorithm was not able to extract RID56 (abdomen) and RID35732 (guidance). The final RPID was correct in the end as the combination of the extracted RIDs (even if incomplete) enabled us to uniquely identify the correct RPID.

On the other hand, for five cases (6 %), we extracted an additional RID. For instance, the institution-specific description CT PELVIS/BONE W was mapped to RPID48 (with Playbook description CT PELVIS W IVCON). In this instance, our algorithm extracted an additional RID: RID13197 (bone), and yet the final RPID was correct.

Finally, there were two instances (3 %) where the number of extracted RIDs was the same; however, it was an inexact match. An example of this is the institution-specific procedure description CT ANGIO BRAIN WWO which was mapped to RPID326 (CT HEAD ANGIO BRN) that contains five RIDs—RID13060 (imaging procedure), RID10321 (CT), RID9080 (head), RID10371 (angiography), and RID6434 (brain). In lieu of RID9080, our algorithm extracted RID28771 (imaging without then with IV contrast).

The difference between Tables 3 and 4 indicate the RID mapping accuracy for the 39 descriptions for which an RPID did not exist. For 36 of these descriptions (92 %), we correctly extracted all the RIDs. For two descriptions, we extracted fewer RIDs (discussed later), while for one, we extracted an extra RID. We extracted the extra RID for description CT TEMPORAL BONE WO for which the correct RIDs were RID13060 (imaging procedure), RID10321 (CT), RID9361 (temporal bone), and RID28768 (imaging without IV contrast). In addition to these, our algorithm extracted RID13197 (bone) which is one of the candidate values for Playbook’s BODY_REGION attribute (note that temporal bone is a candidate value for ANATOMIC_FOCUS attribute). Playbook contains CT TMP BON WO & W IVCON (RPID1546), but not an entry for WO IVCON, and therefore, this description was not mapped to an RPID.

We also evaluated the algorithm performance on overall RID extraction process, i.e., how likely it is for a given RID to be part of the set of RIDs defining an RPID. Table 5 shows the 2 × 2 contingency table. Based on Table 5, our RID extraction accuracy is 94.4 % (95 % CI 94.4–95.9 %).

Table 5.

2 × 2 Contingency table for RID extraction

		Ground truth
		In RPID definition	Not in RPID definition
Algorithm result	In RPID definition	353	7
	Not in RPID definition	14	0

Determining RPID(s) from Extracted RIDs

Running the RPID mapping algorithms on the RIDs extracted from the 77 procedures in the test set for which a ground truth RPID existed resulted in 76 correctly mapped RPIDs (i.e., true positives) and one incorrectly mapped RPID (i.e., a false positive). The false positive was for procedure description CT ANGIO BRAIN WWO which was getting mapped to RPID1006 (CT HEAD NECK ANGIO BRN WO & W IVCON) and RPID1009 (CT HEAD 3D IMAGE ANGIO BRN WO & W IVCON). In fact, only a close candidate exists in Playbook—CT HEAD ANGIO BRN (RPID326), but not an RPID with a direct mapping such that all RIDs are included, or an RPID that refers to “brain” only instead of (or in addition to) “head.” The consensus was that RPID326 is still a reasonable mapping for this procedure, and therefore, it was not considered as “missing from Playbook”. Table 6 shows some of the institution-specific procedure descriptions that were mapped to a correct RPID.

Table 6.

Sample set of institution-specific procedure descriptions that have been mapped to RPIDs

Procedure description	Mapped RPID	RPID short description
CT LOWER EXTREMITY WO RT	RPID26	CT LE RT WO IVCON
CT PELVIS W	RPID48	CT PELVIS W IVCON
CT UPPER EXTREMITY WO LT	RPID55	CT UE LT WO IVCON
CT NECK SOFT TISSUE	RPID61	CT NECK
CT ANGIO ABDOMEN	RPID74	CT ABD ANGIO
CT SCREENING VIRTUAL COLONOSCOPY	RPID86	CT ABD PELVIS COLONGRPHY SCREEN
CT NEEDLE BIOPSY LIVER	RPID119	CT ABD GUIDE LIVER BX
CT NEEDLE BIOPSY PANCREAS	RPID121	CT ABD GUIDE PANC BX
CT NEEDLE BIOPSY RENAL	RPID135	CT ABD GUIDE KIDNEY BX
CT LSPINE	RPID242	CT L SPINE
CT THORACIC SPINE	RPID243	CT T SPINE
CT CHEST ABDOMEN PELVIS WWO	RPID250	CT CHST ABD PELVIS WO & W IVCON
CT CHEST PE W BILAT LE W	RPID426	CT CHST LE PULM ARTS BILAT EMBO W IVCON

As described in “Establishing Ground Truth,” for the remaining 39 descriptions, there was no correct ground truth mapping. As such, the reason the algorithm could not determine a suitable RPID was due to the lack of a directly relevant RPID entry in Playbook despite extracting all the possible RadLex terms from the description 92 % of the time; for instance, we extracted RID13060 (imaging procedure), RID10321 (CT), and RID2638 (lower extremity) from procedure description CT EXTREMITY LOWER, but all the lower extremity related procedures in Playbook are more specific (e.g., CT LE ANKLE and CT LE LT W IVCON). In all those cases, the algorithm did not find an RPID to map to or suggested an incorrect mapping as a result. We did not consider these cases as true negatives or false positives since the mapping to a correct RPID was not available in Playbook. Within these 39 cases that could not be mapped, four broad categories were identified:

1. Non-existing RPIDs in Playbook. For instance, at the time of this study, there are no RPIDs that relate to the concept of POST MYELOGRAPHY, and therefore, none of the institution procedures involving post-myelography were correctly mapped (e.g., CT CSPINE POST MYELO WO). There were 31 procedures for which there was no corresponding RPID. The output of our tool in this case is a list of RPIDs containing the best possible match as long as a minimum of three matching RIDs were found. For the given example, our tool extracts five RIDs—RID13060 (imaging procedure), RID10321 (CT), RID34571 (c-spine), RID10380 (myelography), and RID28768 (without IV contrast) and outputs RPID21 (CT C SPINE WO IVCON) and RPID296 (CT C SPINE MYELOG). There are many other similar procedures, for instance, RPID876 (CT MYELOG), RPID1124 (CT HEAD C SPINE WO IVCON), RPID1268 (CT C SPINE L SPINE WO IVCON), and RPID1363 (CT T SPINE C SPINE MYELOG), but our algorithm shows only the ones that had the highest RID match.

2. The institution-specific procedure description is more generic than Playbook. In this case, Playbook contains a more specific version than the institution procedure, but not the general case. An example is the procedure CT CHEST PE W BILAT EXT. Playbook contains more specific versions related to this procedure, such as CT CHST LE PULM ARTS BILAT EMBO W IVCON (RPID426), but not the general case. In this instance, our tool outputs RPID426 based on the highest possible RID match (although it did not contribute towards our accuracy). There were four instances where there was no generic mapping.

3. The institution description is more specific than Playbook. In this case, Playbook contains a more generic description, for instance, CT LE HIP WO IVCON (RPID376), but not a more specific version such as CT LE HIP WO RT (indicating right laterality). The tool outputs RPID376 in this instance (as discussed in “Algorithm Development”). In our test set, there were four instances that belonged to this category.

4. The design of Playbook allows each RPID to have only a single value per attribute for certain attributes, e.g., only one contrast type or anatomical focus. As mentioned in “Mapping Institution-Specific Procedure Descriptions to RIDs” for two descriptions for which an RPID was not available, our algorithm extracted one fewer RID. The reason for this was that the institution-specific description contained two values for contrast enhancement whereas Playbook, by definition, has provision to select only one contrast enhancement option. For instance, for the description CT CHEST PELVIS W ABDOMEN WWO, the “truth” contained seven RIDs since W IVCON and WO & W IVCON were both included. Our algorithm was designed to extract only one contrast enhancement value, and therefore, we did not extract an RID for the WWO portion of the description.

User Interface

Using the techniques described in “Methodology,” we developed a simple, proof-of-concept tool that can assist an end user to automatically determine the closest RPID(s) for a given institution-specific procedure description. The tool has three modes of operation:

1. Single procedure mapping: This mode allows a user to map a single procedure description to an RPID. This mode can be useful when a new addition is made to an existing list of procedure names. The user can type the description into the textbox provided (currently populated with CT CHEST ABD PEL in Fig. 2) and press the “Find Playbook Entry” button. The tool will then show a list of possible RPID candidates in the datagrid at the bottom of the tool.

2. Bulk mode: This mode allows a user to select a file that contains a list of institution-specific procedure names. In Fig. 2, the user needs to enter the file path into the textbox provided and press the “Process Data File” button. The tool will then output a tab delimited file containing a list of mapped RPIDs for each procedure as well as populate the datagrid with the mapped RPID(s). When multiple RPIDs are present for a given description, the results are shown using a semicolon as the separator.

3. Exploratory mode: This mode is functionally similar to the Excel spreadsheet since the user can select a value for the various attributes from a list of possible candidates (Fig. 3). Once all attributes have been specified, the tool can display the corresponding RPID. In case a unique RPID cannot be identified based on selections, potential candidate RPIDs are shown. In that case, the attribute values can be refined to identify a unique RFID. The purpose of this mode is to assist the user to understand how the selection of different attribute values affects the resulting RPID.

Fig. 2.

Tool interface for mapping procedure descriptions to RPIDs

Fig. 3.

Exploratory mode

Shown in Fig. 2 is a screenshot of the tool demonstrating the first two modes of operation. Figure 3 shows the exploratory mode.

Discussion

Lack of standardized terminology for naming imaging procedures has been identified as a substantial challenge to establishing national registries, for instance, a national registry of dose to facilitate comparisons among different types of CT procedures [10]. The RadLex Playbook from RSNA is an effort towards addressing this gap; this research focused on developing a tool to map institution-specific procedure descriptions to Playbook entries to assist with this standardization effort. We have demonstrated how the developed algorithms can be used to map production procedure descriptions to RPIDs, streamlining the mapping process. The tool we have developed has three operational modes and can be used for educational purposes to assist administrators to understand how different attribute values affect the resulting RPID. We have focused on maximizing recall at the expense of precision so that, for instance, an administrator can use our tool to select from a much shorter list of Playbook entries to assist in the Playbook mapping process. Our tool can also help in accelerating the completion of the list of Radlex Playbook procedures at a given institute by systematically analyzing the mapping results to identify the missing procedures.

Our results indicate that it is feasible to develop algorithms to streamline the RPID mapping process; the 66 % mapping rate, which needs to be improved, shows the potential to streamline the mapping process with production procedure descriptions. In fact, if we consider only the cases where a valid RPID exists, our accuracy is close to 99 % (76 correctly mapped out of 77 valid RPIDs). Once Radlex playbook is extended to contain a complete list of procedures, it would be possible to reach a higher mapping rate and we envision the mapping process to be more automated. As discussed in “Determining RPID(s) from Extracted RIDs,” we identified four broad categories of reasons why the current algorithms could not determine a corresponding RPID. We could have excluded these descriptions that do not have an exact mapping from our analysis, but we wanted to explore the challenges and the extent of how much automated methods can contribute to a real-world problem where incomplete institution-specific procedure descriptions routinely get mapped to RPIDs. The four categories we identified may all have their root in the evolving state of RadLex Playbook (perhaps with the exception of the last one). However, because the algorithms use the atomic RadLex concepts as an intermediary step towards determining the RPID, we can be reasonably confident that as new RPIDs are added, procedures can be correctly mapped to those new procedures so long as the new RPIDs are also defined in terms of constituent RadLex concepts. Even for the correctly mapped procedures, there were several procedure descriptions which were not sufficiently specific. For example, when establishing the “ground truth,” we mapped the institution procedure CT HEAD AND SOFT TISSUE NECK WO to CT HEAD NECK WO IVCON (RPID1123) which is a more general procedure. The algorithm correctly mapped this description to RPID1123. Given that SOFT TISSUE OF THE NECK is a candidate value for the Playbook attribute “anatomy focus,” it may be logical to add a more specific procedure to Playbook. Similarly, for our analysis of the mapping between a procedure description and corresponding RIDs, we only considered the RadLex concepts that are included as part of Playbook (i.e., candidate values). We encountered a few cases where the concept exists in RadLex, but not in the subset that Playbook considers. For instance, the description CT DRAIN RETROPERITONEAL ABSCESS was correctly mapped to RPID127 (CT ABD GUIDE RETRO DRAIN) based on the RIDs that were extracted; in this instance, the concept “abscess” exists in RadLex—RID3711, but is not part of the RadLex subset that Playbook uses. Correctly or incorrectly extracting abscess, thus, has no impact on the RPID mapping. Analyses such as these may assist Playbook to develop a more complete set of procedure descriptions. On the other hand, the fourth category we identified in “Determining RPID(s) from Extracted RIDs” is based on differences between the design of Playbook and the way in which some “procedures” are defined and used in a particular clinical setting. This could be addressed either by defining the site-specific procedure as a union of two RPIDs with no change to Playbook. Alternatively, this could be addressed by modification to the design of Playbook itself, e.g., by allowing a user to specify a second contrast enhancement value (for instance, similar to ANATOMIC_FOCUS_1, ANATOMIC_FOCUS_2 that already exist in Playbook). Such a change may not necessarily be as straightforward since there is a need for a way to specify the sequence of events (i.e., the fact that CHEST and PELVIS need to be imaged W IVCON while the ABDOMEN needs to be imaged W IVCON and WO & W IVCON in our CT CHEST PELVIS W ABDOMEN WWO example).

There have been at least two recent attempts closely related to the work presented herein. First is the DIR Mapping Tool [4] which allows DIR participants to map institution-specific CT procedure descriptions to RPIDs. The tool allows direct assignment of an RPID to a study description as well as building the mapping by specifying the various required attribute values (such as modality and body regions). This is a manual mapping/assignment process compared to the more automated tool presented here.

The second is an effort by a commercial entity named RadMapps [11] which facilitates the mapping of DIR facilities’ exam descriptions to RPIDs. In this paper, we have focused on the development of algorithms allowing the automation of the process as well as of interactive tools, although the final outcome is very similar to that provided by DIR and RadMapps.

The current study has several limitations.

1. We determined the “ground truth” ourselves based on consensus among the authors. We are liaising with Playbook administrators to get the mapped set of RPIDs (per Playbook guidelines) for our dataset. However, as previously mentioned, this requires manually assigning a value to the different Playbook entities for each procedure.

2. Our dataset is from one institution, and therefore, the current version of the algorithms may need to be adapted to be used across institutions. For instance, we interpret WO to mean without IV contrast, whereas at another institute, the default use of WO in a procedure description could indicate without oral contrast. The algorithms can be augmented to support such abbreviation disambiguation depending on context, as well as removing known phrases; for example, WO PHYSICIAN is not a candidate for any of the Playbook attributes and therefore such terms can be removed from the procedure description prior to the RadLex term mapping process.

3. We could determine the correct RPID even when the extracted set of RIDs was incomplete, yet had sufficient information to uniquely identify an RPID (e.g., the CT DRAIN SUBDIA/SUBPHRN ABSCESS example described in “Mapping Institution-Specific Procedure Descriptions to RIDs”). However, as shown in Table 4, the accuracy of RID extraction is imperfect, and therefore, the RPID mapping process via the extracted RIDs may not always yield meaningful results.

4. RadLex and Playbook are routinely being updated, and as a result, it is possible that some of the issues we have discussed herein are, or have been, already addressed at the moment of publication of this article.

Despite these current limitations, the algorithms are designed and implemented within a framework that can be easily extended (e.g., to include additional attributes and to expand the regular expressions to account for more variations) and the tool can be used by administrators to map institution-specific procedure descriptions to RPIDs. RadLex Playbook has enjoyed rapid initial adoption within the radiology community, and we believe that developing tools such as the one presented here could accelerate the adoption process.

Conclusions

Mapping institution-specific procedure descriptions to Playbook entries is an important standardization initiative by RSNA. However, the steps involved in the current mapping process are time consuming and new tools are needed to streamline this effort. The proposed tool shows how the mapping process can be simplified. Further validation will be performed before it can be used in clinical settings.