Deterministic vs. Probabilistic: Best Practices for Patient Matching Based on a Comparison of Two Implementations

Abstract

In order to successfully share patient data across multiple systems, a reliable method of linking patient records across disparate organizations is required. In Canada, within the province of Ontario, there are four centralized diagnostic imaging repositories (DIRs) that allow multiple hospitals and independent health facilities (IHF) to send diagnostic images and reports for the purpose of sharing patient data across the region (Nagels et al. J Digit Imaging 28: 188, 2015). In 2017, the opportunity to consolidate the two regional DIRs that share the south-central and southeast area of the province was reviewed. The two DIRs use two different methods for patient matching. One uses a deterministic match based on one specific value, while the other uses a probabilistic scorecard that weighs a variety of patient demographics to assess if the patients are a match. An analysis was conducted to measure how a patient identity domain that uses a deterministic approach would compare to the accepted “standard.” The intention is to review the analysis as a means of identifying interesting insights in both approaches. For the purpose of this paper, the two DIRs will be referred to as DIR1 and DIR2.

Background

A primary objective of Canada Health Infoway was to achieve the goal of “One Patient, One Record.” A critical part of this goal involved providing clinical users seamless access to diagnostic imaging (DI) exams from non-local enterprises. Regional diagnostic imaging repositories (DIRs) were implemented across Canada as a means of providing clinical users seamless access to outside diagnostic images and reports (DI information) [1]. In many areas, the DIRs are set up as a hub and spoke environment. The individual connected organizations are spoke sites that publish DI information to the hub DIR, and are configured to consume outside DI information directly from the DIR into the site’s local PACS.

Sponsored at a federal level by Canada Health Infoway and at a provincial level by eHealth Ontario, four central DIRs were implemented in the province of Ontario. HDIRS is a not-for-profit organization owned by the contributing hospital organizations that manage two of the DIRs (DIR1 and DIR2). The implementation of the DIRs was broken into two key phases:

• Phase 1—Publish: connect hospitals and clinics to enable the ability to publish images and reports to the DIR

• Phase 2—Consume: provide hospitals and clinics the ability to ingest outside images and reports into their local PACS.

Figure 1 provides a high-level timeline of the Publish and Consume phases for DIR1 and DIR2.

Fig. 1.

Timeline of the Publish and Consume phases for DIR1 and DIR2

The distribution and seamless sharing of DI exams across multiple organizations has demonstrated benefits that include the following [2, 3]:

• Physician reliance on access to outside images and reports

• Reduction in CD imports

• Reduction in repeat imaging for patients

In order to provide a patient’s longitudinal imaging record across multiple organizations that do not share a patient identity pool, a consistent and reliable patient matching method must be in place. As demonstrated in Table 1, both DIRs present challenges to ensure that the patients are identified uniformly across the disparate sites and identity pools. The following section will compare the two methods utilized by each DIR. An analysis was performed comparing the patient matching methods of these 2 DIRs that cover the south-central and south-east area of the province.

Table 1.

Difference between the environments of DIR1 and DIR2

Comparision of DIR1 and DIR2
Item	DIR1	DIR2
Patient identification	Deterministic: match based on provincial Ontario Health Card Number	Probabilistic: EMPI patient matching based on a scorecard of key demographics
Number of contributing sites	98	29
Total number of registered patients	≈ 12,370,000 total patients	≈ 7,500,000 total patients
Annual exam volume	≈ 5 million exams annually	≈ 3 million exams annually
Number of patient identity pools	40	21

The chart (Table 1) contrasts the environments of DIR1 and DIR2 as it relates to patient matching.

There are two main methods employed for patient matching: deterministic and probabilistic. Often a deterministic implementation will rely on biometric identification based on a fingerprint, facial or iris recognition [4], but may lavage on a reliable government ID card. A probabilistic model is more common with the use of an enterprise master patient index (EMPI).

Deterministic matching uses business rules to determine when two or more records match (the rule “determines” the result). In a deterministic matching system, for example, one rule might instruct the system to match two records based on matching Social Security number and address fields. Probabilistic matching uses likelihood ratio theory to assign comparison outcomes to the more likely decision. This method leverages statistical theory and data analysis, which can improve accurate links between records with more complex typographical errors and error patterns than a deterministic system [5].

DIR1 does not use an EMPI to match a patient’s longitudinal records. DIR1 uses the Ontario health number (HN) as a means of creating a deterministic patient match across multiple sites. Contributing sites are required to include the patient’s HN in the Health Level 7 (HL7) Admit Discharge Transfer (ADT) message. The DIR maps the HN to the DICOM tag, “Other Patient ID Sequence” (0010,0002), which provides a global patient identifier across all of the connected sites.

DIR2 leverages an EMPI to establish a probabilistic match of a patient’s longitudinal records. Patient demographics are compared and measured against a scorecard (see Table 2). If a score is higher than 70%, the patients are considered a match, if the score is lower than 63% the records are considered to belong to different patients. A patient matching score that is between 63 and 70% is considered an uncertain link and requires human intervention to review and manually match or break the “uncertain links.”

Table 2.

Patient demographics

DIR2 EMPI Scorecard
Attribute	Weight
Health number	22.46	Exact
Last name	14.54	Distance
First name	11.54	Distance
Middle name	7.95	Distance
Gender	7.39	Exact
Date of birth	17.86	Date-Distance
Postal code	7.64	Distance
Home phone	10.62	Distance

Methods

During the evaluation phase, the following 3 methods were reviewed as a means of measuring a comparison between the patient matching models associated with DIR1 and DIR2.

Assessing False Positive and False Negative Patient Matches

In a deterministic model in which patient matches are determined through known relationships, without room for random variation, there is a greater chance of false positive patient matches in which two different patients are incorrectly matched based on a shared value used for the deterministic identification [6]. DIR1 establishes a patient link based on a matching HN and does not validate any other demographics as the basis of the patient link. Based on an assessment of patient data across both DIRs, it is estimated that roughly 5% of patients will not have an HN. The following groups represent individuals that may not have an Ontario HN:

• Indigenous communities

• Military personnel

• Incarcerated individuals

• Non-landed immigrants

• Non-Ontario residents

A common practice that sites follow when a patient arrives and does not have a valid HN is that the clerical staff will enter a value in the HN to identify the type of patient. For example, clerical staff may register an out of province patient with a “1” as their HN, and they may register an out of country patient with a “0” as the HN. This practice may be useful for the local site to categorize which types of patients are receiving service without a valid HN; however, this introduces the opportunity of a creating false positive patient matches based on invalid HN entries.

The Ontario health card number is a 10-digit numeric sequence that leverages the Luhn algorithm (modular 10) to confirm if the number is valid. As a means of combatting the problem with false positives matches based on invalid HN entries, DIR1 implemented a process to filter non-valid HNs. The patient’s HN is sent to the DIR via an ADT HL7 message. DIR1 designed a mechanism on the HL7 broker to review the HN included in the inbound HL7 message. If the HN does not meet the Luhn algorithm qualifications, the HL7 broker will nullify the HN. The patient will be treated as if no HN was entered for the patient. The longitudinal record that exists for the patient across disparate contributing sites will not be available, as there is no uniform global identifier; however despite this result, implementing the HN validation filter has been valuable as it has mitigated the opportunity of false positive patient matches based on invalid HN entries.

The higher risk associated with DIR2 was a greater chance of “false negative” patient matches [7]. This is found when a patient’s records, across multiple sites, are not linked as a result of a low patient matching score.

Table 2 identifies the scorecard that is used to measure patient matches. If a score is greater than 70% it is considered a positive patient match, if a score is less than 63% the records are considered different patients, and if the score is between 63 and 70% the potential match will go into an uncertain link queue.

• > 70%: positive match

• < 63%: different patients

• 63–70%: uncertain link

Patients in the uncertain link queue are not presented as part of the patient’s longitudinal record, as there is uncertainty associated with the linkage. The uncertain link queue requires manual administration by the local site to manage the patient matches.

After reviewing the uncertain link queue, it was clear that many potential patient matches have fallen short of a positive match score due to a specific configuration on the scorecard settings. For example, patients that have no middle name are scored 0 out of a possible 7.95, due to the patient matching criteria not designed to score two null middle names as a positive score. Additionally, in many cases, inconsistencies in how different sites format the phone number and postal code creates a lower score in these categories and prevents a matching score of greater than 70%.

Evaluation of Administrative Overhead

As previously highlighted, DIR2 patient matches that score between 63 and 70% are identified as uncertain links and requires human intervention to review and manage the uncertain link queue. This is typically managed by a member PACS team from the contributing site.

Surveys were distributed to site administrators to assess the perceived time spent managing the uncertain link queues. An assessment of the how regular the uncertain link queues are maintained from the contributing sites was reviewed to.

Algorithmic Comparison

In order to identify how the patient links established in DIR1’s deterministic model would compare to the matching criteria used in DIR2, an analysis was performed to apply the patient matching scorecard logic highlighted in Table 2 to patients’ records in DIR1. An analysis was conducted in 2014 and 2018 to assess if the results were consistent across the two time periods.

Results/Evaluation

Measuring False Negative and False Positive Patient Matches

A limitation with DIR1’s deterministic model is that there is no method of seamlessly linking patients across disparate sites when a patient does not have an Ontario HN, resulting in a false positive patient mismatch. Since DIR1 relies heavily on the Ontario HN, our initial hypothesis was that DIR2’s model would likely have a significant advantage over a deterministic model in the ability to match patients that do not have the required HN. Based on this prediction, a study was conducted to identify how many patients with no HN are matched successfully in DIR2’s environment.

An analysis of DIR2’s environment discovered that out of 7,526,830 patients, a total of 372,252 (4.95%) patient records were classified as patients without a valid HN. Further analysis of all positive patient matches in the EMPI revealed that only 28,850 (7.8% of the total of 372,252 records without a valid HN) positive patient matches were created for patients without a valid HN.

As a means of assessing the strength of using the HN as a deterministic value, we conducted an analysis to review how many potential false positive patient matches could occur with a patient match based on HN. For the purpose of this analysis, we considered a false positive patient match when the HN is the same for value while the last name, first name, and date of birth all mismatch. The results are listed in Table 3.

Table 3.

Results of potential false positive patient matches

Potential false positive patient matches
DIR	Total number of patient pairs	Last name, first name, and DOB mismatch
DIR1	14,009,674	6,906 (0.05%)
DIR2	6,911,272	3,576 (0.052%)

This analysis provides an indication that HN is a fairly reliable matching value. The number of potential false positives across both DIRs is comparatively even at 0.05% and 0.052%. With a strong data quality, practices in place to ensure follow-up actions are conducted to clean-up false positive matches; leverage the HN as a deterministic matching value across both DIRs is a reasonable consideration.

As it relates to the risk of false positive patient matches, it has been identified that the use of an EMPI improves accurate links between records and reduces opportunities for false positive patient matches [6]. False positive patient matches would be far less likely in DIR2’s implementation, compared to DIR1. DIR1 does not use likelihood ratio theory or any statistical analysis to establish a patient link, and rather relies on an exact match of HN across disparate sites. The assumption is that clerical errors could lead to incorrectly registered HN’s for patients and create a high number of false positive patient matches.

The DIR1 Operations team provides sites regular reports of potential false positive patient matches, based on records that have a matching HN but other key demographics mismatch (name, date of birth, and gender). A review of DIR1’s Service Desk ticket system identified only 2 service tickets in the past 7 years had been opened to address false positive matches identified from these reports.

Administrative Overhead

Survey feedback from sites contributing to DIR2 indicates that managing the uncertain link queue is a time-consuming task. As a result of this feedback, the analysis was performed to measure how well the administrative tasks were adopted for each site. The analysis identified that many sites were not managing the uncertain link queue on a regular basis. To demonstrate this point, Table 4 features 5 sites with the greatest number of uncertain link queues.

Table 4.

Feedback provided based on discussions with the sites

Sites with the highest number of uncertain links
Site	Number of uncertain links	Total no. of patients
HSC	8920	467,886
UHN	7116	1,149,590
WOHS	4385	641396
MSH	4209	571,280
THP	4077	604,714

The table illustrates that the number of patient matches arriving in the uncertain link queue is at an unmanageable level for many sites. Feedback provided based on discussions with the sites identified in Table 4, indicate that if the matching logic could create less uncertain links the administrative process to manage the queue would be more achievable.

DIR1 has a significantly smaller administrative overhead for contributing sites. There are no uncertain link queues sites have to regularly maintain, however as previously highlighted in this paper, there are regular reports that DIR1 sends to contributing sites to inform of the opportunity of possible false positive patient matches. A review of DIR1’s Service Desk ticket system revealed that only 2 service tickets in the past 7 years had been opened to address cleanup of false positive patient matches.

This indicates that DIR2’s approach is considerably more time consuming from an administrative perspective. As a result, many sites do not regularly maintain their uncertain link queues, causing a high number of data quality issues related to potential false negative mismatches.

Algorithmic Comparison

A comparison was conducted in 2014, in which nearly 4 million matched pairs of patients in DIR1 were measured against DIR2’s patient matching criteria scorecard.

2014 results found that based on DIR2’s matching logic, out of 3,888,958 matched pairs of patients in DIR1:

• 35,028 (0.90%) matches identified as uncertain links

• 15,184 (0.39%) matches identified as mismatched

In order to confirm if the 2014 results were consistent, a follow-up comparison was conducted in 2018 against matched pairs of patients in the DIR1. In 2018, the comparison looked at how over 9.5 million matched pairs of patients measured against DIR2’s patient matching scorecard.

2018 results found that based on DIR2’s matching logic, out of 9,606,788 matched pairs of patients in DIR1:

• 81,633 (0.85%) matches identified as uncertain links

• 39, 987 (0.41%) matches identified as mismatched

Both assessments indicate a maximum differential of 1.29%. This suggests that DIR1 and DIR2’s patient methods have comparable results matching patients.

Conclusion

In summary, both methods yield similar results with a total differential in 2014 of 1.29% and in 2018 a total differential of 1.26%. DIR2 provided a slight improvement in positive matches for patients with no HN, as 7.8% of patients with no HN were successfully matched in DIR2. However, matching only 28,850 out of 372,252 patients with no HN is a fairly small match rate when measuring it against the cost of administrative time managing the EMPI. DIR2’s method did slightly reduce false positive matches (0.39–0.41%); however, this came at the expense of increased administrative overhead, and ‘locked out’ false negative uncertain matches.

Based on the overall assessments, there is a high degree of confidence in the HN; however, there are risks that have to be considered when it comes to the opportunity of false positive patient matches. Based on the analysis in this paper, we are currently reviewing the opportunity of implementing a consistent patient matching method across both DIR1 and DIR2. Future opportunities could include leveraging Ontario’s provincial assets for patient identity management and investigating the prospect of enhancing EMPI patient matching with machine learning.