Quantitative Evaluation of PACS Query/Retrieve Capabilities

Abstract

The PACS (Picture Archiving and Communication System) market is heterogenous with dozens of PACS providers having deployed installations in healthcare facilities. The DICOM® query and retrieve interfaces provided by PACS have multiple variations, related to the implemented SOP (Service Object Pair) Classes, transfer syntaxes, extended negotiations, matching attributes, and matching types. These variations can make integration of a new DICOM® consumer with a PACS complex and time consuming. Even if most of PACS products provide a DICOM® conformance statement with a description of its query/retrieve capabilities, there is no collective analysis describing the various PACS query and retrieve capabilities and variations. Also, application developers and healthcare facilities lack a method to evaluate the PACS capabilities and classify its functionalities. GE designed a method to evaluate the PACS capabilities in terms of query retrieve functionalities. Our aim is to analyze several PACS in test and production environments, using our method to provide the DICOM® object consumers a macroscopic knowledge of query/retrieve capabilities of PACS functionalities and its different variations. Our evaluation can also be used by PACS and VNA (Vendor Neutral Archive) developers to evaluate their query/retrieve capabilities, for quality improvement purpose.

Introduction

Most healthcare facilities have an archiving mechanism for patient images and exams. The most known and used archiving method is the Picture Archiving and Communication System (PACS) [1]. PACSs utilize the DICOM® Standard to store, to query, and to retrieve DICOM® objects, through DICOM® C-STORE, C-FIND, C-GET, and C-MOVE services [2]. We have developed an application to formally evaluate the query/retrieve capabilities of a PACS, in order to improve the interoperability of algorithms intended to automatically retrieve DICOM® objects for analysis.

Hypothesis

There are many variations in the marketplace in terms of PACS query and retrieve capabilities.

• The C-GET service is not always supported and may be prohibited by some PACS.

• The C-MOVE service could be performed through multiple SOP Classes, and vary through the supported extended negotiations.

• The C-FIND service possesses the greatest flexibility and variations related to SOP (Service Object Pair) Classes, transfer syntaxes, matching attributes, and types.

For each service, there are three SOP Classes that can be implemented. However, PACS does not always implement all of them. Furthermore, for each SOP Class, several querying levels are defined. In C-FIND service, for each level, a multitude of matching attributes are possible. The possible matching attributes in each level are described within the DICOM® Standard; however, most of them are optional, and thus each PACS provider supports what it considers necessary for their system and for the facilities. Configuration varies from system to system. In fact, two installations of the same PACS version may have different matching attributes support capabilities, based on local configuration, and based on the capacity of the PACS to activate or deactivate supported attributes. Nearly all PACS providers produce a DICOM® Conformance Statement (DCS), where they provide a description of their matching attributes support for each level and for each supported C-FIND SOP Class. However, the DICOM® Conformance Statement describes the product and how it can be configured, but cannot describe how the product has currently been configured in a specific site installation. The capacity of matching attributes has many types: single value matching, range matching, list of UID matching, wildcard matching, and universal matching. PACSs do not provide the same capability for all the attributes they support; furthermore, this support is generally site specific. Thus, most PACS provide in their declarative DCS a description of their matching attributes support. Many describe the supported matching types, but some do not.

Many DICOM® consumers are developed using the safest capabilities. For instance, some searching attributes are required by the DICOM® Standard, and supposing that the PACS is not implementing any special capability may be sufficient for a wide range of DICOM® consuming application. However, for a DICOM® node wanting to consume DICOM® objects across several PACS vendors and versions using complex and/or optimized queries, the task is challenging. An in-depth site survey involving communication with IT and PACS administrators and on-site testing is often required to fully understand PACS query/retrieve capabilities. From the development perspective, developers must assert suppositions regarding the matching attributes supported in PACS; incorrect suppositions will prevent the product from functioning as needed and reduce its usage.

Our aim is analyzing several PACS in test and production environments to provide the DICOM® object consumers a macroscopic knowledge of PACS query/retrieve capabilities they may be able to expect and to encounter in practice. Our method could also be used by PACS and VNA developers to evaluate their own query/retrieve capabilities, for quality improvement purposes.

Background

Picture Archiving and Communication System (PACS)

PACS is used for storing and accessing exams and DICOM® objects for multiple types of modalities, like computed tomography (CT), ultrasound (US), magnetic resonance imaging (MRI), and nuclear medicine exams (NM), as well as other known modalities [1, 16]. In addition to radiology, many healthcare application areas are supporting PACS integration, such as cardiology, oncology, dental domain, and ophthalmology. The storing and retrieval of images within PACS is mostly based on the DICOM® Standard, using the DIMSE-C services C-FIND, C-MOVE, C-GET, and C-STORE service [2]. However, new technologies emerged recently and integrated with the most recent versions of PACS, such as DICOMweb™ protocols for storing and retrieving images (STOW, QIDO and WADO) [3, 25]. Despite the rapid adoption of Web protocols over the last several years, the standard DICOM® services are still widely used and are the reference implementation for any modality communicating with a PACS.

Dozens of PACS providers exist in the market. Many studies have attempted to collect PACS information for sale or for comparison purpose [4–6, 18]. Based on our experience, there are a multitude and a variety of PACS deployed in hospitals and in healthcare facilities. The selection of a PACS provider depends on many factors, such as price, functionalities, and facility needs [16, 18]. Large facilities do not target the same PACS providers as small facilities [11]. The following is a non-exhaustive list of known PACS providers, without order: General Electric, Sectra, INFINITT, Intelerad, Siemens, Agfa, Philips, Fujifilm, IBM, McKesson, DR Systems, eMed, PaxeraHealth, Amicas, Cerner, IDX, Canon, and Carestream. Many other PACS providers exist including open-source PACS [12], like the well-known DCM4CHEE, Conquest, Orthanc, and many others [5]. Some open-source tools and methodologies exist allowing to test PACS capabilities [22, 23]; however, there are not enough to perform the needed analyses in this paper.

Query Retrieve PACS Variations

In general, PACS provides the possibility to perform query and retrieve of exams and DICOM® objects through the C-FIND, C-MOVE, and C-GET services [2]. There are variations from one implementation to another. Even for the same software version, two different installations may have different configurations, and thus different query/retrieve capabilities. The variations between PACS can be divided into query variations and retrieve variations. The current analysis does not include DICOM® SOP Classes for the query/retrieve of non-patient instances, which are considered out of scope (example: Color Palette Query/Retrieve, Hanging Protocol Query/Retrieve, Defined Procedure Protocol Query/Retrieve, etc. [2]).

DICOM® Query Variations

There are three SOP Classes that enable DICOM® query to a PACS, and each PACS may decide to implement one or multiple of the following SOP Classes:

• Patient Root Query/Retrieve Information Model – FIND [1.2.840.10008.5.1.4.1.2.1.1]

• Study Root Query/Retrieve Information Model – FIND [1.2.840.10008.5.1.4.1.2.2.1]

• Patient/Study Only Query/Retrieve Information Model – FIND [1.2.840.10008.5.1.4.1.2.3.1]

For each SOP Class, queries for exams are not performed in the same way. The main differences between the three methods are the query levels supported and the matching attributes supported at each level. The Patient Root Information Model has four query levels [2]: PATIENT, STUDY, SERIES, and IMAGE. The Study Root Information Model has three query levels [2]: STUDY, SERIES, and IMAGE. The Patient/Study Only Information Model has only two query levels: PATIENT and STUDY. For each level, a list of possible matching attributes can be queried. The Patient/Study Only Information Model is retired since 2004 [2]; however, many image managers continue to support it due to possible legacy consumer applications. Likewise, many healthcare facilities have legacy PACS; thus, consumers may need to support the Patient/Study Only Information Model method. The retained query method within the latest DICOM® Standard are the Patient Root Information Model and the Study Root Information Model [2]. The main difference being that the Study Root Information model has a query level STUDY, grouping the tags from the query levels PATIENT and STUDY, coming from the Patient Root Information Model. With this slight difference, there is a major difference in the implementation for DICOM® objects consumers, as the STUDY level is not always directly “query-able” for the PACS implementing only the Patient Root Information Model.

For each query level, a list of matching attributes can be used to perform queries to gather information about the patients, studies, series, or instances, based on the selected query level [20]. Each query level has a matching attribute as a unique identifier. For example, at the SERIES level, the unique identifier is the SeriesInstanceUID. All the unique identifiers are required to be supported as a matching attribute; however, only other few matching attributes are required by the DICOM® Standard, per query level [2]:

• PATIENT level: Patient’s Name, Patient ID (unique key)

• STUDY level: Study Date, Study Time, Accession Number, Study ID, Study Instance UID (unique key)

• SERIES level: Modality, Series Number, Series Instance UID (unique key)

• IMAGE level: Instance Number, SOP Instance UID (unique key)

All the other matching attributes are optional and may or may not be supported by PACS systems. For each tag, there are many ways to support it as a matching attribute. PACS may implement one of six matching types [2, 14]:

• Single value matching: an exact match on the value specified. When matching an attribute that has a value multiplicity of greater than 1 (e.g., Modalities in Study, SOP Classes in Study), if any of the values in the target attribute match the single specified value, then that record matches and DICOM® requires all the values in the attribute be returned (see [2], section C.2.2.3).

• Range value matching: applicable for date and time matching attributes, like StudyDate and StudyTime where either a start or an end or both may be specified.

• List of UID matching: describes how to handle searching and retrieving using multiple UIDs. For example, this matching type can be used in C-FIND queries to search for studies matching at least one of multiple StudyInstanceUID values. However, in C-MOVE queries, providing a list of StudyInstanceUID is a query to retrieve the different studies matching the provided values. In this paper, the C-MOVE behavior of this matching type is out of scope, and was not analyzed.

• Wildcard matching: this is related to string matching and the capability to use “*” and “?”

• Universal matching: zero-length value is specified, which is considered to match any value. Since universal matching does not imply a matching algorithm as its name suggest, but rather is a mechanism to request the selected Attribute Value be returned in corresponding C-FIND responses, it is commonly called a “Return key” within IHE®.

• Sequence matching: a method to handle matching on attributes structured as a Sequence Items (SQ as VR); also, its sub-attributes. The sequence matching identifies how to use the other matching types within a sequence.

Even if the DICOM® Standard mandates some matching attributes to be required in the different query levels, it stays not clear what matching types shall be supported by these attributes.

Here, we note the potential complexity for DICOM® consumers whose needs are not met by the required attributes listed above (e.g., Patient Name, Patient ID, Study Date, Accession Number, and Modality). Every PACS has the potential to implement any matching attribute from the optional matching attributes for each level, and for each matching attribute, it can implement any matching capability from the previous described types; constructing optimized algorithms for collecting data from PACS is complex, and must take into consideration these variations.

Another variation of PACS DICOM® queries is extended negotiations. SOP Class Extended Negotiation allows a consumer SCU to ask the PACS, during association establishment, whether some advanced features are available for use [24]. C-FIND service can have multiple flavors from standardized options, which may be documented in the DICOM® Conformance Statement [2]:

• RELATIONAL: this is an important aspect that must be considered during DICOM® queries. A query to a PACS can be relational or hierarchical. In relational PACS, you can perform C-FIND queries using any supported matching attribute from any query level; also, there is no need to provide the unique identifier of the upper levels. In hierarchical PACS, you can only use tags from the targeted query level and must provide the unique identifiers of the upper levels.

• DATETIME: a variation to support combined date and time matching.

• FUZZY: a variation to support fuzzy semantic matching, used to query person names.

• TIMEZONE: a variation to support time zone considerations.

Note: There is an additional extended negotiation defined by the DICOM® Standard that is not included in this analysis, which is Enhanced Multi-Frame Image Conversion.

In conclusion, we can identify three kinds of variations in PACS capabilities regarding DICOM® queries support:

• Support of extended negotiations: RELATIONAL, DATETIME, FUZZY, and TIMEZONE

• Support of Root Information models: Patient Root, Study Root, and Patient/Study Only Root.

• Support of tags as matching attributes: and for each matching attribute, the supported matching types: single value matching, range matching, list of UID matching, wildcard matching, and universal matching.

Let us consider the following use case. An exam consumer application needs to gather all series performed last week. The easiest method is to perform a C-FIND query in the SERIES query level, using the matching attribute SeriesDate, and providing a range of date corresponding to the last week range. This method is targeting the PACS supporting:

• Patient Root or Study Root information model

• RELATIONAL queries

• SeriesDate supported as query matching attribute, with the type range matching.

Before this paper, there was no organized survey to determine what DICOM® query capabilities a consumer application could safely assume to be available from most hospital PACS; thus, we could not confirm if the defined algorithm is a good one.

DICOM® Retrieve Variations

To perform a retrieve of DICOM® objects from PACS, there are two DIMSE-C services that can be used: C-MOVE and C-GET. C-MOVE requests the PACS to send the studies, series, or instances to a third destination through C-STORE operation; on the other hand, C-GET requests the PACS to send the studies, series, or instances to the same SCU destination, using the opened association [24]. Each of these DIMSE-C services has three related SOP Classes [2]:

• Patient Root Query/Retrieve Information Model – MOVE [1.2.840.10008.5.1.4.1.2.1.2]

• Study Root Query/Retrieve Information Model – MOVE [1.2.840.10008.5.1.4.1.2.2.2]

• Patient/Study Only Query/Retrieve Information Model – MOVE [1.2.840.10008.5.1.4.1.2.3.2]

• Patient Root Query/Retrieve Information Model – GET [1.2.840.10008.5.1.4.1.2.1.3]

• Study Root Query/Retrieve Information Model – GET [1.2.840.10008.5.1.4.1.2.2.3]

• Patient/Study Only Query/Retrieve Information Model – GET [1.2.840.10008.5.1.4.1.2.3.3]

The SOP Classes related to Patient/Study Only Query/Retrieve Information Model are retired from DICOM® Standard since 2004; many PACS systems are still supporting them for legacy consumers; however, it is not clear if there are still some devices supporting only these retired SOP classes.

The main difference between these SOP classes is in the creation of the retrieve query when the extended negotiation RELATIONAL is not supported [2]. In fact, there are two extended negotiations supported by C-MOVE and C-GET services: RELATIONAL retrieve and Enhanced Multi-Frame Image Conversion (see reference [2], sections C.4.3.2.2). In this analysis, we include only the RELATIONAL retrieve.

Thus, there are two main analyzed variations in C-MOVE and C-GET services:

• Support of Root Information models: Patient Root, Study Root, and Patient/Study Only Root.

• Support of RELATIONAL retrieve.

Methods

Evaluation of Query and Retrieve Capabilities

Query/Retrieve capabilities are described in the DICOM® Conformance Statement (DCS), provided by the PACS vendors [7]. The information in the DCS might be used by an engineer to configure the consumer during installation, but the information cannot be directly used by an object consumer, as it is not provided in machine-readable format. The DICOM® object consumer can determine the available SOP Classes, transfer syntaxes, and other extended negotiation features of the PACS using the DICOM® Association Negotiation mechanism, but not the matching attributes and matching types which are listed in the DICOM® Conformance Statement. It is also possible that the site may have configured the PACS to limit the matching attributes, the matching types, or the DIMSE-C services used, like for example, prohibiting the use of C-GET operations. GE Healthcare developed an application to formally evaluate the query/retrieve capabilities of a PACS to improve the automation of algorithms designed to retrieve DICOM® objects. The developed application identifies the following properties for each PACS:

• Supported SOP Classes for query/retrieve services. For each SOP Class:

  • ⚬ Supported transfer syntaxes
  • ⚬ Supported extended negotiations
  • ⚬ Supported matching attributes from selected list of matching attributes (see Table 1). And for each matching attribute, the application identifies the supported matching types:

    • ◾ Single Value matching
    • ◾ List of UID matching
    • ◾ Range Value matching
    • ◾ Wildcard matching
    • ◾ Universal matching

Table 1.

Tested C-FIND matching attributes

Query level	Number of tested attributes
	Common attributes identified by DICOM®	Extra attributes from the analyzed level
PATIENT	16 (of 16 attributes)	0 (no extra attributes tested)
STUDY	24 (of 24 attributes)	4 attributes are tested related to the Procedure Code Sequence, as sub-attributes, coming from Enhanced Code Value Keys Macro: - Code value - Coding scheme designator - Coding scheme version - Code meaning
SERIES	4 (of 4 attributes)	12 from General Series Module Attributes 6 sub-attributes related to sequences from General Series Module Attributes 2 from Performed Procedure Step Summary Macro Attributes 7 from General Equipment Module Attributes
IMAGE	17 (of 18 attributes)	8 attributes related to sequences from the common attributes identified by DICOM®, and coming from Enhanced Code Value Keys Macro 1 attribute coming from General Image Module Attributes (Image Type)

The supported matching types are analyzed only within the C-FIND queries, and the discovery method is described under a pending patent “Systems And Methods For Device Query/Retrieve Capability Discovery.”

The sequence matching type is also considered in this analysis; as described in DICOM® Standard, the sequence matching describes how to handle other matching types within the SQ attribute and the attributes under its items. Thus, for SQ attributes and their sub-attributes, we use the sequence matching type to construct the C-FIND queries, and we test the other matching types.

The matching attributes are generally described in the DICOM® Conformance Statement (DCS) of the PACS. However, the application does not take into consideration the content of the DCS; the identification of the matching parameters is algorithmically calculated through systematic query of the PACS with different parameters, to test real-world capabilities. For each matching attribute and for each matching type, positive and negative tests are performed with the PACS. The generated output from the discovery is then directly computed from the PACS installation, and not declarative from the PACS provider. This capabilities discovery simplifies the integration process with PACS.

There are hundreds of matching attributes that can be tested. In fact, in each query level, the DICOM® Standard describes a list of common attributes, and points out that all other attributes of the analyzed level can be identified as a matching attribute. In this paper, we analyzed only a subset of 101 attributes, mostly composed from the common attributes identified in the DICOM® Standard, and some extra attributes that we estimate important to test. Table 1 describes the distribution of the tested attributes per query level.

No extra attributes are tested in the PATIENT and STUDY level. Most of added attributes were in the SERIES level. We estimate that most of the attributes in the General Series Module are important, need to be analyzed, and may be useful during series filtering, like SeriesDate for example. Also, we estimate that attributes from General Equipment module are important and very helpful to refine the queries to the PACS, especially when we are searching based on the device generating the series. For coded values, we restricted the analysis on the attributes code value, coding scheme designator, coding scheme version, and code meaning, which is largely sufficient to identify a code and its coding scheme.

The output is an XML file describing the listed properties. Figure 1 is a partial presentation in HTML of the findings related to an evaluated PACS.

Fig. 1.

PACS Q/R capabilities discovery result

This application was executed in several PACS installations from different vendors. We collected results in a database of PACS query/retrieve capabilities, to facilitate analysis of PACS from the query/retrieve aspects. In this paper, we consolidated the different results in order to provide a general understanding of PACS systems capabilities.

Quantitative Analysis Methodology

In order to evaluate a PACS regarding its Q/R capabilities, we need to define comparison parameters based on the following information:

• The list of SOP Class supported

• The list of extended negotiations supported

• The list of matching attributes supported for each level and their different matching types.

To compare the different C-FIND matching attributes capabilities between PACS, we used three parameters:

• The global number of supported universal matching keys (N_u)

• The global number of supported single value matching keys (N_si)

• The global number of strong matching keys (N_str)

Forty-nine different PACS installations were tested and analyzed in order to define a macroscopic understanding of the PACS query retrieve capabilities. The different selected PACS installations come from different environments, some are in production, and others from testing and beta testing environments. The evaluated PACS are coming from 27 different vendors, including many known providers.

The analyses include:

• Categorization of PACS through their Q/R capabilities

• Identification of matching keys capabilities and usage

For each analysis of the results of PACS evaluation, two methods were performed:

• Blind analysis: all the PACS are analyzed as equivalent in terms of market share; there is no appreciation of one PACS over the other PACSs.

• Weighted analysis: the results are weighted based on market share [10]. For instance, if a PACS has a robust query capability, and small market share, the results will be lowly weighted with minimal impact, as the tested PACS does not describe the behavior that we may encounter in the marketplace from a macroscopic point of view.

Weighted Analysis Calculation

To determine the market shares of PACS providers, several studies [8, 9, 17] were consulted. The values from the most recent one [9], a summary of a HIMSS report on 4605 hospitals, were used in the weighted analysis.

The formula used for the weighted analyses is as follows [10]:

$$P_{\mathit{opt}}^w=\frac{{\sum }_{i=1}^n{E}_i\times S_i}{{\sum }_{i=1}^n{S}_i}$$

where.

n is the number of targeted PACS vendors in our analysis and matching the study in the reference [9], related to PACS market share.

S_i is the percentage of market share, identified in the reference [9], related to vendor “i.”

E_i averages the analyzed option found in the tested PACSs from vendor “i.” Example of options: support of RELATIONAL extended negotiation, support of a specific SOP Class, etc.

${\sum }_{i=1}^n{S}_i$ a constant that describes the reliability of our analysis to represent the market segmentation, we call it in our analysis: the reliability indicator. Note: this indicator does not depend on the analyzed options.

In our analysis, this value is:

$${\sum }_{i=1}^n{S}_i=0.67$$

The study [9] identifies 11 different vendors sharing 81% of the market. A total of 19% of the market was filled by other PACS providers. From the analyzed 27 PACS providers, 5 matched identified vendors in study [9], so the value of S_i was taken from there and together represented 48% of the market. The other analyzed 22 PACS providers did not appear in study [9]. These 22 providers were assumed to represent the entirety of the remaining 19% of the market, and were assumed to have equal market shares. Thus, an S_i value of 0.86% was used for each of those 22 providers and they are all included in the weighted analysis.

Application Results

Categorization of PACSs Through Q/R Capabilities

Q/R SOP Class Variations

We performed an analysis related to SOP Classes for the C-FIND, C-MOVE, and C-GET services, supported by PACS. The results are shown in Figs. 2 and 3. Figure 2 describes the percentage of supported SOP Classes based on the 49 tested PACS installation. Figure 3 describes the same results weighted following the method described above.

Fig. 2.

Percentage of SOP Class deployment: blind analysis

Fig. 3.

Percentage of SOP Class deployment: weighted analysis

The weighted analysis follows the same pattern as the one for the blind analysis. We note three observations for the PACSs tested: (1) the C-FIND and C-MOVE services were always supported under the Study Root Query/Retrieve information model, (2) the Patient Root Information Model was almost always supported, and (3) the C-GET service was not always supported, with at most 40% support by the tested servers, and less than 30% as identified in the weighted analysis. We can conclude that utilizing C-GET service as a source of information for a DICOM® consumer covers few installations. This percentage is also confirmed in the literature [19].

The Patient/Study Only Query/Retrieve Information Model is retired from the latest versions of DICOM® Standard; however, we note that 50% of tested servers implement a SOP Class related to this information model.

The analysis confirms that all the tested PACS installations are implementing the Study Root Information Model; thus, developers of DICOM® consumers should consider it as their primary method for C-FIND operations. This is also confirmed in the literature [14].

Extended Negotiation Analysis

Figure 4 describes the unweighted repartition of support for extended negotiations related to C-FIND service. The percentages in Fig. 4 are the analysis results of the 49 PACS installations. Figure 5 describes the result of weighting the Extended Negotiation support for C-FIND service by PACS Providers, using the method described above.

Fig. 4.

Extended Negotiation support percentage for C-FIND service: blind analysis

Fig. 5.

Extended Negotiation support percentage for C-FIND service: weighted analysis

In both figures, we note that fewer than 40% of PACS support relational mode, which is considerable; developers of DICOM® object consumers would be advised not to depend on relational C-FIND being available. This result mirrors findings in an article on the Medical Connection website [13]. The support of the other extended negotiations is also significantly less than 30%.

Less than 20% support fuzzy semantic matching and time zone extended negotiation in the blind analysis, and less than 15% in the weighted analysis. We noted that VNAs primarily support the time zone query adjustment option.

The analysis of the RELATIONAL extended negotiation for C-MOVE and C-GET DICOM® services are described in Figs. 6 and 7.

Fig. 6.

RELATIONAL Support Percentage for C-FIND/C-MOVE/C-GET services: blind analysis

Fig. 7.

RELATIONAL Support Percentage for C-FIND/C-MOVE/C-GET services: weighted analysis

We remark that less than 30% of the PACS supporting C-MOVE or C-GET are supporting the RELATIONAL extended negotiations. Not all PACS supporting a C-FIND service in relational mode support a C-MOVE service in a relational mode, forcing the DICOM® consumer to check availability of a relational request before using a C-MOVE service, even if it is supported in the C-FIND service.

C-FIND Matching Attributes PACS Variations

In this analysis, we evaluate the number of supported matching attributes and their related matching types: universal matching, single value matching, and strong matching. We consider a matching attribute to be strong if it supports at least the range matching, the list of UID matching, or the wildcard matching. From the samples of PACS selected, we calculated the normal distribution of supported matching attributes. The results of this analysis are summarized in Table 2, and presented in Fig. 8.

Table 2.

Average and standard deviation of attributes supported by PACS

Matching type	Blind analysis		Weighted analysis
	Average	Standard deviation	Average	Standard deviation
Universal matching	31.08	10.87	33.48	11.65
Single value matching	20.40	7.17	19.89	8.77
Strong matching	17.22	6.14	16.81	3.49

Fig. 8.

Normal distribution for query attributes matching: blind analysis

The average of implemented universal matching attributes is 33.48 in the weighted analysis, which is quite high when compared to that in single value and strong matching.

Figure 8 shows the disparity between PACS implementations. Some of the PACS are providing only few matching attributes, highly reducing the possibilities offered to the DICOM® consumers. For example, in terms of universal matching, we observed PACS providing only few matching attributes (less than 25 matching attributes), and others are providing more than 50 matching attributes. This analysis can be used by PACS providers to compare their Q/R capabilities to other implementations.

Matching Keys Analysis

In this paragraph, we discuss matching attributes implementations, with a focus on attributes specificities, not attributes quantity. In the previous application, we evaluated the number of matching attributes implemented by PACS systems; this allowed comparing between PACS providers. As a DICOM® consumer, we also need a detailed cartography of the frequency of matching attributes implementations, for a better understanding of the PACS, and for optimization of DICOM® object identification and collection.

The matching attributes are divided into three levels: STUDY, SERIES, and IMAGE. The attributes related to the PATIENT level are merged with the STUDY level, following the Study Root Information Model. We performed two analyses, as in our previous analyses: a blind analysis and a weighted analysis, following the PACS market share description [9]. For each analysis, we calculated for each tested matching attribute the following values:

• G_u: the implementation percentage of universal matching type

• G_si: the implementation percentage of single value matching type

• G_str: the implementation percentage of a strong matching type (a range value matching, a list of UID matching, or a wildcard matching type).

The analysis of 49 PACS installations, weighting of market share for the 27 different PACS vendors, provided us the outputs in Table 3.

Table 3.

Matching attributes analysis

Table 3 includes all the matching attributes where the blinded analysis (G_u) is greater or equal to 15%.

As described in the paragraph “DICOM® Query variations”, DICOM® Standard defined a list of required matching attributes for each query level. PACS systems shall support matching on required matching attributes. However, some of these required attributes were not supported by all the tested PACS systems, like AccessionNumber, StudyID, SeriesNumber, and InstanceNumber. This pointed to the nonconformance of these PACS systems to DICOM® Standard.

The most implemented attributes are in the STUDY level; always implemented are StudyDate, StudyTime, AccessionNumber, PatientName, PatientID, StudyID, StudyInstanceUID, ModalitiesInStudy, NumberOfStudyRelatedSeries, NumberOfStudyRelatedInstances, and PatientSex. ModalitiesInStudy is well supported by all tested PACS systems; this finding simplifies exams collections by targeting only studies with the searched modality types. We were surprised that the StudyTime was not near 100% of support, for single value and strong matching. In order to query large archives, some consumers search based on both StudyDate and StudyTime. The absence of this tag from single and strong matching may be problematic for consumers and for some use cases. Also, the PatientBirthDate is supported at 83%, which is problematic; many products utilize patient name and patient birth date to query patient-related clinical information. This method is typical in healthcare domain, as multiple patients may have the same name and differentiating by birth date provides a better combination to identify the desired patient, as in the IHE® XCPD profile [15]. When PACS does not support PatientBirthDate queries, many consumers applications and functionalities are useless. IssuerOfPatientID was supported at only 27%, which is a very low value. In many PACS systems, having patients coming from multiple facilities or multiple hospitals is very common; even within the same facility, importing exams performed by other hospitals or structures is also common. Without providing IssuerOfPatientID, these situations may lead to patient identifiers conflict. Querying using IssuerOfPatientID is the best way to avoid having inconsistent data, which could be dangerous for the patient healthcare in some use cases.

Three attributes are strongly supported in the SERIES level: SeriesInstanceUID, SeriesNumber, and Modality. These attributes are indeed required by the DICOM® Standard for both Patient Root information model and Study Root information model. NumberOfSeriesRelatedInstances is also very well supported. Many other attributes are supported in universal matching, but few are supported in single and strong matching types. It is remarkable that the SeriesDate and SeriesTime are supported by only half of the tested PACS; this analysis allowed us to highlight the poor support of these matching attributes, and thus algorithms utilizing Series search through SeriesDate are limited to only few PACS systems. In the paragraph “DICOM® Query Variations”, we described a use case using the SeriesDate and the relational mode as key concept for the objects gathering algorithm. We saw that the relational mode is supported by less than half of the deployed PACS systems, and the SeriesDate is supported by 32% of PACS, for the single value matching and the strong matching types. This makes the defined algorithm ineffective in most of the encountered PACS systems.

At the IMAGE level, only the InstanceNumber, the SOPInstanceUID, and SOPClassUID are strongly supported as matching attributes. Only the attribute ImageType was supported by more than 50% of tested PACS systems. We remark that the SOPClassUID is not supported at 100% as universal matching, and less than 80% for single value and strong matching, which was unexpected. In fact, SOPClassUID identifies the object type, like CT Image, Secondary Capture Image, or X-Ray Radiation Dose SR. In the same exam, different object types may be present. Some algorithms use the PACS capabilities to filter on SOPClassUID in order to avoid consuming unwanted objects. Without the support of SOPClassUID and the ImageType, issues related to bandwidth and timeouts can be observed in DICOM® consumer applications, as these applications will be obliged to consume the complete exam and to filter on the targeted objects after that.

All required matching attributes are heavily supported, and ten extra matching attributes are supported as return key by more than 80% of tested PACSs. Most of these found supported matching attributes correspond to required return keys by the actor Imager Archive in IHE® SWF.b profile [26], following the transaction Query Image (RAD-14) [21].

This analysis allowed a better understanding of query/retrieve capabilities of PACS systems. It can be used to improve gathering algorithms of DICOM® objects. The analyses of the different PACS capabilities created a unique PACS Q/R capabilities database.

Weighted Analyses Discussion

In this paper, we used the study in reference [9] in order to weight the PACS systems analyses by their market share values. Some weaknesses of this method need to be highlighted. First, the reference [9] seems targeting the US market, which is different from the European or the Japanese market. Some of the highlighted PACS systems are less present outside the USA. Also, in Europe, the PACS market is different from one country to another. From the other hand, the weighted analysis does not take in consideration the PACS providers’ versions and models. Sometimes, the same provider can have multiple PACS products, and dozens of versions with different query/retrieve capabilities. Even after taking into consideration all these parameters, having the weighted analyses allowed us to have a better view on the different performed analyses, and rising the reliability of the found results.

Conclusion

This paper provides a view of what to expect from different PACS Q/R within healthcare facilities, allowing us to understand the variability of PACS Q/R functionalities. Quantitative analysis enables comparison of Q/R functionalities and aids the DICOM® consumer products in establishing optimized gathering and communication algorithms. Our analyses identified the behavior of PACS from different perspectives: the support of Q/R SOP Classes and information models, the support of extended negotiations, and the support of query matching attributes with their different supported matching types. The analyses confirmed that the Q/R PACS interfaces are quite variable and differ from one PACS to another. We also identified several consistent patterns and behaviors. All the tested PACS implement the Study Root Information Model, and most of them implement the Patient Root Information Model. The C-GET service is not always supported. Less than the half of PACS are supporting the RELATIONAL extended negotiation, and fewer support the other extended negotiation options. Required matching attributes are well supported by most of the PACS systems, and ten other matching attributes were supported at more than 80% of the tested PACSs.

The comparison and the analysis between PACS C-FIND queries variations provide PACS providers, and healthcare facilities, the possibility to evaluate their functionalities. The analysis of the matching attributes implementation regarding the supported matching types allowed having a complete cartography of what can be expected in production environments regarding attribute matching. These results are valuable information for DICOM® consumer products, in order to improve their gathering algorithms, improve the connectivity bandwidth, and avoid surprises in production.

Consolidating all the attributes matching types in a unique table allowed having a visual interpretation of the results. Using two methods of analysis, the blind analysis and the weighted analysis, allowed improving reliability of the found results.

Further research utilizing this analysis could include automatic validation of DICOM® conformance statements. Also, the same analysis process can be extended to DICOMweb™ QIDO-RS implementations. The analysis of the significance of Q/R capabilities can also be helpful to evaluate the PACS capabilities and to create a PACS grading method, through a deep analysis of the common DICOM® workflows.

Declarations

Research Involving Human Participants and/or Animals

This research did not involve protected health information from human participants; the methodology did not collect patient information, only system information.

Conflict of Interest

The authors report all the authors are employees of GE Healthcare during the conduct of the study. In addition, one of the authors has a patent Method for SYSTEMS AND METHODS FOR DEVICE QUERY/RETRIEVE CAPABILITY DISCOVERY pending.