Is the Training of Imaging Informatics Personnel in New Zealand Adequate?

Abstract

The purpose of this study of Imaging Informatics Professionals (IIPs) in New Zealand was to assess their experience, background, educational qualifications and needs for support and continuing education. The IIP role includes administration of DICOM modalities, picture archiving and communication systems (PACS), radiology information systems (RIS) and many additional software and hardware systems, including the interface to New Zealand’s nationwide individual electronic medical records (EMR) system. Despite the complexity of current systems, training programmes for IIPs are almost non-existent in Australasia. This cross-sectional qualitative case study used triangulated data sources, via online questionnaire, interview and critical incident analysis. Demographic data was also obtained from the questionnaire. Participants included about one third of the IIPs in New Zealand. Quantitative results were summarised with descriptive statistics or frequency data. Qualitative data was assessed by iterative multi-staged thematic analysis. This study found that the IIP role is undertaken by personnel from diverse backgrounds. Most of the IIPs learned what they know from vendors and on the job. Many feel that their biggest issue is in not knowing what they do not know and therefore not having sufficient understanding of the imaging informatics field. Only one IIP had any formal certification in PACS administration. Most respondents indicated their desire for some form of additional training. The number of IIPs in New Zealand healthcare is very small, so neither a formal training programme nor regulatory body is viable or justified. However, IIPs believe there is a need for education, regulation and recognition that their role is a critical component in healthcare.

Background

Very few people turn up to work each day deliberately planning to do a poor job. However, mistakes are common in most occupations, leading to a substantial overhead in rework which is a significant component of waste in healthcare. An estimate of overall waste in US healthcare was as high as $1.2 trillion of the $2.2 trillion total healthcare cost (PWC). Reduction of rework was discussed by W. Edwards Deming [1] in his landmark book Out of the Crisis. In particular, Deming’s sixth point of his famed 14 points for management states that workers need to be adequately trained to do their jobs and that the responsibility for this training rests with management.

In the health sector, mistakes include incorrect diagnoses, leading to inappropriate patient management and imposing a significant burden on health budgets. Medical imaging examinations are one of the sources of incorrect diagnosis, the proportion of missed lesions being as high as 30 % in some documented studies, whilst false positives lead to a significant number of unnecessary follow-up procedures [2].

With the advent of digital imaging and the associated picture archiving and communication systems (PACS), there was promise of an improvement in the timeliness and accuracy of radiological diagnoses, because of image processing capabilities, computer-aided diagnosis, elimination of lost images, improved workflow, streamlining of second opinions, availability of teleradiology and so on [3–5]. Along with digital imaging, there has grown a new industry, medical imaging informatics, charged with the management of the ever-growing deluge of medical images and related information, via PACS and other interconnecting networks, including radiology and hospital information systems (RIS, HIS) and the World Wide Web [6,7]. These systems bring with them the potential for new categories of mistakes, due to software bugs, data input errors and data corruption, for example. The systems are complex in themselves and also rely on health level 7 (HL7) and digital imaging and communication in medicine (DICOM) standards, among others. To manage these systems, the role of the imaging informatics professional (IIP) has evolved [8].

Current PACS may conform to one of four models, these models having developed in historical sequence from modality centric, to department centric, then hospital centric and currently patient centric, with examples of each to be found in New Zealand [9]. The patient-centric model is of particular importance to New Zealand, with government policy being for an electronic health record (EHR) for every citizen [10].

Regardless of the PACS model they support, the IIP staff are subject to the principles of total quality management (TQM). For example, Deming’s 85/15 rule predicts 85 % of the mistakes in imaging informatics systems will be due to failings in institutional management and the Pareto principle predicts 80 % of the problems experienced will be due to 20 % of the causes [11]. Clearly, IIPs need to be adequately trained to meet their responsibilities [12], so that the maximum benefit in terms of timely and accurate diagnosis, with minimization of harm and costs, may be obtained from imaging informatics, as required for a QA programme in New Zealand [13]. Anecdotal evidence of inadequate training combined with the experiences of one of the authors in the informatics field motivated this study of IIPs in New Zealand, their role and previous experience and their needs and requirements for education resources and support.

Methods

A cross-sectional case study method was employed, using triangulated qualitative data on a bound system of individuals, the imaging informatics professionals in New Zealand, to provide an “intense, holistic description and analysis” [14] of their training, experience and expectations. Written approval for the study was obtained from the Unitec Research and Ethics Committee. Three data sources constituted the triangulation: online questionnaires, interviews and critical incident analysis. The questionnaire and interview schedule are described in detail elsewhere [9].

The number of IIPs in New Zealand was estimated to be approximately 75, distributed among four IP regions and several private radiology practices. Participants were sampled from each of the four regions, giving 11 of the 20 DHBs, 2 of 8 private companies and a sample of 31 individuals. The questionnaire was distributed online (using Survey Monkey) to these participants, with 26 responding (84 % response rate). The questionnaire consisted of 54 questions, with a mixture of quantitative demographic questions, categorical questions and qualitative open-ended questions, including options for additional comments. The categorical items were all structured as questions and employed Likert-type response scales with five, seven or nine options with the same response direction from positive to negative [15]. One of the questions asked for volunteers for a follow-up interview, with 12 affirmative responses.

Of these 12, a representative sample of seven participants was interviewed. The semi-structured interview used a set of open-ended questions, including the possibility for additional questions to follow up themes that arose during the interview. The interviewees were also asked to describe a critical event that they considered to be an exemplar of the issues that they faced as IIPs. The widely spread-out locations of the respondents meant that the interviews were conducted online using Skype and were recorded with a digital recorder. Some shorthand notes were made during each interview. Interview transcription was performed as soon after each interview as practicable, and each completed transcript was vetted by the participant [14].

The data was analysed in a systematic fashion. The demographic data from the questionnaire was evaluated using descriptive statistics as appropriate, and the categorical data represented using frequency analysis. The qualitative survey responses and additional comments and the interview transcripts were analysed thematically using a multi-staged iterative approach, enabling the maximum validity and depth to be obtained from the data [16].

Results

Questionnaire Responses

The IIPs were predominantly in the 30- to 49-year age group. For 70 % of them, their primary qualification was a radiography diploma or degree and most of them had maintained their radiography registration. The balance had either an IT-related certificate, diploma or degree, an unrelated degree or, in three cases, no formal qualification. Sixty-five percent had less than 20 years of work experience post-qualification and only five respondents had obtained postgraduate qualifications of any sort. Eighty-five percent of the IIPs were full time, though many of them fulfilled other roles such as radiography and none of them had been in their current positions for more than 10 years.

The IIPs reported a very wide range of responsibilities. Their duties included:

• Applications training of other staff, for at least nine categories of applications from PACS to advanced visualisation software

• Support for at least 15 DICOM modalities including digital radiography, computed tomography, ultrasound, magnetic resonance imaging, nuclear medicine and related systems such as orthopaedic templating

• Direct or indirect support for at least 23 hardware or software systems, such as PACS, HIS, RIS, archive systems, display units, reporting, networks and printing

The IIPs reported that the initial training for these duties was on the job (96 %), vendor supplied (58 %) or as a result of prior experience (31 %), with only 8 % reporting any formal training. The adequacy of training on a nine-point scale was judged moderately adequate or worse, in 89 % of cases, a typical comment being “Minimal training was provided initially. Most of my knowledge I have learnt on the fly from trial and error”.

Their colleagues were judged to be highly skilled by 68 % of the IIPs with another 32 % reporting extremely or moderately skilled colleagues. However, about 80 % of the IIPs judged their own skills as lacking to some extent, with one fourth of these perceiving their skills to be lacking very or extremely often. Eighty-five percent of the IIPs reported some degree of likelihood that they would call for vendor support on any given day, and correspondingly, 85 % reported that they had easy access to required resources, including vendor support. However, only 8 % of the IIPs had completed any formal training related to the PACS, RIS or informatics systems, only 19 % had attended any users-group sessions and 66 % showed some degree of dissatisfaction with their access to relevant education. Furthermore, 81 % stated that some form of courses or education should be compulsory for new IIPs and also expressed a preference for some form of continuing professional education (CPE). Regarding familiarity with relevant standards, 65 % were familiar with DICOM, 52 % familiar with HL7, but less than 12 % familiar with any other standards such as CDA or FDA. Seventy-three percent of the IIPs expressed a preference for IIPs to have a defined scope of practice, whilst 80 % of those with a radiography qualification felt that an advanced practice role would be appropriate.

All respondents found their role to be challenging to some degree, with 65 % rating it as very challenging, several citing the degree of challenge to be a consequence of lack of knowledge and understanding. Nevertheless, 38 % of the IIPs liked their job to a moderate degree and 46 % a great deal, despite 73 % experiencing work stress moderately, very or extremely often. Lack of knowledge and understanding was cited most commonly as the cause of work stress.

The results of the questionnaire are summarised in Table 1.

Table 1.

Summary of results from the questionnaire responses

Imaging informatics professionals	Result
Are most commonly aged	30 to 49
Have as primary qualification	a radiography diploma or degree (70 %)
Have been in their current position	less than 10 years
Consider the quality of their training as	moderately adequate or worse in 89 % of cases
Judge the skill levels of colleagues as	extremely, highly or moderately skilled in 100 % of cases
Judge their own skills as	lacking to some extent, very often or extremely often in 80 % of cases
Are likely to require vendor support	daily—85 %
Have easy access to vendor support	85 %
Have formal training in informatics, PACS, RIS	8 %
Attend user groups	20 %
Familiarity with relevant standards is	DICOM 65 % HL7 52 % all others <12 %
Find the job challenging	100 % (65 % very challenging)
Liked their job	moderately 38 %, a great deal 46 %
Experience work stress	moderately, very or extremely often 73 %
Expressed a desire for formal education	most of them

Interviews and Critical Incidents

From the interviews and critical incident reports, it was clear that the IIPs generally were interested in computers and technology and that most of them had transitioned to the role with some element of chance rather than by a career choice from the outset. One of the most common interview statements [12] and also a common comment in the questionnaires was “you don’t know what you don’t know”. This was associated with a clear desire for better knowledge and understanding, a number of interviewees describing significant extra work being caused by errors or omissions that resulted from lack of familiarity with the systems and system requirements. For example

I had over 14 k exams from our old system when we migrated to our new system to fix due to changes in ways of handling images. Migration made me aware of this but I was not aware of these issues until the issue came up and bit me. It also made me look like I did not know my system and like an idiot and this I do not appreciate. I know better for next time but again it was a lesson learned after the fact because I did not know what I could have known.

Someone went on leave and made a change on a Friday with no documentation. We had an issue and were called and then put three hours into trying to sort but finally had to contact the person on leave.

I was provided with only 4 hours of training and thrown in to the role. It is hard to know what I don’t know and so when I find out things I have done are not accurate it is no surprise.

Proper planning and management of projects would make a huge difference when putting in new systems and realising that there are reasons why some systems don’t work together or infrastructure that might be required. Too often the management or teams will come and demand systems are built and configured without consulting the people in the know and this creates more issues that might have been flushed out earlier.

I think I do too much fire-fighting rather than preparing and it is a source of frustration.

The cause of high stress level at work . . . mostly due to my lack of knowledge and experience in the role.

Most respondents expressed a desire for formal education in DICOM and HL7. Other education topics suggested included networking, project management, MS office, databases and PC knowledge/troubleshooting.

Discussion

From the triangulated data, a clear and consistent picture has emerged of a profession in its infancy, staffed by people who have gravitated to the role from an IT or radiography background, with little or no formal education specific to imaging informatics. The IIPs generally liked their role but expressed frustration at their lack of knowledge and at the failings of the management structures related to planning of the systems and training of personnel, including the IIPs themselves and the end users. Among the frustrations, there was a consistent pattern of lack of knowledge that led either to errors or to difficulties in correcting errors, leading in turn to considerable wasted effort, or rework as described by Deming [1]. His solutions include points 6 and 13 of his 14 points for management:

• 6. Institute training on the job.

• 13. Institute a vigorous programme of education and self-improvement.

Both of these points are the responsibility of the management of the institutions. However, there is currently a lack of suitable courses and qualifications in Australasia, the only certifications being further afield such as PARCA [17] and CIIP [18]. Creation of stand-alone imaging informatics qualifications in New Zealand would not be viable because of the low numbers of enrolments. However, an Australasia-wide initiative would be worth pursuing, especially if treated as an inter-disciplinary initiative between, for example, existing medical imaging, IT and management courses. Such a course could be offered for distance learning online.

Furthermore, despite the clear advantages of adequately trained IIP staff, there would be little incentive for institutions to support the training of their existing staff or recruitment of new trained staff, unless a formal qualification or some form of scope of practice was to be made compulsory. Such a requirement could be incorporated in legislation or could be a criterion for ISO 9001 accreditation [19].

Conclusions

The findings of this study regarding the roles, tasks and responsibilities of the IIPs in New Zealand and their limited access to the necessary educational resources suggest the following recommendations:

1. The industry provide support for New Zealand to join the Integrating the Healthcare Enterprise (IHE) group.

2. Employers should provide funding and leave for IIPs to attend international conferences.

3. A trans-Tasman or Australasian partnership should be established to develop courses to assist IIPs to pursue a postgraduate certificate, diploma or masters-level degree.

4. Employers should encourage IIPs to take advantage of existing courses that may have some relevance, such as adult education training, computer networking, PC skills and productivity software.

5. Workshops with Australasian content should be made available.

6. Clinical training courses should be provided for those entering the field from a non-clinical background.

7. An overview of imaging informatics should be incorporated into undergraduate medical imaging degree courses.

8. IIPs should prepare for, embrace and actively participate in the move by the National Health IT Board towards their IHE goals.

Formalising an academic pathway and recognising certification will establish an identity and recognition of the critical role imaging informatics professionals play in the New Zealand health system, and will lead to a reduction in healthcare costs and to the potential improvements in patient care to be expected from imaging informatics.

Compliance with Ethical Standards

Written approval for the study was obtained from the Unitec Research and Ethics Committee.