Development and Validation of SafeHIT: An Instrument to Assess the Self-Reported Safe Use of Health Information Technology

Abstract

Background  Implementing health information technology (HIT) may cause unintended consequences and safety risks when incorrectly designed and used. Yet, the tools to assess self-reported safe use of HIT are not well established.

Objective  This study aims to develop and validate SafeHIT, an instrument to assess self-reported safe use of HIT among health care practitioners.

Methods  Systematic literature review and a semistructured interview with 31 experts were adopted to generate SafeHIT instrument items. In total, 450 physicians from various departments at three Malaysian public hospitals participated in the questionnaire survey to validate SafeHIT. Exploratory factor analysis and confirmatory factor analysis (CFA) were undertaken to explore the items that best represent a specific construct and to confirm the reliability and validity of the SafeHIT, respectively.

Results  The final SafeHIT consisted of 14 constructs and 58 items in total. The result of the CFA confirmed that all constructs demonstrated adequate convergent and discriminant validity.

Conclusion  A reliable and valid theoretically underpinned measure of determinants of safe HIT use behavior has been developed. Understanding external factors that influence safe HIT use is useful for developing targeted interventions that favor the quality and safety of health care.

Background and Significance

Health information technology (HIT) involves processing, storing, retrieving, sharing, and using health information in an electronic environment. HIT includes electronic health records (EHRs) or electronic medical records (EMRs), personal health records, electronic prescribing (E-prescribing), computerized provider order entry (CPOE), telehealth, clinical decision support system, and hospital information systems (HIS). HIT aims to enhance the quality and safety of health care, medical outcomes, financial performance, and administrative efficiencies. 1 2 3 Nevertheless, there is substantial evidence of unintended consequences and patient safety risks of HIT globally. A retrospective analysis of all safety events between September 2005 and November 2011 revealed that software use errors associated with human factors problems resulted in three deaths. 4 Human factors issues were four times more likely to harm patients than technical issues. 4 Moreover, an analysis of EHR-related patient safety incidents in fully digital Finnish hospitals revealed that problems with human–computer interaction were the most often reported in HIT-related incidents. 5 HIT safety events arise in a complex health care environment due to several reasons, including hardware and software failure, poor usability or functionality, user behaviors, disruptions in workflow, and organizational characteristics. 6 7 8 Understanding the contributing factors is an important first step to minimizing the risks of HIT.

A sociotechnical approach is frequently recommended for patient safety improvement efforts. 9 10 11 Sociotechnical refers to the interconnection between social structure and technology. The concept of the human–machine system will not be accomplished if the system focuses primarily either on the human or machine. Safety incidents emerge from the interactions between people and the elements of technology, tasks, environment, and organization in which they work. 11 12

HIT use behavior refers to the manner in which health care practitioners utilize the capabilities of HIT. 13 The manner in which HIT has been used can be described in several ways, namely workarounds, vigilance, and procedure compliance. The HIT use behavior has implications on the perceived quality and safety of HIT use. Safe HIT use is defined as the prevention of adverse events or outcomes arising from leveraging HIT in the health care process. It aims to avoid and lessen risks, mistakes, and injuries to patients during the delivery of medical care. To date, research on HIT use has profoundly relied on qualitative interviews to understand the factors influencing health care practitioners' HIT use behavior. Although interviews are convenient for gaining an in-depth understanding, they require a lot of time and resources and frequently permit a small sample size. A quantitative instrument, on the contrary, might be a quicker method to gain information among a larger sample.

To the best of our knowledge, the instruments to assess self-reported safe use of HIT among health care practitioners do not exist in the literature. Although there are validated patient safety instruments, 14 15 these instruments tend to measure general attitudes, culture, and climate rather than assess social–technical antecedents to safe HIT use behavior. A survey was administered to measure the attitudes of health professionals toward EHRs in primary health care settings. 16 The survey focuses on the attitude about benefits, barriers, and user satisfaction of EHRs, which influence the health care professional's acceptance, adoption, and use of an EHR rather than the safe HIT use behavior. Besides, the Safety Assurance Factors for EHR Resilience (SAFER) guides are designed to aid health care organizations in conducting self-assessments to optimize the safety and use of EHRs by identifying and addressing potential safety hazards related to technology, workflow, and organizational policies and culture. 17 The guides focus on the areas of foundational guides (high-priority practices and organizational responsibilities), contingency planning guides (contingency planning, system configuration planning, and system interfaces), and clinical process guides (patient identification, CPOE with decision support, test results reporting and follow-up, and clinician communication). Each of the guides contains a checklist of recommended practices for EHR safety. The SAFER guides do not focus solely on safe EHR use behavior but instead provide recommendations for improving the safety and reliability of EHR systems. Moreover, the SOPS Health Information Technology (Health IT) Patient Safety survey assesses health care organizations' safety culture related to HIT use. 15 The survey addresses EHR patient safety and quality issues, EHR system training, EHR and workflow/work process, EHR system support and communication, and overall EHR system rating. On the contrary, SafeHIT covers a broader aspect of social-technical antecedents to safe HIT use behavior, including person, organization, technology, and task dimensions. In addition, the safe HIT use behavior items in SafeHIT focus on vigilance, workarounds, and procedure compliance in using HIT. Therefore, this study developed and validated SafeHIT, a measurement tool to assess self-reported safe use of HIT among health care practitioners. The instrument contains three components: (1) antecedents of the safe use of HIT, (2) HIT use behavior, and (3) perceived quality and safety of HIT use. The safe use of HIT antecedents was considered by considering the socio-technical viewpoint: person, organization, technology, and task dimensions.

Objectives

The objectives of this study are as follows.

• To develop SafeHIT, an instrument that assesses self-reported safe use of HIT among health care practitioners.

• To validate the reliability and validity of the SafeHIT instrument.

Methods

The SafeHIT items were developed and validated based on three stages which involved (1) item generation, (2) scale development, and (3) scale evaluation. 18

Stage 1: Item Generation

A systematic literature review (SLR) and semistructured interviews with experts were conducted to identify the appropriate constructs for the instrument items generation. An SLR was conducted on relevant databases, including Science Direct, Medline, EMBASE, and CINAHL. The terms electronic medical record, health information system, health information technology, medical mistake, and safety were all included as keywords in the search. 19 The semistructured interview involved 31 physicians from three government hospitals in Malaysia. They have used HIS for patient care for 1.5 to 10 years. 20 21

SafeHIT items were informed by the work system in the Systems Engineering Initiative for Patient Safety model and the extended DeLone and McLean information system (IS) success model. Content analysis and thematic analysis of the SLR and semistructured interview allow for articulation of the four key determinants of safe HIT use behavior: (1) person, (2) technology, (3) organization, and (4) tasks. Both SLR and semistructured interviews revealed that knowledge for the person component; system quality for the technology component; organization resources and teamwork for the organization component; and task-related stressors for the task component emerged as the contributing factors. 19 20 In addition, it is discovered from the SLR that information quality and service quality for the technology component and training from the organization component contribute to the safe HIT use. 13 Contrary to the SLR findings, it was discovered from the semistructured interview that physical layout and noise does not influence the safe HIT use. 14 Moreover, the interview results revealed that (1) vigilance, (2) workarounds, (3) procedure compliance, and (4) copy and paste habits emerged as the attributes for safe HIT use behavior. 15 In terms of HIT use outcomes, the findings show that the implementation of HIT leads to positive and negative impacts on patient safety and quality of patient care.

Next, the content for the SafeHIT instrument is developed. The instrument is structured into 11 main sections with a total of 70 items that utilize a seven-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). Table 1 presents the meaning of each construct in SafeHIT, its items resources, and the number of items.

Table 1. Constructs in SafeHIT.

Component	Construct	Definition	Items resources	Initial number of items
Person	Competence	The extent to which users know how to use HIT. 37 Competence is measured according to the knowledge of HIT and typing skills.	• Semistructured interview and findings of previous studies 38 39	5
Technology	System quality	The degree of quality performance of information system. 40 SafeHIT focuses on four system quality factors which are usability, compatibility, reliability, and response time.	• Instruments 41 • Semistructured interview and findings of previous studies 42 43	11
	Information quality	The degree of quality information provided by the information system. 40 The SafeHIT focuses on completeness, relevancy, and timeliness dimensions of information quality.	• Instruments validated 44	7
	Service quality	The degree of quality service delivered by the IT department to users. 40 The SafeHIT focuses on responsiveness, assurance, and tangible dimensions of service quality.	• SERVQUAL instrument tested 45	5
Organization	Organization resources	The extent to which organizational resources are available to support safe patient care. 46 The organization resources include staff, computer equipment, time, and policies or procedures associated with the safe HIT use.	• Instruments developed and validated 14 46 • Semistructured interviews and findings of previous study 47	5
	Training	The extent to which training related to HIT is given to HIT users. 48	• Instruments developed and validated 49 50 51	5
	Teamwork	The extent to which health care practitioners work together to achieve a specific and shared goal. 52	• Instruments developed and validated 14 53 • Semistructured interviews and findings from previous study 54	5
Task	Task-related stressor	The extent to which task-related factors affect an individual and require extra coping strategies which are present during a job. 55 The SafeHIT focuses on four types of task-related stressors, namely time pressure, workload, cognitive load, and interruptions.	• Instruments validated 55 56	5
HIT use	Workarounds	Alternative procedures are employed by health care practitioners to accomplish a task in response to a misfit between HIT and existing work processes. 57 58	• Semistructured interviews and findings from previous studies 58 59 60	5
	Vigilance	Careful action or attention to avoid potential errors or risks.	• Semistructured interviews and findings from previous studies 61 62	4
	Copy and paste	To copy patient-related information from the previous entry and put it in the new entry of HIT.	• Semistructured interviews and findings from previous studies 63 64	3
	Procedure compliance	The act of complying with a procedure of HIT usage.	• Semistructured interviews and findings from previous studies 36 65	5
HIT use outcomes		The extent of belief of HIT users that using HIT contributes to the quality of care and the safety of the patient. 13 66	• Instruments developed and validated 14 49 • Semistructured interviews and findings from previous study 67	5

Abbreviation: HIT, health information technology.

Stage 2: Scale Development

Pretesting questions, pilot tests, survey administration, and factor extraction were conducted in the scale development phase.

Pretesting Instrument

A pretest was performed to test the face and content validity of the SafeHIT. A group of four experts of lecturers and three physicians from a teaching university hospital and a government hospital was chosen to pretest the SafeHIT. The lecturers are involved in teaching and research in the field of health care and ISs. All the physicians have experience in using HIT. Once respondents were briefed about the research, a set of hardcopy SafeHIT was given to them for review in 1 week.

In terms of face validity, all experts indicated that the appearance and layout of SafeHIT were acceptable. Additionally, they indicated that the questions were easy to understand and answer. Concerning the content validity, five questions with codes C1, C2, TS2, TS4, and OR1 had to be rephrased to ensure clarity and similar meaning. One item with code ServQ4 was revised and broken down into two items due to double meanings. The panel of experts confirmed that the SafeHIT was a valid and effective measurement. The items did measure the theoretical construct.

Pilot Test

A pilot study was performed in this research to ensure the reliability and validity of the constructs of interest. 22 The final draft of the SafeHIT with modifications from the pretest and the instrument evaluation form was given to 37 physicians for pilot testing. The number of participants was small because each hospital's Clinical Research Centre (CRC) was restricted from accepting a certain number of SafeHIT for the pilot study.

Thirty-one SafeHIT were returned over a period of 2 weeks. However, five copies were excluded as there were some items that were not answered and suspicious response patterns or straight lining. Reliability analysis showed that all Cronbach's alpha values for each construct were more than 0.7, except for competence (0.650) and organization resources (0.685). A Cronbach's alpha ranging from 0.6 to 0.7 is considered ‘acceptable’, 0.7 to 0.9 ‘good’, and above 0.90 is considered ‘excellent’ internal consistency. 23 Therefore, the internal consistency reliability of the majority of the constructs is considered good. Table 2 presents the results of the reliability analysis for each construct in the SafeHIT.

Table 2. Reliability of each construct in the SafeHIT.

Construct	Cronbach's alpha
Competence	0.650
System quality	0.865
Information quality	0.803
Service quality	0.914
Organization resources	0.685
Teamwork	0.783
Training	0.897
Task-related stressor	0.764
HIT use	0.783
HIT use outcomes	0.732

Abbreviation: HIT, health information technology.

Half of the respondents remarked that the length of the SafeHIT was just right, while 45.83% indicated it was too long. The remaining 4.17% of the respondent did not comment on the length of the SafeHIT. More than half of the respondents (58.33%) indicated the overall usability of SafeHIT was good, very good, and excellent. Slightly over two-third of the respondents (70.83%) rated the questions were clear, and they are able to understand the questions. Three-quarters of the respondents (75%) indicated that SafeHIT provided answer choices that reflected their true thoughts, opinions, or experiences.

Questionnaire Survey Administration

It is most plausible for physicians to disregard the mail questionnaires due to their hectic schedules. Hence, self-administered survey research, as suggested by, 24 was adopted for this research. After obtaining ethics approval from the Medical Research and Ethics Committee and access approval from each hospital director of the selected hospitals, questionnaire survey requests were sent to each head of department (HOD). The improved SafeHIT based on the pretest and pilot test results was distributed to 450 physicians. Finally, a total of 346 responses were collected.

Factor Extraction

Exploratory factor analysis (EFA) was conducted in this research to identify items that best represent a specific construct. All 346 usable responses were randomly split into two equal sample sizes for EFA and confirmatory factor analysis (CFA). 25 A sample size of 173 was considered to be acceptable to perform an appropriate EFA. 26

EFA was conducted according to the five steps in the EFA protocol as suggested. 27 This research applied the Kaiser–Meyer–Olkin (KMO) Measure of Sampling Adequacy and Bartlett's Test of Sphericity to assess the suitability of the data for factor analysis. Principal axis factoring was selected as the factor extraction method because it is appropriate for this research to identify the underlying constructs that reflect what the items share in common. 28 Next, parallel analysis was used in this research to determine the number of factors to be extracted or retained. Consequently, Promax was used to assist in the interpretation of the extracted factor because it permits factors to correlate. 28 Items with a factor loading of 0.45 and above were considered significant as the sample size of this research is 173. 28

Stage 3: Scale Evaluation

CFA was conducted by using SmartPLS 4 software to confirm the reliability and validity of the refined SafeHIT. Composite reliability is applied as the measure of internal consistency reliability. Composite reliability values of 0.60 to 0.70 is considered acceptable in exploratory research; however, they should be more than 0.708 in more advanced stages of research. 29 Convergent validity and discriminant validity are important measures of construct validity. Outer loadings of the indicators and average variance extracted (AVE) are employed to judge convergent validity. On the contrary, discriminant validity is accessed by using the Fornell–Lacker criterion.

Results

Descriptive Analysis of Demographics

Of the 450 SafeHIT distributed to three hospitals, 364 were returned over a period of two and half months. In total, 346 SafeHIT were usable for further analysis. A total of 17 SafeHIT were found to be unusable due to suspicious response patterns or straight lining, missing answers, and more than one answer for an item. Table 3 presents the details of the demographic profile of the respondents.

Table 3. Demographic details of the respondents.

Characteristic	Item	Frequency	Percent (%)
Gender	Female	213	61.56
	Male	130	37.57
	Nil	3	0.87
Age	≤24 y	1	0.29
	25–34	306	88.44
	35–44	27	7.80
	45–54	1	0.29
	Nil	11	3.18
HIT experience	<1 y	42	12.14
	1–5	266	76.88
	6–10	22	6.36
	>10	0	0.00
	Nil	16	4.62
Department	Emergency and trauma	73	21.10
	Pediatric	51	14.74
	Medical	39	11.27
	Obstetrics and gynecology	37	10.69
	Surgery	28	8.09
	Orthopaedic	25	7.23
	Otorhinolaryngology	25	7.23
	Ophthalmology	14	4.05
	Hematology	9	2.60
	Psychiatric	8	2.31
	Nephrology	8	2.31
	Cardiology	7	2.02
	Cardiothoracic	5	1.45
	Oral and maxillofacial surgery	5	1.45
	Rehabilitation	5	1.45
	Neurosurgery and neurology	4	1.16
	Urology	3	0.87

Abbreviation: HIT, health information technology.

The survey result demonstrates that a majority of the respondents were female respondents (61.56%). Most of the respondents were below 35 years of age (88.73%), and only 0.29% of respondents were between 45 and 54 years of age. The result indicates that respondents were generally from the young generation. Thus, technology should not be an unfamiliar topic for them. Interestingly, more than two-third (76.88%) of the respondents had experience using HIT within the range of 1 to 5 years. Only 12.14% of the respondents had experience using HIT for less than a year. The result implies that the respondents had adequate experience to undertake the specific tasks available in the HIT. Regarding the department attached by the respondents, it was observed that respondents attached to the emergency and trauma department were the largest groups (21.10%), followed by pediatric (14.74%), next, the medical department (11.27%), followed closely by obstetrics and gynecology (10.69%). The result typically suggests that a majority of the respondents are attached to the departments that are commonly available in most hospitals.

Exploratory Factor Analysis

The KMO measure of each component was between 0.651 and 0.914, and Bartlett's test is significant ( p  < 0.001), indicating that the data are adequate for factor analysis. All the items below the cut-off loading of 0.45 were removed. Supplementary Appendix A (available in the online version) presents the results of EFA.

For the person component, the EFA produced two distinct factors. Three items (C1, C2, and C3) with loadings of 0.821, 0.974, and 0.719 were loaded onto Factor 1. It was also observed that items C3 and C4 possessed moderately high loadings of 0.598 and 0.883, respectively, which were cleanly loaded onto Factor 2. Factor 1 was redefined appropriately as knowledge since the items best reflect more knowledge of HIT. On the contrary, two items in Factor 2 were identified as typing skills as the items were clearly identifiable with typing skills. This is in line with the definition of competence, which builds on a foundation of basic skills and knowledge. 30

The generated factor analysis from the technology component consists of three distinct factors. Nine items from the “system quality” scale were cleanly loaded onto Factor 1 with loadings ranging from 0.554 to 0.882. However, SQ10 (HIT of this hospital is able to integrate with other in-house systems and medical machines) and SQ11 (HIT of this hospital occasionally system down occurs when I use the HIT) were dropped because they did not meet the cut-off point of 0.45. Factor 1 remains to be labeled as system quality. Factor 2 comprises all seven items from the “information quality” scale, with loadings ranging from 0.637 to 0.962. Similarly, all five items from the “service quality” scale were loaded clearly onto Factor 3. The loading results read from 0.631 to 0.919. Therefore, Factor 2 and Factor 3 remain to be labeled as information quality and service quality , respectively.

The results of the factor analysis for the organization component found three distinct factors. All five items from the “training” scale were loaded clearly onto Factor 1 with loading between 0.5360 and 0.995. Factor 1 remains to be labeled as training. All items that measure “teamwork” were loaded onto Factor 2 except for TW5. The four items have loadings between 0.568 and 0.896. Item TW5 (as an employee of this hospital, I am always aware on new orders requested through the HIT) did not meet the cut-off point. Thus, it was removed from further analysis. Adequately, Factor 2 is denoted as teamwork. Likewise, Factor 3 comprises four items from the “organization resources” scale with loadings between 0.504 and 0.980. OR5 (This hospital has specific written policies or procedures to prevent me from making errors when using the HIT) was dropped due to a loading value below the 0.45 threshold. Therefore, Factor 3 remains to be labeled as organization resources.

The results of the factor analysis for the tasks component found one factor. All five items from the “task-related stressor” scale were cleanly loaded onto Factor 1, with loading ranging between 0.720 and 0.825. Factor 1 remains to be labeled as a task-related stressor.

Three distinct factors were generated from HIT use. Five items (W1: I coordinate patient care activity by using the HIT rather than through verbal, W2: I write notes on the paper containing information printed from the HIT, W3: I immediately transcribe patients' written progress notes to the HIT afterward rounds, V4: I sometimes forget to key in a patient's notes to the HIT, and WP4: I always log off from HIT after each usage) were dropped from further analysis due to loadings below the threshold of 0.45. Three items (V1, V2, and V3) from the “vigilance,” and two items (CP2 and CP3 and WP1 and WP2) from “copy and paste” and “procedure compliance,” respectively, were loaded onto Factor 1. Most of the items loaded on Factor 1 comprised of terms such as attention, double check, or carefully which were deemed to reflect vigilance. Hence, Factor 1 is denoted as vigilance. Factor 2 comprises two items (WP3 and WP5) from “procedure compliance,” and one item (CP1) from “copy and paste.” Since items from the “procedure compliance” scale had higher loadings, Factor 2 is appropriately labeled as procedure compliance. This is in line with the recommendation 28 that variables with higher loadings influence a great extent in naming to represent a factor. The remaining two items (W4 and W5) were loaded onto Factor 3 and remained as workarounds as the items were clearly identifiable with workarounds.

The results of the factor analysis for HIT use outcomes demonstrate two distinct factors. NB1 and NB2 were loaded onto Factor 1 with loadings of 0.718 and 0.791, respectively. It was also observed that items NB3, NB4, and NB5 possessed moderately high loadings of 0.698, 0.780, and 0.926, respectively. These items were loaded onto Factor 2. Factor 1 is relabeled appropriately as patient safety since the items best reflect errors and mistakes. On the contrary, NB5, which is concerned with patient-care quality, records higher loadings (0.926) as compared to the other two items (NB3 and NB4). Therefore, Factor 2 is labeled as patient-care quality. This is in line with the recommendation 28 that variables with higher loadings influence a great extent in naming to represent a factor.

Confirmatory Factor Analysis

The primary objective of CFA is to confirm the reliability and validity of the refined SafeHIT derived from the EFA. Construct validity was assessed in terms of convergent validity and discriminant validity. Table 4 indicates the results of convergent validity. All indicators loading was above the 0.7 standards except for SQ7 (0.625) and CP1 (0.577) that fell below the less conservative 0.5 cut-offs. Hence, all indicators established indicator reliability. Moreover, the composite reliability of all constructs exceeded the recommended threshold of 0.7, which suggested high levels of internal consistency reliability. All constructs' AVE ranged from 0.53 to 0.888, which exceeded the 0.5 rule of thumb. Therefore, the results demonstrate that the measures of the 14 reflective constructs have satisfactory levels of convergent validity.

Table 4. CFA results of convergent validity.

Construct	Indicators	Loading	Composite reliability	AVE
Knowledge	C1	0.891	0.937	0.832
	C2	0.927
	C3	0.918
Typing skill	C4	0.866	0.889	0.800
	C5	0.922
System quality	SQ1	0.849	0.937	0.651
	SQ3	0.841
	SQ4	0.864
	SQ5	0.879
	SQ6	0.828
	SQ7	0.625
	SQ8	0.792
	SQ9	0.720
Information quality	IQ1	0.880	0.945	0.742
	IQ2	0.873
	IQ3	0.833
	IQ5	0.848
	IQ6	0.856
	IQ7	0.856
Service quality	ServQ1	0.856	0.921	0.701
	ServQ2	0.889
	ServQ3	0.851
	ServQ4	0.813
	ServQ5	0.771
Organization resources	OR1	0.827	0.871	0.629
	OR2	0.736
	OR3	0.730
	OR4	0.871
Training	T1	0.819	0.926	0.715
	T2	0.872
	T3	0.864
	T4	0.884
	T5	0.785
Teamwork	TW1	0.838	0.910	0.717
	TW2	0.867
	TW3	0.850
	TW4	0.832
Task-related stressor	TS1	0.768	0.906	0.660
	TS2	0.787
	TS3	0.859
	TS4	0.864
	TS5	0.779
Vigilance	CP2	0.841	0.929	0.653
	CP3	0.811
	V1	0.793
	V2	0.820
	V3	0.754
	WP1	0.783
	WP2	0.851
Workarounds	W4	0.725	0.759	0.613
	W5	0.837
Procedure compliance	CP1	0.577	0.769	0.530
	WP3	0.790
	WP5	0.796
Patient safety	NB1	0.824	0.858	0.751
	NB2	0.907
Patient-care quality	NB3	0.875	0.941	0.888
	NB5	0.960

Abbreviations: AVE, average variance extracted; CFA, confirmatory factor analysis.

In terms of discriminant validity testing, the Fornell–Larcker criterion assessment was applied. Table 5 exhibits the results of the Fornell–Larcker criterion assessment. The square root of each construct's AVE is on the diagonal (bold). The nondiagonal elements represent the correlations between the constructs. It was found that the square roots of the AVE for each construct were all higher than the correlations of these constructs with other constructs. Hence, the discriminant validity of all constructs is supported. Altogether, the measurement constructs demonstrated adequate convergent validity and discriminant validity. This further suggests that the measurement items used for each construct are reliable and valid. All constructs are theoretically and empirically distinct.

Table 5. CFA results of discriminant validity.

	Information quality	Knowledge	Organiza-tion resources	Patient safety	Patient-care quality	Procedure compliance	Service quality	System quality	Task-related stressor	Teamwork	Training	Typing skill	Vigilance	Work arounds
Information quality	0.862
Knowledge	0.862	0.912
Organization resources	0.526	0.292	0.793
Patient safety	0.416	0.075	0.022	0.867
Patient-care quality	0.242	0.365	0.411	0.233	0.942
Procedure compliance	0.507	−0.056	−0.104	0.348	−0.054	0.728
Service quality	−0.026	0.336	0.364	0.129	0.323	−0.117	0.837
System quality	0.468	0.605	0.439	0.145	0.558	−0.112	0.418	0.807
Task-related stressor	0.674	0.104	−0.113	−0.120	−0.026	−0.202	0.073	−0.012	0.812
Teamwork	0.110	0.341	0.432	0.238	0.440	−0.058	0.592	0.408	0.025	0.847
Training	0.590	0.372	0.430	−0.090	0.252	−0.107	0.329	0.455	−0.038	0.269	0.845
Typing skill	0.328	0.337	0.207	0.010	0.201	−0.005	0.139	0.328	−0.016	0.133	0.244	0.895
Vigilance	0.171	0.405	0.108	0.149	0.427	0.029	0.262	0.404	0.185	0.447	0.113	0.224	0.808
Work arounds	0.420	−0.115	−0.124	0.106	−0.065	0.240	−0.239	−0.189	−0.399	−0.200	-0.295	-0.102	-0.117	0.783

Abbreviation: CFA, confirmatory factor analysis.

Discussion

SafeHIT is developed and validated in this study to extend the current approach to assess self-reported safe use of HIT among health care practitioners. Specifically, SafeHIT is developed to capture social-technical elements of the people, technology, organization, and task components as well as HIT use behavior that potentially contributes to the safety and quality of patient care. The final SafeHIT comprising 58 items is presented in Supplementary Appendix B (available in the online version) The measure of safe HIT use determinants consisted of nine constructs: knowledge and skills from the person component; system quality, information quality, and service quality from the technology component; organization resources, training, and teamwork from the organization component; and task-related stressor from the task component. Moreover, safe HIT use behavior consisted of three constructs: vigilance, procedure compliance, and workarounds. HIT use outcomes consisted of two constructs, namely, patient safety and patient-care quality. Consistent with the safe HIT use literature contributing factors of safe HIT use are multidimensional 31 and safe HIT use behavior corresponds to vigilance, procedure compliance, and workarounds. 32 33

As argued, 34 it is critical to examine the complex interplay between sociotechnical elements involved in the patient care to prevent or lessen the chances of safety incidents from happening. The SafeHIT can be used to quantitatively examine the key determinants of safe HIT use behavior from a wide range of sociotechnical factors such as user characteristics (e.g., knowledge and skill), technology context (e.g., system quality, information quality, and service quality), organization context (organization resources, training, and teamwork), and task-related stressor over a large sample size. Besides, the current study has made a theoretical contribution by conceptualizing the SafeHIT instrument as having 14 constructs and by empirically testing the instrument. The SafeHIT could potentially be a tool for developing theoretically underpinned large-scale interventions to develop feasible strategies to address key contributing factors or barriers to safe HIT use behavior.

The research findings have several implications for hospital management who adopt HIT for patient care. SafeHIT provides a multidimensional aspect of a sociotechnical perspective to identify determinants of safe HIT use behavior across a large sample. The SafeHIT could be used as a diagnostic tool to provide feedback to facilitate hospital management in addressing safe HIT adoption strategies and to plan the policies to establish safe HIT adoption and implementation. Furthermore, it might be beneficial to identify the potential risks of HIT adoption for the purposes of redesigning HIT and the organization. Interventions can be considered for HIT users with patterns of lower scores in specific HIT use dimensions. For example, if a majority of HIT users exhibit a high level of vigilance but a low level of knowledge of HIT, an educational intervention could be designed and put into action to deal with the potential issue. Additionally, HIT providers and developers could adopt and adapt the SafeHIT to examine whether HIT business model modifications impact user behavior.

Although our results indicate that SafeHIT is a valid measure to assess self-reported safe use of HIT among health care practitioners, the current study had several limitations. First, the evaluation was done by health care practitioners in Malaysian hospitals. Hence, this study cannot be generalized to other cultural contexts. Future research may be needed to test SafeHIT among health care practitioners in other countries. A larger sample from different countries and the use of probability sampling would permit increased confidence in the reliability and external validity of the measure. Second, the number of measures for typing skills, workarounds, patient safety, and patient-care quality was reduced to two items per construct. However, the measures have proved valid and reliable properties. The AVE of all constructs exceeded the 0.5 rule of thumb to establish convergent validity. 28 In addition, the square roots of the AVE for each construct were all higher than the correlations of these constructs with other constructs. Hence, the discriminant validity of all constructs is supported. Third, SafeHIT does not include workflow measures that may force a workaround. In the context of health IT use, workflow interruption or disruption happens when the health IT does not fit into their clinical workflow, poor usability, insufficient computers, and insufficient training. 35 36 All these measures that may force workarounds are included in SafeHIT, although a specific construct on workflow is absent. Nevertheless, the SafeHIT can be extended in the future to include workflow measures to evaluate workflow's causal effect on workarounds.

Conclusion

This study employed a rigorous scale development procedure to establish reliable and valid properties for assessing the psychosocial factors affecting practitioner HIT use behavior. The SafeHIT can be a useful diagnostic tool to identify the contributing factors to safe HIT use and, subsequently, develop effective intervention strategies. Furthermore, the 14 constructs measurement scale adds to the extant literature by establishing a basis for further theoretical advances on safe HIT use. Further research should be carried out to fully understand the SafeHIT measure's use and limitations.

Clinical Relevance Statement

The SafeHIT can be used to quantitatively examine the key determinants of safe HIT use behavior from a wide range of sociotechnical factors over a large sample size. Besides, SafeHIT could be used as a diagnostic tool to provide feedback to facilitate hospital management in addressing safe HIT adoption strategies and to plan the policies to establish safe HIT adoption and implementation. Furthermore, it might be beneficial to identify the potential risks and consequences of HIT adoption for the purposes of redesigning HIT and the organization.

Multiple-Choice Questions

1. SafeHIT is a measurement tool to assess safe HIT use behavior of which of the following social-technical dimensions:

   1. Vigilance, procedure compliance, and workarounds

   2. System quality, information quality, and service quality

   3. People, technology, organization, and tasks

   4. Patient safety and patient-care quality

   Correct Answer: The correct answer is option c. SafeHIT assesses safe HIT use behavior across the social-technical dimensions of people, technology, organization, and tasks.

2. The benefits of using SafeHIT include all the following except:

   1. SafeHIT provides a multidimensional aspect of a sociotechnical perspective to measure the patterns of safe HIT use behavior.

   2. SafeHIT is beneficial to qualitatively identify the complex interplay between sociotechnical elements involved in the patient care.

   3. SafeHIT provides feedback to facilitate hospital management in addressing safe HIT adoption strategies and to plan the policies to establish safe HIT adoption and implementation.

   4. SafeHIT is beneficial to identify the potential risks and the consequences of HIT adoption for the purposes of redesigning HIT and organization.

   Correct Answer: The correct answer is option b. There are many benefits of using SafeHIT: (1) providing a multidimensional aspect of sociotechnical perspective to measure the patterns of safe HIT use behavior, (2) providing feedback to facilitate hospital management in addressing safe HIT adoption strategies and planning the policies to establish safe HIT adoption and implementation, and (3) beneficial to identify the potential risks and the consequences of HIT adoption for the purposes of redesigning HIT and organization. SafeHIT is beneficial to qualitatively identify the complex interplay between sociotechnical elements involved in the patient care is not the benefit of using SafeHIT.

Funding Statement

Funding This work was supported by Universiti Teknikal Malaysia Melaka (UTeM) research grant (grant number: JURNAL/2020/FTMK/Q00055) and the Ministry of Higher Education (MOHE) Malaysia.