Development of a Healthcare Information System for Community Care of Older Adults and Evaluation of Its Acceptance and Usability

Abstract

Objective

The need for health and social care for community-dwelling elderly is on the rise as the population ages. Through the provision of comprehensive services by multiple professionals in local communities, elderly people can receive continual care in a non-medical setting, which is favorable for early detection and intervention of potential problems. However, the lack of digitalization in primary care affects the effectiveness of the services and precludes full exploitation of the data. This study proposed an information system dedicated to caring for community-dwelling elderly people and investigated its acceptance and usability.

Methods

An information system was designed for elderly care centers in the community, where data generated during care delivery, involving socio-demographic data, bio-measurements and health assessments and questionnaires, were digitized and stored for information management and exchange. A study was conducted to evaluate the acceptance and usability of the system after routine use of 6 months. The users of the system at an elderly care center were recruited to respond to a technology acceptance questionnaire and a system usability questionnaire.

Results

The mean scores of the acceptance and usability questionnaires reached 5.1 out of the highest possible score of 7. The constructs of the acceptance questionnaire had good reliability. The social influence and facilitating conditions constructs had a significant correlation with the behavioral intention construct.

Conclusions

The proposed information system demonstrated good acceptance and usability, which supported the feasibility of implementing it in community care centers for older adults. Further research will be conducted to address the limitation of sample size by extending the system to other elderly care centers, forming a large user base for a more in-depth and comprehensive performance evaluation.

Introduction

The unprecedented challenges that population ageing would bring to the healthcare system and the society at large have been widely discussed and recognized. Primary care plays an important role to address the challenges through the delivery of elderly care services at the community level. On the one hand, health literacy can be promoted through seminars, workshops, or educational programs to equip community-dwelling elderly people with essential self-care management abilities. On the other hand, elderly care centers nowadays provide various routine assessment services and checkups, delivered by community-based health and social care professionals in a non-medical setting. This enables early detection of potential issues and timely administration of interventions to prevent the issues from developing into more serious problems. The community-oriented elderly care facilitates the decentralization of healthcare and helps relieving the burden on secondary care in hospitals. It is also supportive of the ageing-in-place ¹ initiatives advocated for long-term care.

To promote community-based elderly care, electronic information systems can provide support for managing the well-being of older adults. While electronic health records (EHRs) have been conventionally used in hospitals for health data management and information exchange, the use of electronic records for community care is relatively less common. Early efforts were made by community health centers to use the information in EHRs for quality improvement in preventive and chronic care. ² EHRs were also adopted by nurses for clinical processes and patient care in community settings. ³ However, community health records are unique in that they contain various determinants of health at the population level, including physical, social, and lifestyle, which can complement hospital EHRs to provide a more comprehensive view of individuals and enable holistic care.^4,5 Among the determinants, social factors (e.g. education and employment at the individual level, or neighborhood and socioeconomics at the community level) are of particular concern that affect health-related outcomes and attribute to a significant percentage of death. ⁶ In the case of the caring for older adults, stakeholders of community health records involve both health and social care professionals, as well as community and informal caregivers, that the existing EHRs are not effectively designed for interoperability ⁷ and data sharing among these stakeholders. ⁸ The disparity between hospital-based EHRs and community health records thus leads to the so-called health and social divide. ⁹ Indeed, it has been pointed out that existing community health information systems are often built for administrative purposes rather than facilitating professional care and service delivery. ¹⁰ Current use of community health records remains informal, lacking common structures and standards. ⁵

Nevertheless, the vast experience gained through decades of hospital EHR development lends itself to electronic information systems for elderly care. In particular, health information exchange standards and clinical terminology standards developed for hospital EHRs are important success factors that enable interoperable community-based health and social care networks. Among the health information exchange standards, the Health Leven Seven (HL7) protocol is a popular one that is adopted internationally to specify the interface between different healthcare applications. It has evolved from the HL7 version 2, which is flexible but prone to inconsistency, to the HL7 version 3, which turns out to be too complicated to implement, ¹¹ and then to the contemporary Fast Healthcare Interoperability Resources (FHIR) ¹² which emerges as a viable solution for interoperability. FHIR follows the Representational State Transfer (REST) principles to yield scalable and reliable interfaces. The FHIR specification defines the resource types and the web services. Since the advent of HL7 FHIR, apart from increasing adoption in hospital EHRs, it has been leveraged for other applications such as personal health records that can communicate back with EHRs, ¹³ clinically integrated patient-centered platforms to retrieve patient radiology information from EHRs by external software, ¹⁴ or research data management system for neuroimaging and biosignal processing. ¹⁵ In community health and social care, FHIR has been used to develop a vital sign tele-monitoring system for elderly people with chronic diseases at nursing homes and care centers. ¹⁶ It is also employed to build a platform to support professional practice and services delivered at local care centers, whereby social care information of community-dwelling elderly people can be retrieved efficiently from the FHIR-enabled care networks. ⁸

The current study was based on a community-oriented electronic system, called Elderly Healthcare Information System (EHIS), developed to facilitate the delivery of health and social care services for elderly clients of community centers. FHIR and clinical terminology standards were employed to enable interoperability with other centers and institutions, as well as hospital EHRs, thereby building an information network of elderly care at the community level to support professional practice and enhance the quality of primary care.

Interoperability of the EHIS with existing hospital EHRs is necessary and advantageous. In the context of this study, the information available from EHRs, including patient history, medications, allergies, and outpatient appointments, is useful for primary care providers of community centers to determine the most appropriate care and care plan for a specific elderly client. On the other hand, it is useful to communicate back the information collected from the community centers which is valuable in various aspects: the community health records contain data that are collected more frequently than episodic care; the data are collected through various assessments and interventions of multi disciplinary primary care in non-medical settings; and the data contain social determinants of health such as living environment and support. ⁶ These community-collected information complements the EHR data in hospitals to provide a longitudinal and holistic view of the elderly, and promote continuity of care. With interoperability enabled, data sharing between the EHIS and EHRs can be streamlined to create synergy for enhancing the quality of elderly care.

The proposed EHIS also aims to facilitate the workflow and the collection of data produced during the delivery of elderly care services, and the subsequent data management and report generation. A factor that determines whether the EHIS can achieve this goal is user acceptance and usability of the system, which are indeed important for health information systems in general.^17,18 According to the review by Yen and Bakken, ¹⁸ the Technology Acceptance Model and the IBM Computer System Usability Questionnaire (CSUQ), respectively, are among the most frequently used approaches for evaluating the acceptance and usability of health information technology. Venkatesh et al. developed a generalized version of the technology acceptance model, named the Unified Technology Acceptance and Use of Technology (UTAUT), by integrating eight different models of technology acceptance. ¹⁹ The strength of UTAUT in modeling acceptance of information systems was supported by a meta-analysis of 74 studies. ¹⁷ In a recent review study of 247 papers, ²⁰ UTAUT and CSUQ were also identified as the most used questionnaires for measuring the acceptance and usability of mobile health applications. Hence, a survey was conducted in the present study with questionnaires developed based on UTAUT and CSUQ to evaluate the acceptance and usability of the proposed EHIS. Health and social care providers who used the EHIS at community care center were recruited to rate the system.

The rest of the paper is organized as follows. After a description of the design and development of the EHIS in the next section, the paper presents the survey conducted to evaluate the system's acceptance and usability in the Methods section, which covers the participants, instruments, procedure, and data analysis. The findings of the survey are reported in the Results section, followed by discussions on the findings with comparison to related studies, the limitations and the future research. Finally, a conclusion is given. To facilitate reading, a list of abbreviations used in this paper is provided in Table 1.

Table 1.

List of abbreviations used in the paper.

Abbreviation	Meaning
α	Cronbach's alpha
r s	Spearman's correlation coefficient rho
API	Application Programming Interface
CSUQ	Computer System Usability Questionnaire
EHIS	Elderly Healthcare Information System (the system proposed in this study)
EHR	Electron Health Records
eHRSS	Electronic Health Record Sharing System
FHIR	Fast Healthcare Interoperability Resources
HAPI	Health Level Seven Application Programming Interface
HL7	Health Level Seven
JSON	JavaScript Object Notation
LOINC	Logical Observation Identifiers Names and Codes
REST	Representational State Transfer
SNOMED-CT	Systematized Nomenclature of Medicine Clinical Terms
UTAUT	Unified Technology Acceptance and Use of Technology

Elderly healthcare information system

The proposed EHIS aims to meet the needs of community elderly care organizations, enabling them to leverage the healthcare data for providing timely and necessary services. To realize the idea, partnership has been established with a major non-governmental organization that offers elderly care services in local communities. The organization has a large client base. Through iterative modifications based on user feedback, the EHIS was designed to align with the existing workflow at the frontline. As a pilot, the EHIS was deployed in one elderly care center of the organization.

Workflow and data source

The elderly care center provides services to community-dwelling elderly through a membership scheme. Registration is required for new clients where basic information and socio-demographic data are collected upon registration. Clients are then scheduled for healthcare activities at the center. Attendance is taken at the beginning of each visit where basic vital signs like body temperature, pulse rate, and blood pressure are measured. The center provides comprehensive elderly care services delivered by a multidisciplinary team, including social workers, program workers, care workers, nurses, occupational therapists, and physiotherapists, who are the possible users of the EHIS. Depending on the needs, different assessment tools and instruments of the respective professions are used to evaluate the health and social conditions of the clients. Given the variety of assessments that a client may need, it would require several visits to complete the initial examination. The assessments involve general health status, mental health conditions, ability in conducting activities of daily living, self-care ability, living environment and support, physical functioning, as well as the administration of standardized questionnaires like Geriatric Depression Scale ²¹ and Mini-Mental State Examination. ²² These assessments constitute a major source of data for the EHIS. For outreach in-home elder care services, where clients receive one-on-one services in the comfort of the home, the healthcare staff would perform a subset of assessments that can be conducted in residence.

Based on the conditions of individual clients and the assessment results, custom-designed care plans are stipulated such that appropriate interventions and health promotion schemes are arranged for both the clients and their caregivers. Each individual case is also scheduled for regular reviews, with attention to the progress and the results of the follow-up actions (e.g. further assessments or treatments) ordered in the care plan and the ensuing update of specific health and social conditions.

System framework

The overall system framework of the proposed EHIS is shown in Figure 1. The EHIS is divided into two main subsystems: the Database Subsystem and the Interoperability-Enabling Subsystem. The former consists of a database management system that stores and manages the data collected from the elderly care center, whereas the latter turns the EHIS into an interoperable system and enables the integration of external devices through the Add-on Connectivity Module. A key feature of the EHIS is the interoperability that permits communication and exchange of information across different elderly care centers, which would facilitate continuity of care of cross-community clients, as well as the aggregation of big data for enhancing the health and care of older adults. By decoupling the two subsystems, existing legacy information systems can be readily upgraded to the corresponding FHIR versions without major modifications.

Figure 1.

Overall framework of the EHIS: database subsystem (bottom), interoperability-enabling subsystem (middle), and add-on connectivity module (top).

Database subsystem

The Database Subsystem adopts a client-server architecture that the database management system runs on a server to perform data storage and access. A web server is created to enable wireless access to the database in the center via tablet devices. For outreach home care services, the center employs a third-party online survey system to record client data using mobile phones or tablets. The data are exported from the survey system and integrated conveniently into the database through data import.

Interoperability-enabling subsystem

In the Interoperability-Enabling Subsystem, health information exchange standard and standardized terminologies are incorporated into the database developed by a tiered architecture. The architecture is implemented with HL7 FHIR version 4.0.1. Refer to the Interoperability-Enabling Subsystem depicted in Figure 1, an instance of open source HL7 application programming interface (HAPI) FHIR server is commissioned to provide the required FHIR capability. All calls made with FHIR are handled and processed by the HAPI FHIR server. Data from the Database Subsystem are periodically exported as a set of intermediate files, which are subsequently processed by middleware (or wrappers) and translated into REST requests, and eventually accepted by the HAPI FHIR server. The following FHIR resources are mainly used in the Interoperability-Enabling Subsystem to provide the necessary features to support the operations of the elderly care center:

• Patient: elderly clients of the center.

• Observation: measurements such as body temperature and blood pressure.

• Questionnaire: instruments and questionnaires used by center staff during health assessments.

• Questionnaire Response: responses obtained from the clients during the assessment.

The middleware executes the following operations when generating the FHIR requests: (i) check if a record already exists in the FHIR server and send either an add or an update request to the FHIR server; (ii) insert additional information such as clinical terminology to the data prior to making a request; and finally, (iii) send the requests to the FHIR server and handle any errors. Any external system accessing data through the FHIR standard would interact only with the FHIR server, leaving the Database Subsystem untouched. To facilitate data collection and avoid transcription error, wrappers are also developed to convert data collected from external measurement devices into FHIR conformation specification, which can be accessed via communication protocol based on RESTful API with JavaScript Object Notation (JSON) structure. The tiered architecture, decoupled from the database, thus provides a fast method of introducing FHIR to existing legacy systems without modifying them. Interoperability is further attained by adopting the international clinical terminology standard Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) to code the data formatted in local standards or custom data structures, and by adopting the Logical Observation Identifiers Names and Codes (LOINC) to encode the assessment instruments such as Geriatric Depression Scale and Mini-Mental State Examination.

Add-on connectivity module

With the add-on connectivity module, the Interoperability-Enabling Subsystem can serve as a connectivity platform for the integration of health data acquired from external measurement devices. These device-collected data are encoded by FHIR with wrappers and stored in the HAPI FHIR server. The platform supports three types of devices, namely, Bluetooth-supported devices, analog devices, and wearable devices. Bluetooth-supported devices that are compliant with the Bluetooth Low Energy Generic Attribute Profile specifications can be readily connected to the system. For example, the direct transfer of body temperature measurements to the Interoperability-Enabling Subsystem is realized with a Bluetooth digital infrared thermometer. For analog measurement devices, electronic circuits are built to interface with the devices and transfer the data into the Interoperability-Enabling Subsystem through a Bluetooth connection. The circuits amplify the analog signal using an instrumentation amplifier and digitize it using a 10-bit analog-to-digital converter. For example, digitization and storage of analog hand grip strength data, acquired using an electronic dynamometer during physical fitness assessment, are implemented with this approach. As wearable devices are gaining popularity as health monitors for elderly, the Interoperability-Enabling Subsystem is extended to such applications by using the data acquisition platform of the device manufacturers to extract health and activity data (e.g. heart rate by minute, sleep time by date from the wearable device Fitbit Charge 3). With explicit prior consent of data sharing from elderly clients, the wearable device data are extracted using the API provided by device manufacturers and uploaded to the Interoperability-Enabling Subsystem. Dedicated wrappers for FHIR encoding are developed for the data of these three types of devices to ensure interoperability and the encoded data are then stored in the HAPI FHIR server. Overall, the strategy of external device integration can reduce potential errors due to manual data transcription and streamline the workflow.

Implementation

An EHIS prototype was built for a center of the collaborating elderly care organization. The Database Subsystem of the EHIS provides 12 input forms used by the center to collect demographic data, client profile, and health assessments. The data were stored in a generic relational database. The Interoperability-Enabling Subsystem adopted FHIR, LOINC, and SNOMED-CT. The FHIR-formatted input forms and responses were pushed (using the FHIR REST APIs with JSON) to the FHIR server for storage and utilization. Examples of the JSON structures for the FHIR Patient and Observation resources used in the system are given in Appendices 1 and 2.

FHIR middleware for the data input forms, e.g. Mini-Mental State Examination or Geriatric Depression Scale, was developed to wrap the data in the Database Subsystem with FHIR and convert them into FHIR-compliant format so that the data could be imported into the FHIR server of the Interoperability-Enabling Subsystem. The middleware for each form consisted of two parts. The first part converted the form's questions into a Questionnaire resource in FHIR format. The second part converted the form's responses into a Questionnaire Response resource in FHIR format. Examples of the JSON structure of these two resources for the Geriatric Depression Scale are given in Appendices 3 and 4. In the elderly care center, the short version of the Geriatric Depression Scale with 15 items is used. Since the standard codes for the Geriatric Depression Scale are already available from LOINC (LOINC code: 48543-3), the scale was readily encoded in the EHIS prototype. Similarly, the standard codes for the Mini-Mental State Examination are also available from LOINC (LOINC code: 72107-6) and the encoding of the questionnaire was straightforward. For custom input forms where the terms are not yet coded in existing terminology standards, SNOMED-CT was adopted for coding the individual terms. In summary, data from the Database Subsystem were provided to the FHIR middleware as a regular set of intermediate files containing either the subset of changed data or the complete set of data. The individual FHIR middleware was integrated to establish a self-running autonomous system to process any new and updated data from the Database Subsystem.

Methods

A survey was conducted in the pilot study to evaluate the acceptance and usability of the EHIS discussed above. The participants, instruments, procedures, and data analysis of the study are presented as follows.

Participants

The EHIS prototype was developed for an elderly care center with 120 active clients. There were 12 health and social care practitioners who used the system in their daily work. The professions included care worker, program worker, social worker, and occupational therapist Given the small population, total population sampling was adopted where all users of the EHIS prototype were invited and recruited to participate in the study. The study was approved by the institutional review board. Implied consent was adopted. The participation consisted of only filling out a set of anonymous questionnaires. The purpose of the study and the data collected were to be used for research were stated at the beginning of the questionnaires to inform the participants. Consent to participate in the study was implied by completion of the questionnaires.

Instruments

Two survey questionnaires were designed based on the UTAUT ¹⁹ and the IBM CSUQ ²³ respectively. The UTAUT scale is widely used to evaluate the degree of acceptance and use of information technology by the intended users. ²⁴ It has been used to evaluate the level of the acceptance of nursing information system, ²⁵ information technology adopted in community health centers, ²⁶ healthcare wearable devices, ²⁷ and a mobile tuberculous treatment monitoring system. ²⁸ Based on the UTAUT model, ¹⁹ a 7-point Likert survey questionnaire with 29 items was developed for evaluating the EHIS. The model consists of eight constructs, including Performance Expectancy, Effort Expectancy, Social Influence, Facilitating Conditions, Attitude, Self-Efficacy, Anxiety, and Behavioral Intention. A summary of the items corresponding to the constructs is given in Appendix 5. In the questionnaire, “1” indicates the lowest possible agreement with a survey item and “7” indicates the highest.

The IBM CSUQ (reliability = 0.91–0.96, content and construct validity validated) ²⁴ is a popular psychometric instrument used to assess the extent of satisfaction with the usability of computer systems. It has been used for evaluating user satisfaction toward nursing information systems, ⁷ home-based telemedicine systems for older adults, ²⁹ and clinical information systems for cognitive disorders and mental health. ³⁰ The CUSQ is a 7-point Likert questionnaire consisting of four subscales with a total of 19 items. The first three subscales are System Usefulness, Information Quality, and Interface Quality, accounting for the first 18 items of the questionnaire. The last item concerns overall satisfaction toward the system's usability. The fourth subscale of the CSQU is Overall, which is the average score of all the 19 items. A summary of the corresponding items is given in Appendix 6. Like the UTAUT, “1” indicates the lowest possible agreement with an item of the CSUQ and “7” indicates the highest.

Procedure

The prototype was deployed in the elderly care center in September 2020 and came into routine use in March 2021. A study was then conducted in August 2021 to evaluate the acceptance and usability of the system. Participants were provided with a hard copy of the survey questionnaires and a self-seal envelope. They were given one month to complete the questionnaires anonymously. The participants were required to seal the completed questionnaires in the envelope provided and pass them to a coordinator at the center who would then forward them to the researcher by registered mail.

Data analysis

Demographics of the participants were analyzed in terms of gender, age, and work experience with the community elderly care center. For acceptance evaluation, a descriptive analysis of the scores rated by the participants on the eight constructs of the UTAUT was conducted. The corresponding values of the mean, standard deviation, and 95% confidence interval of each construct were evaluated. In addition, the distribution of responses to each construct was also analyzed with histograms. The responses, on a 7-point Likert scale, were further categorized into three levels of agreement, namely, with “Negative” referring to the choices of 1, 2, or 3, “Neutral” to the rating of 4, and “Positive” to the choices of 5, 6, or 7. The distribution of responses under these three categories was studied. Similarly, for usability evaluation, the scores given to the four subscales of the CSUQ were analyzed in terms of mean, standard deviation, and 95% confidence interval. The distribution of the responses to each subscale was also studied with histograms and following the same three-level categorization of the extent of agreement as discussed above.

Furthermore, reliability analysis using Cronbach's alpha was conducted to study the internal consistency of the items under each construct of the UTAUT survey. Since the sample size was small, structural equation modeling could not be used to analyze the UTAUT model. Instead, correlation analysis was conducted to examine the correlation between the Behavioral Intention construct with four constructs respectively, i.e. Performance Expectancy, Effort Expectancy, Social Influence, and Facilitating Conditions, which are the relationships suggested by the UTAUT theory. Here, the null hypotheses were that there was no correlation between Behavioral Intention with these four constructs. The analysis was conducted using Spearman's correlation coefficients on the mean scores summarizing the items under each construct.

Results

Demographic data

All the 12 EHIS users of the center responded to the survey after using the system for 6 months. Eleven of the participants (92%) were female. Five of the participants (42%) were aged between 26 and 35 years, three (25%) between 46 and 60 years, two (17%) between 36 and 45 years, one between 18 and 25 years, and one above 60 years. Of the 12 participants, eight (67%) had 1 to 5 years of work experience with the center, while two had less than 1 year of experience, and the remaining two had 6 to 10 years. Eight (67%) of the participants were care workers, the rest were program workers, social workers, and occupational therapists, respectively.

Technology acceptance

The responses of the 12 participants to the survey questions based on the UTAUT are shown in Table 2. The mean score is above 5 for five constructs (Effort Expectancy, Social Influence, Facilitating Conditions, Self-Efficacy, and Behavioral Intention); close to 5 for two (Performance Expectancy and Attitude); and close to 3 for the Anxiety construct. It is clear that, except the Anxiety construct, the distributions of the responses skew to the positive end of the scale. Most of the responses fall into the “Positive” category, with Social Influence and Behavioral Intention being more conspicuous, where almost 90% of the ratings given are 5 or above. The positively skewed distributions of the responses in the seven constructs can also be seen in Figure 2, where the mode is “5.” For the Anxiety construct, in addition to having a low mean score, the distribution of responses is also negatively skewed. Since the items under the Anxiety construct were presented in negative statements like “I feel apprehensive about using EHIS…,” “It scares me to think that I could lose a lot of information …,” “I hesitate to use EHIS …,” or “The system is somewhat intimidating …,” a low score indeed indicated disagreement with these negative statements. That is, the level of anxiety in using EHIS was low. To this end, the Anxiety scores were reversed to account for responses to the negative statements in order to reconcile with the other constructs. With this correction, the mean score of all the items of the acceptance questionnaire is 5.1.

Table 2.

Responses of EHIS users on the survey based on UTAUT.

Construct	Mean	SD	95% CI	Neutral	Positive	Negative
Performance expectancy	5.0	0.77	4.77–5.45	21%	75%	4%
Effort expectancy	5.3	0.95	4.69–5.89	21%	79%	0%
Social influence	5.5	0.77	4.98–5.96	8%	89%	3%
Facilitating conditions	5.6	0.92	4.98–6.15	18%	82%	0%
Attitude	4.8	1.00	4.20–5.47	29%	60%	11%
Self-efficacy	5.3	0.94	4.69–5.89	21%	75%	4%
Anxiety*	3.2	1.29	2.32–3.97	21%	23%	56%
Behavioral intention	5.4	1.98	4.80–6.04	11%	86%	3%

*Questions were in negative statements.

Figure 2.

Distributions of responses to the constructs of UTAUT. PE: performance expectancy; EE: effort expectancy; SI: social influence; FC: facilitating conditions; AT: attitude; SE: self-efficacy; AX: anxiety; BI: behavioral intention.

System usability

The responses to the survey questions based on the CSUQ are shown in Table 3. The mean score is above 5 for System Usefulness and close to 5 for Information Quality and Interface Quality. The mean score of Overall is 5.1. Generally speaking, the majority of the responses are “Positive.” This is also evident from the distributions of the responses shown in Figure 3. The distributions are all positively skewed. The mode is “5” for System Usefulness, Information Quality, and Overall; while the mode is “4” for Interface Quality. The mean score of all the items of the usability questionnaire is 5.1.

Table 3.

Responses of EHIS users on the survey based on CSUQ.

Subscale	Mean	SD	95% CI	Neutral	Positive	Negative
System usefulness	5.3	0.81	4.75–5.78	17%	82%	1%
Information quality	4.9	0.77	4.38–5.26	27%	68%	5%
Interface quality	4.9	1.14	4.22–5.67	47%	50%	3%
Overall	5.1	0.46	4.58–5.55	26%	71%	3%

Figure 3.

Distributions of responses to the subscales of CSUQ. SYSUSE: system usefulness; INFOQUAL: information quality; INTERQUAL: interface quality.

Reliability of UTAUT constructs

The internal consistency between the items under each construct of the UTAUT survey is shown in Table 4. Except for the Self-Efficacy construct, the Cronbach's alpha coefficients (α) of the constructs are all greater than 0.7, five of them are even greater than 0.8, and two reaching over 0.95, indicating a high level of internal consistency reliability. For Self-Efficacy, the Cronbach's alpha is 0.582 which is within the acceptable range of 0.35 to 0.7. ³¹

Table 4.

Reliability of constructs.

Constructs	Cronbach's α	Number of Items
Performance expectancy	0.875	4
Effort expectancy	0.964	4
Social influence	0.789	3
Facilitating conditions	0.897	5
Attitude	0.887	4
Self-efficacy	0.582	2
Anxiety	0.792	4
Behavioral intention	0.955	3

Correlations with behavioral intention

The relationships between the Behavioral Intention construct and the four UTAUT constructs—Performance Expectancy, Effort Expectancy, Social Influence, and Facilitating Conditions—are shown with Spearman's correlation coefficients (r_s) in Table 5. The results show that Behavioral Intention has a statistically significant correlation with Social Influence (r_s = 0.733, p < 0.01) and Facilitating Conditions (r_s = 0.648, p < 0.05), respectively, whereas the correlations between Behavioral Intention and Performance Expectancy and Effort Expectancy are not significant. Therefore, the null hypotheses that there was no correlation between Behavioral Intention and Social Influence, and that there was no correlation between Behavioral Intention and Facilitating Conditions, were both rejected. On the other hand, the results did not reject the null hypotheses that Behavioral Intention had no correlation with Performance Expectancy and Effort Expectancy respectively.

Table 5.

Spearman's correlation coefficients between behavioral intention and four constructs.

		Performance expectancy	Effort expectancy	Social influence	Facilitating conditions
Behavioral intention	Spearman's rs	0.310	0.431	0.733**	0.648*
	p-value	0.326	0.162	0.007	0.023

*p < 0.05, **p < 0.01 (2-tailed)

Discussion

The proposed EHIS makes ample use of contemporary standards in healthcare industry, including FHIR, LOINC, and SNOMED-CT, which enables system interoperability and can potentially improve the accessibility of information among elderly care centers. The study demonstrates a viable approach that can be practically applied to build electronic information systems, or to upgrade the existing ones, to support elderly care in the community. The proposed EHIS has been piloted at a community elderly care center and is in routine use at frontline. The survey conducted with all the 12 users after 6 months of use shows that the system has good acceptance and usability. The overall mean score of both the UTAUT and CSUQ questionnaires reached 5.1 out of 7. Regarding the acceptance of the EHIS, the study collected responses to the eight constructs and the mean scores ranged from 4.8 to 5.4 (the Anxiety score was reversed due to negative statements in the questions). In the present study, the eight constructs adopted followed those in the original paper of Venkatesh et al., ¹⁹ while the five constructs—Performance Expectation, Effort Expectancy, Social Influence, Facilitating Conditions, and Behavioral Intention—are commonly used among studies applying UTAUT to examine technology acceptance^25–28. With respect to these five constructs, the corresponding mean score of the EHIS in this study (which is 5.2) is higher than that in some previous studies. For example, the mean scores of the five constructs for a nursing information system ²⁵ developed to support the documentation of patient care delivery in surgical and medical wards ranged from 4.2 to 4.9. The overall mean of the five constructs was 4.8, with Performance Expectancy rated the lowest. Similarly, the mean scores of a mobile tuberculous treatment monitoring system ²⁸ were within 4.2 to 5.4. The overall five-construct mean score was 4.9, with Facilitating Conditions rated the lowest. On the other hand, in a study on the acceptance of health information technology in Thailand's community health centers, ²⁶ the overall mean score of the five constructs reached a highly favorable value of 5.8 out of 7. Note that the study considered the use of health information technology in general, rather than a specific system or device. In another study adopting the five constructs to investigate the acceptance of fitness and medical wearable devices by consumers ²⁷ (not on a specific device but devices in general), a low overall mean score of 3.8 was only obtained (i.e. the level of popularity was mediocre). Hence, with the ratings attained by the EHIS, the acceptance was considered to be satisfactory.

However, despite the positive responses to the survey based on UTAUT, the number of negative responses to the Attitude construct was relatively high (11%) and so were the neutral responses (29%). The items under the construct evaluated user's attitudes toward using the EHIS which concerned their affective reactions, i.e. “Using EHIS is a good idea,” “EHIS makes work more interesting,” “Working with the EHIS is fun,” “I like working with EHIS.” Undesirable affective reactions may have a negative influence on the user's perception of the ease of use of the EHIS, i.e. Effort Expectancy, and in turn affecting their intention to use the system, i.e. Behavioral Intention. ³² The findings indicate the need to improve attitude toward using the EHIS. Possible approaches include running workshops to explain the benefits of EHIS (e.g. streamline daily operations and improve elderly care delivery); or developing attractive and convenient user interface (to be further discussed below), or organizing focus group interviews to obtain insights from user's perspectives for improving the system.

On the other hand, regarding the usability of the EHIS, the performance of the proposed EHIS, in terms of the four subscales of the CSUQ, was similar, each with a mean score of around 5.0 out of 7. The results reflect that the level of satisfaction of the participants toward the EHIS was satisfactory when compared to the results of a number of related studies. With reference to a study examining the usability of four home-based telemedicine systems for elderly people, ²⁹ the mean scores of the four CUSQ subscales for a system made by a well-known manufacturer of videoconferencing products (Polycom) were within the range of 4.2 to 5.3. The overall score was 4.8 and the score of Information Quality subscale was rated the lowest. The best system, Doxy.me, had an overall score of 5.5 and the range of scores of the four subscales was 5.0 to 5.9. Note that the Polycom system used in the study is a generic web-based videoconferencing application, whereas the Doxy.me solution is designed specifically for telemedicine practice and consultations. The latter outperformed in the Interface Quality subscale, and the initialization time was the shortest. Besides, the usability of the EHIS was deemed on the high side when compared with a study evaluating the usability of nursing sub-systems of two hospital information systems ⁷ (used in 186 and 60 hospitals respectively for nursing services documentation, pharmacy and laboratory order entry, document sharing, etc.), where a medium level of satisfaction was concluded based on the results of CSUQ, with the mean scores of the four subscales ranging within 3.0 to 4.0 for one system, and within 3.6 to 5.6 for the other (converted to a 7-point scale for comparison here). Compared to an information system developed for clinics serving patients with cognitive disorders and mental health problems, ³⁰ the usability was determined to be at the mild-to-moderate level of satisfaction, where the overall score rated by all the 10 users was 4.6 and the administrative staff rated the lowest at 4.3 (the scores quoted here were reversed since a 7-point scale in ascending order of disagreement was adopted in that study). Extra workload due to the entry of additional data and complications caused by the need of cross-referencing multiple sources were among the reasons for the administrative staff's low satisfaction. For the proposed EHIS, an iterative design process was adopted where users were engaged to solicit comments during the development cycles. This is helpful to ensure that the system is in line with the existing workflow of the center, thereby reducing the learning curve of the new EHIS system and avoiding potential burdens on the users. The EHIS users are supposed to input data as usual during the process of service delivery, where the data are now directly entered into the system electronically rather writing on paper forms. Hence, with the overall scores of around 5.0 rated by frontline users of different roles, the usability of the EHIS was considered to be satisfactory.

While the results of the CUSQ survey were positive, about half of the users (47%) made a neutral response in the Interface Quality subscale. The findings underscore that the user interface of the EHIS prototype needs further improvement. Since the current design of the EHIS aimed to address the basic needs of frontline users, the user interface was rather plain and static. It should be pointed out that an uncomfortable and unattractive user interface can reduce work efficiency and hinder system adoption in practice, despite good system usefulness and information quality. Given the complexity of health information, a user interface with a dynamic document layout that is responsive to user settings interactively is a promising option. ³³ Such a user-controllable layout allows interactive information organization on screen (e.g. show, hide, collapse) to meet the specific needs during consultation and data exploration.

Regarding the UTAUT model, the internal consistency between the items within the constructs of the questionnaire was high, indicating good reliability of the constructs. Although the Cronbach's alpha of the Self-Efficacy construct (α = 0.582) was within an acceptable range, ³¹ it was not higher than the generally accepted standard of 0.7, suggesting that improvement of the items may be necessary. Besides, the results of correlation analysis were in partial agreement with the hypotheses of the UTAUT theory that the four constructs Performance Expectancy, Effort Expectancy, Social Influence, and Facilitating Conditions are significant determinants of Behavioral Intention. Such a relationship was shown with statistical significance for the Social Influence and Facilitating Conditions constructs only. While the subscales of Performance Expectancy and Effort Expectancy were highly reliable (α = 0.875 and 0.964, respectively), the corresponding Spearman's correlation with Behavioral Intention was not statistically significant. The findings suggested further investigation of the model and possible correlations between constructs (e.g. Effort Expectancy and Social Influence may have indirect effects on Behavioral Intention through Performance Expectancy ³⁴ ) and also the effect of moderating variables that may affect the relationships. The investigation can be conducted using structural equation modeling with larger sample size. On the other hand, the results show that Social Influence and Facilitating Conditions have a statistically significant correlation with Behavioral Intention. This highlights the importance of support by peers and the management (e.g. adopting a strategy to build champion users as facilitators to support novices), training resources (e.g. hands-on courses, manuals, and demonstration videos), as well as technical infrastructure and support (e.g. network and computer facilities, hotline service), in the adoption and deployment of the proposed EHIS.

The overall feedback of the pilot study is on the positive side, as indicated by the results of the UTAUT and CSUQ surveys. A care worker commented that “the system is easy to understand and operate. If it can be fully deployed across all elderly care centers of the organization, it will be a very good suggestion.” Although the elderly care center participating in the study is relatively small, useful experience is gained to further improve the EHIS and to prepare for larger-scale deployment. Expansion to multiple centers under the same organization is deemed straightforward with minimal local modifications, attributed to the resemblance of the services provided, the workflow and assessment methods adopted across the centers.

It is essential for the EHIS to interoperate with EHRs in hospitals in order to overcome the issue of health data silo and to provide a complementary and holistic view of elderly care. In Hong Kong, government-run public hospitals have been playing a major role in secondary care and a centralized EHR system is used by the hospitals. To provide a territory-wide platform for two-way data sharing between public and private healthcare providers, the government had started to develop the Electronic Health Record Sharing System (eHRSS) since 2008. ³⁵ The system was launched in 2016, regulated by legal ordinance on data protection, privacy, and security. Local systems of private or non-government healthcare providers can communicate with eHRSS via the turn-key clinical management system CMS On-ramp that is made available by the government for data sharing and integration. ³⁶ The CMS On-ramp is an open, portable, and technology-independent application complying with the requirements of system interoperability and information standards. In addition to private hospitals, eHRSS access is extended to community-based healthcare providers, e.g. private clinics, elderly homes, non-governmental organizations, and social service agencies, to support primary healthcare development. While the current stage of the project is to disseminate the proposed EHIS to more elderly care centers and realize a community care information network for older adults, the proposed EHIS is eHRSS ready—it can be interfaced with eHRSS by implementing the API of the CMS On-ramp to enable data sharing and integration capability with the EHRs.

Contributions

The main contribution of this study is that a health information system is proposed to meet the informatics needs of community centers in the caring for older adults in non-medical settings.

The system enables digitalization of the care delivery process where electronic versions of the instruments and assessment questionnaires administrated at the centers, and the care plans formulated, are available to record the data digitally. Connectivity to consumer healthcare monitoring and wearable devices is also enabled. The acceptance and usability of the system demonstrated in the study support the feasibility of the proposed approach as a means to digitalize professional practice and elderly care service delivery in the community. The system's interoperability for effective data management and information exchange with other centers or hospitals makes contributions to filling the information gaps of social and primary care informatics, which is beneficial for providing a holistic view of elderly care and establishing large multi-center datasets for healthcare predictive intelligence.

Limitations

The study on the acceptance and usability of the EHIS had a number of limitations. First, the sample size of the pilot study was small, which precluded more detailed analysis and might reduce the validity of the results. In the evaluation of technology acceptance using the UTAUT, descriptive statistics and histograms were only used to analyze the data, with attempts to conduct reliability and correlation analysis. In-depth investigation using structural equation modeling or path analysis^26,28,37 would require a larger sample size. Therefore, validation of the UTAUT model or prediction of Behavioral Intention could not be performed in the present study. Besides, the influence of work experience and professional roles were not investigated due to the small sample size. In the study, the usability and acceptance of the EHIS were evaluated by comparing it with similar studies that evaluated health information technologies using the UTAUT and CSUQ questionnaires. However, as the advocate of community healthcare information systems specifically for older adults is relatively recent, few studies on similar systems are available for a more direct comparison. On the other hand, the present study was confined to the evaluation of acceptance and usability using the UTAUT and CSUQ surveys, which could be rather general that they may not expose specific issues that the users were less satisfied with. For example, a program worker who participated in the study raised a concern that “while client identification number is usually assigned upon first-time membership registration and regarded as invariant, but according to their routine practice, the number can be replaced afterward by the number of a government healthcare subsidy scheme if the client is a recipient.” This reveals that the EHIS still has room for improvement to cater to such subtleties in the workflow.

Future research

Future research will be carried out to address the limitations discussed above. The proposed EHIS prototype will be deployed in more community centers under the same elderly care delivery organization that the project has been collaborating with. The sample size can be then increased for a larger scale and more in-depth investigation, e.g. studying the association of acceptance and usability with work experience and roles, and conducting structural equation modeling with the UTAUT model. In addition to UTAUT and CSUQ surveys, instruments measuring other aspects of the EHIS will be employed, e.g. Questionnaire for User Interface Satisfaction ³⁸ and User Experience Questionnaire, ³⁹ to obtain a comprehensive view of the EHIS. Semi-structured interviews along with open questions will also be designed to obtain qualitative feedback, and identify potential issues in user–system interactions, or in operations that do not align well with routine practice. In addition, formal methods in cognitive science such as observation and think-aloud testing ⁴⁰ will be explored for usability testing. These future studies will provide insights into how the EHIS can be further improved in terms of acceptance and usability, enabling the system to better meet the need for elderly care in the community.

On the other hand, the proposed EHIS aims to provide a practical approach that can promote the digitalization of community healthcare services and enable the development of primary care information systems. To achieve this long-term goal, the EHIS system will be scaled up and extended to other care delivery organizations to realize a digital primary care network for older adults, from which a large multi-center dataset can be established to support smart health management, decision support, and predictive intelligence. For the latter, intelligent reminders of potential health and social issues of elderly people can be realized by leveraging the community healthcare datasets collected to assist in early diagnosis and intervention. In fact, primary care data contain rich information for secondary research. For example, elderly profile (e.g. demographics and medical history) and assessment data collected from community-based healthcare services have been used to predict the quality of life ⁴¹ and the risk of dementia ⁴² by making use of machine learning techniques.

Conclusion

An EHIS was proposed to promote the digitalization of elderly care and services provided by community-oriented care centers. The system adopted international standards of health information exchange and clinical terminology to enable interoperability and data sharing. Feasibility was supported by an evaluation study showing that the users accepted the use of the system and were satisfied with its usability. The system is in routine use now. The present study was limited by a small sample size and confined to the evaluation of acceptance and usability using self-reported questionnaires. Future work will be conducted to extend the use of the system to more elderly care centers and evaluate the system more comprehensively with mixed methods, including user interface design and user experience, semi-structured interviews as well as observations and think-aloud approach.

Appendix 1

Example of FHIR Patient resource in the proposed EHIS.

Appendix 2

Example of FHIR Observation resource implemented in the proposed EHIS.

Appendix 3

Table 8.

Example of FHIR Questionnaire resource, implemented for the short version of the Geriatric Depression Scale.

Appendix 4

Example of FHIR QuestionnaireResponse resource, implemented for the short version of the Geriatric Depression Scale.

Appendix 5

Summary of items in the survey questionnaire based on UTAUT.

Construct	No. of items	Description
Performance expectancy	4	Usefulness, speediness, productivity, and convenience that EHIS would offer in performing the work
Effort expectancy	4	Clarity and understandability of the interactions with EHIS; easiness of use and of learning to use the EHIS, and easiness of becoming skillful at using EHIS
Social influence	3	Views of colleagues on the need to use EHIS and management's support
Facilitating conditions	5	Availability of resources, knowledge, assistance for using EHIS; and willingness to use EHIS voluntarily
Attitude	4	Appreciation toward EHIS; degree of interestedness and pleasure with EHIS
Self-efficacy	2	Ability to use EHIS independently
Anxiety	4	Apprehensiveness, worry, hesitation, and intimidation in using EHIS
Behavioral intention	3	Intention, prediction, and plan to use EHIS in the next 12 months

Appendix 6

Summary of items in the survey questionnaire based on CSUQ.

Subscale	No. of items	Description
System usefulness	8	Satisfaction toward easiness, simplicity, and comfort of use of EHIS; agreement with the efficiency, effectiveness, speediness, and productivity that the use of EHIS would enhance
Information quality	7	Clarity of error messages, recovery of mistakes, clarity of online help or on-screen messages, easiness of access of information required, on-screen information organization, and effectiveness of the information in completing the work
Interface quality	3	Availability of expected functions and capabilities; degree of satisfaction and pleasure with the interface
Overall	19	Average of the first 18 items above and the last item concerning the overall satisfaction toward the system's usability