Evaluation of the Design and Development of the HeLe Newborn Hearing Screening Tele-Audiology Systems for the Philippines

Abstract

Introduction

There is increasing interest in innovation development and management in the Philippines, especially in the last decade. In the advocacy for universal hearing health, the HeLe, “Hearing for Life’’ Research Program was implemented. HeLe developed novel telehealth technologies and field tested a proof-of-concept service delivery model to improve provision of newborn hearing screening and intervention services in the Philippines.

Objective

As the HeLe research period concludes, this appraisal was organized to document and assess the health information technology systems of the HeLe.

Methods

The evaluation follows the elements of the Centers for Disease Control and Prevention (CDC) guidelines for evaluation of public health surveillance systems. It centers on the status of the eHealth-based components of the HeLe NHS interventions: HeLe NHS module in the Community Health Information Tracking System (CHITS) electronic medical records system, the Tele-Audiology module in National Telehealth System (NTS), and the HeLe NHS registry. The evaluation is based on interviews of key HeLe research staff and documentation review.

Results

The HeLe system has a stable, SQL-Server-based architecture. It is a secure, web-based system with clean separation of back-end database and front-end Web, using Secure Socket Layer (SSL) technology. Standardization of data via mapping ensures reliable, comparable measures. HeLe demonstrates that NHS data collected by the HeLe NHS device can be sent to, stored in, and extracted from the CHITS electronic medical record system and exchanged across platforms. Where actual patient and NHS data were available, this HeLe system is validated to be efficacious to capture and seamlessly exchange data across various eHealth platforms. These eHealth technologies are described to be at Technology Readiness Level 5, “technologies are validated in a relevant environment”. The HeLe program, however, needs to address completeness in documentation as a standard practice, if only to ensure better management of risks introduced by novel eHealth systems in patient care. The CDC public health surveillance checklist used for this assessment is useful in identifying gaps in research management for the HeLe inventors. It is recommended to be incorporated to be standard and implemented early in the next iteration of the HeLe research.

Conclusions

Overall, the HeLe technologies, in this initial stage of research, have achieved the purpose for which they were developed. As a novel technology-based NHS system, HeLe is a potentially powerful tool to assist in monitoring newborn hearing disease caseloads by community-based primary care clinics, NHS facilities, and hospitals that provide definitive medical services. As other health systems strengthening reforms take root in the Philippines, secure exchange of data electronically across the country would depend on sound technologies, including those used in hearing health. This paper can be instructive to the emerging research community in the eHealth and biomedical development space especially in resource-challenged settings. Likewise, lessons can reinforce institutional support from research agencies, clinicians, and state/ county or subnational health departments for policy and resource mobilization to better manage those identified with congenital hearing loss.

INTRODUCTION

The research program Hearing for Life (HeLe)^a developed components of a newborn hearing screening (NHS) and intervention system: a working prototype of the NHS device, an NHS module in an electronic medical record system, and a tele-audiology referral system, as well as a computer-based training or e-learning system with an in-person skills development workshop for newborn hearing screener training and accreditation. Aside from these products, the Program is also completing several studies on clinical validation of the HeLe biomedical NHS device, evaluation of the tele-audiology systems, and analysis of the pilot implementation of the blended HeLe NHS training course. It field tested a proof-of-concept service delivery model enabled by telehealth technologies to improve provision of NHS and intervention services in the Philippines.

Permanent bilateral hearing loss in the country occurs in about 0.13% of live births.¹ Each child with untreated hearing loss will cost a family about 4.3 million pesos to pay for special education, special care, as well as lost income during adulthood. This economic burden from delays in speech and intellectual development is preventable if the hearing loss is detected and treated early. Early detection and intervention help individuals reach their full potential in addition to the costs of about 3 to 4 million pesos annually.²

The Philippine government recognized the need for early detection and intervention of hearing loss when it enacted the Universal Newborn Hearing Screening and Intervention Law in 2009. This is Republic Act (RA) 9709 which requires that all newborns be screened for hearing loss, and if present, treated early. In 2020, however, about a decade after its enactment, less than 10% of Filipino newborns are screened.³ This is attributed to several factors, namely, the lack of accredited screening facilities and personnel, the costs of the screening device (including maintenance and repair), challenging Philippine geography, and births outside hospitals. To address these challenges, the HeLe research and development program is implemented and funded by the Philippine-California Advanced Research Institute (PCARI) Cycle 2 research grant, under the Commission on Higher Education (CHED).

There is increasing interest in innovation development and management in the Philippines with the expanding number of players, especially in the last decade. This paper discusses the status of the eHealth-based components of the HeLe NHS interventions, as the research period concludes. It will describe elements of the design and development of such, and their efficacy in health data capture and transfer across different electronic platforms and various types of NHS facilities. In addition to the HeLe inventors, this paper is of potential use to the growing number of researchers in the eHealth and biomedical device development space, especially in resource-limited settings.

METHODS

The evaluation follows the elements of the Centers for Disease Control and Prevention (CDC) guidelines for evaluation of public health surveillance systems.⁴ It will describe elements of the design and development of such, and their efficacy in health data capture and transfer across different electronic platforms and various types of NHS facilities. The evaluation was based on interviews and documentation review. Interviews with key HeLe research staff explored aspects of the organization, infrastructure, and management of the HeLe tele-audiology system. Contact via email was used extensively between and after interviews. Additional information was obtained from the system documentation, publications, presentation materials, and access to the HeLe system.

This evaluation was conducted on the 20^th to the 22^nd month of a 24-month research period. At that point, the HeLe NHS module in CHITS^b electronic medical records system, the Tele-Audiology Module in NTS^c telehealth platform, and the HeLe NHS registry have been developed, tested, and validated for their health information management functions. The NHS device prototype has been developed and is still undergoing clinical validation.

In its test environment, dummy mother, newborn, and corresponding NHS data were encoded in the CHITS. From the CHITS-NHS module, these dummy data were electronically transferred or ‘pushed’ to the Tele-Audiology Module of the NTS telemedicine platform and the NHS registry.

HeLe was implemented in eight (8) Rural health units (RHU) in the provinces of Bulacan, Romblon, and Iloilo, the Philippine General Hospital Ear Unit, two regional medical centers of the DOH, and one private hearing services center.

At the time of this study, 27 newborns underwent NHS in three (3) RHU, using a commercially available Otoacoustic Emission Test (OAE) device. Since the OAE device (Grason-Stadler GSI Corti™) is not interoperable with the CHITS EMR, OAE-generated NHS data is manually encoded into the CHITS EMR. These were partitioned from dummy data in the CHITS NHS module. Its transfer to the NHS Registry and tele-audiology referral were evaluated.

Ethical Considerations

Ethical approval was obtained from the University of the Philippines Manila Research Ethics Board before its implementation.

RESULTS

The HeLe is a collaborative initiative demonstrating a cross-disciplinary coalition of local community organizations, national, and international academic partnerships to ensure that newborns with congenital hearing loss will be able to hear, speak, and live like normal children.

The HeLe NHS module in CHITS electronic medical records system, the Tele-Audiology Module in NTS telehealth platform, and the HeLe NHS registry have been developed and tested. Where actual patient and NHS data were available, this HeLe system is validated to be efficacious to capture and seamlessly exchange data across various eHealth platforms. These eHealth technologies are described to be at Technology Readiness Level 5, “technologies are validated in a relevant environment”.

System Description

Purpose

As mandated in the RA 9709 or the Universal Newborn Hearing Screening and Intervention Act of 2009, measures of prevention and early diagnosis of congenital hearing loss among newborns, and the provision of referral, follow-up, recall, and early management to infants with hearing loss. However, even with the law, less than 10% of babies born every year have been screened for hearing loss.

To address these limitations, the Program was envisioned with the aim of increasing the number of babies screened for hearing problems, through early diagnosis and intervention to deaf Filipino infants using technology innovations. In addition, the research Program will contribute to NHS advocacy the development of scalable telehealth protocols and systems to support NHS, with involvement of local community-based health workers and medical institutions, and efficient, scalable approaches for onboarding stakeholders and training health workers.

Objective

The goal of the HeLe Program is to increase the number of babies screened for hearing problems through novel technologies such as the HeLe NHS Device (an AABR device prototype) used within the HeLe Tele-Audiology system and made available at the rural health units to provide early diagnosis and intervention to deaf infants.

Necessarily, among main objectives of this HeLe research is to develop and evaluate its biomedical hearing screening device and eHealth knowledge products. It will also conduct and present an analysis of the effects of the envisioned HeLe technologies, the effects of the implemented health worker capacity building program, and its eHealth–enabled, community–based service delivery network in the provision of NHS and intervention services on the pilot sites.

Organizations / Collaborators

The HeLe Program is collaboratively led by the Philippine National Ear Institute (PNEI), the National Telehealth Center (NTHC) of the National Institutes of Health (NIH), University of the Philippines (UP) Manila, the UP Diliman College of Engineering Electrical and Electronics Engineering Institute (EEEI) with the University of California (UC) Davis Center for Health and Technology, and UC Berkeley CITRIS-Banatao Institute.

Stakeholders

Immediate stakeholders of HeLe are the newborns (patients) and their families, the academe/ research community (UP, UC, PCARI-CHED), participating health leaders and professional staff of Rural Health Units (RHUs) and Local Government Units, hospital and program administrators (public and private), physicians (primary care; otorhinolaryngologists and pediatric medical specialists), nurses, midwives, clinical audiologists, and the Department of Health (DOH) Technical Working Group on NHS with the Philippine Newborn Hearing Screening Reference Center of the UP-NIH.

In addition to this set of stakeholders, results, and lessons from HeLe implementation will be of value to the following:

• NHS service providers, practitioners and professional societies, relevant networks,

  • ◦
    Association of Municipal Health Officers of the Philippines (AMHOP), the Doctors-to-the-Barrios, and other government primary care health worker professional societies
  • ◦
    Provincial governments and their provincial hospitals; other DOH hospitals that will provide NHS Category A to C
  • ◦
    private NHS and hearing intervention centers and clinicians
  • ◦
    Philippine Academy of Neurotology, Otology and Related Sciences Rural (PANORS), Collaboration for Newborn Hearing Screening Advocacy (CoNHScA), the Philippine Academy of Neurotology, Otology, and Audiology (PANOA), the Ear Study Group of the Philippine Society of Otolaryngology-Head and Neck Surgery

• the local Philippine biomedical device industry

• local Philippine digital health solutions providers (for telehealth platforms, and electronic medical records systems),

• the Philippine Health Insurance Corporation (PhilHealth), which pays for evidence-informed health care services or packages delivered by accredited providers. In this case, PhilHealth reimburses services rendered under the Neonatal Care Package, wherein NHS falls under.

• the Philippine Congress and DOH, which have over-all mandate for development and implementation of critical health policy in the country.

Funding

This HeLe Program is a multidisciplinary and cross-agency effort, funded through PCARI-CHED and counterpart resources of the PNEI and NTHC of the NIH-UP Manila, EEEI of the UP Diliman College of Engineering. No research funding was sourced from industry: private hearing medical centers, hearing screening device manufacturers or distributors, eHealth/ digital health solutions providers.

Pilot Sites

There are 12 medical facilities actively participating in this Program: The Philippine General Hospital Ear Unit, two regional medical centers of the DOH, a private hearing services center, and eight local government primary care health centers (alternately referred to as rural health units, RHU). These RHU serve rural remote and disadvantaged populations and tested other telehealth innovations of the NTHC prior to the HeLe program.

Memorandum of agreement (MOA) were implemented between participating facilities. These MOAs specify the purpose of the research and limits of data access, the data security plan, and constraints on data dissemination, presentation, and publication.

Likewise, the facilities were linked into an NHS service delivery network through arrangements stipulated in the MOA and guided by principles of early identification of newborns with (potential) hearing loss and appropriate referral. This follows provisions of RA 9709 wherein NHS centers will be linked with facilities that provide confirmatory and intervention services for hearing impairment. With the campaign for universal NHS, the HeLe research engaged RHU as NHS centers, bringing NHS services closest to the communities where target infants live, and where the majority of births occur. This HeLe MOA was deemed necessary in this research to ensure (and document) the data flow across health facilities in a tele-audiology system.

Data Flow

Operational policies of RA 9079 defined facility capacity to provide specific services for hearing diagnosis and intervention: Category A (NHS only), B (NHS and confirmation of hearing loss), C (services rendered by Category B center plus services for hearing amplification), and Category D (services rendered by Category B center plus services for surgical and medical intervention). The HeLe Program established NHS Category A-capable facilities (in eight RHUs and one DOH hospital) and demonstrated telereferral between Category A and B facilities.

The Program uses three systems to capture, relay, and display needed NHS data: Community Health Information Tracking System (CHITS) electronic medical record system, National Telehealth System (NTS) as the telemedicine platform, and HeLe Device to capture biologic signals for newborn hearing screening.

The HeLe NHS device is a novel automated acoustic brainstem response detection device. It is a paired device: a headpiece acoustic stimulator, and sensor and a computer that processes the electrical signals and displays the results. It is also a telemedicine tool, inspired by the RxBox device, intended for cardiovascular care, and maternal-child care.

The HeLe NHS device is designed as a component of an integrated system that collects and sends newborn hearing screening data to the RHU’s database in their CHITS EMR, as well as to the HeLe NHS registry. When needed, a telereferral is made to a Category B NHS facility through the NTS telemedicine platform. Figure 1 depicts the data flow across the HeLe technologies and between health facilities.

Figure 1.

Data flow in the Newborn Hearing Screening Tele-Audiology Systems.

ABR – Auditory Brainstem Response Test; EMR – Electronic Medical Records (in this case, the CHITS-EMR: Community Health Information Tracking System); NHS – Newborn Hearing Screening; TLB – Total Live Births.

These will be described in greater detail below.

The (patient) data requirements are of three types: general demographic data, newborn hearing screening data, and confirmatory data. CHITS captures patient-level data that, when aggregated, provides data necessary to make sound and evidenced-based decisions for sub-national and national health priorities.

The CHITS documents the same data consistent with the Newborn Hearing Screening Registry Card, mandated to implement the law (RA 9709). CHITS’ Newborn Hearing Screening Module is developed as essentially an electronic version of the NHS Registry Card. CHITS allows single data entry, thus, its Newborn Hearing Screening (NHS) Module will automatically be pre-filled, and pulls relevant data from previously captured and documented data in other CHITS modules: i.e., Patient Consult Module (for mother or neonate), Mother’s Maternal Care, and Mother’s Lyingin (or parturition/ childbirth) Module, and the Newborn Module.⁵ Once NHS is conducted, data generated by the HeLe NHS device is pushed and documented in the CHITS’ NHS Module.

The CHITS Newborn Module captures the same data on the services rendered by the medical facility under PhilHealth’s Newborn Care Package, including NHS. Submission of data will allow reimbursement of services rendered to PhilHealth members by the RHU by PhilHealth.

The HeLe device interfaces with the CHITS through Fast Healthcare Interoperability Resources (FHIR) to allow interoperability and exchange of data. NHS data – generated by the HeLe device and pushed to the CHITS – will indicate whether one or both ears of a neonate passed the NHS test, or, if either or both ears failed the test. Newborn patients who failed their hearing screening test can be referred to Category B hearing confirmatory centers. These telereferrals are sent from CHITS and will then be received by the Tele-Audiology Module of the NTS telemedicine platform as incoming referrals to the hearing confirmatory centers. After the needed (hearing confirmatory) service has been rendered by the receiving Category B facility, its designated user can send a return slip back to the referring facility. The referring physician from the RHU or Category A facility can also retrieve the confirmatory details of the index patient via CHITS.⁶

Theoretically, this Category B facility, via the NTS Tele-Audiology Module can send a new telereferral to an NHS Category C or D facility for definitive intervention on the patient. However, this is not within the scope of this current HeLe research.

NHS data – generated by the HeLe device and pushed to the CHITS, which in turn will push this NHS data into the web-based HeLe NHS Registry. For the latter, this will allow surveillance and basis for recalibration of NHS implementation activities. Further, lessons from this technology development were envisioned by the HeLe researchers to inform enhancements to the existing NHS Registry maintained by the Philippine Department of Health Technical Working Group on NHS (DOH TWG on NHS).

Information Architecture

End-user Requirements

The HeLe involves several types of technology end-users, not just health professionals. The HeLe researchers have categorized these; see Table 1 for the user type, their definition, and corresponding roles and privileges as they interact with any of the HeLe technologies.⁷

Table 1.

HeLe User Types, Definition with Roles and Privileges

User Type	Definition	Roles/ Privileges
User / End-users	Health workers who are trained to use the CHITS, NTS, and HeLe NHS device.	Interacts with patient and uses the HeLe Device, CHITS, web applications for data collection and clinical decision-making (i.e., to refer to Category B facility, and inform parents of patients of NHS results, including when referral for confirmatory testing is warranted). Reports technology issues encountered to the HeLe Systems Facility Administrator or Level 1 User.
Level 1 / HeLe Systems Facility Administrator (IT Designate)	Health worker of the HeLe Program site who is identified as the Facility Administrator (IT-Designate) and trained to deal with minor technical/ technology issues.	Can troubleshoot minor issues* stipulated in the actual IT training and Training manual disseminated. * These issues relate to ascertaining if the hardware is/are connected to the power source, whether there is internet connectivity. Regularly perform the HeLe device, CHITS Maintenance internet connectivity testing, using checklists. Acts as liaison for the Program site with the HeLe (research team) Level 2 staff.
Level 2 / Help Desk	HeLe Research Personnel who receives, tracks, resolves minor issues, or refers major issues to the Level 3 IT or device engineers. Maintains the Centralized Issue Tracking System (Help Desk) activities.	Can troubleshoot most of the minor issues and some major issues that are not related to CHITS and NTS telehealth software debugging and HeLe device internal circuitry. Manager of Centralized Issue Tracking System (CITS) accounts.
Level 3 / IT and device engineers (“Supplier/ Distributor”)	HeLe CHITS/ NTS software developers, System Administrators, HeLe Device engineers and software developers.	Troubleshoots major to critical issues related to any of the HeLe software and device/ hardware.

Roles and privileges refer to responsibilities in NHS data management, and in technology issue management. For the latter, any concern or issue related to the innovative technologies must be reported, which in turn should be resolved within the participating health facility or the HeLe research software and hardware engineers. (See section below on Training and Support).

HeLe provides a common user interface and a communications protocol accessible with Internet browser software of version 4.0 or above of either Firefox Web Browser, Chromium Web Browser, and Google Chrome Web Browser.

Data Quality / Error Checking

Data quality is checked repeatedly at various stages of development of the HeLe system of innovations. The HeLe Tele-audiology system enhances data quality through built-in error checking capacity when data is transmitted. In addition, the system also makes use of confirmation dialog boxes to encourage users to verify the inputted data before saving; thus, minimizing human error. Sending duplicate/ multiple referral entries for the same patient are also not allowed by the system. See Figure 2 for the flow.

Figure 2.

Data quality flow.

NTS – National Telehealth System platform; OBS – Observation Table

Data Accuracy

The HeLe technology developers were conducting data validation, concurrent to this assessment. That is, they were checking for accuracy of the data transmitted from CHITS to NTS to CHITS to HeLe NHS Registry. See Figure 3 for the sample data flow of EMR to the Registry. With the available data, this process did not uncover any major problems. Some issues (i.e., non-transmission due to software bugs) were resolved immediately (within the day) by the software developers.

Figure 3.

Data accuracy flow.

HIE – Health Information Exchange; NTSP – National Telehealth System platform

The NTS Tele-Audiology System would eventually be deployed. A Newborn Hearing Screening Day was conducted in HeLe sites. At the time of this study, a total of 27 babies were screened using both the “Baah” Test and Otoacoustic Emission Test (OAE) using a commercial device recognized by the DOH (Grason-Stadler GSI Corti™). No issues were encountered when the encoding of patient’s data (demographic and NHS data in the CHITS EMR) was demonstrated to be automatically ‘pushed’ to the CHITS NHS module. A minor issue arose during the referral of babies to the NTS’ Tele-Audiology Module; it was reported and immediately resolved. The issue was minor, i.e., the lack of internet connectivity due to improper set-up of the router with the desktop computer’s central processing unit. This warranted troubleshooting by the RHU health worker who was trained to assess why the HeLe system was not functioning as intended and perform adjustments to correct the way the machines are connected (Level 1, see Table 1). Minor issues are not unexpected since technology use is still not standard practice in rural primary care health facilities in the country. Skills and confidence to resolve minor issues at the facility level have to be built among the local technology staff or health workers.

Privacy, Confidentiality, and Security

When HeLe is reported to stakeholders or discussed in public forums, privacy and confidentiality standards are met for the HeLe system by excluding patient identifiers such as name, date of birth, patient identification and/or PhilHealth identification numbers. Regarding information security, several standard network procedures are in place to ensure that confidential information is protected. These include firewall protection, password authentication to allow only registered users access to the site, and 128-bit data encryption using Secure Socket Layer (SSL) Technology with clean separation of back-end database and front-end website. Registered users of the system are also given roles with certain privileges to limit their account access. In addition, use of GNU Privacy Guard (GPG) asymmetric encryption for dumping and storing of database backups are also being implemented. An Operating System level encryption was also placed in the newly deployed HeLe servers. This will provide an additional layer of security to the system, as it will require the users to enter a passphrase for the system to boot. See Table 2 for the Security Protocols used for the CHITS 2.0 with Newborn Hearing Screening Module (HeLe-CHITS), and the NTS 2.0 with Tele-audiology Module (NTS).

Table 2.

Security Protocols Used for the HeLe-CHITS and NTS

Security Protocols	Description	Other security information details
Containerization	The HeLe-CHITS and NTS systems were set up in separate systems using containerization technology (containers). This way they can be approached as single units of services; gain the benefits of system isolation and snapshot capability.	System isolation. Containers are enclosed systems in a host system. This encapsulation serves as a layer of protection between: the host system and the container the container and the other containers hosted on the same system All containers were created in “unprivileged” mode --so in an event the container gets compromised, it can be immediately quarantined without affecting the services of the host system and the other containers. Snapshot capability. Container snapshots are records of the state of the container at a given time. Somehow this can be treated as backup copies of the container; can be used to reinstate the services hosted on the container with ease and minimal restoration time.
Reverse proxy	A reverse proxy (front-end container) was also set up in a separate container to also act as a layer of protection for the HeLe-CHITS and NTS containers (backend containers).	There are many uses of a reverse proxy: it acts as a barrier between backend containers and the Internet it protects the identities of the backend containers it is used in conjunction with Transport Layer Security (TLS) setup for the backend containers The reverse proxy has also been enabled to redirect Hypertext Transfer Protocol (HTTP) transactions to HTTPS (see next TLS section below).
Transport Layer security(TSL)	In combination with the reverse proxy setup, TLS was included to secure the reverse proxy service, and offload the TLS processes for the backend containers.	Essentially, this gives the same protection to the backend containers, i.e., it secures the transactions between: the user and the HeLe-CHITS service the user and the NTS service the HeLe-CHITS service and the NTS service vice versa for above specified transactions These web transactions (HTTP) become verified and encrypted via the TLS protocol, also known as HTTP Secure (HTTPS).
Secure Shell (SSH)	For the cloud version systems, SSH was installed in each of the containers for remote access and management. Developers are given SSH accounts to their respective assigned service-containers to update the: container OS container application stack application code	The cloud version systems are the following: HeLe-CHITS Demo Site, https://demo.HeLe.chits.ph/openmrs NTS 2.0 Demo Site, https://demo.nts2.telehealth.ph NTS 2.0 Deployment Site, https://HeLe-nts.telehealth.ph A password generator was used to produce passwords for the SSH accounts. These passwords were sent using secured communication channels: email with PGP, and/or Signal Private Messenger. As for retrieving application code and files, SSH key pairs were set up between: the developer workstation and GitLab server to securely transfer the application code from the developer workstation to GitLab the container and GitLab server to securely transfer the application code from GitLab to the container Like the TLS protocol, the SSH protocol secures the following transactions: access to, and management of the remote container file transfer between systems
Host-based firewalls	All containers: HeLe-CHITS, NTS, and the reverse proxy, have been set up with hostbased firewalls.	Each firewall has a different set of rules that allows which ports are opened. Ports are service openings where the application transactions go through. By default, all firewall rules are set to deny incoming transactions from all ports except the SSH port. Additional rules are set on each container which are as follows: HTTP and HTTPS ports are allowed on the reverse proxy container HTTP is allowed on the HeLe-CHITS container but only the reverse proxy container can access it HTTP (port number 8080) is allowed on the NTS container but only the reverse proxy container can access it

The National Privacy Commission (NPC) has templates in the NPC Privacy Toolkit available as a guide in their website⁸ to conduct a privacy impact assessment in the future. This (privacy assessment) is recommended in addition to what is already described, for the next iteration of the HeLe research. As practice, it is best that researchers align with current national policy and administrative measures, especially that of privacy concerns.

Back-up and Restore

When this assessment was conducted, there was no available document for review of the back-up process or protocol for restoring the database. It is highly recommended that an incremental backup of the database is done every night with removable tape. Then, a full backup is done every month after receiving data from the sites.

However, the IT CHITS Infrastructure Maintenance document discussed the detailed physical integrity protection (listed below).⁹ The physical integrity of the local CHITS Server should be secured to ensure the recommended working condition of CHITS. The Staff are reminded on what to do during an unforeseen event and how to prevent them, these include:

• Ensuring the safety of servers from potential hazards such as power surges, fire, and direct sunlight.

• Ensuring devices are checked periodically to prevent short circuits.

• Terminals, especially the server, should be connected to an Uninterrupted Power Supply (UPS) for the users to have ample time to save work and shutdown the server properly. The UPS will also help in regulating power surges and static electricity that can start a fire.

• Ensuring that the devices are free from dust. Dust and dirt may sink into the device’s fan, circuitry, and chips and might cause malfunctions.

• Daily generation and storage of backup files to ensure patient data can be restored in case the server is severely damaged.

Usability and Accessibility

During this evaluation – which is at least 80% of the HeLe research study time – the results of a structured usability test have yet to be published. The Usability of the System and User satisfaction of Mag-Ina Telereferral System (MInTS)¹⁰, also developed by the NTHC, was a simple exercise and can similarly be conducted for HeLe. It is highly recommended to review the US Health and Human Services-issued web design guidelines regarding user-friendliness and accessibility.¹¹ Usability studies can identify gaps in the system that need to be addressed, to achieve utmost user acceptance as the HeLe scales the Technology Readiness ladder.

Another ongoing study within the HeLe research program is a user-centric design (UCD) methodology applied to the development of the next iteration of the HeLe NHS device user interface. While the initial prototype was designed by clinicians adept with NHS and whilst the HeLe research team is cognizant of UCD method as premium to innovations, the practice is yet to be mainstreamed. Results of this ongoing UCD study should be instructive for the next version of the HeLe.

For the HeLe end-users, the two major components are already accessible to them, having prior use of these technologies. The CHITS EMR is hosted locally; other workstations can access it via local area network (LAN). On the other hand, NTS is an online web system and can be accessed in any device’s browser if it has internet connectivity.

Since the NTS Tele-Audiology Module for HeLe is essentially a telereferral for confirmatory studies, the internet connectivity in the locale is deemed sufficient for the store-and-forward telecommunications requirement of the HeLe.

Training and Support

The HeLe Capacity Building Program is offered as a blended learning course designed and developed for community-based primary care health providers, especially those serving isolated and/or disadvantaged populations. The education model consists of two stages: online learning supported with Web 2.0 technologies and face-to-face three-day seminar-workshop. The latter allows skills development on the NHS under supervision by audiologists and NHS-trained staff.

The computer-based training (CBT) is a four-module course on NHS and eHealth tools for NHS implementation; this is hosted in the UP Manila Virtual Learning Environment. The CBT can be used online or offline at the convenience of health workers, and ideally completed as a prerequisite to the face-to-face workshop. The pilot implementation of the CBT course on the NHS among primary care providers in low resource settings revealed key findings: first, CBT can be used to improve awareness and knowledge gaps on the NHS. Second, community healthcare providers are receptive to an online CBT even with unstable Internet connection; and an offline CBT course may be more applicable in areas where Internet connection is poor.¹²

Identification, diagnosis, and management of major or minor issues are tasks of the HeLe researchers; a Centralized Issue Tracking System is developed and maintained. Documentation and analysis of these issues are intended to surface operational and technology concerns and demonstrate maneuvers to troubleshoot such issues. Major issues are related to software and / or device problems that can only be addressed by the HeLe developers. Minor issues are those that relate to concerns that are not major but are often due to health personnel concerns such as forgetting passwords or how to plug the devices to power source, correcting incorrect set up of hardware.

Currently, there is a user manual for the HeLe innovations that is given to the participating health workers and IT-designated staff during the HeLe training program. An electronic version is also made available and can be downloaded from the HeLe research repository.

Help is only available by contacting the HeLe research staff/ Help Desk coordinator through a hotline mobile phone number. There is no built-in help utility online.

The HeLe System issue escalation protocol has three levels in relation to the roles and responsibilities of each level of issue as seen in Figure 4. End user or health workers report issues encountered to the HeLe Facility Administrator or Level 1 technology support staff. These are expected to [1] define and assess / diagnose issues raised by any Health Center staff, and [2] troubleshoot minor and typical technical problems expected, and wherein they were trained for. Should he/ she need assistance, the issue would be reported to the Help Desk (Level 2). Network and hardware concerns should be typically addressed. The HeLe Level 2 technology support staff is responsible for managing and maintaining the issue registry. Software concerns will be escalated and typically addressed within 24-hours by the Level 3 software developers. See Figure 2, depicting the Centralized Issue Tracking System (CITS) workflow.⁷

Figure 4.

Centralized Issue Tracking System (CITS) workflow.

Interface

The HeLe-Electronic Medical Record (HeLe-EMR) Fast Healthcare Interoperability Resources (FHIR) interface allows the HeLe device, an electronic medical diagnostic device, to read data and transfer records to an EMR in a separate device.¹³ The FHIR was used as the standard messaging protocol for the EMR and device to communicate with each other. FHIR as a communication standard was created by the Health Level Seven International (HL-7) health-care standards organization. For this stage of HeLe, however, the researchers have not sought certification that the FHIR is officially that of the HL-7 organization. Nevertheless, the HeLe implementation guide document explains the interactions of the HeLe device with any EMR. For this phase of HeLe, and as a starting point for the design, the EMR used is the Community Health Information Tracking System (CHITS), also developed by the NTHC. The standards-based interoperability of the device and EMR is demonstrated in this HeLe research. It is envisioned for the HeLe NHS device to be EMR-agnostic, but EMR should be developed using accepted communication standards, as demonstrated in this Program. The interface is expected to evolve as current requirements change and new requirements are discovered.

DISCUSSION

The HeLe Research Program is timely and contributes to heightening the advocacy for universal NHS among Filipino children. This is also considering findings of Ramsey, Svider and Folbe of no improvement globally in hearing loss burden over 25 years, as measured by age-standardized disability-adjusted life years (DALYs).¹⁴

The HeLe system of technologies evaluated was able to fulfill its intended purpose with both strengths and limitations. The strength of the system is in its relational database that allows efficient, stable data management, and indicator calculation. Both the back-end database and the java-based front-end permit easy scalability. The design facilitates implementation at additional NHS service units and clinics.

The HeLe is a proof-of-concept that relevant NHS data can be exchanged across platforms (from an EMR with an NHS device, and from the EMR to telereferrals to NHS confirmatory centers). But this is confined to the HeLe system, data exchange capability is limited. Viewed from a macro-level, HeLe created an island of NHS data (among facilities involved in the research), not unlike that of manual paper-based clinical information systems. What has yet to be defined at the national level are the standard datasets for the NHS, the specific results of tests that can (or should) trigger electronic reporting to public health agencies, and other stakeholders. Standards that will allow exchange of NHS data across patient electronic information systems of institutions and agencies have not been specified. The HeLe research findings lend voice to the ever-increasing advocacy for standards to be set and adhered to amidst the expanding use of electronic health information systems across the country. Innovations can be more valuable and be given a better chance to be scaled and incorporated into standard practice if these are standards-based.

HeLe also has some limitations of completeness, as previously described. At its most basic, the clinical results displayed by HeLe are only as complete and accurate as the data entered into the medical record system, notwithstanding the lack of nationally prescribed standard data sets needed for public health management and payment of services. Completeness of documentation of security protocols, and that of data management also need to be improved, if only to ensure better management of risks that is introduced by novel eHealth systems in patient care.

The system also cannot capture all the complexities of clinical practice, like real-time referral for services elsewhere, and movement of patients between facilities. While perhaps sufficient for this initial stage of technology research and development, this should be considered a major element in the design of the next phase of the HeLe research.

HeLe demonstrated how a NHS laboratory information management system (LIMS) can be integrated with a child health program information systems. It supports the single-data entry, many uses adage premised on the protection of children. That is, by assuring more reliable data, better data quality capture, reduction of errors, speed: notification of screening results to the community-based Rural Health Unit, would mean more prompt referral for confirmatory studies and, when needed, definitive interventions provided by appropriate medical facilities.

A recent systematic review affirms such a premise.¹⁵ This was in 14 populations, albeit in developed countries: the United States (US), United Kingdom, Austria, and Australia. The paper, however, did not report use of EMR nor interoperability with the NHS devices. All states of the US now have early hearing detection and intervention (EHDI) programs, implementing its own EHDI tracking and surveillance information system (EHDI-IS)¹⁶, monitored by the Centers for Disease Control and Prevention (US CDC). While EMR use is ubiquitous across the country, some facilities still are paper-based and manually record patient encounters. For the EHDI, the US CDC provides guidance on what relevant NHS data to collect, and the variety of data sources, paper-based or electronic.¹⁷

More relevant to the Philippines, a systematic review of EHDI programs in low- and middle-income countries in Asia revealed three of 82 studies describing “the use of ICT for storing and forwarding results, database management, and sending reminders for follow-up screening”.¹⁸

This HeLe evaluation centered on the NHS program and technical / technology inputs, based on the CDC guidelines for evaluating public health surveillance systems. We evaluated information quality (relevance, accuracy, completeness, timeliness) and system quality (availability, usability/ accessibility, functionality, comprehensive data capture, error reduction, and security). The CDC public health surveillance checklist was useful in identifying gaps in research management for the HeLe inventors. It is recommended to be incorporated to be standard and implemented early in the next iteration of the HeLe research.

Alternatively, a nine-dimension framework was also developed by the US Public Health Informatics Institute (PHII) to assess the value of integrating newborn (metabolic) screening laboratory information management systems with child health information systems.¹⁹ Authors posit that it “represents an important first step” in identifying metrics that can be consistently applied in the evaluation of public health information systems. This PHII framework, like the CDC guidelines, similarly captures the twin aggrupation of inputs and assessment of the quality of the information and technology system. However, the PHII framework is more holistic in that it offers a more expansive logic model that captures the value of innovations being developed. Current literature on Philippine biomedical devices or eHealth innovations center on a single dimension: i.e., the functionality or usefulness of the technology, or how it was developed, or its usability. This assessment was able to document other essential technology development components or processes in the production of the HeLe Tele-Audiology System before it was deployed to field users. The study can be instructive to other researchers especially in resource-poor settings. Both the CDC and the PHII evaluation guidelines can help future eHealth and biomedical device research and development, not just HeLe. Any evaluation framework must be applied early in the investigative process. In articulating the essentials of the research, it enables researchers to document and facilitate the production of evidence that the innovation works and is of public health importance.

CONCLUSIONS

The HeLe system of novel technologies demonstrates a stable SQL Server-based architecture. It is a secure system with clean separation of back-end database and front-end Web using SSL technology. Standardization of data ensures reliable, comparable measures. HeLe demonstrates successfully that data can be extracted from the electronic medical record system for facility-specific reporting. Its potential for use nationwide by trained health workers in community-based primary care clinics in identifying newborns with potential hearing loss and transmitting NHS data to (and from) nearby rural referral hospitals electronically via real-time telereferral is demonstrated.

The HeLe program however needs to address completeness in documentation as a standard practice, if only to ensure better management of risks introduced by novel eHealth systems in patient care. This CDC technology development assessment checklist is useful in identifying gaps in research management for the HeLe inventors. It is recommended to be incorporated to be standard and implemented early in the next iteration of the HeLe research. The broader PHII framework is also recommended and used early in the research process to ensure facilitated valuation of novel public health information systems.

Overall, the HeLe system demonstrates a potentially powerful tool to assist hospitals and community-based health units in monitoring Newborn Hearing Screening caseloads specifically. It contributes general lessons in health systems research that would support the future envisioned to have facilities securely exchanging data electronically. The HeLe system has achieved the purpose for which it was developed. Expanded functionality for mobile phone tele-audiology systems together with SMS and web-based tele-consultation will be dependent on institutional support from research agencies, Philippine government agencies, and regional health departments (if data is to be sent electronically from the rural health units) for resource mobilization and development of data exchange capability.

This paper can be instructive to the emerging research community in the eHealth and biomedical development space especially in resource-challenged settings. Likewise, lessons can reinforce institutional support from research agencies, clinicians, and state/county or subnational health departments for policy and resource mobilization to better manage those identified with congenital hearing loss.

Recommendations

The HeLe team should improve documentation with version approval signatures and develop an online user manual. As noted earlier in the assessment, assistance is not available online. It is highly recommended that different types of help content be made available that are context dependent, meaning that the help information provided would vary based on what the user is attempting to do.

The backup and recovery process for each system must be also documented and periodically reviewed.

This assessment also reveals gaps in the HeLe system, i.e., its user interfaces, that need to be addressed to achieve utmost user acceptance. A usability test and heuristic evaluation should be performed; design guidelines regarding user-friendliness and accessibility are well documented. An evaluation of the use and effectiveness of HeLe is currently underway, but the application of usability heuristics to the system will yield additional recommendations about the design of the user interfaces, queries, and outputs. Reassessment of the infrastructure should be done to ensure an intact feedback loop. Establishment of a rigid issue escalation protocol to address immediate issues on hardware or software. There is a need for users to be retrained in using the System especially in assessment form. There is a need for facilities from the referring and receiving end, to develop a policy (sustainability, technical support, knowledge transfer) in the use of HeLe.

The accuracy of the newborn hearing screening data should be further evaluated using a three-phase process. In the first phase, independent queries should be made to identify any errors in programming that might lead to errors in counts. All the measures for which query results did not match the data should be explored further until the differences are resolved. In the second phase of data validation, the patient charts should be manually reviewed to identify any data capture issues. The purpose of this phase is to evaluate whether any discrepancies between the medical records and the electronic records could explain any unexpected indicator values returned by HeLe. In the third phase, health care providers must review the indicator results and discuss any results that did not match what they would expect. To date, data validation including the data being sent from CHITS to NTS to CHITS to Registry are on-going. This process did not uncover any major problems. Some minor issues were resolved immediately.

As to HeLe’s future expansion, the use of the International Classification of Diseases Tenth Revision Procedure Coding System (ICD-10 PCS), Clinical Modification (ICD 10-CM) for physicians and hospitals, Current Procedural Terminology (CPT), Logical Observation Identifier Names and Codes (LOINC), and Systematized Nomenclature of Medicine, Clinical Term (SNOMED CT) concepts should be undertaken as the major step toward full integration of newborn hearing screening activities into a network of systems. Integration to other systems and accommodation needs of other initiatives should also be addressed. Many of these issues have both technical and social solutions. A great deal of work in both realms remains to be done before the newborn hearing screening systems can be said to have achieved its full potential.

Statement of Authorship

All authors certified fulfillment of ICMJE authorship criteria.

Author Disclosure

The study on which this report was based was undertaken with the financial and administrative support of the Commission on Higher Education, Philippines – California Advance Research Institutes, and the National Institutes of Health, University of the Philippines Manila. These, however, did not have any influence on the content of this paper.

Authors have no relationships with manufacturers of medical devices.