TABLE 1. Selected results from DQS Plus assessments, by dimension, related to the electronic immunization registry, Panama, May 2014 and Honduras, November 2015.

EIR dimension/Item in the dimension		Panama	Honduras
System scope
	Included population	All population groups.	Children under 5 years old, though SINOVA has the capacity to record data for any person.
	Routine program, supplementary immunization activities, vaccines not included in the national immunization schedule	PAI Software allows inclusion of different vaccination strategies (health facility, vaccination campaigns, outreach modalities: school vaccination program, health workers, vaccination in businesses, farms, etc.). It also allows registering vaccines outside the national EPI schedule (for the private sector or foreigners).	SINOVA allows registering routine doses, in health facility or in outreach. Vaccination campaigns are registered as outreach vaccination activities. No possibility to record vaccines outside the national EPI schedule.
	How is the EIR to be used during outreach activities	Paper-based recording. The electronic tool itself is not used outside health facilities.	Paper-based recording. The electronic tool itself is not used outside health facilities.
	Previous cohorts (from paper or electronic systems)	No attempts at adding legacy data were made.	No attempts at adding legacy data were made.
	Vaccination history of new people as they are being added into the EIR	This is not done systematically.	This is not done or envisioned.
Normative and legal context
	National eHealth strategy in place	At the time of the development of the PAI Software, no eHealth strategy was in place. In 2014, the national eHealth strategy and policies were being developed. At the same time, an electronic health record (EHR) project for primary health care for the Ministry of Health (MoH) (38) and the Social Security system was being implemented.	A plan for an Integrated Health Information System was available but it is outdated.
	EIR system compliance with national norms	There was no legislation or a normative framework for the use of PAI Software. Data collection and reporting was available in the 1999 national EPI Manual of Panama, but not in more recent versions.	There were no national regulations describing or regulating the EIR. The SINOVA instructions were only mentioned as part of the project procedures.
	Mandatory use of the EIR (including private and other sectors)	Mandatory for MoH public health facilities, but not for Social Security or private providers.	Mandatory in the health regions where SINOVA is being implemented. Social Security facilities do not use the SINOVA, but this possibility has been raised. SINOVA use by private providers is not planned.
	Legislation framework for data privacy and confidentiality	National Law 68 includes a prohibition on sharing data with personal identifiers through any means (physical or electronic) unless it is for epidemiological or academic research use by an authorized entity. There is no data encryption, but access to the systems is through name and user validation.	There is no national law or regulation from the MoH for the privacy and confidentiality of the data contained in SINOVA. Persons interviewed indicated that in practice the confidentiality of the data in registries is strongly guarded. There is no data encryption or user validation.
Architecture
	Integration with other health information systems; integration with birth registration or civil registration systems; integration with other EPI information systems	PAI Software does not interoperate with any other EPI software, or with other software from the MoH, Social Security, or national Department of Vital Statistics. The data produced by PAI Software are not integrated into data from the national Health Information System.	SINOVA interoperates with the official immunization registration software used in non-SINOVA pilot regions. It does not interoperate with other information subsystems in the MoH. Although there is an agreement signed with the national Registry of Persons for the electronic exchange of data, a process to verify personal data has not yet been implemented.
	Software type	Visual FoxPro 6.0, standalone and client-server model.	Standalone model and client-server, in Visual Basic .NET 2010.
	Database type	Visual FoxPro 6.0 database.	Microsoft SQL Server 2012 database engine.
	Online–offline options	Only offline.	Only offline.
	Periodicity of data updates and database synchronization	Monthly, but monitoring not done systematically.	Monthly.
	Location of the database	The database, divided by calendar year, is stored locally in computers at the national EPI.	The databases are in a national server at the MoH Information Management unit, in servers in the project regions, and in computers in selected municipality headquarters within those regions.
	Technical specifications for computers for the system	Minimum configuration required: Pentium 4 processor Windows 98 or above, except for Windows 8 512Mb RAM 512Mb of free space in the hard drive	Minimum configuration required: CORE i3 processor or higher 4 GB RAM Memory 500 GB Hard Disk Windows 7 or higher For regional level servers: 4 core processor E3-1220 Xeon RAM memory, 4GB RAID configuration of 2 disks of 500 GB 7.2 K RPM serial ATA3 GBPS 3.5 Windows Server Standard 2012 OLP operating system with CALS to access the server For the national level server: Windows Server 2012 + 4 CALS for server access Intel Xeon Processor E5-2620-2.00GHZ RAM Memory 3 x 4 GB DDR3-1333 MHz Dual Ranked RDIMM Hard drive 1 TB 7.2 K RPM
	Inclusion of the module for short message service (SMS) or mobile health (mHealth)	No version of the PAI Software that would enable linkages with mHealth is available.	No version of SINOVA that would enable linkages with mHealth is available.
Maintenance and sustainability
	Information management	Through a coordinated effort between the national Department of Statistics and the national EPI of Panama’s MoH, with support from the EPI of one health region.	Currently, any SINOVA adjustment depends on the original developer. However, the Technology Support Area in the MoH Information Management Unit is to assume this role.
	Plan for scale-up and capacity	PAI Software is not scalable and no plan to scale it was in place.	The current software version was designed to be scalable.
	Data security	Database backup procedures are not clearly defined, as a protocol, policy, or in any other written document.	There is no protocol, policy, or any other standard procedure for computer security or for backup copies, although a backup process is done weekly by the head of SINOVA in the Health Statistics Unit and in the regions.
	Management of software updates and improvements	PAI Software is usually only updated when new vaccines are introduced into the immunization schedule. Only one person at a regional statistics office knows how to update PAI Software. The database is updated, and an executable file is created. The file is copied onto a compact disk (CD) and sent to the regions; it is also sent as a downloadable file through a link in the MoH website. Statistics personnel at the regional level receive the CD or download the update and then send these updates to health facilities through various media. In the health facility, the executable file is copied and run. The file name is renamed to add the year of the update. At the time of the DQS Plus, May 2014, it was not possible to know whether all the facilities had the same version of PAI Software. The last plan to make improvements to PAI Software was in August 2010.	For any update, SINOVA’s developer sends the update to the Information Management Unit system manager and to the regions.
	Management of errors, users’ questions, and duplicates of records or of persons in the EIR	The person who identifies an error (usually a data entry clerk) notifies the regional level. If this level cannot resolve the problem, the regional level reports it to the region that supports PAI Software in lieu of the national EPI. The entity solving the problem depends on the nature of the error. There are no strategies to help users, such as a help desk. There are no norms or defined procedures for identification and correction of duplicate records or persons in PAI Software.	Management of incidents or users’ questions was not included in SINOVA guidelines, but a plan to do so existed. In practice, health regions and municipalities have a dedicated notebook for incident recording. This information is sent to the national level, including screenshots if needed. The technical support activities provided to the user to solve problems in the SINOVA functionality are remote and provided by the SINOVA development consultant using TeamViewer software. If there are computer operating problems, the support is given by the health region’s computer specialist. SINOVA includes validations that do not allow duplicate data to be entered. In practice, there may be duplication, such as the same person with different names, a repeated identification number, or the same person registered in different regions/municipalities. Future plans include cleaning the national database and then returning the clean database to the lower levels on a periodic basis.
	Documentation up to date	No technical documentation on the architecture or other key features of the software is available. Nevertheless, there is a user manual that is revised with each software update and distributed simultaneously.	The technical documentation on the architecture and internal operation of the software and the user manual are outdated. A clear definition of what software technical documentation (entity relationship diagrams, data dictionary, Unified Modeling Language (UML)) that needs to be made available to the MoH has not been specified.
	Financial plan for maintaining the EIR	There is no budget for maintenance, updates, or improvements to the PAI Software.	There is no budget for maintenance, updates, or improvements to the SINOVA software.
Human resources
	Profile of data entry personnel	Profile for data entry clerks: technical or bachelor’s degree statistical level 1, 2, or 3 as per MoH organizational structure must be certified by the Health Statistical Association of Panama	There is no standardized profile for the human resources that manage SINOVA, but whenever there is new staff to do data entry, they are trained in vaccination schedules and other relevant EPI information.
	Profile of person responsible for validating the data and monitoring potential record duplication	No defined official profile for the person doing data validation.	No defined official profile for the person doing data validation, but this task has been assigned to the head of the Information Management Area and the person in charge of the EPI in the health regions.
	Profile of software developers; profile of trainers; profile of people in charge of maintaining hardware and telecommunication infrastructure; profile of database administrator	Profiles required not described. Available personnel lacked minimal technical skills needed for developing and administering a database, for training on the use of the software, or for maintaining the hardware and telecommunications used by PAI Software. No defined official profile for tech support.	No technical profile defined for database administration personnel, software development, and/or maintenance and training in the use of software, nor for technicians for hardware and telecommunications maintenance.
	Help desk	No help desk strategies for PAI Software users.	No help desk strategies for SINOVA users, but there are multidisciplinary coordinating teams, at national and regional levels.
Modules included in the system
	Immunization registry	PAI Software has an immunization registry module and a module for aggregated data on doses administered.	SINOVA has an immunization registry module only.
	Logistics and supply chain management	No	No
	Cold chain inventory	No	No
	Surveillance of adverse events following immunization	No	No
	Vaccine-preventable disease surveillance	No	No
	Training module	No	No
	Other modules	No	No
	EIR functionalities
	Following individual vaccination schedules	No	Yes
	Coverage monitoring	No	Yes
	By age	Yes	Yes
	By condition (pregnancy, chronic diseases, etc.)	Yes	No
	By geographical area	Yes	Yes
	By ethnicity/minority group	No	Yes
	By health facility vaccinating	Yes	Yes
	By vaccinator	No	No
	By health system affiliation	Yes	No
	Vaccine lot number monitoring	Yes	No
	Recalls/reminders (automated generation)	No	No
	Management reports	Yes	Yes
	Ad hoc reports	Yes	No
	Predefined reports	Yes	Yes
	Validation modules	Yes	No
	Duplicate record management	No	No
	Map generation	No	No
	Georeferenced data	No	No
	Access for external users	No	No
	Communication between EPI and EIR users	No	No
	Information dissemination	No	No
	Clinical decision support on immunization	No	No

Source: DQS Plus results from Panama in 2014 and Honduras in 2015.