Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 May 1.
Published in final edited form as: Int J Infect Dis. 2008 Dec 18;13(3):410–418. doi: 10.1016/j.ijid.2008.09.015

Personal digital assistants to collect tuberculosis bacteriology data in Peru reduce delays, errors, and workload, and are acceptable to users: cluster randomized controlled trial

Joaquín A Blaya a,b,*, Ted Cohen c, Pablo Rodríguez d, Jihoon Kim e, Hamish SF Fraser b,c
PMCID: PMC2673336  NIHMSID: NIHMS94726  PMID: 19097925

Summary

Objectives

To evaluate the effectiveness of a personal digital assistant (PDA)-based system for collecting tuberculosis test results and to compare this new system to the previous paper-based system. The PDA- and paper-based systems were evaluated based on processing times, frequency of errors, and number of work-hours expended by data collectors.

Methods

We conducted a cluster randomized controlled trial in 93 health establishments in Peru. Baseline data were collected for 19 months. Districts (n = 4) were then randomly assigned to intervention (PDA) or control (paper) groups, and further data were collected for 6 months. Comparisons were made between intervention and control districts and within-districts before and after the introduction of the intervention.

Results

The PDA-based system had a significant effect on processing times (p < 0.001) and errors (p = 0.005). In the between-districts comparison, the median processing time for cultures was reduced from 23 to 8 days and for smears was reduced from 25 to 12 days. In that comparison, the proportion of cultures with delays >90 days was reduced from 9.2% to 0.1% and the number of errors was decreased by 57.1%. The intervention reduced the work-hours necessary to process results by 70% and was preferred by all users.

Conclusions

A well-designed PDA-based system to collect data from institutions over a large, resource-poor area can significantly reduce delays, errors, and person-hours spent processing data.

Keywords: Handheld computer, Personal digital assistant, Evaluation studies, Developing countries, Tuberculosis

Introduction

Clinical and research organizations must often collect data from large numbers of patients who are distributed over wide geographic areas. New technologies may play an important role in ensuring that high-quality data can be quickly and reliably collected under these challenging field conditions. Ideally, organizations or individuals that need to record large amounts of data in dispersed locations would be able to electronically capture this data at the point of collection. In clinical and research settings within developed countries, personal digital assistants (PDAs) have shown some promise as a new technology that can increase the quality and efficiency of data collection, though performance has varied between studies.1-14 This heterogeneity may suggest that the design and implementation of the PDA intervention play a key role in a system's success. In resource-poor settings, initial experiences have demonstrated several situations in which PDAs15-25 and cellular phones26,27 are of benefit. However, to date, we have found no quantitative studies of the impact of mobile technologies on the time to collect and process data, the frequency of discrepancies, or the number of person-hours required for data collection.

We worked with an organization that monitors multi-drug-resistant tuberculosis (MDR-TB) patients in Peru to implement a PDA-based system and to study the impact of this system on data collection. In this treatment program, patients are required to submit a monthly sputum sample at their local health center. Timeliness and accuracy of reporting laboratory results for these samples is essential to determine if a patient is responding to treatment and, if not, to alert physicians to the possible need for medication changes.28 We expected that laboratory monitoring of treatment response would result in reduced culture-conversion times and better treatment outcomes for patients, as well as reduced transmission of the disease in the community.

The laboratory monitoring process begins with a smear microscopy test at the local health center. The sputum sample and smear result are then sent to the corresponding regional laboratory for culture (Figure 1). In some cases, the sputum sample is sent directly to the regional laboratory and smear microscopy and culture are performed. The four-member bacteriology team visits approximately 100 of these health centers and five regional laboratories that care for MDR-TB patients. In each health center the team records the smear test result on a paper sheet and in each regional laboratory the team records both the culture result and the smear result sent by the health center on a similar paper sheet. These sheets are then brought to a central office where the culture and smear results are verified, copied onto additional clinical and administrative forms, and then typed into the web-based Partners in Health Electronic Medical Record system (PIH-EMR).29 In Lima, the team makes at least bi-weekly visits to all 105 sites distributed over 2672 km2.

Figure 1.

Figure 1

Open in a new tab

Peruvian laboratory structure, and workflow of the bacteriology data collection team with the current paper system (white lines) and with the PDA-based system (red lines).

The major disadvantages of this paper-based method are the delays in processing and entering laboratory results, data quality issues stemming from multiple opportunities for transcription errors, and the heavy workload involved in the process. A preliminary study found that the mean time from the test result date to entry in the PIH-EMR was 55.3 days. A routine quality control examination found error rates as high as 10.1%, and the bacteriology team was consistently back-logged because of the increasing number of patients on treatment.

At the time of this study many of these laboratories and health centers had neither Internet nor an appropriate web-based laboratory information system; as such, a PDA-based system represented the most appropriate technology to improve the process of monitoring laboratory results. Since the implementation of this system in March 2006, the Peruvian Ministry of Health has expanded Internet access to an increasing number of health establishments, and a controlled trial of a web-based laboratory information system is currently underway.30

The study described here evaluated a PDA-based system that we implemented in an attempt to alleviate these problems.31 The specific aims of this study were: (1) to compare the processing time using the electronic system to the paper-based system; (2) to compare the frequency of errors entered with and without the electronic system; and (3) to assess the system's usability and its acceptability by users.

Methods

Intervention

We designed and implemented an electronic bacteriology collection system using low-cost PDAs (Palm Zire 31 and 21) as the initial point of data entry at the clinical site in an effort to decrease delay time and errors.31 These handhelds were chosen due to their low cost, small size, and monochrome screens, which extend battery life and, more importantly, allow them to be disguised for the security of team members. The commercial software Pendragon® Forms was used to create the PDA forms because it allowed for rapid prototyping of forms and was able to synchronize to the Oracle® database of the PIH-EMR29,32 through an open database connectivity (ODBC) connection over the Internet. Each form implemented was designed iteratively by the bacteriology team and the developer to ensure suitability for workflow.

Bacteriology team members using the new system visited a health center or laboratory and copied data directly from the laboratory register or chart using the PDA (Figure 2). Upon return to the central office, these users uploaded the data from the PDA to the PIH-EMR. A module was added to the PIH-EMR that permitted the automated processing of data and included web pages that displayed information in a tabular format identical to the previous paper forms. We found that users with low to moderate computer experience preferred this view because it allowed them to see all of the information at once in a familiar format. Additional web pages were created to perform the data quality checks previously done by the team and to transfer the data to the clinical section of the PIH-EMR, as can be seen in Figure 3. Errors, which prevented a result from being transferred to the clinical system, are in red text. Warnings, which still allow the transfer of the result, are in black text. Examples of these checks include:

  1. Smears not transferred to the clinical system after 14 days (Smears Past Due in Figure 3);

  2. Cultures not transferred to the clinical system after 70 days (Cultures Past Due in Figure 3);

  3. Reporting any missing data for a specimen (first error in Figure 3);

  4. Checking that every culture has a corresponding smear and every smear a corresponding culture (second error in Figure 3);

  5. Preventing duplicate entry of a test result;

  6. Alerting for any overdue smear or culture that has not been transferred.

Figure 2.

Figure 2

Open in a new tab

PDA form example.

Figure 3.

Figure 3

Open in a new tab

Decision support page within the web-based medical record system, PIH-EMR, to automate verification and cross-checking of smear and culture results collected by the bacteriology team.

The intervention was piloted for one year before the time—motion study began. This study was approved by the Partners Human Research Committee and the Peruvian National Institute of Health.

Study design

After collecting baseline data for 19 months from four of five health districts in Lima, Peru, we randomly assigned two to the intervention, while two were maintained as controls. During the intervention period, we collected data on the same endpoints in both control and intervention arms (Figure 4). This allowed us to perform a prospective comparison between the intervention and the control arms (between-districts comparison) as well as a historical comparison comparing each arm to itself before the intervention began (within-districts comparison). This complementary design using two comparison groups allowed us to minimize the risk that the changes measured were due to secular changes in the regions studied or to baseline differences between the arms. Since the potential sources of bias should be independent, observing similar effects in both comparisons should offer reassurance that our conclusions are valid.

Figure 4.

Figure 4

Open in a new tab

Cluster randomized controlled trial scheme with within-districts (before and after) and between-districts comparisons.

Processing time

We defined the processing time as the number of days from the bacteriology result date to its entry into the PIH-EMR (Figure 5). There were many activities that the bacteriology team had to perform during this time, including visiting the health establishment to collect the result, processing and verifying that result back at the office, and entering the data into the PIH-EMR for clinical and administrative use. Though the collection time, from the bacteriology result date to collection date, was not affected by the intervention, it was included in the processing time because it could not be separated in the retrospective data used for the within-districts comparison. We therefore analyzed the between-districts data and found that there was no statistically significant difference between the mean collection times in the intervention and the control districts. This allowed us to conclude that the time taken for the team to visit the health establishment and collect the result did not contribute to the difference in processing times seen in the between-districts comparison.

Figure 5.

Figure 5

Open in a new tab

Definition of processing time.

Collection errors

We defined a collection error as an occurrence of information entered into the PIH-EMR not matching the original laboratory notebook (gold standard). We recorded all variables collected for culture and smear microscopy. These included result date, identification number, result, and if the result was assigned to the wrong patient (misidentification errors).

We expected a decrease in all types of errors since data had to be entered only once in the PDA-based system compared to three times in the paper-based system. The additional forms in the paper-based system were necessary to organize the information for both clinical and administrative purposes. In the PDA-based system, these forms were placed online and generated automatically. Further, the PDA-based system had a full patient list from which the user could select a patient name. We believed that having this utility would reduce the number of misidentification errors since the users would not have to remember all active patients when they searched for results.

Usability and acceptability of the system

A survey that had previously been used in Peru,26 was administered to measure the usability and acceptability of the system. The survey was modified for our intervention, validated with other employees from our organization Partners In Health and Socios en Salud, and given to the bacteriology team. The responses were short answers or given on a five-point Likert scale anchored by 1 = very negative, 5 = very positive. The survey examined four themes: the amount of time each user spent collecting information, the amount of training required for each of the two systems, the effect of the PDA on the user's interaction with healthcare personnel, and the number of technical problems.

Data abstraction

For the between-districts comparison, we collected all culture and their respective smear microscopy results for the 6 months after the full implementation of the PDA-based system (result dates between March 24 and September 24, 2006).

For the within-districts before-and-after comparison, we collected all culture and smear microscopy results entered into the PIH-EMR during the routine operation of the bacteriology team before the intervention from January 1, 2004 to July 31, 2005. Two exclusion criteria were used and the quantities and percentage of results eliminated are shown in parenthesis: (1) the PIH-EMR entry date was before the result date (two smears 0.01%, 18 cultures 0.2%); (2) the processing time was greater than 1 year. This eliminated results collected during retrospective searches and not during routine collection (100 smears 0.8%, 223 cultures 2.1%).

We compared the data entered in the PIH-EMR with the original laboratory register by visiting each clinical site. Twenty-five percent of results were reviewed a second time and we found excellent agreement (99%) with the original data. All questions about errors were resolved by a consensus of one of the authors (JAB) and the bacteriology team.

Statistical analysis

To compare processing times, we used a mixed effects model33,34 to test for the fixed effect of intervention controlling for the period (pre- and post-implementation) effect. In our design, district was a random effect since the individual observations within the district might be correlated. Therefore, we compared the processing time between intervention and control groups adjusting for baseline levels. The response variable, processing time, was log-transformed as it had a right-skewed distribution. The intervention effect was tested with the period as a block in the model. The intracluster correlation coefficients (ICCs) calculated for culture and smear microscopies were 0.025 and 0.102, respectively. For the collection errors, we fit a generalized linear mixed model (GLMM)35,36 to test for the effect of the intervention, since response variables ‘collection error’ and ‘misidentification error’ were binary (1 for presence of error, 0 otherwise). In the second model, ICCs were 0.049 and 0.064.

As the collection processes differed for culture and smear microscopy, the analysis of processing time was done separately. Smear microscopy was usually performed at a local health center and the result communicated to a regional laboratory where the culture was performed. The smear microscopy data were collected from both locations and cross-checked before being entered into the PIH-EMR. Culture results were always collected from the regional laboratory. For the collection errors, both the process of extracting results from clinical settings and the variables collected were similar, so culture and smear microscopy results were combined. Additional data fields were implemented in the PDA system at the request of the users; however, these fields were not taken into account for the collection errors.

Results

Characteristics of the intervention and control districts are summarized in Table 1. The number of monthly results collected by the bacteriology team since 2004 (pre-intervention) increased for both sets of districts. The control districts had more health centers from which data were collected (58 vs. 35) and more monthly results collected (2255 vs. 785) compared with the intervention districts. The number of years working in the bacteriology team (mean 4.5 vs. 4.9 years) and years of internet experience (mean 4.3 vs. 4.6 years) were similar before and after the PDA-based system was implemented, primarily because three team members participated in all periods of the study.

Table 1.

Descriptive statistics of samples for study

Before
 After
Intervention districts Control districts Intervention districts Control districts
Smear microscopies for processing time 5846 6376 2791 3435
Cultures for processing time 4876 5954 2890 3263
Smears and cultures for collection errors 677 N/A 1112 970
Health centers from which data were collected 35 58 35 58
Mean monthly smear and culture results collected 315 460 785 2255
Mean years as team member 4.5 4.5 4.9 4.9
Mean years using Internet 4.3 4.3 4.6 4.6
Culture collection time (days)
    Mean 43.2 43.2 9.9 35.1
    Standard deviation 39.8 40.3 10.1 45.6
    Median 30.5 30.8 7.7 22.5
    IQR 35.2 41.5 7.7 26.1
Smear collection time (days)
    Mean 32.6 42.5 15.0 34.3
    Standard deviation 34.0 43.2 12.2 38.2
    Median 21.5 27.7 11.6 24.6
    IQR 30.1 40.7 11.3 19.8
Collection errors 4.3% (29/677) 2.6% (29/1112) 6.1% (59/970)
Misidentification errors 0.44% (3/677) 0.09% (1/1112) 0.62% (6/970)
Open in a new tab

IQR, interquartile range.

Bacteriology team member characteristics are identical within the before and after comparisons because users were the same and they rotated between the intervention and control districts.

Processing times

The effect of the intervention on processing time was highly significant for both culture and smear (p < 0.001, p < 0.001). In the random effects model for cultures, the period effect was also significant (p < 0.001) and the ICC was 0.025. For the smears the period was also significant (p < 0.001) but the ICC was slightly higher, 0.102.

The median culture processing time for the intervention districts was 65.8% less (7.7 vs. 22.5 days) in the between-districts comparison and 74.8% less (7.7 vs. 30.5 days) in the within-districts comparison (Figure 6a). For smears, the PDA-based system was associated with a 52.8% (11.6 vs. 24.6 days) reduction in delay measured in the between-districts study and 45.8% (11.6 vs. 21.5 days) in the within-districts study (Figure 6b). We also found that the control districts had a decrease in processing time for both cultures (22.5 vs. 30.8 days) and smears (24.6 vs. 27.7 days) after the PDA-based system was implemented in the intervention districts.

Figure 6.

Figure 6

Open in a new tab

Box plot for processing time of (a) cultures and (b) smears in log scale (left y-axis) and days (right y-axis). These show that for both culture and smear results there was a statistically significant decrease (p < 0.001) in the processing time with the PDA-based system (intervention districts after) compared to the same districts before the implementation (intervention districts before) and districts with the paper-based system (control districts after). The Kaplan—Meier survival curves for the initial 100 days for (c) culture and (d) smear microscopy show that the PDA-based system was able to drastically decrease the number of outlying results with a processing time of over 90 days.

Furthermore, the timing of data entry with the PDA-based system was more predictable than the paper-based system. The interquartile range (IQR) for culture processing time in the intervention districts (7.7 days) was smaller than that for the between-districts (26.1 days) and the within-districts (35.2 days) comparisons. This effect was also observed for the smear microscopy results (11.3 vs. 19.8 and 30.1 days, respectively).

Finally, this system was able to almost eliminate outliers defined as processing time of over 90 days (Figure 6, c and d). At baseline, 9.2% and 8.2% of cultures had a processing time of at least this long for the intervention and control districts, respectively. This decreased to 0.1% in the intervention district post-implementation compared to 5.4% in the control district post-implementation. The same phenomenon was observed for smear results where the pre-implementation values were 6.0% and 9.1% for the intervention and control districts, respectively, and they decreased to 0.1% and 4.8% post-implementation.

Collection errors

After fitting GLMMs, we found that the intervention had a significant effect on the total frequency of collection errors (p = 0.005); the fraction by which errors were reduced was 57.1% for the between-districts comparison and 39.1% for the within-districts comparison. The proportion of results with errors in the intervention districts was 2.6% (29/1112 results) compared to 6.1% (59/970 results) and 4.3% (29/677 samples) in the control districts and the baseline intervention districts, respectively.

Despite finding 80−85% fewer results with misidentification errors in intervention districts for both the between-districts and within-districts comparisons, we could not conclude that the intervention significantly lowered the frequency of this serious type of error (p = 0.074). This is largely attributable to small numbers of these types of errors overall; intervention districts had an error rate of 0.09% (1/1112 samples) compared to 0.62% (6/970 samples) in the control districts and 0.44% (3/677 samples) in the baseline data for the intervention districts. Unlike processing time, the period effect was not found to be significant for either type of error (p = 0.554, p = 0.064).

Usability and acceptability of the system

The user feedback for the electronic system was positive, with all four users preferring the PDA-based to the paper-based system. After less than five days of practice, each of the users became comfortable with using the PDA to enter information. The users noted the ability to quickly verify and transfer results electronically instead of working with large quantities of paper and to access an updated patient list automatically uploaded to the PDA instead of having to manually create it every week, as favorite features of the electronic system. The users requested that the study be concluded early so that the system could be expanded to all health districts in Lima as soon as possible.

All users found it easier to learn to use the PDA (mean 4.0 out of 5) than the paper-based system (mean 3.5 out of 5), to collect results with the PDA (4.5 vs. 3.5), and to process results (4.75 vs. 3.0). All users said that the intervention affected their relationship with the local health center personnel in a positive or very positive way. Two of the users expressed that it improved their relationship because it seemed more professional and they could explain its use.

Discussion

Many organizations must collect information from a population that is distributed over a large area. In a previous publication, we reported on the design and implementation of a PDA-based system to collect TB bacteriological data from many institutions.31 In this full evaluation, we found that the use of this system was associated with a substantial reduction in the delays from collection to entry of laboratory results, a decreased frequency of errors, and a reduced workload for those involved in data collection and processing.

This system was able to reduce the median processing time by 46−74% depending on the type of result and comparison. Also, the intervention was able to almost eliminate large delays of over 3 months from 6−9% to 0.1%. Though a baseline comparison was not possible for the errors, the intervention districts had 39% and 57% fewer errors than the baseline intervention and control districts, respectively. We believe this improvement resulted from eliminating manual data entry and providing electronic verification tools. Finally, the intervention lowered the person-hours spent processing and verifying data and was well-liked by users. One user wrote “With the paper system our work was always late. With the PDA system our work is up to date”. Providing more timely and accurate bacteriology data to clinicians should allow them to monitor their patients better and reduce the amount of time that patients are infectious. This is the first quantitative evaluation showing that a user-friendly PDA-based system to collect data in resource-poor settings can significantly reduce processing time, decrease the frequency of collection errors, and lower the effort required for processing.

We also found that the control districts had a decrease in the mean delay of 27% for cultures and 11% for smears compared to the pre-intervention delay. In reviewing the results with the team, they asserted that the main reason for this decrease was that they had more time to work in the control districts because their workload in the intervention districts was reduced.

Another possible measure of the success of this system is its continued use and expansion.37 After the study period, the PDA-based system was transferred to our Peruvian partners. This process consisted of training their technical team and providing monthly technical advice via telephone. At the request of the users, they have expanded the system to the control districts, one additional district in Lima, and five provinces of Peru. Additionally, four new users have been recruited to work with the leaders of the bacteriology team preparing and performing the training. Finally, at the request of the clinical staff, the same system is currently being extended to incorporate the collection of patient weight and height data. All activities and costs for these additional activities have been published elsewhere.37

Limitations of the study

Though this study was small with four users in four health districts, the use of dual comparison groups (between-districts and within-districts before-and-after) helped us to minimize potential biases due to secular trends and baseline between-district differences. Further, we took other steps to reduce sources of bias by rotating the users of the system and ensuring that no other changes in collection were made during the study. Finally, this was a formative rather than summative evaluation since the developers were involved, though the expansion and continued maintenance of the system by local staff independent of the original developer shows its sustainability.

Future studies

This first evaluation assessing the ability of PDAs to improve timeliness in resource-poor settings emphasizes the need for further research as to the methodology and best practices in designing and implementing such systems, so that recommendations can be made for the optimal system depending on the setting and infrastructure. Further evaluations should also assess costs, including comparisons of development and maintenance of electronic and paper systems. This is especially important for organizations, domestic or international, working with scarce resources.

Conclusions

This study shows that a well-designed PDA-based system can provide great improvements in community data collection for clinical and administrative purposes, even in resource-poor settings. These systems can provide higher quality data with fewer communication delays and person-hours required, though the effort, time, and attention to detail required to create these systems must be taken into account. These benefits might also be seen in the use of cellular phones, especially smart phones. However, their user interface and connectivity with a larger records system must be studied further. Organizations working at the community level or requiring data from institutions spread over a large area should consider the advantages of using mobile data collection systems.

Acknowledgments

We thank the users (Mayra Napa, Yrene Torres, Veronica Albitres, and Briam Chavez), Darius Jazayeri, Michael Seaton, and Wilmer Gomez for technical assistance, Walter Curioso, Jaime Bayona, Ynes Pereda, and Carole Mitnick for assistance in the study, and Neal Lesh, Claire Mack, and Brian DeRenzi for reviewing the manuscript. We also thank Satellife and the South African MRC for their helpful suggestions. We thank the Gates Foundation for their support in the development of the PIH-EMR and the MIT Graduate Students Office for the Albert Memorial Fellowship to JAB. Sponsors had no involvement in the study.

Footnotes

Conflict of interest: No conflict of interest to declare.

References

  • 1.Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application. Int J Med Inform. 2005;74:519–26. doi: 10.1016/j.ijmedinf.2005.01.003. [DOI] [PubMed] [Google Scholar]
  • 2.Carroll AE, Saluja S, Tarczy-Hornoch P. Development of a personal digital assistant (PDA) based client/server NICU patient data and charting system. Proc AMIA Symp. 2001:100–4. [PMC free article] [PubMed] [Google Scholar]
  • 3.Carroll AE, Saluja S, Tarczy-Hornoch P. The implementation of a personal digital assistant (PDA) based patient record and charting system: lessons learned. Proc AMIA Symp. 2002:111–5. [PMC free article] [PubMed] [Google Scholar]
  • 4.Thomas SM, Overhage JM, Warvel J, McDonald CJ. A comparison of a printed patient summary document with its electronic equivalent: early results. Proc AMIA Symp. 2001:701–5. [PMC free article] [PubMed] [Google Scholar]
  • 5.Caro JJ, Sr, Caro I, Caro J, Wouters F, Juniper EF. Does electronic implementation of questionnaires used in asthma alter responses compared to paper implementation? Qual Life Res. 2001;10:683–91. doi: 10.1023/a:1013811109820. [DOI] [PubMed] [Google Scholar]
  • 6.Jamison RN, Raymond SA, Levine JG, Slawsby EA, Nedeljkovic SS, Katz NP. Electronic diaries for monitoring chronic pain: 1-year validation study. Pain. 2001;91:277–85. doi: 10.1016/S0304-3959(00)00450-4. [DOI] [PubMed] [Google Scholar]
  • 7.Hyland ME, Kenyon CA, Allen R, Howarth P. Diary keeping in asthma: comparison of written and electronic methods. BMJ. 1993;306:487–9. doi: 10.1136/bmj.306.6876.487. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Saleh KJ, Radosevich DM, Kassim RA, Moussa M, Dykes D, Bottolfson H, et al. Comparison of commonly used orthopaedic outcome measures using palm-top computers and paper surveys. J Orthop Res. 2002;20:1146–51. doi: 10.1016/S0736-0266(02)00059-1. [DOI] [PubMed] [Google Scholar]
  • 9.Lal SO, Smith FW, Davis JP, Castro HY, Smith DW, Chinkes DL, et al. Palm computer demonstrates a fast and accurate means of burn data collection. J Burn Care Rehabil. 2000;21:559–61. doi: 10.1097/00004630-200021060-00015. discussion 558. [DOI] [PubMed] [Google Scholar]
  • 10.Yon BA, Johnson RK, Harvey-Berino J, Gold BC. The use of a personal digital assistant for dietary self-monitoring does not improve the validity of self-reports of energy intake. J Am Diet Assoc. 2006;106:1256–9. doi: 10.1016/j.jada.2006.05.004. [DOI] [PubMed] [Google Scholar]
  • 11.van Gerven JM, Schoemaker RC, Jacobs LD, Reints A, Ouwersloot-van der Meij MJ, Hoedemaker HG, et al. Self-medication of a single headache episode with ketoprofen, ibuprofen or placebo, home-monitored with an electronic patient diary. Br J Clin Pharmacol. 1996;42:475–81. doi: 10.1046/j.1365-2125.1996.43613.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Buck DS, Rochon D, Turley JP. Taking it to the streets: recording medical outreach data on personal digital assistants. Comput Inform Nurs. 2005;23:250–5. doi: 10.1097/00024665-200509000-00008. [DOI] [PubMed] [Google Scholar]
  • 13.Kvien TK, Mowinckel P, Heiberg T, Dammann KL, Dale O, Aanerud GJ, et al. Performance of health status measures with a pen based personal digital assistant. Ann Rheum Dis. 2005;64:1480–4. doi: 10.1136/ard.2004.030437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Fletcher LA, Erickson DJ, Toomey TL, Wagenaar AC. Handheld computers. A feasible alternative to paper forms for field data collection. Eval Rev. 2003;27:165–78. doi: 10.1177/0193841X02250527. [DOI] [PubMed] [Google Scholar]
  • 15.Bridges.org. Evaluation of the SATELLIFE PDA Project, 2002: testing the use of handheld computers for healthcare in Ghana, Uganda, and Kenya. Satellife; Boston, MA: 2003. [Google Scholar]
  • 16.Yasin Z, Choi S, Fraser H. Improving access to TB medical records in remote clinics in Peru using a personal digital assistant based application. Proc AMIA Symp. 2002:1207. [Google Scholar]
  • 17.Sato dos Santos G, Cajigas Gonzalez I, Gomez-Uribe C, Ohno-Machado L. Remote health surveillance: a case study using PDAs and GPS.. The 2nd International Conference on Open Collaborative Design for Sustainable Innovation (dyd02); Bangalore, India. December 2002. [Google Scholar]
  • 18.Anantraman V, Mikkelsen T, Khilnani R, Kumar VS, Pentland A, Ohno-Machado L. Open source handheld-based EMR for para-medics working in rural areas. Proc AMIA Symp. 2002:12–6. [PMC free article] [PubMed] [Google Scholar]
  • 19.Diero L, Rotich JK, Bii J, Mamlin BW, Einterz RM, Kalamai IZ, et al. A computer-based medical record system and personal digital assistants to assess and follow patients with respiratory tract infections visiting a rural Kenyan health centre. BMC Med Inform Decis Mak. 2006;6:21. doi: 10.1186/1472-6947-6-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Missinou MA, Olola CH, Issifou S, Matsiegui PB, Adegnika AA, Borrmann S, et al. Piloting paperless data entry for clinical research in Africa. Am J Trop Med Hyg. 2005;72:301–3. [PubMed] [Google Scholar]
  • 21.Gupta PC. Survey of sociodemographic characteristics of tobacco use among 99,598 individuals in Bombay, India using handheld computers. Tob Control. 1996;5:114–20. doi: 10.1136/tc.5.2.114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Selanikio JD, Kemmer TM, Bovill M, Geisler K. Mobile computing in the humanitarian assistance setting: an introduction and some first steps. J Med Syst. 2002;26:113–25. doi: 10.1023/a:1014853825636. [DOI] [PubMed] [Google Scholar]
  • 23.Shirima K, Mukasa O, Schellenberg JA, Manzi F, John D, Mushi A, et al. The use of personal digital assistants for data entry at the point of collection in a large household survey in southern Tanzania. Emerg Themes Epidemiol. 2007;4:5. doi: 10.1186/1742-7622-4-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Satellife . Handhelds for health: Satellife's experiences in Africa and Asia. Satellife; Boston, MA: 2005. [Google Scholar]
  • 25.Jaspan HB, Flisher AJ, Myer L, Mathews C, Seebregts C, Berwick JR, et al. Methods for collecting sexual behaviour information from South African adolescents—a comparison of paper versus personal digital assistant questionnaires. J Adolesc. 2007;30:353–9. doi: 10.1016/j.adolescence.2006.11.002. [DOI] [PubMed] [Google Scholar]
  • 26.Curioso W, Karras B, Campos P, Buendia C, Holmes K, Kimball A. Design and implementation of Cell-PREVEN: a real-time surveil-lance system for adverse events using cell phones in Peru. AMIA Annu Symp Proc. 2005:176–80. [PMC free article] [PubMed] [Google Scholar]
  • 27.Parikh TS. Using mobile phones for secure, distributed document processing in the developing world. IEEE Pervasive Computing. 2005;4:74–81. [Google Scholar]
  • 28.World Health Organization . Treatment of tuberculosis: guidelines for national programmes. WHO/CDS/TB/2003.313. World Health Organization; Geneva: 2003. [Google Scholar]
  • 29.Fraser H, Jazayeri D, Mitnick C, Mukherjee J, Bayona J. Informatics tools to monitor progress and outcomes of patients with drug resistant tuberculosis in Peru. Proc AMIA Symp. 2002:270–4. [PMC free article] [PubMed] [Google Scholar]
  • 30.Blaya JA, Shin SS, Yagui MJ, Yale G, Suarez CZ, Asencios LL, et al. A web-based laboratory information system to improve quality of care of tuberculosis patients in Peru: functional requirements, implementation and usage statistics. BMC Med Inform Decis Mak. 2007;7:33. doi: 10.1186/1472-6947-7-33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Blaya J, Fraser HS. Development, implementation and preliminary study of a PDA-based tuberculosis result collection system. AMIA Annu Symp Proc. 2006:41–5. [PMC free article] [PubMed] [Google Scholar]
  • 32.Fraser H, Blaya J, Choi S, Bonilla C, Jazayeri D. Evaluating the impact and costs of deploying an electronic medical record system to support TB treatment in Peru. AMIA Annu Symp Proc. 2006:264–8. [PMC free article] [PubMed] [Google Scholar]
  • 33.Campbell MJ. Cluster randomized trials in general (family) practice research. Stat Methods Med Res. 2000;9:81–94. doi: 10.1177/096228020000900202. [DOI] [PubMed] [Google Scholar]
  • 34.Pinheiro J, Bates D. Mixed effects models in S and S-plus. Springer; New York: 2002. [Google Scholar]
  • 35.Wolfinger R, O'Connell M. Generalized linear mixed models: a pseudo-likelihood approach. J Stat Comput Sim. 1993;48:233–43. [Google Scholar]
  • 36.Schall R. Estimation in generalized linear models with random effects. Biometrika. 1991;78:719–27. [Google Scholar]
  • 37.Blaya J, Rodriguez P, Fraser H. Cost and implementation analysis of a personal digital assistant system for laboratory data collection. Int J Tuberc Lung Dis. 2008;12:921–7. [PubMed] [Google Scholar]

RESOURCES