Evaluating children’s location using a personal GPS logging instrument: limitations and lessons learned

Abstract

Global positioning system (GPS) technology is increasingly used to assess geographically varying exposure in population studies. However, there has been limited evaluation of accuracy and completeness of personal GPS data. The ability of a GPS data logger to assess location of children during usual activity was evaluated. Data collected for 4 days from 17 children wearing GPS loggers, recorded every 15 s, were evaluated for completeness by time of day during weekend and weekdays, and for accuracy during nighttime at home. Percentage of possible GPS-recorded points and of 5-min intervals with at least one recorded location were examined. Mean percentage of total possible 15-s interval locations recorded daily was less than 30%. Across participants, the GPS loggers recorded 1–47% of total possible location points on weekends and 1–55% on weekdays. More complete data were measured during travel to school (average 91%). The percentage of daily 5-min intervals with recorded data was as high as 53%. At least one location was recorded during 69% of 5-min intervals before school (0630–0800 h), 62% during school (0800–1400 h) and 56% after school (1400–1700 h). During night time (0000–0600 h), on average, location was recorded for less than 25% of 5-min intervals and accuracy was poor. The large proportion of missing data limits the usefulness of GPS logging instruments for population studies. They have potential utility for assessing on-road travel time and route. GPS technology has limitations, and lessons learned from this evaluation can be generalized to the use of GPS in other research settings.

INTRODUCTION

Epidemiological studies often need to define the places in which participants spend time and the amount of time spent in a specific location. Characterizing time and location is important for assessing determinants of physical activity, effects of the built environment and air pollution exposure among other outcomes.^1–4 The time–location information can be assessed with self-reporting methods such as diaries and questionnaires.^5,6 However, relying on self-report has well-recognized limitations such as inaccurate recall and limited compliance, and these limitations may be greater in children.^7,8 With improvements in technology, the use of global positioning systems (GPS) provide an objective method for time–location assessment. Portable GPS devices with memory to store substantial amounts of data over time are now available at relatively low cost, suitable for large population studies. Compared with self-report, GPS devices have reduced participant burden and have great potential for collecting a continuous record of accurate information on location over time.

GPS has been shown to be useful for research in fields ranging from navigation to farming.¹ Increasingly, GPS has been used in studies of transportation, physical activity and time–location tracking to assess determinants of environmental exposures.^9–12 In physical activity research GPS has the potential for determining the spatial context of activity, factors such as distance traveled and speed, and there is potential for GPS data to augment information from accelerometer-measured physical activity.^1,13,14 Personal GPS-logging instruments have also been used in the assessment of air pollution exposure.^15,16

More accurate and more complete GPS data may be required for some outcomes and not others. For example, to distinguish indoor from outdoor locations requires accuracy to within less than a few meters, whereas to show that a subject is in a park requires accuracy only to tens of meters or greater. Similarly, relatively complete data are required to identify the geographic context of activities that occur infrequently, such as vigorous physical activity in a relatively sedentary population. We have previously evaluated the impact of operating characteristics and different environments on accuracy, missing data and time to signal acquisition of several commercially available GPS loggers.¹⁷ In scripted activities we found that performance varied markedly for all instruments by outdoor and indoor locations, and by building construction materials and height. We have now evaluated the accuracy and signal capture of a single brand of GPS logger to accurately assign time–location to a sample of children under conditions that might be common during usual activities in a population-based study.

METHODS

Study Population

Participants were recruited from two 4th grade classrooms at a school in Palos Verdes, CA, to wear GPS loggers for 5 days. Forty students were eligible to participate, 18 (45%) agreed and returned signed parental informed consent. However, one participant joined on the final day after observing classmates participating (and this child was not included in the analysis). Of the remaining 17 contributing to analyses, three joined on the third day of the study.

Study Area Features

Participants lived in suburban areas in and around Palos Verdes, CA. Most lived in single-family homes with no surrounding tall buildings. The participants’ classrooms were also in a suburban area in a one-storey building. There were no large buildings surrounding the school parcel.

Sampling Period and Protocol

The GPS loggers were set to record location every 15 s. The devices were turned on before giving them to the study participants and the participants were asked not to turn the loggers off or on. They were instructed to charge the instrument every night. Participants received a reminder flyer to put on their refrigerators, and parents were called twice during the study period to remind them to encourage the child to charge the instrument.

Participants were asked to wear the GPS devices everyday during waking hours, except while bathing or swimming, during the study period. Devices were distributed on Friday, 2 May 2008 and returned the following Wednesday morning. Data were analyzed starting at Friday, 2000 h.

Specifications of Instrument

We used Wintec WBT-201 devices in our study (Figure 1a). We selected this instrument because of its size, cost, battery life and data storage capability, and because it had previously been used in an epidemiological study of a large group of children with asthma.¹⁸ The key to selection of this instrument was the small size (6 × 3.8 × 1.6 cm, weight 48 g) that enabled children to easily carry it on a belt. Pretesting demonstrated comparable reception and accuracy when carried in a small pouch on the belt, in a pocket, purse or backpack. Children were willing to carry it all day at school; the instrument is of moderate cost ($75 per unit), which means that loss of the instrument would not have major economic impact even with a larger population study. We have previously compared the Wintec BT-201 with other GPS personal data logging instruments available at the time this study was undertaken and found it to have similar performances to those instruments.¹⁷ In order to address inter-unit variability, all Wintec BT-201 loggers used in the current study were evaluated for accuracy by driving along a scripted route in a suburban area, along main roads and side streets, before distribution to study participants. All instruments had similar signal capture and accuracy (results not shown). After charging, a piece of Styrofoam was then taped over the device’s power button to prevent children from turning it off, and the covered device was placed in a small pouch and attached to a belt before distribution (Figure 1b). The manufacturer reports that the horizontal position accuracy is 1–5 m when it can perform a wide-area augmentation system correction, and otherwise is ~ 10 m. The device stores up to 131,072 waypoints (recorded locations), allowing ~ 22 days of data archiving with 15-s recording epochs. The data output includes latitude, longitude, date, time, speed and elevation. A limitation to the instrument was a battery life of 15 h; hence, daily recharging was necessary to obtain multi-day information.

Figure 1.

(a) The Wintec WBT-201, a wearable global positioning system (GPS) data logger. (b) Wintec WBT-201, covered with a piece of Styrofoam to prevent tampering, and belt pouch.

Data Processing

GPS data were downloaded using Time Machine X programming software provided by the Wintec manufacturer. The output format of the GPS logger is TK files, and these files can be converted to KMZ (for viewing in Google Earth) and GPX (GPS eXchange file) formats. The data were exported to Esri’s ArcGIS software using SAFE’s FME software to convert the GPX data to shape files. By viewing the attribute tables in ArcGIS, we identified errors based on location (e.g., x and y coordinates, placing the child and accompanying GPS unit in the Pacific Ocean), as well as improbable changes in elevation or speed (Figure 2). The elevation errors were identified from individual waypoint elevations (e.g., a child and GPS unit at 10,735 m above sea level) or rapid changes in elevation (> 300 m) between consecutive waypoints spanning short distances (<500 m) or unrealistically rapid movement between consecutive waypoints (e.g., 300 km/h). These erroneous waypoints were then deleted from the data set.

Figure 2.

Representation of global positioning system (GPS) raw data, showing example of data with travel between locations on land and well into the ocean between 15-s recordings.

For each subject we calculated the number of 15-s interval waypoints recorded each day and the total number of waypoints recorded during the study period. We also calculated the proportion of all 5-min intervals with at least one GPS location reading. Time and location point data were assessed by day (over each 24 h period from midnight to midnight) and by periods of interest during each day (overnight 2000–0630 h, travel to school 0630–0800 h, at school 0800–1400 h, after school 1400–1700 h and evening 1700–2000 h). In addition, we examined overnight data between 0000 and 0600 h. Review of these nighttime data, for which we assumed the instrument would have been at each participant’s home in a single location, demonstrated high variability in recorded location; hence, locations were determined by averaging location data over 5-min rolling averages (e.g., 0100–0105 h, 0101–0106 h, etc.) to determine whether more accurate location data could be obtained. Finally, we required a 5-min period-specific 50% data completeness threshold for inclusion in this analysis. For each 5-min rolling average, there should have been 20 waypoints logged; hence, if less than 10 logged points were recorded, that interval was not assigned a location. Using this approach, we evaluated whether the precision and accuracy of location was improved by excluding intervals with less complete data.

Geocoding Addresses

Participants’ home addresses and the address of the school were geocoded using the USC Geocoder.¹⁹ This research platform incorporates the standard components found within typical geocoding system architectures, including the representation and storage of reference data layers, a feature matching algorithm based on a deterministic candidate scoring scheme, and a set of feature interpolation algorithms.²⁰ GPS data were overlaid with census parcel data to view locations recorded within home and school parcels.

Data Analysis

To summarize data, percent possible GPS waypoint recordings for each participant were calculated by dividing the number of 15-s waypoint readings recorded by the total number possible and multiplying by 100 ((waypoints recorded/possible recordings) × 100). Each 24 h period had 5760 total possible GPS waypoints (1440 min × 4 records per minute). Percent possible waypoints were calculated for each study participant by day and by time of day, and for the total study period. We performed similar calculations for each study period (e.g., 0630–0800 h specific data) and calculated the percent possible time monitored ((time monitored in minutes/possible time in minutes) × 100). We used analysis of variance to investigate heterogeneity of percent possible waypoints or percent possible 5-min time intervals by day of the week, time period, or by participant. All analyses were performed using SAS version 9.2 (SAS Institute, Cary, NC, USA).

RESULTS

Fourteen of the 17 participants included in the analysis had GPS units for all study days. However, of the 14 participants 1 lost the instrument and 1 reported a malfunctioning device, and these children were provided new units on day 3 (Monday morning). Three additional children enrolled on day 3. Therefore, there were 12 participants with data logged for the entire study period and 5 participants with data for 2 days beginning Monday.

There was substantial missing data (Figure 3). Of the 5760 possible daily recordings, the mean percent waypoints per day across study participants was less than 30% for each study day. Across subjects, on weekends the GPS loggers recorded from 1 to 47% of the total. A similar range was recorded on weekdays, from 1 to 55%. Sunday had the lowest percent waypoints recorded, with a mean of 16% compared with 26–28% for the other days. The heterogeneity of percent waypoints recorded across days was significant (P < 0.001).

Figure 3.

Percent possible waypoints recorded by day. Note: 24 h day: 0000 h day 1 to 0000 h day 2; boxplot of median, interquartile range, maximum and minimum; “+” is mean.

The proportion of the day for which at least one location was recorded in each 5-min interval was higher (Figure 4). Mean percentages for Saturday and Sunday with 40% and 31%, respectively, were lower than mean percentages for Monday and Tuesday with 53% and 41%, respectively, (across days P < 0.001). There was considerable variability between participants on any given day, from 0% (a participant with a day of missing data) to 93%.

Figure 4.

Percent possible 5-min intervals with at least one location recorded, by day. Note: 24 h day: 0000 h day 1 to 0000 h day 2; boxplot of median, interquartile range, maximum and minimum; “ +” is mean.

Completeness of at least one recording in each 5-min interval varied markedly by time of day. On weekdays (Figure 5a), the data were most complete between 0630 and 0800 h (69%), during school hours from 0800 to 1400 h (62%) and during afternoon from 1400 to 1700 h (57%), and were least complete from 2000 to 0630 h (28%; P < 0.001 for heterogeneity across time of day). On the weekend (Figure 5b), data were most complete for 1700–2000 h (58%) and least complete for 0630–0800 h (20%), although the period from 2000 to 0630 h was almost as incomplete (21%; P < 0.001). There was substantial between-participant variability in completeness of data regardless of the time of day (generally ranging from 0 to almost 100%).

Figure 5.

(a) Percent possible 5-min intervals recorded with at least one location, weekdays. Note: 2000 h Sunday to 0800 h Wednesday; boxplot of median, interquartile range, maximum and minimum; “+” is mean. (b) Percent possible 5-min intervals with at least one location recorded, weekend. Note: Friday 2000 h to Sunday 2000 h; boxplot of median, interquartile range, maximum and minimum; “+” is mean.

Data were least complete between 0000 and 0600 h, both for all possible 15-s waypoints (Figure 6a), on average less than 12% (range 0–62%), and for at least one recording in each 5-min interval (Figure 6b), on average less than 25% (range 0–74%). Some participants had a large number of waypoints outside the home parcel, and poor precision and accuracy to the location during the 0000 to 0600 h period when the GPS loggers should have been stationary and charging (see, e.g., Figure 7a showing scatter outside of the residential parcel and into the next block). The accuracy and precision for this instrument’s location was not improved by using the 5-min rolling average location (Figure 7b at the same residence). Using the 5-min rolling average with a 50% data completion threshold resulted in no waypoints located in the parcel (Figure 7c). There was, however, considerable variability between children, with waypoints for some children located primarily within the home parcel during the night (Figure 8a), and this precision was generally improved by taking the 5-min rolling average (Figure 8b) and further improved using the 5-min rolling average with a 50% data completion threshold (Figure 8c). However, there were no children for whom accuracy and precision were sufficient to locate the child within the home (not just within the parcel).

Figure 6.

(a) Percent possible waypoints recorded 0000–0600 h. Note: boxplot of median, interquartile range, maximum and minimum; “+” is mean. (b) Percent possible 5-min time intervals recorded with at least one location, 0000–0600 h. Note: boxplot of median, interquartile range, maximum and minimum; “+” is mean.

Figure 7.

(a) Nighttime (0000–0600 h) 15-s location recordings with a large amount of scatter outside a residential parcel (shaded). (b) Location recording when restricted to 5-min rolling average during overnight hours. (c) Location recording with 5-minute rolling averages and a 50% data threshold.

Figure 8.

(a) Nighttime (0000–0600 h) 15-s interval locations with few points outside the home parcel (shaded). (b) Improved location recording when restricted to the 5-min rolling average during overnight hours. (c) Precision and accuracy was further improved when restricted to 5-min rolling averages with a 50% data threshold.

During the school day, recorded locations showed considerable scatter, but these were almost exclusively within the school boundaries, which were relatively large (see, e.g., Figure 9).

Figure 9.

A study participant’s locations recorded while at school.

The 0630–0800 h weekday period with the most complete data collection (Figure 5a) included the morning commute to school for each child. In order to understand whether time spent in transit was more complete than other activities, we examined each participant’s data individually for the trip from home to school. We calculated GPS-recorded location completeness during travel to school by dividing the number of 15-s interval locations recorded during travel by the total number of intervals spent in transit. On average, across participants, the GPS recorded location for 91% of the transit time (SD = 16, range 31–100%). On-road travel route was generally readily identified even if there were some waypoints that were not recorded (Figure 10a and b).

Figure 10.

(a) Typical freeway travel record. (b) Travel path approaching school (shaded) on small roadways for one participant.

We investigated whether the variability in missing data was consistently accounted for by specific individuals. Across complete study days (N = 4 for this analysis), there was considerable variability in the percent of time recorded for each participant (Figure 11). For example, participant 1 had a range of 23–90% time recorded across the four study days. Participants 2 and 3 had less variability across the study days (20–39% and 65–78%, respectively; the P-value for heterogeneity between subjects was 0.05).

Figure 11.

Percent of possible 5-min time intervals with at least one location recorded by subjects, across days. Note: restricted to participants with 4 days of data collection: Saturday, Sunday, Monday and Tuesday; boxplot of median, interquartile range, maximum and minimum; “+” is mean.

DISCUSSION

In this evaluation of time and location patterns using a commercially available low-cost GPS logger, we observed a large proportion of time with missing data among children engaged in their usual pattern of activities. An important observation was the variability in completeness of data in different environments and at different times of day. A notable exception to the generally large proportion of missing data was during time spent in on-road travel, for which data were, on average, greater than 90% complete. This made it possible to identify an individual route traveled by each participant (Figures 10a and b), even where data were missing for short periods. For other locations (e.g., at school and after school), it was possible to identify a larger proportion of time during which a child’s location could be assigned by requiring only one data point in each 5-min interval. However, during school hours relatively imprecise GPS data provide little additional information beyond that provided by school attendance records. In addition, there was large variability in record completeness both between-child (within day) and within-child (across day). The worst data completeness was during night time, when children were at home with the instrument charging. Precision and accuracy of measurement was poor during this period; it was improved in some cases by using a rolling 5-min average of location, but was still not sufficiently accurate to distinguish indoor location from outdoor location within the home parcel.

An important implication of our results is the potential bias that could result from missing data that varies by time of day and location. Valid information about the proportion of time spent in different locations and activities could be obtained from the GPS logger if missing data were randomly distributed over the day, but could bias inference about where children spend time if data were not missing at random.²¹ For example, accelerometers assessing physical activity have been used with GPS loggers to evaluate the location and context of physical activity. Requiring accelerometry-recorded physical activity data to be matched to GPS location will result in incomplete data (based on our findings). If physical activity occurs indoors at a gymnasium where there is poor signal capture, the physical activity data will not have matched GPS data, and exclusion of these data could result in an underestimation of the role of indoor gymnasium time in physical activity. The reported proportion of missing matched-accelerometer and GPS data in selected physical activity studies has been variable, for example, from 28 to 76%.^9,10 Another scenario in which the use of GPS loggers could result in biased results is in the assessment of exposure to air pollution, which is known to vary markedly by proximity to traffic corridors and time of day.^22–26 Our results indicate that any exposure assignment made based simply on GPS-recorded location could overestimate the contribution of heavy on-road exposure (with little missing information) to the total time sampled and underestimate the contribution of exposure at home at night.

We considered whether the large and variable proportion of missing data might be specific to the instrument we selected. Although this is possible, our previous evaluation of the Wintec BT-201 and six other commercially available instruments undergoing scripted challenges did not identify any device to have consistently superior performance with regard to spatial accuracy and signal loss.¹⁷ Equipment malfunction or discharged batteries could result in no location data being recorded, but it is unlikely that all instruments in our study malfunctioned. Using any data in each 5-min interval as a criterion for non-missingness fairly markedly improved the percentage of time for which data were available. This would not have occurred if continuous periods with an uncharged battery had accounted for these missing data periods. We also observed a high percentage of missing data during the 0000–0600 h when participants were highly likely to have been at home. If the devices were plugged in for charging, loss of battery life should not have been the cause of nighttime missing data.

We previously found that microenvironment was an important determinant of signal loss and accuracy across various instruments.¹⁷ For example, all but one of seven instruments we evaluated had high signal-loss rates in a large concrete masonry building. Signal loss is known to occur because of interference in the satellite signals in concrete or steel buildings, and these results were consistent with the 99% signal loss in a windowless room in a concrete building in another study.²⁷ Multi-path errors that arise from the reflection of satellite signals from other surfaces, including buildings, cars, trees, the ground or water may also contribute to missing or inaccurate data.²⁸ When the available satellites are in one part of the sky or nearly lined up (relative to the GPS antenna), the precision of the GPS is reduced. Embankments or buildings that partially block the line-of-sight of the GPS unit can reduce the accuracy of the location measurement. Therefore, time spent outside is likely to have more complete information than time spent indoors. When the GPS has open sky and receives signals from satellites that are dispersed in elevation and angular distance, the positional coordinates of the GPS have the greatest accuracy.³ Therefore, GPS may perform well during on-road travel, because there is a view of the open sky and less signal interference. Even if an obstruction results in loss of signal, as the vehicle moves it will be likely to pass through locations with open sky where a waypoint will be recorded accurately. Tall buildings in central cities may also interfere with signals, and this has been called the “urban canyon” effect.²⁹

Our poor nighttime signal capture when the device was assumed to be indoors is consistent with other studies, as a generic limitation to current GPS instrumentation is in distinguishing time spent indoors from outdoor time. Other investigators have interpolated prolonged periods with missing signal to indoor locations, based on the location of the last previously recorded location.^5,14 In a refinement of this approach, time with no signal was classified as time indoors at home, if signal loss occurred at home, unless the subsequent first recorded location was greater than 1 km from the residence (suggesting that the signal was lost for some other reason than having gone indoors).⁸ However, in our data it was common for the next location recorded after a prolonged loss of signal to be many kilometers away from the previously recorded location, probably because when GPS devices lose their signal, it takes time to reinitialize (in order to find the orbit and clock data for the relevant satellites). This loss of signal can be brief (e.g., when driving through a tunnel), but after a prolonged loss of signal can take up to 13 min to reinitialize even with a clear satellite view, according to the Wintec WBT-201 user’s manual. We previously found the Wintec BT-201 to have longer signal acquisition time, particularly after a cold start, than some other instruments evaluated.¹⁷ Therefore, it is possible that instruments with more rapid signal acquisition would be better suited to innovative approaches to interpolating indoor location, but we concluded that interpolating missing GPS data collected with the Wintec BT-201 would require judgment and assumptions that may result in location misclassification. To automate this procedure in a way that could be applied in large population studies would entail considerable uncertainty in location assessment.

The limitations that we have identified in currently available instrumentation suggest some guidelines for the evaluation of new personal GPS logging technology and for selection of an appropriate instrument for use in large population studies. A key lesson learned is the importance of time to signal acquisition, especially after a cold start. Our study and others suggest that GPS loggers have high signal capture and good accuracy during on-road travel, an activity of interest, for example, to transportation planners. Rapid signal acquisition is critical for identifying the path of on-road travel early in a trip that might start from inside a home or a building where signal was lost for a prolonged period. In this regard, devices that provide assisted GPS capabilities may, under certain conditions, improve the startup performance, or time to first fix, of a GPS satellite-based positioning system and may use network resources, such as cell phone towers, during periods of sparse satellite coverage and/or poor atmospheric conditions to improve the subsequent positioning results obtained from the units. Rapid acquisition would also make it more feasible to develop post hoc procedures to identify indoor periods based on when signals disappear and reappear at the same location. Paradoxically, for an instrument with rapid signal acquisition, an indoor signal with poor accuracy might be worse than no signal at all, because a missing signal could be assigned to the indoor location when it disappeared and a rapid reacquisition at the same location could be assigned to outdoor activity at the same location. Because of the dependence of instrument performance on characteristics of the evaluation microenvironment, it is probably not sufficient to rely on the manufacturer’s specifications. However, signal acquisition time could be evaluated quickly for new instruments using this rapidly evolving technology in scripted microenvironments with open sky and with different types of satellite obstruction; for example, inside and outside of different types of buildings.¹⁷ In the meantime, it is important for investigators using personal GPS logging technology to acknowledge and, if possible, assess the potential bias associated with missing data. One useful step would be for investigators to report information on sampling frequency (e.g., every 15 s in our study), averaging time and other restrictions used to impute location more precisely, and the proportion of missing or inaccurate data by microenvironment (e.g., signal lost at home or work) and time of day. This would facilitate comparison of results with other studies, and assessment of the possibility that missing data might explain differing results between studies.

Newer GPS loggers have begun to address limitations that we have identified in the Wintec BT-201 and similar personal GPS logging instruments. The QStarz, for example, provides a record of when location is not identifiable owing to lack of satellite signal, and also provides a continuous record of information on the satellites available and those contributing to each location assignment. Research is urgently needed to develop algorithms using this type of information on measurement uncertainty to characterize accuracy and to distinguish outdoor from indoor time. We also learned that the size and weight of the Wintec BT-201 was quite acceptable to our study participants. Other considerations in evaluating new instruments are battery life between charges. The 15 h of battery life¹⁷ and data storage capabilities of 22 days with readings at 15-s intervals were acceptable for our study, but larger storage capacity would be needed for longer periods of data collection, and longer battery life would obviate the need for nightly recharging and corresponding data loss if a subject should forget. Other criteria to consider in evaluating new instruments include the chipset and the type of antenna. There has been increased interest in using GPS installed within cell phones to evaluate location, but the measurement has been shown to be considerably less accurate than standalone GPS devices, at least for older phones.^30,31

Even with the current limitations to data collection, GPS data might complement diary time and activity information with well-known limitations for large population studies of children. For example, personal GPS-logging instruments might be used to identify locations in specific time intervals of interest, such as during travel or after school. GPS personal loggers have been used to examine the walk to school and to evaluate time outdoors after school.¹⁰ Although overall participation rate by children eligible for our study was only 45%, the late enrollment by some children after seeing their classmates’ enthusiasm with the study suggests that participation rate could be improved by including a run-in period before the final enrollment and start of data collection.

Position accuracy and instrument precision under static and dynamic conditions in a variety of environments is critical for time–location analysis. We conclude that personal GPS data logging instruments have promise for identifying time spent in transit. For other locations, these data may best be used as a complement to other data sources. Epidemiological studies of effects of environmental exposures using currently available devices to identify exposure based on location require consideration of the potential of missing data to bias the interpretation of results.