Skip to main content
NCBI home page
UKPMC Funders Author Manuscripts logoLink to UKPMC Funders Author Manuscripts
. Author manuscript; available in PMC: 2023 Oct 30.
Published in final edited form as: Environ Int. 2018 Nov 22;122:3–10. doi: 10.1016/j.envint.2018.11.042

A picture tells a thousand … exposures: Opportunities and challenges of deep learning image analyses in exposure science and environmental epidemiology

Scott Weichentha a,*, Marianne Hatzopoulou b, Michael Brauer c
PMCID: PMC7615261  EMSID: EMS189845  PMID: 30473381

Abstract

Background

Artificial intelligence (AI) is revolutionizing our world, with applications ranging from medicine to engineering.

Objectives

Here we discuss the promise, challenges, and probable data sources needed to apply AI in the fields of exposure science and environmental health. In particular, we focus on the use of deep convolutional neural networks to estimate environmental exposures using images and other complementary data sources such as cell phone mobility and social media information.

Discussion

Characterizing the health impacts of multiple spatially-correlated exposures remains a challenge in environmental epidemiology. A shift toward integrated measures that simultaneously capture multiple aspects of the urban built environment could improve efficiency and provide important insights into how our collective environments influence population health. The widespread adoption of AI in exposure science is on the frontier. This will likely result in new ways of understanding environmental impacts on health and may allow for analyses to be efficiently scaled for broad coverage. Image-based convolutional neural networks may also offer a cost-eff ective means of estimating local environmental exposures in low and middle-income countries where monitoring and surveillance infrastructure is limited. However, suitable databases must first be assembled to train and evaluate these models and these novel approaches should be complemented with traditional exposure metrics.

Conclusions

The promise of deep learning in environmental health is great and will complement existing measurements for data-rich settings and could enhance the resolution and accuracy of estimates in data poor scenarios. Interdisciplinary partnerships will be needed to fully realize this potential.

1. Introduction

Environmental pollution has an important impact on the overall global burden of disease with economic impacts measured in billions of dollars each year (Landrigan et al., 2018). Traditionally, spatial and temporal variations in human exposures to environmental pollutants have been estimated separately for individual pollutants (e.g. noise, air pollution) using ground-based monitors or statistical models that combine various land-use, built environment, and/or remote sensing data to predict parameters of interest (Brauer et al., 2012; Weichenthal et al., 2016). While there will always be a need for research focused on individual chemical contaminants, it remains difficult to separate the individual health effects of multiple spatially correlated exposures (e.g. gaseous pollutants, particulate pollutants, noise, heat, etc.) in urban environments. Alternatively, integrated measures that capture the collective impact of the broader built environment on chemical/physical exposures may provide new insights into how multiple neighbourhood characteristics simultaneously interact to impact population health (Brauer & Hystad, 2014). New methods will be needed to facilitate this approach. One option is the application of deep learning methods including deep convolutional neural networks to estimate environmental exposures from large databases of street-level digital images (Gebru et al., 2018), aerial images from remote sensing (Jean et al., 2016), and other complementary information such as cell phone mobility data (Nyhan et al., 2016; Nyhan et al., 2018), personal activity sensors/other sensors (Asimina et al., 2018; Henriksen et al., 2018; Xie et al., 2018), or social media data (Schootman et al., 2016; Tao et al., 2016; Arthur et al., 2018; Costa et al., 2018). The aim of this brief review is to provide an overview of how deep learning image analyses may be used to integrate multiple data streams to predict variations in environmental pollutants influenced by the urban built environment. In doing so, we highlight the opportunities and expected challenges along the way.

As an illustrative example, consider traditional approaches to estimating spatial variations in environmental pollutants using physically based models (e.g. dispersion models) or geostatistical methods (e.g. land use regression models) (Fig. 1). Physically based models require input data on sources and their activity levels, and factors determining atmospheric fate and transport such as modifying factors of the environment (e.g. 3D building data) and meteorology (Gan et al., 2012; Penn et al., 2015). In the case of geostatistical approaches, multi-variable linear models are built by conducting large-scale spatial monitoring campaigns, extracting geographic information system (GIS) data around each sampling location, and using these parameters as pre-dictors of pollutant concentrations in final models. These approaches generally work well (Ryan & LeMasters, 2007), but the GIS parameters needed for model development and application are often available on a limited spatial scale and models are not generalizable across cities (Patton et al., 2015). In the case of physical models, large amounts of input data are required which may be difficult to access and/or not collected in all locations.

Fig. 1. Principles of physically-based and geostatistical models.

Fig. 1

Open in a new tab

Alternatively, information on land use, traffic, the built environment, and interactions between these factors are also encoded in images which can be captured both locally (i.e. at street level) (Fig. 2) and in a spatially continuous fashion through satellite imagery (Fig. 3). As such, large databases of paired pollutant-image samples may serve as a resource to train deep learning networks to predict environmental exposures on local scales. Moreover, this approach has the potential for scaling up to global coverage through the widespread availability of images in urban areas and by combining with remote sensing information. As images are readily available (and inexpensive to collect), this approach would not be subject to the spatial limitations of GIS data and could reveal previously unrecognized patterns in how environmental exposures are influenced by the collective built environment as it exists in real-life (as opposed to a selected subset of GIS parameters). Once developed, such models may also serve as an efficient means of predicting future/past exposures based on known or anticipated changes in land use, traffic, or the built environment or to prioritize areas for detailed monitoring and/or surveillance. However, such applications will depend on the availability of historical satellite/street- level images and thus may be more useful moving forward than for estimating exposures many years in the past.

Fig. 2. Heat maps can be used to highlight areas of images used to make predictions (simulated example).

Fig. 2

Open in a new tab

Fig. 3. Deep convolutional neural networks can have multiple inputs and multiple outputs. The top panel illustrates a model developed using local images, satellite images, and audio data. Pre-trained models can be applied to new images to make predictions for multiple exposures (bottom panel).

Fig. 3

Open in a new tab

Here we discuss the promise and challenges of applying deep learning image analysis in the fields of exposure science and environmental epidemiology. We begin by providing a brief overview of deep learning methods, deep convolutional neural networks, and how they are applied to make predictions. Next, we discuss the small number of studies to date that have used these methods to evaluate factors related to environmental exposures and/or the urban built environment and highlight recent examples from other disciplines that may serve as motivation for future applications. We conclude by discussing probable data sources for training and evaluating deep learning models for estimating environmental exposures and highlight future opportunities and challenges. Other recent papers have discussed topics related to machine learning/data mining in air pollution epidemiology (excluding deep learning) (Bellinger et al., 2017), geospatial artificial intelligence (Vopham et al., 2018), and neural networks for time-series estimation of air pollution concentrations (Arhami et al., 2013; Ding et al., 2016; Li et al., 2017; Fan et al., 2017; Li et al., 2016; Qi et al., 2017; Kang et al., 2018; Zhu et al., 2018).

2. How does deep learning work?

Deep learning is a form of supervised machine learning whereby large data sets are used to automatically learn representations linking inputs and outputs using non-linear transformations of raw data over successive layers of the network (Fig. 3) (Goodfellow et al., 2017; Chollet & Allaire, 2018; LeCun et al., 2015). The number of layers of representations in a given model determines the “depth” of the model. During the training procedure, the computer is shown repeated examples of inputs and expected outputs and learns weights for each layer that define the input-output function of the machine (LeCun et al., 2015). These weights are learned/adjusted using a feedback signal over an iterative training process (or training loop) which aims to minimize error between predictions and the known output labels in the training dataset (a procedure called stochastic gradient descent) (LeCun et al., 2015). After training, models are tested on new data to evaluate the generalizability of model predictions to situations not encountered during the training process.

A deep convolutional neural network for image classification starts with the image data (a matrix of pixel values) followed by a series of convolution and pooling layers: representations extracted during this process are fed into a final fully connected layer to output a prediction/class label (LeCun et al., 2015; Rawat & Wang, 2017). The role of convolution layers is feature extraction. This is performed by a series of convolution operations (hence the name) that apply small “filters” (i.e. smaller matrices) across the original input matrices (starting with the input image) to generate new matrices called feature maps. Different values of the filter matrix will produce different feature maps; units in a given feature map are connected to local regions in the feature maps of previous layers through a set of trainable weights called a filter bank (LeCun et al., 2015). It is these “filters” that are learned by the machine to facilitate predictions through an iterative training process that compares predictions to known values and adjusts weights to minimize prediction error (LeCun et al., 2015; Rawat & Wang, 2017). A detailed description of this process is available elsewhere (Rawat & Wang, 2017).

3. A not so black box

An important function of deep convolutional neural networks is the possibility to investigate which image characteristics are used to make predictions. For example, in a network used to predict air pollution or noise from images, specific aspects such as traffic may be prominent (Fig. 2). A detailed description of how this is implemented is provided elsewhere (Chollet & Allaire, 2018; Simonyan et al., 2013). Briefly, it is possible to extract the feature maps from a model to examine portions of the image that are used to make predictions. However, these images become more abstract as the depth of the layer increases, and as a result become less intelligible to the human user. Other options include generating heat maps that highlight areas of the input image used to make a prediction (Chollet & Allaire, 2018). This capability suggests the possibility for convolutional neural networks to predict exposures as well as their determinants (Fig. 3). Moreover, this capability may help to identify new exposures of interest if certain image characteristics (e.g. neighbourhood features) are found to directly influence health outcomes. In this way, deep learning models may help to identify priority regions for in-depth monitoring of environmental pollutants and characteristics and thus contribute to our understanding of how environmental factors impact public health.

4. Outdoor air pollution estimation using deep convolutional neural networks

A small number of studies have applied deep convolutional neural networks to estimate ambient air pollution concentrations from image data (other studies have used images to estimate “haze” conditions via light scattering (Li et al., 2015a; Liu et al., 2016) but are not discussed here). Specifically, Chakma et al. (Chakma et al., 2017) used images from Beijing, China between 2013 and 2017 to classify airborne concentrations of fine particulate air pollution (PM2.5) as good (< 75 μg/ m3), moderate (75–115 μg/m3), or severe (> 115 μg/m3) and reported a classification accuracy of 68.74% based on data from 2364 images. In a related study, Zhang et al. (Zhang et al., 2018) applied deep convolutional neural networks to estimate PM2.5 and PM10 concentrations (across 6 categories) using images from Beijing. This analysis was based on > 30,000 images and the authors reported average classification errors ranging from approximately 0.3–0.4 across 6 categories. While the categories of PM2.5 classification were very broad (1: < 35 μg/m3; 2: 25–75 μg/m3; 3: 75–115 μg/m3; 4: 115–150 μg/m3; 5: 150–250 μg/ m3; 6: > 250 μg/m3), these results support the feasibility of classifying temporal variations in ambient PM2.5 concentrations using images.

5. Motivating examples from other disciplines

While few studies have applied deep convolutional neural networks to estimate environmental exposures from images, there is a recent history of using images for virtual audits of built environment factors that are known to influence environmental exposures. For example, Chow et al. (Chow et al., 2014) developed the Environmental Profile of a Community Health (EPOCH) tool to evaluate built environment features using a set of standardized images. Using 5 photos per neighbourhood, most items covering neighbourhood density, aesthetics, disorder, pedestrian safety, and bicycle infrastructure had high levels of inter-rater reliability (intra-class correlation ≥ 0.70). In a second example, Mooney et al. (Mooney et al., 2014) used Google StreetView images to assess 9 previously identified built environment measures of physical disorder and reported that virtual audits were internally consistent, with composite disorder measures correlated with census unemployment and housing vacancy measures. Similarly, remote sensing and street level images have been combined with other readily available (web-based) information to develop a reliable desktop park quality audit tool (POSDAT) (Edwards et al., 2013) which was successfully applied to assess the quality of 200 parks (Rugel et al., 2017). Similarly, Naik et al. (Naik et al., 2017) used StreetView images to evaluate changes in quality of the built environment over time and demonstrated that these changes were predicted by different neighbourhood char-acteristics (e.g. education of residents) available from census or other data available at more coarse spatial scales.

Additional examples are also available in medicine. For example, Poplin et al. (Poplin et al., 2018) applied deep convolutional neural networks to predict a number of cardiovascular risk factors using retinal image data from > 200,000 patients in the United Kingdom and the United States. In particular, this model was able to predict risk factors including age, gender, smoking status, and blood pressure and provided reasonable estimates of the risk of major cardiovascular events within 5 years (area under the receiver operator curve = 0.70). A number of other examples are also available in the medical literature including the use of deep convolutional neural networks to detect/classify skin cancer lesions (Esteva et al., 2017), diabetic retinopathy (Gulshan et al., 2016), and invasive breast cancer on slide images (Cruz-Roa et al., 2017). More recently, Christiansen et al. (Christiansen et al., 2018) described the use of deep convolutional neural networks to predict the locations of fluorescent labels in unlabelled light microscope images and reported that this method was capable of accurately identifying cell states (alive/ dead), types, and subcellular features. Numerous other examples are discussed in a recent review on deep learning for computational biology (Angermueller et al., 2016).

6. Data sources

A number of data sources are outlined below that may serve as a resource to begin developing large databases to implement deep convolutional neural networks (and other machine learning methods) in exposure science and environmental epidemiology. This list is not meant to be exhaustive, buy may serve as a starting point in this process.

6.1. Google street view images

Pairing environmental exposure measurements Google Street View images may be the most efficient way to begin developing databases for the purpose of using deep learning models. In particular, incorporating environmental exposure measurements (e.g. noise, air pollution, temperature, light) with directly onto Google Street View vehicles as standard practice could greatly facilitate this process as recently demonstrated for methane leak detection in multiple cities (von Fischer et al., 2017) and for air pollutants in Oakland, California (Apte et al., 2017). A global database of paired image/exposure data from these vehicles would provide an ideal resource for training deep learning models of environmental exposures worldwide.

6.2. Satellite images and remote sensing

Satellite remote sensing data has revolutionized global exposure estimation for ambient air pollution and these data (combined with satellite images) may also serve as an important resource as inputs for deep convolutional neural networks. Currently, remote sensing data for aerosol optical depth are combined with chemical transport models (e.g. GEOS-Chem) to estimate ground level PM2.5 concentrations (Brauer et al., 2012). However, chemical transport models cannot capture all possible sources of pollution and the combined use of remote sensing data with satellite images may be advantageous if images can provide complementary data reflecting emission sources missing from chemical transport models. Moreover, as satellite images capture a larger spatial scale than local images, it may be advantageous to include both local and satellite images as input data to capture the potential impact of local and regional sources of environmental pollutants/risk factors. Indeed, true color remote sensing images have been used to detect plumes from wildfire smoke (Henderson et al., 2011) and to derive predictors (e.g. vegetation, impervious surfaces) to be used in continental-scale (Novotny et al., 2011; Vienneau et al., 2013; Knibbs et al., 2014) or global land use regression models (Larkin et al., 2017). Similarly, greenness measures, which have increasingly been associated with health benefits, are most commonly derived from remote sensing (Defries & Townshend, 1994), with more recent applications of streetlevel images (Li et al., 2015b; Li & Ratti, 2018).

6.3. Beyond images

The smart city paradigm has given rise to ubiquitous networks of data pertaining to traffic counts and human mobility which may provide a rich source of information in estimating population exposures to environmental pollutants. For example, traffic counts are typically collected in real-time at a limited set of locations in an urban area; deep convolutional neural networks can help with the development of spatiotemporal interpolations of traffic counts or mode share across an urban area. Similarly, GPS data collected by taxis, ride hailing services (such as Uber) or the general public can be used to generate measures of traffic speeds and congestion, which typically influence the amount of air contaminants emitted by on-road vehicles. Finally, various appli-cations have been developed to collect daily mobility information (Patterson & Fitzsimons, 2016). These data can be used to impute mobility patterns for the population and refine existing measures of environmental exposures (Fallah-Shorshani et al., 2018).

7. Future opportunities and challenges

7.1. Opportunities

7.1.1. Multiple data streams for multiple exposures

Deep learning models can have multiple inputs and multiple outputs. Therefore, if multiple exposures are assigned to large image databases, deep convolutional neural networks could be used to predict all of these factors simultaneously (Fig. 3). Furthermore, it may be possible to combine inputs from multiple data streams including images (local and satellite), location data, point of interest data, traffic data, audio data (discussed below), social media text/trends (e.g. smog alerts, forest fires), and other relevant factors to jointly model multiple environmental exposures. Moreover, recent evidence suggests that it is possible to predict street-level images using satellite data (Deng et al., n.d.) and this may provide a cost-effective means of generating a continuous database of ground level images using remote sensing. Once developed, such models could be integrated into smartphone applications to provide real-time environmental exposure estimation using data collected from the smartphone's camera, GPS, microphone, and social media applications. In addition, similar tools could be applied by knowledge users to inform urban planning/policy development and thus have a positive impact on population health.

7.1.2. Audio data

Like images, sound also encodes important information related to environmental exposures (and their sources) and can be easily collected using smartphones or other methods. Deep convolutional neural networks can be used to classify audio files and have been applied to classify lung sounds (Aykanat et al., 2017), environmental sounds (e.g. car horn, engine idling) (Piczak, 2015), and acoustic scenes (e.g. home, parks, grocery store, metro station) (Han & Lee, 2015). Interestingly, other studies have classified environmental sounds by first converting audio files into spectral images and applying deep convolutional neural networks to the resulting image files (Boddapati et al., 2017). As with image data discussed above, audio data may provide an interesting avenue for the evaluation of environmental exposures or as an alternative method of tracking time-activity patterns determined by “acoustic scenes”.

Images as Integrators of Multiple Exposures: Linking Images Directly to Health Outcomes.

Given small-scale spatial information on disease rates (e.g. countylevel disease rates) it may be possible to directly evaluate the relationship between images and disease incidence on an ecological level as recently illustrated for obesity and the neighbourhood built environment in the United States (Maharana & Nsoesie, 2018). This approach would have all of the same limitations as traditional ecological analyses but could provide important insights into how our collective environments impact public health if we can identify specific portions of images used for predictions. However, this approach will also have potential ethical challenges as labeling a given neighbourhood as “good” or “bad” based on image analyses may be of little use if corrective interventions are not readily apparent or easily inferred from study results. Indeed, the ethical implications of the wide spread use of artificial intelligence are an important societal concern and Google recently released a list of ethical principles to guide the development of this technology including: 1) Being socially beneficial; 2) Avoiding the creation/reinforcement of unfair biases; 3) Apply strong safety and security practices; 4) Being accountable to people; 5) Incorporating privacy design principles; 6) Upholding high standards of scientific excellence; and 7) Being available for uses that support these principles (Pichai, 2018). These principles, while already prominent in environmental health research must also be considered when applying artificial intelligence approaches.

7.2. Challenges

7.2.1. Building databases

The validity of predictions made from any modeling process depends entirely on the quality of information entered in to the model. As such, while moving toward the construction of paired image/audio/exposure databases it is important to ensure that the exposure information is of the highest quality and corresponds to the temporal/spatial resolution of the image. Real-time monitoring equipment and small portable cameras/audio recorders will make it easy to generate large databases of short-term temporal variations for some exposures; however, large databases of paired image/audio/exposure data for long-term exposures over large spatial gradients will be more difficult to collect. It would be advantageous for the research community to develop an open and efficient means of pooling data for the purpose of linking images to environmental exposure measures on a global scale.

A number of other challenges are also apparent. For example, in order to be effective, we must be able to interpret and generalize associations discovered using deep learning methods. It is not enough to predict the presence of a hazard/risk if we cannot identify the underlying causes or suitable interventions. As noted above, relationships identified through the application of deep convolutional neural networks may help to highlight combinations of neighbourhood characteristics likely to have a positive/negative impact on health; however, traditional environmental/biological monitoring will still be needed to identify the specific chemical/physical contaminants of concern. Moreover, images themselves have limitations as most databases reflect daytime hours, outdoors, and thus exclude behavioural aspects of exposure including time spent indoors. Ultimately, the success of this approach will rely on high quality training data that captures the wide range of environments/exposures we encounter on a daily basis.

8. Conclusions

Artificial intelligence is changing the world we live in and already impacts many aspects of our daily lives with applications spanning disciplines from medicine to engineering. To date, few studies have integrated deep learning methods into environmental exposure assessment, but user-friendly tools are now available to support such applications. The opportunities and challenges are great, but new methods are needed to evaluate the combined health impacts of multiple spatially-correlated exposures and deep convolutional neural networks may provide new insights into this important question. In particular, as images, audio, and location data all encode important information related to environmental exposures, their combined use in deep learning models are a logical starting point to begin this process. In doing so, we must not lose sight of the importance of high-quality exposure measurements as the validity of input-output relationship in deep-learning models will depend on accurate and reliable estimates of the exposures of interest.

Acknowledgements

We acknowledge intellectual and financial contributions of the Canadian Urban Environmental Health Research Consortium (CANUE), and the study team of the Pathways to Equitable Healthy Cities project (supported by the Wellcome Trust). In particular, discussions with Dr. Esra Suel and Dr. Eleanor Setton were helpful in articulating potential applications of image analysis for environmental exposure assessment. Mr. Tristan Goodbody created all of the figures included in this manuscript.

Funding

Dr. Weichenthal received research support from an NSERC Discovery Grant, a CIHR Foundation Grant, and a GRePEC salary award funded by the Cancer Research Society, the Quebec Ministry of Economy, Science and Innovation, and FRQS (Fonds de Recherche du Québec- Santé). Dr. Brauer is supported in part by the British Columbia Lung Association Professorship.

Footnotes

Conflict of interest

None.

References

  1. Angermueller C, Parnamaa T, Parts L, Stegle O. Deep learning for computational biology. Mol Syst Biol. 2016;12:878. doi: 10.15252/msb.20156651. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Apte JS, Messier KP, Gani S, Brauer M, Kirchstetter TW, Lunden MM, et al. High-resolution air pollution mapping with Google Street View cars: exploiting big data. Environ Sci Technol. 2017;51:6999–7008. doi: 10.1021/acs.est.7b00891. [DOI] [PubMed] [Google Scholar]
  3. Arhami M, Kamali N, Rajabi MM. Predicting hourly air pollutant levels using arti ficial neural networks coupled with uncertainty analysis by Monte Carlo simulations. Environ Sci Pollut Res. 2013;20:4777–4789. doi: 10.1007/s11356-012-1451-6. [DOI] [PubMed] [Google Scholar]
  4. Arthur R, Boulton CA, Shotton H, Williams HTP. Social sensing of floods in the UK. PLoS One. 2018;13:e0189327. doi: 10.1371/journal.pone.0189327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Asimina S, Chapizanis D, Karakitsios S, Kontoroupis P, Asimakopoulos DN, Maggos T, et al. Assessing and enhancing the utility of low-cost activity and location sensors for exposure studies. Environ Monit Assess. 2018;190:155. doi: 10.1007/s10661-018-6537-2. [DOI] [PubMed] [Google Scholar]
  6. Aykanat M, Kilic O, Kurt B, Saryal S. Classification of lung sounds using convolutional neural networks. EURASIP J Image Video Process. 2017;65 doi: 10.1186/s13640-017-0213-2. [DOI] [Google Scholar]
  7. Bellinger C, Jabbar MSM, Zaiane O, Osornio-Vargas A. A systematic review of data mining and machine learning for air pollution epidemiology. BMC Public Health. 2017;17:907. doi: 10.1186/s12889-017-4914-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Boddapati V, Petef A, Rasmusson J, Lundberg L. Classifying environmental sounds using image recognition networks. Progr Comput Sci Appl Logic. 2017;112:2048–2056. [Google Scholar]
  9. Brauer M, Hystad P. Commentary: cities and health…let me count the ways. Epidemiology. 2014;25:526–527. doi: 10.1097/EDE.0000000000000110. [DOI] [PubMed] [Google Scholar]
  10. Brauer M, Amann M, Burnett RT, Cohen A, Dentener F, Ezzati M, Henderson SB, Krzyanowski M, Martin RV, Van Dingenen R, van Donkelaar A, et al. Exposure assessment for estimation of the global burden of disease attributable to outdoor air pollution. Environ Sci Technol. 2012;46:652–660. doi: 10.1021/es2025752. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Chakma A, Vizena B, Cao T, Lin J, Zhang J. Image-based air quality analysis using deep convolutional neural network; 2017 IEEE International Conference on Image Processing (ICIP); Beijing. 2017. pp. 3949–3952. [DOI] [Google Scholar]
  12. Chollet F, Allaire JJ. Deep Learning With R. Manning Publications; Shelter Island, NY: 2018. [Google Scholar]
  13. Chow CK, Corsi DJ, Lock K, Madhavan M, Mackie P, Li W, et al. A novel method to evaluate the community built environment using photographs-en-vironmental profile of a community health (EPOCH) photo neighbourhood evaluation tool. PLoS One. 2014;9:e110042. doi: 10.1371/journal.pone.0110042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Christiansen EM, Yang SJ, Ando DM, Javaherian A, Skibinski G, Lipnick S, et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell. 2018;173:792–803. doi: 10.1016/j.cell.2018.03.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Costa D, Duran-Faundez C, Andrade D, Rocha-Junior J, Peixoto J. TwitterSensing: an event-based approach for wireless sensor networks optimization exploiting social media in Smart City Applications. Sensors. 2018;18:1080. doi: 10.3390/s18041080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cruz-Roa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih NNC, et al. Accurate and reproducible invasive breast cancer detection in whole slide images: a deep learning approach for quantifying tumour extent. Sci Rep. 2017;7:46450. doi: 10.1038/srep46450. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Defries RS, Townshend JRG. NDVI-derived land cover classifications at a global scale. Int J Remote Sens. 1994;15:3567–3586. [Google Scholar]
  18. Deng X, Zhu Y, Newsam S. What Is It Like Down There? Generating Dense Ground Level Views and Image Features From Overhead Imagery Using Conditional Generative Adversarial Networks. arxiv.org/abs/1806.05129 . [Google Scholar]
  19. Ding W, Zhang J, Leung Y. Prediction of air pollutant concentration based on sparse response back-propagation training feedforward neural networks. Environ Sci Pollut Res. 2016;23:19481–19494. doi: 10.1007/s11356-016-7149-4. [DOI] [PubMed] [Google Scholar]
  20. Edwards N, Hooper P, Trapp GSA, Bull F, Boruff B, Giles-Corti B. Development of a Public Open Space Desktop Auditing Tool (POSDAT): a remote sensing approach. Appl Geogr. 2013;38:22–30. [Google Scholar]
  21. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–118. doi: 10.1038/nature21056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Fallah-Shorshani M, Hatzopoulou M, Ross NA, Patterson Z, Weichenthal S. Evaluating the impact of neighbourhood characteristics on diff erences between residential and mobility-based exposures to outdoor air pollution. Environ Sci Technol. 2018;52:10777–10786. doi: 10.1021/acs.est.8b02260. [DOI] [PubMed] [Google Scholar]
  23. Fan J, Li Q, Hou J, Feng X, Karimian H, Lin S. A spatiotemporal prediction framework for air pollution based on deep RNN. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci. 2017 doi: 10.5194/isprs-annals-IV-4-W2-15-2017. [DOI] [Google Scholar]
  24. Gan WQ, McLean K, Brauer M, Chiarello SA, Davies HW. Modeling population exposure to community noise and air pollution in a large metropolitan area. Environ Res. 2012;116 doi: 10.1016/j.envres.2012.04.001. [DOI] [PubMed] [Google Scholar]
  25. Gebru T, Krause J, Wang Y, Chen D, Deng J, Aiden EL, et al. Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proc Natl Acad Sci. 2018;114:13108–13113. doi: 10.1073/pnas.1700035114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Goodfellow I, Yoshua B, Courville A. Deep Learning. MIT Press; Cambridge, MA: 2017. [Google Scholar]
  27. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–2410. doi: 10.1001/jama.2016.17216. [DOI] [PubMed] [Google Scholar]
  28. Han Y, Lee K. Acoustic Scene Classifi cation Using Convolutional. Neural Networks and Multiple-width Frequency-delta Data Augmentation. 2015:arXiv:1607.02383v1 [cs.SD] [Google Scholar]
  29. Henderson SB, Brauer M, MacNab YC, Kennedy SM. Three measures of forest fire smoke exposure and their associations with respiratory and cardiovascular health outcomes in a population-based cohort. Environ Health Perspect. 2011;119:1266–1271. doi: 10.1289/ehp.1002288. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Henriksen A, Haugen Mikalsen M, Woldaregay AZ, Muzny M, Hartvigsen G, Hopstock LA, et al. Using fitness trackers and smartwatches to measure physical activity in research: analysis of consumer wrist-worn wearables. J Med Internet Res. 2018;20:e110. doi: 10.2196/jmir.9157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Jean N, Burke M, Xie M, Davis WM, Lobell DB, Ermon S. Combining satellite imagery and machine learning to predict poverty. Science. 2016;353:790–794. doi: 10.1126/science.aaf7894. [DOI] [PubMed] [Google Scholar]
  32. Kang GK, Gau JZ, Chiao S, Lu S, Xie G. Air quality prediction: big data and machine learning approaches. Int J Environ Sci Dev. 2018;9:8–16. [Google Scholar]
  33. Knibbs LD, Hewson MG, Bechle MJ, Marshall JD, Barnett AG. A national satellite-based land-use regression model for air pollution exposure assessment in Australia. Environ Res. 2014;135:204–211. doi: 10.1016/j.envres.2014.09.011. [DOI] [PubMed] [Google Scholar]
  34. Landrigan PJ, Fuller R, Acosta NJR, et al. The Lancet commission on pollution and health. Lancet. 2018;391:462–512. doi: 10.1016/S0140-6736(17)32345-0. [DOI] [PubMed] [Google Scholar]
  35. Larkin A, Geddes JA, Martin RV, Xiao Q, Liu Y, Marshall JD, et al. Global land use regression model for nitrogen dioxide air pollution. Environ Sci Technol. 2017;51:6957–6964. doi: 10.1021/acs.est.7b01148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–444. doi: 10.1038/nature14539. [DOI] [PubMed] [Google Scholar]
  37. Li X, Ratti C. Mapping the spatial distribution of shade provision of street trees in Boston using Google Street View panoramas. Urban For Urban Green. 2018;31:109–119. [Google Scholar]
  38. Li Yuncheng, Huang J, Luo J. Using user generated online photos to estimate and monitor air pollution in major cities; Proceedings of the 7th International Conference on Internet Multimedia Computing and Service (ICIMCS ‘15) ACM; New York, NY, USA. 2015a. (Article 79) [DOI] [Google Scholar]
  39. Li X, Zhang C, Li W, Ricard R, Meng Q, Zhang W. Assessing street-level urban greenery using Google Street View and a modified green view index. Urban For Urban Green. 2015b;14:675–685. [Google Scholar]
  40. Li X, Peng L, Hu Y, Shao J, Chi T. Deep learning architecture for air quality predictions. Environ Sci Pollut Res. 2016;23:22408–22417. doi: 10.1007/s11356-016-7812-9. [DOI] [PubMed] [Google Scholar]
  41. Li X, Peng L, Yao X, Cui S, Hu Y, You C, Chi T. Long short-term memory neural network for air pollutant concentration predictions: method development and evaluation. Environ Pollut. 2017;231:997–1004. doi: 10.1016/j.envpol.2017.08.114. [DOI] [PubMed] [Google Scholar]
  42. Liu C, Tsow F, Tao N. Particle pollution estimation based on image analysis. PLoS One. 2016;11:e0145955. doi: 10.1371/journal.pone.0145955. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Maharana A, Nsoesie EO. Use of deep learning to examine the association of the built environment with prevalence of neighbourhood adult obesity. JAMA Netw Open. 2018;1:e181535. doi: 10.1001/jamanetworkopen.2018.1535. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mooney SJ, Bader MDM, Lovasi GS, Neckerman KM, Teitler JO, Rundle AG. Validity of an ecometric neighborhood physical disorder measure constructed by virtual street audit. Am J Epidemiol. 2014;180:626–635. doi: 10.1093/aje/kwu180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Naik N, Kominers SD, Raskar R, Glaeser EL, Hidalgo CA. Computer vision uncovers predictors of physical urban change. Proc Natl Acad Sci. 2017;114:7571–7576. doi: 10.1073/pnas.1619003114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Novotny EV, Bechle MJ, Millet DB, Marshall JD. National satellite-based land-use regression: NO2 in the United States. Environ Sci Technol. 2011;45:4407–4414. doi: 10.1021/es103578x. [DOI] [PubMed] [Google Scholar]
  47. Nyhan M, Grauwin S, Britter R, Misstear B, McNabola A, Laden F, et al. “Exposure track”—the impact of mobile-device-based mobility patterns on quantifying population exposure to air pollution. Environ Sci Technol. 2016;50:9671–9681. doi: 10.1021/acs.est.6b02385. [DOI] [PubMed] [Google Scholar]
  48. Nyhan MM, Kloog I, Britter R, Ratti C, Koutrakis P. Quantifying population exposure to air pollution using individual mobility patterns inferred from mobile phone data. J Expo Sci Environ Epidemiol. 2018 doi: 10.1038/s41370-018-0038-9. (Apr 27, Epub ahead of print) [DOI] [PubMed] [Google Scholar]
  49. Patterson Z, Fitzsimons K. DataMobile: smartphone travel survey experiment. J Transp Res Board. 2016;2594:35–43. [Google Scholar]
  50. Patton AP, Zamore W, Naumova EN, Levy JI, Brugge D, Durant JL. Transferability and generalizability of regression models of ultrafine particles in urban neighbourhoods in the Boston area. Environ Sci Technol. 2015;49:6051–6060. doi: 10.1021/es5061676. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Penn SL, Arunachalam S, Tripodis Y, Heiger-Bernays W, Levy JI. A comparison between monitoring and dispersion modeling approaches to assess the impact of aviation on concentrations of black carbon and nitrogen oxides at Los Angeles International Airport. Sci Total Environ. 2015;527-528:47–55. doi: 10.1016/j.scitotenv.2015.03.147. [DOI] [PubMed] [Google Scholar]
  52. Pichai S. AI at Google: our principles. 2018. Jun 7, [Accessed date: 3 July 2018]. Available: https://blog.google/technology/ai/ai-principles/
  53. Piczak KJ. Environmental sound classification with convolutional neural networks; IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), 2015; Boston, MA. 2015. [DOI] [Google Scholar]
  54. Poplin R, Varadarajan AV, Blumer K, Liu Y, McConnell MV, Corrado GS, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2:158–164. doi: 10.1038/s41551-018-0195-0. [DOI] [PubMed] [Google Scholar]
  55. Qi Z, Wang T, Song G, Hu W, Li X, Zhang ZM. Deep air learning: interpolation, prediction, and feature analysis of fine-grained air quality. IEEE Trans Knowl Data Eng. 2017 doi: 10.1109/TKDE.2018.2823740. [DOI] [Google Scholar]
  56. Rawat W, Wang Z. Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput. 2017;29:2352–2449. doi: 10.1162/NECO_a_00990. [DOI] [PubMed] [Google Scholar]
  57. Rugel EJ, Henderson SB, Carpiano RM, Brauer M. Beyond the Normalized Difference Vegetation Index (NDVI): developing a Natural Space Index for population-level health research. Environ Res. 2017;159 doi: 10.1016/j.envres.2017.08.033. [DOI] [PubMed] [Google Scholar]
  58. Ryan PH, LeMasters GK. A review of land-use regression models for characterizing intraurban air pollution exposure. Inhal Toxicol. 2007;19(Suppl. 1):127–133. doi: 10.1080/08958370701495998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Schootman M, Nelson EJ, Werner K, Shacham E, Elliott M, Ratnapradipa K, et al. Emerging technologies to measure neighborhood conditions in public health: implications for interventions and next steps. Int J Health Geogr. 2016;15:20. doi: 10.1186/s12942-016-0050-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Simonyan K, Vedaldi A, Zisserman A. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv:13126034 [csCV] 2013 [Google Scholar]
  61. Tao Z, Kokas A, Zhang R, Cohan DS, Wallach D. Inferring atmospheric particulate matter concentrations from Chinese social media data. PLoS One. 2016;11:e0161389. doi: 10.1371/journal.pone.0161389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Vienneau D, de Hoogh K, Bechle MJ, Beelen R, van Donkelaar A, Martin RV, et al. Western European land use regression incorporating satellite-and ground-based measurements of NO2 and PM10. Environ Sci Technol. 2013;47:13555–13564. doi: 10.1021/es403089q. [DOI] [PubMed] [Google Scholar]
  63. von Fischer JC, Cooley D, Chamberlain S, Gaylord A, Griebenow CJ, Hamburg SP, et al. Rapid, vehicle-based identification of location and magnitude of urban natural gas pipeline leaks. Environ Sci Technol. 2017;51:4091–4099. doi: 10.1021/acs.est.6b06095. [DOI] [PubMed] [Google Scholar]
  64. Vopham T, Hart JE, Laden F, Chiang YY. Emerging trends in geospatial artificial intelligence (geoAI): potential applications for environmental epidemiology. Environ Health. 2018;17:40. doi: 10.1186/s12940-018-0386-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Weichenthal S, Van Ryswyk K, Goldstein A, Shekarrizfard M, Hatzopoulou M. Characterizing the spatial distribution of ambient ultrafine particles in Toronto, Canada: a land use regression model. Environ Pollut. 2016;208:241–248. doi: 10.1016/j.envpol.2015.04.011. [DOI] [PubMed] [Google Scholar]
  66. Xie J, Wen D, Liang L, Jia Y, Gao L, Lei J. Evaluating the validity of current mainstream wearable devices in fitness tracking under various physical activities: comparative study. JMIR mHealth uHealth. 2018;6:e94. doi: 10.2196/mhealth.9754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Zhang C, Yan J, Li C, Wu H, Bie R. End-to-end learning for image-based air quality level estimation. Mach Vis Appl. 2018;29:601–615. [Google Scholar]
  68. Zhu D, Cai C, Yang T, Zhou X. A machine learning approach for air quality prediction: model regularization and optimization. Big Data Cogn Comput. 2018;2:5. [Google Scholar]

RESOURCES