COVID-19 contact tracking based on person reidentification and geospatial data

Abstract

Efficient contact tracing is a crucial step in preventing the spread of COVID-19. However, the current methods rely heavily on manual investigation and truthful reporting by high-risk individuals. Mobile applications and Bluetooth-based contact tracing methods have also been adopted, but privacy concerns and reliance on personal data have limited their effectiveness. To address these challenges, in this paper, a geospatial big data method that combines person reidentification and geospatial information for contact tracing is proposed. The proposed real-time person reidentification model can identify individuals across multiple surveillance cameras, and the surveillance data is fused with geographic information and mapped onto a 3D geospatial model to track movement trajectories. After real-world verification, the proposed method achieves a first accuracy rate of 91.56%, a first-five accuracy rate of 97.70%, and a mean average precision of 78.03% with an inference speed of 13 ms per image. Importantly, the proposed method does not rely on personal information, mobile phones, or wearable devices, avoiding the limitations of existing contact tracing schemes and providing significant implications for public health in the post-COVID-19 era.

1. Introduction

COVID-19 is a global infectious disease pandemic that has had a profound impact on the global economy (Jardou and Lawson, 2021). Most countries and regions have implemented measures to prevent and control the spread of the virus (Zhou et al., 2020). However, due to its high infectivity, long incubation period, a significant number of asymptomatic carriers, and the emergence of new mutant strains, the epidemic has not been adequately contained, and global efforts to prevent and control it remain under great pressure (Abeler et al., 2020).

Lockdowns, contact tracking, nucleic acid testing, sanitation, and quarantines are necessary measures to contain the spread of the outbreak (Bradshaw et al., 2021). Once a case is confirmed, it is crucial to identify the contact group retrospectively to understand the transmission route and timeline. This is because isolating individuals in close contact and sealing key areas can significantly curb the spread of the epidemic (Patel et al., 2020). Thus, contact tracking is the first step in epidemic prevention and control (Ferretti et al., 2020). Simulation models by Hellewell et al. (2020) suggest that effective contact tracking and case isolation can control a new outbreak of COVID-19 within three months in most cases. In light of this, contact tracking is a fundamental strategy adopted by many countries to contain the spread of COVID-19 (Nuzzo et al., 2020).

However, contact tracking is a labor-intensive process, and individuals carrying the disease (index cases) may visit multiple public places and contact hundreds of people (Jali, 2021). In traditional contact tracking, public health workers need to interview index cases and then trace each contact one by one (Pollmann et al., 2021). Recent studies (Sharma et al., 2020, Martin et al., 2020, Idrees et al., 2021) have shown that digital contact tracking (DCT) can be a fast and effective tool in controlling the COVID-19 outbreak. Most existing contact tracking systems utilize communication base stations, Global Positioning System (GPS), and Bluetooth technologies to develop relevant mobile applications or wearable tracking devices (Ali et al., 2020, Ali et al., 2021, Brar et al., 2022). However, communication-based tracking is limited by its low accuracy and can only be used for rough tracking based on the geospatial information of the base station location (Tizzoni et al., 2014). GPS-based tracking can achieve higher accuracy but raises concerns about privacy infringement since personal location data are sensitive and private (Sharma et al., 2020). Bluetooth-based technology strikes a balance between accuracy and privacy issues by identifying contacts through communication between mobile devices over short distances (McLachlan et al., 2020). Nevertheless, the effectiveness of this method depends on the smartphone availability rate of local people and their willingness to install apps and turn on Bluetooth, which is challenging to achieve in the current situation (Zhao et al., 2020). Moreover, this method heavily relies on smartphones, excluding those for whom smartphone usage is inconvenient, such as the elderly and the blind (Hernández-Orallo et al., 2020).

In contrast, person reidentification technology has demonstrated significant advantages in close contact tracking (Narayan et al., 2017). Person reidentification, as an artificial intelligence development, aims to recognize and track a target individual in an open environment and across multiple camera scenes (Wu et al., 2019). Some powerful models have even outperformed humans in identifying public datasets (Ye et al., 2021). Currently widely used surveillance cameras offer the possibility of deploying and implementing person reidentification. Combining surveillance data with the geospatial information of these cameras can provide geospatial big data for infectious disease prevention and treatment (Wu et al., 2021). However, the accuracy of the person reidentification model is limited by natural lighting changes and the size and shooting angle of the human body in video surveillance (Cheng et al., 2020, Jiao et al., 2020). Additionally, the geographic information of existing surveillance cameras is limited to a 2D space (i.e., the latitude and longitude of the camera), making it challenging to function in the main areas of epidemic spread, such as indoors (Kumar et al., 2021).

To overcome the aforementioned challenges, this study proposes a geospatial big data method that combines person reidentification and geospatial information to automatically track COVID-19 contacts. First, a real-time person reidentification model is constructed to identify the same person in different surveillance cameras based on a public dataset. Then, the surveillance data are fused with the geographic location of the cameras and mapped to a 3D geospatial model to track the contact’s movement trajectory. Finally, the proposed method is validated in a real indoor scene in an office building, where a database of personal full-body images and contact information of staff and visitors is collected in advance. When a COVID-19 contact appears in the office building, the proposed method can automatically identify the contact’s real-time location by selecting the contact’s image in the database as the identification sample. Retrieving surveillance videos from the past 14 days, the system automatically marks the cameras that have detected the contact and draws the target person’s trajectory based on the time series and geospatial information of the cameras. This information can be used to identify key places visited by high-risk contacts and find secondary contacts through activity tracking. Contact time and distance can also be utilized to determine the risk level, making population screening more targeted. The proposed method does not require users to install specific mobile applications or wear wearable devices, avoiding the need for additional devices. As a result, this method provides an automatic and effective approach for tracking COVID-19 contacts, which has significant implications for epidemic prevention and public health. The main contributions of this paper are as follows:

• •A COVID-19 contact tracking method based on person reidentification and geospatial data was proposed, feasibility and advantages compared with existing methods were discussed, providing a novel idea for epidemic prevention and control.
• •A multi-module person reidentification model is utilized to overcome the challenges posed by changes in natural lighting and the multi-scale and multi-angle of a person in video surveillance.
• •The geographic information of the cameras is combined with the surveillance data to form geospatial big data, which is mapped to a 3D space to track the contact’s movement trajectory.
• •The proposed method relies solely on video surveillance and geographic information, eliminating the dependence of existing tracking schemes on additional equipment and sensitive data.

The structure of this study is outlined as follows. Section 2 provides a detailed explanation of how the proposed method tackles the challenges mentioned earlier, along with the experimental preparations, including the dataset, experimental settings, facilities, and evaluation metrics. In Section 3, the experimental results are presented. Section 4 examines and discusses the experimental results in greater detail. Lastly, Section 5 provides conclusions drawn from this study and suggests potential avenues for future research.

2. Materials and methods

In this section, the proposed methods are explained separately to illustrate how the proposed methods work and their advantages in tracking COVID-19 contacts. The overall scheme of the proposed method is shown in Fig. 1 , which is mainly composed of a person reidentification model and a geospatial information model.

Fig. 1.

Overall scheme of the proposed method, which integrates person reidentification and geospatial information to track COVID-19 contacts. The system utilizes a multi-module person reidentification model and video surveillance to locate the target person, and maps the surveillance data to a 3D model generated based on geographic information to draw the 3D movement trajectory of the target person for contact tracing.

The proposed method comprises four parts:

(1) Data acquisition: The data used in this study include public datasets and data for verification. To enhance the robustness of the person reidentification model, the fusion of multiple public datasets is utilized for training. The other part of the data is collected in real scenarios, including video surveillance data, geospatial information of the cameras, and 3D architectural models of the buildings. These data are acquired as case studies to verify the performance of the proposed method.

(2) Person reidentification: In this study, a person reidentification model with multiple modules to handle the challenges posed by changes in natural lighting and the multi-scale and multi-angle of a person in video surveillance was adopted. The person reidentification model is a computer-vision-based technology used to determine whether a specific person appears in an image or video sequence, which in this study is to identify whether the person is a COVID-19 contact from video surveillance.

(3) Geospatial information: By combining the geospatial information of the surveillance camera with the video surveillance data and mapping it to the three-dimensional model, the movement trajectory of the target person can be drawn accurately.

(4) Direct and indirect contact tracking: Based on the trajectory of the target person drawn in step (3), the key activity locations and close contacts of the target person can be discovered in a timely manner. By repeating the above steps, we can expand the search scope and automatically and efficiently find the direct and indirect contacts.

Algorithm 1

Pseudocode of the proposed methodology.

Input: Training set $x=\{x_1,x_2,\dots ,x_n\}$, The resnet50-IBN network R of the pre-trained model is loaded, Self-Constrained spatial transformation network (SC-STN).

Output: Optimized model R.

1: while Max epochs not reached do

2:  ResNet50-IBN architecture was employed to extract the feature map, denoted as $F=R(x)$.

3:  Extracted features are divided into three branches, denoted by $\{F_1,F_2,F_3\}$, for further processing.

4:  Feature maps are transformed in finite space using the parametric controllable method, where the transformed feature is denoted as $S_n=\mathrm{SC}-\mathrm{STN}(F_n)$ with $n\in \{1,2,3\}$.

5:  Two local branches divide the feature maps horizontally to extract local features while preserving the global feature maps. The final output features consist of two sets: $\{S_{21},S_{22}\text{;}{S}_{31},S_{32},S_{33}\}$ and $\{F_1,F_2,F_3\}$.

6:  The ultimate features and predicted probability distributions are obtained using averaging pooling, channel compression, batch normalization, and full connection layer.

7:  Triples loss $L_{\mathit{triplet}}$ and Cross entropy loss $L_{\mathit{ce}}$ are used for global features, while only cross entropy loss $L_{\mathit{ce}}$ is used for local features.

8:  Update the network parameters.

9: end while

2.1. Data acquisition

The person reidentification model is fundamentally a data-driven deep learning model, and the diversity of training data plays a crucial role in its generalization and robustness (Huang et al., 2021). To enhance the robustness and generalization of the trained model, unlike existing person reidentification methods (Ning et al., 2020, Zhao et al., 2021), which typically assess the model’s performance on several datasets separately, this study employs the fusion of multiple datasets as the training set when constructing the model for contact tracking. Furthermore, the test data was collected from a real office building to evaluate the performance of the proposed contact tracking method as a typical case.

2.1.1. Public dataset

Market1501 (Ristani et al., 2016), MSMT17 (Qian et al., 2019), and CUHK03 (Li et al., 2014) are the three commonly used person reidentification public datasets. The Market1501 dataset was collected on the campus of Tsinghua University, photographed in the summer, and constructed and made public in 2015. The MSMT17 dataset is a large-scale dataset collected by twelve outdoor cameras and three indoor cameras, covering different weather and different time periods of video surveillance. The CUHK03 dataset was collected at the Chinese University of Hong Kong and published in 2014. These three datasets were all collected from real scenarios and manually labeled, covering common scenarios in the natural environment. The details of these three datasets are shown in Table 1 .

Table 1.

Details of the person reidentification public datasets.

Dataset	Identities	Cameras	Images	Label method
Market1501	1501	6	32217	Hand/DPM
MSMT17	4101	15	126441	Faster RCNN
CUHK03	1467	10	13164	Hand/DPM

2.1.2. Train data collection

To meet the practical deployment needs of large-scale personnel search and matching, and to enhance the reidentification performance of the model in complex real-world scenarios, constructing a large-scale training dataset is essential. In this study, we implemented a multi-dataset fusion strategy by combining the Market1501, CUHK03, and MSMT17 datasets to develop a large-scale person reidentification dataset. The dataset comprises 78,453 images of 3,902 individuals.

2.1.3. Test data collection

To evaluate the performance of the proposed method that combines person reidentification and geospatial information for contact tracking, a two-part test dataset was used. The first part consists of surveillance data collected in a real scenario, which is used to quantitatively assess the performance of the person reidentification model. Multiple cameras were used to capture data of people from various angles, distances, and lighting conditions. As suggested by Qin et al. (Qin et al., 2020), the test dataset was limited to surveillance data collected within 14 days, which is within the incubation period of over ninety percent of confirmed COVID-19 cases. The test dataset contained 2,280 images of 273 person identities, with 397 images randomly selected as queries and the remaining 1,883 images used as a gallery. The second part of the test data includes corresponding case data, including geographic information and epidemiological investigation data, collected from an office building located in the central region of Guangzhou, China.

Three distinct identities from the test set are presented in Fig. 2 . These examples demonstrate that the same person may vary in size and brightness due to differences in the shooting distance. Furthermore, the same person may exhibit different postures at different shooting moments, while different individuals may resemble each other in appearance. Such variations in size, color, and appearance may lead to misidentification of the model. By conducting experiments on the test data collected in the actual use scenario, the robustness and generalization of the model can be more effectively assessed. It is important to note that, in this study, all images were resized to $384\times 128$ pixels (height $\times$ width) for consistency during the experiment.

Fig. 2.

Examples of different identities in the test set used for evaluating the performance of the proposed person reidentification method. Each column represents images of the same person taken from different camera views, while each row represents images of different individuals captured by the same camera.

In addition to video surveillance data, various case data were collected in the real-world scenario to evaluate the effectiveness of the proposed method. Unlike traditional epidemiological surveys, which are typically limited to 2D maps displaying longitude and latitude, this study selected indoor scenarios for evaluation. The collected case data are diverse, including recorded geospatial information of the cameras during system deployment, dynamically captured video surveillance data stored on the server hard disk, and 3D building interior structure acquired through geographic information system and graphic design drawings. Additionally, images of individuals entering the building and their corresponding identity information were obtained at the building entrance.

The HIKVISION webcams with $1/{2.7}^{″}$ inches progressive scan CMOS sensors were selected as the surveillance cameras for this experiment. These cameras can transmit images from different perspectives with a resolution of $1280\times 720$ pixels simultaneously to a remote server via WiFi or a network cable. The experiments were conducted on an Ubuntu 18.04 operating system, using open source libraries including Python opencv, numpy, and Python programming language. The model training was performed on eight NVIDIA Tesla P100 graphics processing units (GPUs).

2.1.4. Evaluation criteria

Consistent with the methodology employed in Karanam et al. (2018), this study employed three evaluation metrics, namely first accuracy rate (Rank-1), first-five accuracy rate (Rank-5), and mean average precision (mAP) to assess the performance of the person reidentification model. Specifically, the test set images were split into two sets: query and gallery. For each query, one person image was randomly selected, and the top ten images with the highest similarity to the person were searched in the gallery. The evaluation metrics, including Rank-1, Rank-5, and mAP, were then calculated based on the similarity ranking results.

2.2. Person reidentification

Person reidentification is widely regarded as a subproblem of image retrieval (Nguyen et al., 2020), and it aims to retrieve a person’s image from different camera views based on the monitored image of the same person. By complementing the visual constraints of current fixed cameras, this model can be combined with person detection and tracking technologies, and can be widely applied in areas such as intelligent video surveillance and intelligent security. Fig. 3 illustrates the workflow of person reidentification, which includes both person detection and identification.

Fig. 3.

Schematic diagram of the proposed person reidentification workflow. The workflow includes three modules: the detection module, the query module, and the gallery module. The detection module generates the initial proposals of the person’s location in the image. The query module extracts the image of the target person from the video surveillance and matches it with the images in the gallery module to confirm the identity of the person. The comparison between the query and gallery modules plays a crucial role in the person reidentification process.

As discussed in Section 1, the effectiveness of contact tracking through person reidentification in natural environments can be affected by factors such as lighting changes, person size variations, and shooting angles. To address these challenges, the proposed method incorporates the concept of multi-module design (He et al., 2020) in the person reidentification model. The model utilizes three distinct modules to optimize the representation of human features and enhance the reidentification ability of the model in complex environments. The person reidentification model, illustrated in Fig. 1, consists of a convolutional neural network (CNN) backbone network with an instance-batch network (IBN) module, an improved spatial transformer networks (STN) module, and a multi-granularity network (MGN) module.

2.2.1. CNN backbone with IBN module

Fig. 1 displays that the video surveillance frames are input to the person reidentification model, which is a sequence of continuous images. To extract the crucial features from video surveillance frames, CNNs, which are widely employed in image recognition, are utilized. In this study, ResNet50 (He et al., 2016), one of the most versatile networks in CNN, is chosen as the backbone network for the model.

Addressing the lighting challenges in natural scenarios is of utmost importance in the proposed method. COVID-19 contacts can engage in daily activities both indoors and outdoors, which may cause significant changes in their appearance in video surveillance due to lighting variations. To mitigate this issue, the proposed method utilizes the CNN backbone with the IBN module (Pan et al., 2018) to normalize the images with different lighting conditions to similar lighting intensities. The architecture of the CNN backbone with the IBN module is presented in Fig. 4 . The IBN consists of two components: instance normalization (IN) and batch normalization (BN). The IN component learns the features that are not sensitive to shape changes, such as color, style, virtual or reality, while the BN component retains texture-related features. This module effectively reduces the impact of lighting changes on model identification by mitigating the influence of style changes.

Fig. 4.

Architecture of CNN backbone with IBN module. The figure illustrates the overall structure of the CNN backbone with the integrated instance normalization and batch normalization (IBN) module. The IBN module is designed to enhance the generalization and robustness of the model by effectively reducing the domain shift caused by variations in the input data. The architecture includes multiple convolutional layers and pooling layers, followed by fully connected layers and a ReLu activation function for classification. The figure provides a visual representation of the network’s structure and its component modules.

2.2.2. Improved STN module

After eliminating the influence of lighting changes on the performance of the model, the multi-scale and multi-shot angle of person in video surveillance is still a huge challenge. The same person will show different sizes at different distances from the camera, and when the person moves between different cameras, they may also show different appearances due to the different deployment positions of the cameras. To solve these challenges, this study follows the idea of Peng et al. (2020) and adopts an improved STN module, named Self-Constrained spatial transformation network (SC-STN), to solve the problem of multi-scale and multi-shot angle. The original STN (Jaderberg et al., 2015) uses adaptive affine transformation to reduce the influence of image spatial diversity, while SC-STN introduces self-constrained branches on the basis of STN, estimates detection errors, and limits the range of affine transformation matrix coefficients. In this way, the optimizing of the affine matrix can be constrained in a limited space, making the learning more stable and avoiding feature loss. The architecture of the SC-STN module is shown in Fig. 5 .

Fig. 5.

Architecture of SC-STN module. The module receives the feature maps extracted by the ResNet50-IBN backbone network and is composed of three branches that operate on the same feature maps in a parallel manner. The global branch preserves the global features of the input image, while the two local branches divide the feature maps horizontally to obtain the local features at different scales. The feature maps are transformed in a parametric controllable manner using the STN module, and the resulting feature maps are concatenated and processed by 1x1 convolutional layers for compression. The final output features and predicted probability distributions are obtained through a combination of averaging pooling, channel compression, batch normalization, and full connection layer operations.

The STN is composed of three key parts, namely the localization network, grid generator, and sampler. The localization network aims to estimate the parameters of a (2$\times$3) vector for a two-dimensional affine transformation. The grid generator constructs a sampling grid based on the predicted transformation parameters. The sampler takes both the sampling grid and the feature map as inputs to generate the output. Specifically, the sampler samples values from the input feature map according to the constructed sampling grid, and these values are used to construct the output feature map. The transformation process of STN can be mathematically formulated as:

$$\left[\begin{array}{ccc}{\theta }_{s1}\hfill & {\theta }_{r1}\hfill & {\theta }_{t1}\hfill \\ {\theta }_{r2}\hfill & {\theta }_{s2}\hfill & {\theta }_{t2}\hfill \end{array}\right]\left[\begin{array}{c}\hfill x^{\mathrm{Source}}\hfill \\ \hfill y^{\mathrm{Source}}\hfill \\ \hfill 1\hfill \end{array}\right]=\left[\begin{array}{c}{x}^{\mathrm{Target}}\hfill \\ y^{\mathrm{Target}}\hfill \end{array}\right]$$ (1)

where the transformation equation of STN involves three coefficients ${\theta }_s,{\theta }_r$, and ${\theta }_t$, representing scaling factor, rotation factor, and translation factor respectively. However, since rotation is not useful for feature alignment, the rotation coefficient is set to 0 in SC-STN. Furthermore, SC-STN introduces an additional coefficient to constrain the variation range of the coefficient matrix during matrix regression.

2.2.3. MGN module

To address the challenges posed by complex backgrounds in person reidentification, this study proposes the use of the MGN module for multi-granular feature fusion, building upon the concept presented in Wang et al. (2018). The module combines global and local features in video surveillance to overcome the limitations of global features, which can only capture the most obvious person features and are less effective in large and complex scenarios. The MGN module, as shown in Fig. 6 , consists of multiple branches, with each branch containing global average pooling (GAP), BN, and fully connected (FC) layers. The feature map extracted by the CNN backbone is divided into two or three blocks horizontally, and each branch takes one block as input, allowing the model to pay more attention to local features. This approach enables the extracted local features to obtain more refined and salient features, thereby increasing the model’s coverage. Additionally, the focal points of multi-granularity local features are different, which can achieve information complementation. The MGN module also combines global features of different scales to measure features. During training, each feature vector represents a person feature. During testing, all features are connected, effectively reducing the impact of image content changes on recognition in complex scenarios.

Fig. 6.

Architecture of MGN module, which consists of a backbone network for extracting the global feature map, followed by two parallel branches for extracting local features. Each branch divides the feature map horizontally into two or three parts to obtain local features with different scales. The output features are compressed, normalized, and connected to a fully connected layer for prediction.

The global feature map of the input image is extracted by the backbone network, while the MGN extracts local features of the image through horizontal uniform segmentation of the global feature map. To make local features have different scales, the feature map is divided into two and three parts respectively by MGN. The local feature sizes obtained by the two branches of MGN can be calculated by:

$$S_n=H/n\times W\times C(n=2/3)$$ (2)

where H denotes the height of the feature map, W is the width of the feature map, C represents the number of channels of the feature map, and $S_n$ refers to the local feature obtained. It should be noted that each branch also preserves global characteristics. To compress the features, global average pooling and 1 × 1 convolution layer are applied to each feature. After compression, batch normalization is performed on the features, and the prediction results are obtained by inputting the full connection layer. For a detailed understanding of the process, the original MGN can be referred to.

2.2.4. Loss function

During the training of the person reidentification model, the triplet loss function is utilized to optimize the Euclidean distance between feature vectors. This function helps in the comparison of paired image features and reduces the distance between positive sample pairs while increasing the distance between negative sample pairs, ultimately improving the quality of feature representations. As stated in Schroff et al. (2015), triplet loss is a popular loss function used to train the CNN-based model. The main objective of triplet loss is to learn a better feature representation, i.e., if the input feature vectors are very similar, triplet loss can learn the feature representation well when the difference between the two input vectors is small. The mathematical expression of triplet loss is:

$$L_{\mathrm{triplet}}=-{\displaystyle \sum _{i=1}^P}{\displaystyle \sum _{a=1}^R}{\left[\alpha +{\displaystyle \underset{p=1\dots r}{\max }}{‖F_a^{\mathit{ii}}-F_p^i‖}_2-{\displaystyle \underset{{\displaystyle \genfrac{}{}{0ex}{}{{\displaystyle \genfrac{}{}{0ex}{}{n-1}{j-\dots >}}}{j\ne /i}}}{\min }}{‖F_a^{\mathit{ii}}-F_n^j‖}_2\right]}_{+}$$ (3)

where P represents the total number of person images in a batch size, and R represents the number of images with the same identity tag in a batch size. $\alpha$ is a threshold parameter used to constrain the distance between positive and negative sample pairs in the feature space. $F_a^i,F_p^i$, and $F_n^j$ represent the characteristics of the anchor, positive sample, and negative sample, respectively. Here, the positive sample feature refers to the image with the same label as the anchor, while the negative sample feature refers to the image with a different label from the anchor.

Moreover, to optimize the cosine distance and measure the difference information between the predicted probability distribution and the true data label distribution, cross-entropy loss is employed. This loss function is defined as:

$$L_{\mathit{ce}}=-{\displaystyle \sum _{i=0}^n}p(x_i)\mathit{log}(q(x_i))$$ (4)

where n represents the total number of person pictures in a batch, $p(x_i)$ is the real probability distribution, and $q(x_i)$ indicates the predicted probability distribution.

2.3. Geospatial information

The person reidentification model can automatically identify a target individual across multiple surveillance cameras. However, the surveillance data is stored as discrete video data, and it is challenging to efficiently track potential contacts. To address this issue, the geospatial locations of the surveillance cameras are known in advance, which allows them to form a network within a certain range. By incorporating this network with geographic information systems (GIS), real-time monitoring and information recording of fixed areas can be achieved. This integration enables the formation of geospatial big data by combining geospatial information with surveillance data. The big data can be utilized to draw the 3D movement trajectory of the contacts, thus facilitating contact tracing.

2.3.1. 3D model generation

To establish the correlation between the discrete surveillance data and the geospatial information of cameras, this study leverages the 3D model of the monitored area established in virtual space to map the processed surveillance data from the person reidentification model. To obtain the basic geographic information of the monitored area, the open source OpenStreetMap project (Haklay and Weber, 2008) is utilized, and a 3D model of the monitored area is generated using the CityEngine$\circledR$ software. Furthermore, to determine the location of the contact more accurately, the existing 2D geographic information obtained from the map is fused with the floor plan of the building to obtain geospatial information such as the floor on which the contact is located in the office building. The schematic of the 3D model generation is illustrated in Fig. 7 .

Fig. 7.

Schematic of the 3D model generation process. The process involves two main steps: data collection and 3D model generation. In the data collection step, OpenStreetMap data and multiple video surveillance cameras data are used to capture the movements of individuals in the target area.The 3D model generation step involves mapping the filtered surveillance data onto a 3D model of the target area, using geographic information to ensure accuracy. This figure illustrates the process of generating the 3D model, which is an essential step in the proposed method for contact tracking using geospatial big data.

2.3.2. Trajectory drawing

Based on the generated 3D model, the challenge of integrating geospatial information and video surveillance data into geospatial big data to track contacts remains. To address this challenge, this study utilizes the timeline as an index to concatenate discrete camera geographic information with video surveillance data. After utilizing the person reidentification model to identify the target person in a video surveillance, the index of the camera and the identified time are recorded in the form of text. By finding the corresponding camera in the generated 3D model and connecting the geospatial position of each camera one by one according to the timeline, the movement trajectory of the target person can be drawn. The approach is illustrated in Fig. 8 .

Fig. 8.

Schematic of the motion trajectory drawing based on a timeline. The proposed method utilizes the surveillance data to locate the target person, and subsequently maps the data to a 3D model generated based on geospatial information. The motion trajectory of the target person is then drawn on a timeline, which allows for easy visualization and analysis of the person’s movements over time. The proposed method represents a novel approach to motion trajectory drawing that leverages both geospatial and temporal information.

Based on the trajectory of the target person obtained through geospatial big data, this study aims to identify key activity locations and close contacts of the target person in a timely manner. By repeating the above steps, the search scope can be expanded and direct and indirect contacts can be identified automatically and efficiently.

3. Experimental results

3.1. Performance of the person reidentification model

To quantitatively assess the performance of the person reidentification model, the proposed model trained on the fused public dataset was evaluated on the test set. A comparison experiment was also conducted to evaluate the proposed multi-module person reidentification method with some existing methods, such as DGNet, ISPNet, OSNet, and PCB. The results of the comparison experiment are presented in Table 2 . The proposed multi-module person reidentification method achieved high accuracy with 91.56% Rank-1, 97.70% Rank-5, and 78.03% mAP on the test set. Moreover, the average inference speed was 13 ms, which is approximately 77 frames per second (FPS), exceeding the 30 FPS acquisition speed of the camera. Therefore, the person reidentification model is capable of achieving real-time contact tracking.

Table 2.

Results of the comparison with existing methods.

Method	Rank-1	Rank-5	mAP
PCB (Sun et al., 2018)	75.45%	93.61%	59.15%
OSNet (Zhou et al., 2019)	79.54%	91.30%	57.54%
ISPNet (Zhu et al., 2020)	66.13%	82.81%	46.72%
DGNet (Zou et al., 2020)	83.38%	95.91%	63.23%
Ours	91.56%	97.70%	78.03%

This study presents the visualization result of the person reidentification model in Fig. 9 . The main client interface of the video surveillance system is displayed in Fig. 9(a), where up to 17 cameras can be viewed on one screen. After integrating the person reidentification model in the system, the camera screen containing the target person is displayed in the center of the main interface. If the target person moves outside the surveillance range, the system searches for the target in the surveillance domain of other cameras. When the target is located, the surveillance screen in the center can be switched to the target’s camera screen at any time. Moreover, the matching result and similarity score between the detection result and the sample are automatically pushed, as shown in Fig. 9(b). To adjust the camera angle, a camera angle adjustment button is available at the bottom of the main screen.

Fig. 9.

Visualization result of the proposed person reidentification method. In Figure (a), surveillance images from multiple cameras are shown, with one designated as the main image and the others as candidate images. Figure (b) demonstrates that the target person in the query is identified with a high degree of consistency with an individual captured by the camera in one of the candidate images. The corresponding candidate image is then displayed on the main screen, and a pop-up window presents the recognition results for contact tracking.

3.2. Case analysis of contact tracking

A comprehensive evaluation of the proposed method was conducted through a case analysis on the test set using the constructed person reidentification model and camera’s geographic information. To this end, the 3D model of the monitored area was generated based on the geographic information in the test set, and the locations of the cameras were marked in the model accordingly, as illustrated in Fig. 10 . This analysis enabled us to assess the method’s effectiveness in identifying and tracking the target person in a complex and dynamic environment.

Fig. 10.

Example of the 3D model generated from the surveillance data in the current case, where the trajectories of the target person are drawn for contact tracking. The model is generated based on geographic information and mapped with the surveillance data, providing a spatial context for analyzing the movement patterns of the target person.

To combine the discrete camera geospatial information and surveillance data, the real-time surveillance images collected by the cameras were integrated into the 3D model of the monitored area. This approach resulted in the formation of geospatial big data from originally discrete camera geospatial information and the surveillance data stored on the hard disk. As shown in Fig. 11 , when a target person is detected by one of the cameras, the camera index and the time of identification are recorded in the form of text. The newly indexed camera locations are linked on the map in chronological order to plot the movement trajectory of the target person in the 3D model.

Fig. 11.

Example of mapping the surveillance data to the generated 3D model and drawing the 3D movement trajectory of the target person for contact tracking in the current case.

Fig. 12 presents a case study based on the test set, wherein the proposed contact tracking method was employed to identify the target person in video surveillance by enclosing the person with different colored bounding boxes. Based on the surveillance video data collected over a period of time, the 3D movement trajectory of the target person was also plotted, showcasing the effectiveness of the proposed approach. To provide a visual understanding of the proposed method, additional examples can be found at https://youtu.be/kxf5Ajm2R_I.

Fig. 12.

example of the proposed contact tracking method, which combines person reidentification and geospatial information to track COVID-19 contacts. The figure demonstrates how the method can accurately identify and track the movement of a target person across multiple camera views in a complex environment. The identified contact points (represented by the blue and red circles on 3D model) provide critical information for epidemiological investigations and public health management. The proposed method significantly enhances the efficiency and scalability of contact tracing efforts, and holds great potential for real-world applications.

To evaluate the efficacy of the proposed method for epidemiological investigations, a professional epidemiological investigator who participated in the COVID-19 epidemic investigation in Liwan, Guangzhou in May 2021 was enlisted to participate in a case analysis. The investigator recorded and photographed all individuals entering the building within the past 14 days at the building’s entrance, enabling a comprehensive record of all individuals present in the video surveillance. Using the accelerated video surveillance approach, the investigator employed the proposed method to identify close contacts of the target person by analyzing instances where the person and the target appeared in the same shot. The investigator required approximately 37 h to identify all close contacts of the target person in the eight-channel video using existing methods, whereas the proposed method required only 5.5 h to identify close contacts of the target person in the past 14 days. Notably, the proposed method can designate the close contacts of the target person as the new target, enabling the identification of more indirect contacts and greatly expanding the scope of screening, which is not possible with existing epidemiological investigation methods.

4. Discussion

4.1. Ablation study of the person reidentification

The present study compared the performance and inference speed of several backbone networks on real datasets. In particular, ResNet101 was considered a larger network than ResNet50, while ResNest was regarded as an improvement over ResNet. As reported in Table 3 , although ResNet101, ResNest50, and ResNest101 displayed better performance, their inference speed decreased significantly, thereby rendering them unsuitable for real-time deployment. As a result, ResNet50 was adopted as the backbone network, since it provided a reasonable compromise between inference speed and performance.

Table 3.

Comparison of different backbone networks and inference speed.

Method	Rank-1	Rank-5	mAP	Speed (ms/img)
ResNet50	91.56%	97.70%	78.03%	13.3
ResNest50(w/o IBN)	91.82%	97.95%	76.49%	38.2
ResNet101	92.07%	97.95%	78.10%	40.3
ResNest101(w/o IBN)	91.29%	98.21%	78.04%	87.2

As previously mentioned, this research proposes a multi-module approach to address the challenges encountered in practical scenarios, such as lighting, shooting angle, and multi-scale variations. To demonstrate the effectiveness of each module, an ablation study was conducted by removing one of the modules and evaluating the performance of the proposed method. Table 4 shows the results of the ablation study. It is evident that each module plays a vital role in addressing complex environmental factors, and the proposed method achieves superior performance with all modules integrated.

Table 4.

Ablation study of each module.

Method	Rank-1	Rank-5	mAP
ResNet50_IBN_SC-STN_MGN	91.56%	97.70%	78.03%
w/o SC-STN	90.03%	98.21%	77.64%
w/o IBN	88.75%	97.70%	77.57%
w/o MGN	86.70%	96.16%	73.69%

4.2. Limitations

While the previous experiments have demonstrated the efficacy of the proposed method for tracking COVID-19 contacts, this study acknowledges certain limitations. Firstly, the method unavoidably infringes on a portion of individual privacy. Additionally, the coverage of cameras is a potential factor that could affect the applicability of the proposed method.

Owing to the need to track the movement trajectory of individuals for epidemiological investigations, people in public areas will unavoidably appear in surveillance footage to determine their possible exposure to COVID-19. Nonetheless, existing methods of contact tracing, such as Bluetooth or mobile applications, cannot escape the risk of privacy breaches. By contrast, the person reidentification approach employed in this study relies on information such as clothing and posture of individuals, which is less sensitive compared to face recognition. Moreover, the proposed method leverages geospatial big data technology to enhance the efficiency of traditional epidemiological investigations that already commonly utilize video surveillance. Despite the potential infringement of privacy, which is an unavoidable limitation in tracking individuals’ trajectories, the proposed method represents a necessary compromise to ensure public health, as current approaches to contact tracing do not fully resolve this issue.

One limitation that cannot be overlooked in video surveillance methods is the existence of camera blind spots. This limitation is unavoidable due to the limited deployment density of cameras and the constraints on their angles and coverage range. It poses a challenge to tracking contacts based on video surveillance. However, this limitation can be mitigated by optimizing the camera placement. Epidemiological statistical surveys have shown that small and confined spaces such as elevators and offices are the main channels for the transmission of COVID-19, while relatively open areas pose less risk (Pinheiro and Luís, 2020). Therefore, cameras can be strategically deployed in high-risk areas to ensure that all contacts can be comprehensively identified.

5. Conclusion and future works

This study presents a novel geospatial big data method for contact tracking of COVID-19, which combines person reidentification and geospatial information. Unlike existing methods that depend on mobile applications or Bluetooth connections, the proposed method utilizes a multi-module person reidentification model that takes advantage of widely deployed video surveillance for target person localization. Then, the surveillance data are mapped to a 3D model based on geographic information, and the 3D movement trajectory of the target person is traced for contact tracking. This represents the first method that integrates person reidentification and geospatial big data. By avoiding the reliance on mobile phones and wearable devices, this method can be deployed on various scales, from office buildings to urban cities. The proposed method significantly enhances the efficiency of epidemiological investigations and holds great potential for ensuring public health.

Based on the experimental results presented in Table 2, we observed that the proposed method achieves promising performance in person reidentification task by leveraging multiple publicly available datasets for training and testing on real-world scenarios. Notably, when trained with a single dataset (Market-1501), the method achieved Rank-1 and mAP results of 84.65% and 66.06%, respectively. This suggests that enlarging the scale of the training dataset could lead to further improvement of the method’s performance. These findings provide further evidence of the potential of the proposed method for addressing the challenges of person reidentification in real-world applications.

Moreover, an ablation study was carried out to evaluate the effectiveness of each module in the proposed person reidentification model, demonstrating the method’s robustness against various challenges encountered in complex environments, including lighting variations, shooting angles, and multi-scale issues. Additionally, different backbone networks were assessed and selected based on practical scenarios. Finally, a case study was presented to showcase the proposed method’s effectiveness in tracking COVID-19 contacts, and visualizations of the results were provided to serve as a reference for future applications of the method.

In the future, we plan to extend the proposed method by exploring various ways to optimize the person reidentification model to handle larger datasets. This may involve the utilization of deep learning algorithms or other machine learning techniques. Furthermore, we aim to evaluate the proposed method on a more diverse range of scenarios, including crowded public spaces, transportation systems, and other areas where contact tracing may be critical. In doing so, we will investigate the impact of different environmental factors on the performance of the method, such as lighting and weather conditions. Through these efforts, we seek to further validate the proposed method and enhance its robustness for real-world applications.

Moreover, we plan to investigate the potential of improving the accuracy of the proposed method by incorporating additional data sources, such as weather and traffic data. This will enable us to enhance the contextual awareness of the system and improve its ability to handle complex scenarios, such as crowded public spaces and transportation systems. To achieve this, we will conduct experiments to evaluate the impact of integrating such data sources on the performance of the proposed method. Furthermore, we will explore the use of other deep learning models and develop new algorithms to process and integrate such data sources. These efforts will allow us to further validate and improve the proposed method for real-world applications.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.