Abstract
Recent literature has reported a high percentage of asymptomatic or paucisymptomatic cases in subjects with COVID-19 infection. This proportion can be difficult to quantify; therefore, it constitutes a hidden population. This study aims to develop a proof-of-concept method for estimating the number of undocumented infections of COVID-19. This is the protocol for the INCIDENT (Hidden COVID-19 Cases Network Estimation) study, an online, cross-sectional survey with snowball sampling based on the network scale-up method (NSUM). The original personal network size estimation method was based on a fixed-effects maximum likelihood estimator. We propose an extension of previous Bayesian estimation methods to estimate the unknown network size using the Markov chain Monte Carlo algorithm. On 6 May 2020, 1963 questionnaires were collected, 1703 were completed except for the random questions, and 1652 were completed in all three sections. The algorithm was initialized at the first iteration and applied to the whole dataset. Knowing the number of asymptomatic COVID-19 cases is extremely important for reducing the spread of the virus. Our approach reduces the number of questions posed. This allows us to speed up the completion of the questionnaire with a subsequent reduction in the nonresponse rate.
Keywords: COVID-19, NSUM, network scale-up method, hidden population
1. Introduction
Since December 2019, China and subsequently the whole world have been dealing with a pandemic due to a betacoronavirus related to the Middle East respiratory syndrome virus (MERS-CoV) and the severe acute respiratory syndrome virus (SARS-CoV2), named COVID-19 by the World Health Organization (WHO) [1]. The virus quickly spread globally [2,3]. In the Italian territory, the outbreak started with cases of pneumonia of unknown etiology at the end of January 2020.
Recent literature has highlighted a high percentage of undocumented cases among COVID-19-infected subjects. Such cases are mostly asymptomatic or paucisymptomatic, as their lack or scarcity of symptoms does not reach the attention of the healthcare system. Undocumented cases have been found to expose a higher proportion of the population due to the lack of quarantine measures [4] and to be hard to recognize, as asymptomatic or mildly symptomatic patients often do not seek medical attention due to a lack of symptoms [5]. While challenging, the prevalence estimation for asymptomatic and mildly symptomatic cases is very important given the highly contagious nature of the virus. Zou et al. [6] reported that the viral load in asymptomatic patients was similar to that in symptomatic carriers. Therefore, both asymptomatic and symptomatic patients may have the same transmissibility potential. Confirmed positive but asymptomatic people also need to be isolated to limit the contact with others. Consequently, accurate epidemiological monitoring of COVID-19 prevalence in asymptomatic people may also further decrease viral contagion. Moreover, it will help in the proper distribution of resources, tailoring the prevention program to the outbreak’s containment [7].
Several studies have tried to reveal undocumented cases. For example, using a networked dynamic metapopulation model and a Bayesian inference in mobility data within China, Li et al. [3] estimated that 86% of all infections were undocumented (95% CI: 82–90%) before the 23 January 2020 travel restrictions. Mizumoto et al. [8], in their study conducted on the Diamond Princess cruise ship, showed that there was a considerable proportion of asymptomatic individuals among all infected cases, which was 17.9% (95% credible interval: 15.5–20.2%). Other estimates of undiagnosed patients with COVID were among the evacuated citizens. Nishiura et al. [5,7] estimated a proportion of 33.3% asymptomatic cases (95% CI: 8.3–58.3%) among Japanese citizens evacuated from Wuhan. Undocumented infections seem to facilitate the geographic spread of SARS-COVID-19 [3]. In the Veneto region, in the municipality of Vo’, which was one of the initial outbreak sites in Italy, the choice to test the overall population helped to identify the proportion of positive COVID-19, revealing 37.7% asymptomatic patients (95% CI 25.5–51.9%) [9]. Properly estimating the number of COVID-19 positive cases, even if asymptomatic, is important since person-to-person transmission can occur from asymptomatic COVID-19 cases to the community, as shown in previous studies [10,11].
Traditional (i.e., direct) methods to detect positive cases are based on the seroepidemiological testing procedures of potentially exposed or infected populations. Real-time PCR tests or other laboratory tests may identify asymptomatic infections [12]. However, these approaches are time-consuming and require considerable financial resources [13]. Other indirect sample estimation methods, instead, are not suitable since they feature limitations, such as the use of independent samples, direct access to the source of data, or data of each country, as in capture-recapture technique, multiplier method, synthetic estimation and multivariate indicator methods, respectively [14,15].
Since undocumented infections are undefined, they constitute a hidden population. The network scale-up method (NSUM), which was first proposed by Bernard et al. [16,17], is among the recommended methods to estimate the hidden population available in the literature. NSUM has been widely used to estimate the size of hard-to-reach populations due to the stigmatizing nature of its knowledge, such as HIV [18], injuries [19], men who have sex with men (MSM) [20], and others. It relies on the idea that the probability of knowing someone in a specific subpopulation is related to the relative size of that subpopulation, i.e., the proportion computed based on the population size of all individuals.
2. Materials and Methods
The Hidden COVID-19 Cases Network Estimation (INCIDENT) study aims to develop a proof-of-concept study for estimating the number of undocumented COVID-19 infections using a Bayesian approach of the traditional NSUM.
2.1. Study Design
This is a cross-sectional survey-design study to assess the prevalence of undocumented COVID-19 symptoms in Italy using an anonymous online questionnaire. The data were collected starting on 15 April 2020 and will continue until the winter of 2020.
2.1.1. Procedures
To avoid unnecessary interactions, this study was structured only for electronic distribution given the social distancing and the limitations imposed by the Italian government.
The questionnaire was created through LimeSurvey (LimeSurvey GmbH, Hamburg, Germany), a professional open-source online survey tool. The respondents had to be at least 16 years old and were required to sign an informed consent.
The study design mimicked snowball social network sampling but had a nonrandom entry point in the population. The snowball sampling method was chosen since it is widely applied and evaluated as particularly useful in studies that consider hidden populations as a target [21,22]. The questionnaire was advertised via social networks, mobile messaging systems, emails, and newspapers (Supplementary Materials).
2.1.2. Ethics
The INCIDENT study was approved by the Ethics Committee of the University of Torino, protocol No 458163, 10 October 2020.
2.2. The Network Scale-Up Method in the Literature
NSUM was first applied to estimate the death toll of the 1985 Mexico City earthquake [17] due to the missing reports regarding fatalities by official registry. Since then, NSUM has been widely used to estimate the size of a subpopulation that consists of hard-to-identify individuals [23], such as individuals with high-risk behaviors that lead to stigmatization and discrimination, such as individuals living with HIV/AIDS [24,25,26,27,28,29,30,31], MSM [20,32], sex workers [23,33], drug addicts [33,34,35,36,37,38], or alcohol users [34,39]. NSUM has also been used to estimate the number of treatment failures [40], people with disabilities [41], number of abortions [42,43], and suicide attempts [43]. Teo et al. [26] suggested the use of NSUM for estimating the hidden population to improve surveillance, prevention, and treatment after proper methodological adjustments.
Table 1 shows articles using NSUM to estimate hidden populations. The search was based on a review in PubMed (Figure 1).
Table 1.
Studies that estimate the hidden population using the network scale-up method or its modification.
| Author | Year * | N Known Population | Hidden Population | N Respondents | Method | Adjustment | Place |
|---|---|---|---|---|---|---|---|
| Ahmadi [34] | 2019 | 1 | Drug/alcohol users before driving | 363 | NSUM | Iran | |
| Bernard [17] | 1991 | 6 | Deaths in earthquake | NSUM | Mexico | ||
| Carletti [45] | 2017 | 20 | Oncological patients | 299 | NSUM | Italy | |
| Ezoe [20] | 2012 | 3 | Men who have sex with men | NSUM | Transmission Error | Japan | |
| Feehan [27] | 2016 | 22 | Populations at risk for HIV/AIDS | 4669 | Blended Scale-up | Rwanda | |
| Guo [25] | 2013 | 3 | Populations at risk for HIV/AIDS | 2957 | NSUM | China | |
| Habecker [51] | 2015 | 18 | Moved to Nebraska in US during last 2 years, do not approve of interracial dating, heroin users | 618 | Mean Of Sums NSUM | United States | |
| Haghdoost [52] | 2015 | ** | Population of breast, ovarian/cervical, prostate, and bladder cancers | 3052 | NSUM | Iran | |
| Heydari [40] | 2019 | 25 | Treatment failure | 2550 | NSUM | Iran | |
| Jafari [28] | 2014 | 29 | Populations at risk for HIV/AIDS | 500 | NSUM | Transmission Bias, Barrier Effects | Iran |
| Jing [23] | 2018 | 48 | Female sex worker | RRT, NSUM | Response Bias | China | |
| Kadushin [38] | 2006 | 3 | Heroin users | NSUM | United States | ||
| Kazemzadeh [53] | 2016 | ** | High-risk behaviors | 563 | CM, NSUM | Iran | |
| Killworth [24] | 1998 | 24 | HIV prevalence, women who have been raped, the homeless | 1554 | NSUM | United States | |
| Maghsoudi [33] | 2014 | 20 | Injection drug users, female sex workers | 600 | NSUM | Barrier Effect | Iran |
| Maltiel [29] | 2015 | 29 | Populations at risk for HIV/AIDS | 500 | Bayesian NSUM | Transmission Bias, Barrier Effects | Brazil |
| Mccormick [48] | 2010 | 12 | Personal network size | 1370 | Latent Non-Random Mixing Model NSUM | Transmission Bias, Barrier Effects, And Recall Bias. | Brazil |
| Mohebbi [41] | 2014 | ** | People with disabilities | 3052 | NSUM | Iran | |
| Moradinazar [54] | 2019 | ** | Suicides and suicide attempts | 500 | NSUM | Iran | |
| Narouee [35] | 2019 | ** | Injection drug users | 1000 | NSUM | Barrier Effect | Iran |
| Narouee [55] | 2020 | Rural area | 1000 | MLE—NSUM | |||
| Nikfarjam [39] | 2017 | ** | Alcohol use | 12,000 | NSUM | Transmission Bias, Barrier Effects | Iran |
| Nikfarjam [36] | 2016 | ** | Illicit drug users | 7535 | NSUM | Iran | |
| Rastegari [42] | 2014 | ** | Abortions | 12,960 | NSUM | Transmission Bias, Barrier Effects | Iran |
| Sajjadi [50] | 2018 | 6 | Students with high-risk behaviors | 801 | NSUM | Transmission Bias, Barrier Effects | Iran |
| Salganik [30] | 2011 | 20 | Populations at risk for HIV/AIDS | NSUM, GSU | Transmission Bias, Barrier Effects | Brazil | |
| Shokoohi [56] | 2010 | 6 | Network | 500 | NSUM | Iran | |
| Shokoohi [31] | 2012 | ** | Populations at risk for HIV/AIDS | 500 | NSUM | Iran | |
| Snidero [44] | 2012 | 33 | Foreign body injuries | 1081 | NSUM | Italy | |
| Teo [26] | 2019 | 24 | Populations at risk for HIV/AIDS | 199 | Bayesian NSUM | Singapore | |
| Vardanjani [57] | 2015 | ** | Cancer | 195 | Generalized NSUM | Iran | |
| Wang [32] | 2015 | 22 | Men who have sex with men | 3097 | NSUM | China | |
| Zahedi [37] | 2018 | ** | Drug users | 2157 | NSUM | Barrier Effect | Iran |
| Zamanian [58] | 2016 | 25 | Age-gender distribution of women | 1275 | NSUM | Iran | |
| Zamanian [59] | 2019 | 25 | Abortion | 1500 | NSUM | Barrier Effect | Iran |
N, Number; MLE, maximum likelihood estimation; NSUM = network scale-up method; RRT = randomized response technique; CM, crosswise model. * year of publication, ** Rastegari et al. [60].
Figure 1.
Flowchart study review.
In the Italian context, NSUM has been used to estimate the number of children with foreign body injuries [44] and to assess the perceived quality of care (PQC) in an oncological center. In the latter study, estimation of PQC was lower than in the traditional questionnaire; in some cases, the level of dissatisfaction was 20-fold higher [45]. Among other selected applications, Paniotto et al. [46] estimated the number of drug addicts, sex workers, and MSM in their study, showing that NSUM estimates were similar or lower compared to other estimation methods. Additionally, estimated populations of seropositive, homeless, and female victims of violence in Killworth et al. [24] were comparable to official data. One limitation of NSUM is that social and physical barriers, such as ethnicity, occupation, or location of residence, may influence the likelihood that respondents know people in hidden populations. This is known as the barrier effect. On the other hand, individuals may not know everything about other people in their personal network. This instance, in which a contact does not share information with the respondent, is termed transmission bias. Other limitations in applying these methods include recall bias [29] and response bias [47]. Several authors have tried to address such limitations, as shown in Table 1. McCormick et al. [48] adjusted NSUM for recall bias, various authors adjusted for barrier effects [49] and transmission errors [28,29,30,39,42,48,50], and Jing et al. [23] adjusted for response bias.
2.2.1. NSUM Questionnaire
This questionnaire is structured into three sections: (1) four questions for the demographic characteristics of the respondent (gender, age, nationality, and region of residence), (2) four target questions related to COVID-19 disease that were defined by consulting the available literature on COVID-19 [61], and (3) one question used for the estimation of social network size that was randomly drawn from 15 known populations (see Supplementary Materials for the full report of the questionnaire). The data source for the known population size is the Italian National Institute of Statistics (ISTAT) [62] (Table 2).
Table 2.
List of known-size populations used in the questionnaire for the estimation of respondents’ network sizes.
| Sub-Population of Known Size | Population Size | Reference Year | Source |
|---|---|---|---|
| People who separated | 99,611 | 2016 | Demographic model |
| Foreign residents | 5,255,503 *** | 2019 | Demographic model |
| Victims of car accidents with injuries | 3334 | 2018 | Demographic model |
| People who graduated | 8530 ** | 2018 | MIUR |
| People working part-time | 3,689,153 *** | 2019 | Demographic model |
| Three-member families | 4954 | 2019 | AVQ |
| Cohabiting couples | 14,110 | 2019 | AVQ |
| People who married | 195,778 | 2018 | Demographic model |
| Children born | 440,780 | 2018 | Demographic model |
| People above 14 with smoking habits | 10,122 | 2017 | AVQ |
| People using the mass media (newspapers, magazines, TV, radio, etc.) | 86,142 * | 2017 | AVQ |
| People who attend places of worship | 14,264 | 2018 | AVQ |
| People who walk to work | 2750 | 2018 | AVQ |
| People who go to school by bus | 8743 | 2018 | AVQ |
| Three-year-olds and above who used a PC and Internet | 62,232 | 2017 | AVQ |
* Number watching TV 53953, radio 32189. ** http://dati.ustat.miur.it/dataset/laureati/resource/43df861d-7345-481a-9803-2eb236aa022e (accessed on 16 April 2020), (difference between 2018 and 2017, 326332-317802). *** cumulative data. MIUR, Ministry of Instruction, University and Research, http://dati.ustat.miur.it/dataset/laureati (accessed on 16 April 2020); AVQ, Aspects of daily life survey, http://dati.istat.it/ (accessed on 16 April 2020).
All questions concerning the specific subpopulations were introduced with the sentence: “How many people do you know …?”. In this study, we used the definition of knowing someone based on those provided by Bernard et al. [63] and already used in previous Italian studies [44,45].
2.2.2. NSUM Assumption
The NSUM estimation method, as explained by Bernard et al. [63] “rests on the assumption that people’s social networks are, on average, representative of the general population in which they live and move.” For example, if a responder assesses to knowing 100 subjects on average and two of them are COVID-19 positive, we estimate a prevalence of 2/100 COVID-19-positive subjects considering as reference point the personal network size. This estimated prevalence is combined with the known size of the general population to estimate the size of hard-to-reach populations, such as the COVID-19-infected population. The accuracy of the estimated size of the hidden population increases as the number of people who answer the question increases.
The NSUM, however, has some limitations. For example, people may not know all the characteristics of their personal network (i.e., a respondent may not know that a member of his or her network is affected by COVID-19.) This is called the transmission bias [48]. In addition, social and physical barriers, such as ethnicity, race, occupation, and location of residence, can cause variations in the probability that respondents know people in hidden populations; this is called the barrier effect [24]. Despite these biases, NSUM has two major advantages. First of all, this method does not ask the respondent for information on its characteristics. For example, stigmatized or hidden populations may be reluctant to disclose their status even in an anonymous survey [51]. Secondly, it is not necessary to directly interview the members of a hidden population, but the NSUM allows the use of considerably cheaper and easier-to-implement sampling techniques that make use of established sampling frames [38].
2.2.3. NSUM Estimator
Letter T is the size of the general population, is the number of subjects in the hidden population known by individual i in the subpopulation, k and is the estimated average size of the social network related to the individual i. The scale-up estimator is based on the assumption that the number of subjects known to the respondent in the k-th subpopulation follows a binomial distribution [64] where . The scale-up estimator of the hidden population size is obtained by the following equation:
| (1) |
To estimate the size of the hidden population, we follow three steps:
-
(1)Estimate the average size of the personal network, , by asking how many people the respondent personally knows about the k known populations (e.g., the number of people who were married in 2019). This number will then be divided by the number of people who got married in 2019 in Italy (), where is the total size of subpopulation k.
(2) -
(2)
Define the number of hidden COVID-19 cases present in each social network, for example, by asking the respondent how many people he/she knows with COVID-19.
-
(3)
Calculate the COVID-19 population size obtained by multiplying the estimated proportions of the population in each subpopulation by the general Italian population. For example, if a respondent knows 10 subjects with COVID-19 cases and has a personal network of 100 people and the total population is 1,000,000, the estimated number of hidden COVID-19 cases will be approximately calculated as: 10/100 × 1,000,000.
2.3. Bayesian NSUM Estimation
Under maximum likelihood estimation, several known populations should be used to reduce the variance of the estimates. This prolongs the time required to complete the questionnaire by increasing the likelihood of dropouts and nonresponses. Estimating the size of the network by considering the known population as partially unknown could be a solution to shorten the length of the survey. For this reason, the Bayesian estimation methods proposed by Maltiel et al. [29] will be extended to estimate the unknown network size using the Markov chain Monte Carlo (MCMC) algorithm.
Moreover, the original personal network size-estimation method proposed by Killworth and colleagues [64] was based on fixed-effects, maximum likelihood estimators in which the network size was considered a nonrandom component. Instead, Maltiel et al. [29] extended this approach in a Bayesian setting by treating personal network sizes as random variables. This allows us to generalize the model to account for the variation in respondents’ propensity to know people in particular subgroups (barrier effects), such as their tendency to know people like themselves, and their lack of awareness to recognize their contacts’ group memberships (transmission bias).
2.3.1. Extended Random Degree Model
The NSUM formulation proposed by Maltiel et al. [29] assumes that the estimate of an individual’s network degree, , improves if a respondent knows a considerable number of subjects in a subpopulation. Network estimation is embedded into a Bayesian hierarchical modeling framework where the lognormal distribution best fits the network estimates across multiple datasets [29].
| (3) |
values are the number of subjects that the i-th subject knew in the k-th subpopulation. Observed values are assumed to be a realization of a binomial random variable whose parameters are defined by the personal network degree () and the overall known proportion of subjects in the subpopulations (), where is the size of the k-th known sub-population and T is the overall population size.
The parameters of the random degree will be estimated in a Bayesian manner using the uninformative priors for the k-th subpopulation, as in the Maltiel et al. [29] work:
| (4) |
priors (hidden population parameter) have been used in the literature for Bayesian estimation of population size with vague prior [29]. The μ and σ prior distributions were derived by Maltiel et al. [29], identifying the best fit to the network estimates across multiple datasets.
In our NSUM formulation, the number of subjects known by the respondent in the k-th subpopulation is unknown, except for the target question that identifies the hard-to-reach subpopulation and the question drawn of the known subpopulation extracted for each respondent from the list of the known subpopulation.
The model is reformulated assuming is partially unknown. The number of subjects that the i-th respondent knows in the k-th subpopulation for each MCMC iteration will be drawn from a binomial random variable () except for the target question that identifies the hard-to-reach subpopulation and the known population.
2.3.2. Performance of the Modified Maltiel Estimators
The algorithm was initialized at the first iteration from the first resampled values defining the starting values using the Killworth method [64].
The performance of the proposed NSUM estimator was evaluated in a simulation study considering different study size scenarios (1000, 1500, 2000, 2500, 3000).
Answers were generated by assuming the sizes of known subpopulations as indicated in the McCarty NSUM study [64] for each scenario. Data were generated using the original nsum.simulate() function proposed by McCarty in the NSUM R package [65]. The data were generated 300 times. For each replication, Maltiel’s NSUM model and our NSUM proposal were estimated with 500 iterations, discarding the first 50 (burn-in iterations).
The total Italian population in January 2020 was 60,244,639 [62]. The assumed prevalence of COVID-19-positive cases is 1.37% (800,000 cases, like the peak of 805,947 officially diagnosed positive cases on 22 November 2020).
To compute the modified version of Maltiel’s short-form questionnaire for each responder, we randomly sampled one of the responses characterizing the known populations from the full generated database, leaving the other responses as missing.
The 95% credible interval (CI) was computed.
The performance of the estimators was computed by calculating the average CI over simulations with the average prevalence estimate and the bias for each study size.
Computations were performed using R 3.6.2 [66] software.
2.4. Statistical Analysis
Data will be presented in aggregate form, and it will not be possible to trace information or make comparisons on an individual level.
Continuous variables will be summarized using the median (quartiles I and III), and qualitative variables will be synthesized using percentages and absolute numbers. Comparisons between groups will be identified using the Wilcoxon–Kruskal–Wallis test for continuous variables and the Pearson chi-square test for categorical variables.
3. Results of the Simulation Study
The INCIDENT study is still ongoing. A total of 1963 questionnaires have been collected until 6 May 2020, 1703 were completed except for the random questions, and 1652 were completed in all three sections. The respondents were primarily female (1206, 61%), and the prevalent residents were the areas most affected by COVID-19, including Veneto, Lombardia, and Piemonte.
Performances of the Modified Maltiel’s Estimators
The estimated prevalence remains approximately constant as the sample size increases for both Maltiel’s method (close to 1.37%) and the modified version (close to 1.57%). The bias remains approximately −0.05% and 0.2% for the original version and the modified version of the method, respectively. The length of the interval decreases as the sample size increases. A sample size of 2000 subjects guarantees a CI length of 0.044% for the modified method (bias = 1.197) and 0.038 (bias = −0.051) for the original approach (Table 3).
Table 3.
Simulation results for the modified Maltiel’s method performance.
| Study Size | Maltiel’s Method | Modified Maltiel’s Method | |||||
|---|---|---|---|---|---|---|---|
| Benchmark Prevalence | Prevalence% | 95% CI Length | Bias | Prevalence% | 95% CI Length | Bias | |
| 1000 | 1.37 | 1.324 | 0.056 | −0.056 | 1.561 | 0.074 | 0.191 |
| 1500 | 1.37 | 1.327 | 0.044 | −0.053 | 1.57 | 0.056 | 0.200 |
| 2000 | 1.37 | 1.329 | 0.038 | −0.051 | 1.567 | 0.044 | 0.197 |
| 2500 | 1.37 | 1.329 | 0.031 | −0.051 | 1.566 | 0.038 | 0.196 |
| 3000 | 1.37 | 1.33 | 0.025 | −0.05 | 1.574 | 0.032 | 0.204 |
For each study size, the average prevalence (%) and the 95% CI were reported for Maltiel’s method and the modified version. The benchmark prevalence represents the true value used to generate the data. The bias (average estimated prevalence—benchmark) was reported for each method.
The modified version of the method demonstrates a slightly higher bias than the original method; however, prevalence estimates are similar between the two methods.
4. Discussion
The results of the simulation study proposed in this protocol revealed that our NSUM model demonstrates a slightly higher bias than the original method. However, prevalence estimates are similar between the two methods.
The NSUM as modified in this study has numerous advantages. First, questions for the scale-up social network estimator can be easily integrated into the survey. As shown in our literature review, most of the time needed to complete the questionnaire is used to estimate the size of the social network [26,28,30,32,40,43,44,45,51]. Some authors have preferred to estimate the size of the active network population only once, as in a previous Iranian population study [60], in order to reduce the time for survey completion.
In our study, we posed questions only related to the hidden population [31,35,36,37,39,41,42,54,57]. Few studies have chosen to use questions related to known populations [17,20,25,26]. Ahmadi et al. [34] proposed only one known population, as in our work. However, with traditional NSUM, our questionnaire would have needed 15–20 known populations; with our method, only one question is needed. This allows to speed up the completion of the questionnaire by reducing the dropout and nonresponse rate.
Moreover, the modified NSU method allows estimation of the size of the personal network independently from the survey on COVID-19. This is possible because data for the estimation of the size of the social network are collected on a separate subpopulation from the general survey.
Direct methods for the estimation of undocumented cases are resource and time consuming due to the requirement of large-scale testing procedures. Viable alternatives that may overcome such limitation are the introduction of statistically strong sampling design for estimating the parameters of the epidemics, such as the one proposed by Alleva et al. [67]. Other methodological statistical studies have tried to estimate the true number of COVID-19-infected people indirectly. Palatella et al. [4] tried to estimate the number of cases based on PCR test alone and Noh and Danuser [68] based on government daily counts of confirmed cases. However, these estimates could underestimate the real proportion since the data are related to the specific population.
In literature, there are different indirect sample estimation methods, but those are not suitable for the purpose of our study. For example, capture-recapture technique requires at least two independent and representative samples [14]. Multiplier and enumeration methods, on the other hand, require direct access to the target population [15].
5. Conclusions
NSUM shows its advantages when we utilized it for the same survey in different contexts, as in our study with snowball sampling. Therefore, data for the estimation of the social network size can be collected in a standardized way both spatially and temporally. This would be extremely difficult if the methods instead required direct access to the hidden population as the target of the investigation. Despite the bias of the traditional NSUM, our modified version shows advantages, considering that only one question was used for defining the social network.
The NSUM can help tracing the undocumented cases that could be known in the people’s social network but do not draw the attention of the healthcare system. Our method would be useful when government testing is not widespread enough to reach the overall population, such as with COVID-19.
Acknowledgments
We would like to address special thanks to the INCIDENT study group for their support of the project and participation in the study activities: Honoria Ocagli, Danila Azzolina, Giulia Lorenzoni, Silvia Gallipoli, Matteo Martinato, Aslihan S. Acar, Paola Berchialla, Dario Gregori, Marco Ghidina, Nicolas Destro, Corrado Lanera, Federica Zobec and Solidea Baldas. We also would like to thank Avis Montegrotto Terme for having contributed to the distribution of the questionnaire. Most importantly, we gratefully acknowledge the study participants for their vital contributions to the study.
Supplementary Materials
The following are available online at https://www.mdpi.com/article/10.3390/ijerph18115713/s1, Questionnaire.
Author Contributions
Conceptualization, D.G. and P.B.; methodology, D.G., P.B. and D.A.; formal analysis, D.A.; investigation, M.M.; resources, S.G.; data curation, H.O., S.G.; writing—original draft preparation, H.O.; writing—review and editing, D.A., G.L. and A.S.A; supervision, D.G.; project administration. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the University of Torino, protocol No 458163, 21 October 2020.
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy.
Conflicts of Interest
The authors declare no conflict of interest.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.WHO Statement Regarding Cluster of Pneumonia Cases in Wuhan, China. [(accessed on 16 April 2020)]; Available online: https://www.who.int/china/news/detail/09-01-2020-who-statement-regarding-cluster-of-pneumonia-cases-in-wuhan-china.
- 2.Cascella M., Rajnik M., Cuomo A., Dulebohn S.C., Di Napoli R. StatPearls. StatPearls Publishing; Treasure Island, FL, USA: 2020. Features, Evaluation and Treatment Coronavirus (COVID-19) [PubMed] [Google Scholar]
- 3.Li R., Pei S., Chen B., Song Y., Zhang T., Yang W., Shaman J. Substantial Undocumented Infection Facilitates the Rapid Dissemination of Novel Coronavirus (SARS-CoV2) Science. 2020 doi: 10.1126/science.abb3221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Palatella L., Vanni F., Lambert D. A Phenomenological Estimate of the True Scale of CoViD-19 from Primary Data. Chaos Solitons Fractals. 2021;146:110854. doi: 10.1016/j.chaos.2021.110854. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Nishiura H., Kobayashi T., Yang Y., Hayashi K., Miyama T., Kinoshita R., Linton N.M., Jung S., Yuan B., Suzuki A., et al. The Rate of Underascertainment of Novel Coronavirus (2019-NCoV) Infection: Estimation Using Japanese Passengers Data on Evacuation Flights. J. Clin. Med. 2020;9:419. doi: 10.3390/jcm9020419. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Zou L., Ruan F., Huang M., Liang L., Huang H., Hong Z., Yu J., Kang M., Song Y., Xia J., et al. SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients. N. Engl. J. Med. 2020 doi: 10.1056/NEJMc2001737. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Nishiura H., Kobayashi T., Suzuki A., Jung S.-M., Hayashi K., Kinoshita R., Yang Y., Yuan B., Akhmetzhanov A.R., Linton N.M., et al. Estimation of the Asymptomatic Ratio of Novel Coronavirus Infections (COVID-19) Int. J. Infect. Dis. 2020 doi: 10.1016/j.ijid.2020.03.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Mizumoto K., Kagaya K., Zarebski A., Chowell G. Estimating the Asymptomatic Proportion of Coronavirus Disease 2019 (COVID-19) Cases on Board the Diamond Princess Cruise Ship, Yokohama, Japan, 2020. Eurosurveillance. 2020;25:2000180. doi: 10.2807/1560-7917.ES.2020.25.10.2000180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lavezzo E., Franchin E., Ciavarella C., Cuomo-Dannenburg G., Barzon L., Del Vecchio C., Rossi L., Manganelli R., Loregian A., Navarin N., et al. Suppression of a SARS-CoV-2 Outbreak in the Italian Municipality of Vo’. Nature. 2020 doi: 10.1038/s41586-020-2488-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bai Y., Yao L., Wei T., Tian F., Jin D.-Y., Chen L., Wang M. Presumed Asymptomatic Carrier Transmission of COVID-19. JAMA. 2020;323:1406–1407. doi: 10.1001/jama.2020.2565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wei W.E. Presymptomatic Transmission of SARS-CoV-2—Singapore, January 23 March 16, 2020. MMWR Morb. Mortal. Wkly. Rep. 2020;69 doi: 10.15585/mmwr.mm6914e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Laboratory Testing for 2019 Novel Coronavirus (2019-NCoV) in Suspected Human Cases. [(accessed on 2 July 2020)]; Available online: https://www.who.int/publications-detail-redirect/10665-331501.
- 13.Evans A.S. Viral Infections of Humans: Epidemiology and Control. Springer Science & Business Media; Berlin, Germany: 2013. [Google Scholar]
- 14.International Working Group for Disease Monitoring and Forecasting Capture-Recapture and Multiple-Record Systems Estimation II: Applications in Human Diseases. Am. J. Epidemiol. 1995;142:1059–1068. doi: 10.1093/oxfordjournals.aje.a117559. [DOI] [PubMed] [Google Scholar]
- 15.World Health Organization . Guidelines on Estimating the Size of Populations Most at Risk to HIV. World Health Organization; Geneva, Switzerland: 2010. [Google Scholar]
- 16.Johnsen E.C., Killworth P.D., Robinson S. Estimating the Size of an Average Personal Network and of an Event Subpopulation. Small World. 1989:159–175. doi: 10.1016/0049-089X(91)90012-R. [DOI] [Google Scholar]
- 17.Bernard H.R., Johnsen E.C., Killworth P.D., Robinson S. Estimating the Size of an Average Personal Network and of an Event Subpopulation: Some Empirical Results. Soc. Sci. Res. 1991;20:109–121. doi: 10.1016/0049-089X(91)90012-R. [DOI] [Google Scholar]
- 18.Verdery A.M., Weir S., Reynolds Z., Mulholland G., Edwards J.K. Estimating Hidden Population Sizes with Venue-Based Sampling. Epidemiology. 2019;30:901–910. doi: 10.1097/EDE.0000000000001059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Snidero S., Morra B., Corradetti R., Gregori D. Use of the Scale-up Methods in Injury Prevention Research: An Empirical Assessment to the Case of Choking in Children. Soc. Netw. 2007;29:527–538. doi: 10.1016/j.socnet.2007.03.003. [DOI] [Google Scholar]
- 20.Ezoe S., Morooka T., Noda T., Sabin M.L., Koike S. Population Size Estimation of Men Who Have Sex with Men through the Network Scale-up Method in Japan. PLoS ONE. 2012;7:e31184. doi: 10.1371/journal.pone.0031184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Sadler G.R., Lee H.-C., Seung-Hwan Lim R., Fullerton J. Recruiting Hard-to-Reach United States Population Sub-Groups via Adaptations of Snowball Sampling Strategy. Nurs. Health Sci. 2010;12:369–374. doi: 10.1111/j.1442-2018.2010.00541.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Biernacki P., Waldorf D. Snowball Sampling: Problems and Techniques of Chain Referral Sampling. Sociol. Methods Res. 2016 doi: 10.1177/004912418101000205. [DOI] [Google Scholar]
- 23.Jing L., Lu Q., Cui Y., Yu H., Wang T. Combining the Randomized Response Technique and the Network Scale-up Method to Estimate the Female Sex Worker Population Size: An Exploratory Study. Public Health. 2018;160:81–86. doi: 10.1016/j.puhe.2018.04.002. [DOI] [PubMed] [Google Scholar]
- 24.Killworth P.D., McCarty C., Bernard H.R., Shelley G.A., Johnsen E.C. Estimation of Seroprevalence, Rape, and Homelessness in the United States Using a Social Network Approach. Eval. Rev. 1998;22:289–308. doi: 10.1177/0193841X9802200205. [DOI] [PubMed] [Google Scholar]
- 25.Guo W., Bao S., Lin W., Wu G., Zhang W., Hladik W., Abdul-Quader A., Bulterys M., Fuller S., Wang L. Estimating the Size of HIV Key Affected Populations in Chongqing, China, Using the Network Scale-up Method. PLoS ONE. 2013;8 doi: 10.1371/journal.pone.0071796. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Teo A.K.J., Prem K., Chen M.I.C., Roellin A., Wong M.L., La H.H., Cook A.R. Estimating the Size of Key Populations for HIV in Singapore Using the Network Scale-up Method. Sex. Transm. Infect. 2019;95:602–607. doi: 10.1136/sextrans-2018-053747. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Feehan D.M., Umubyeyi A., Mahy M., Hladik W., Salganik M.J. Quantity Versus Quality: A Survey Experiment to Improve the Network Scale-up Method. Am. J. Epidemiol. 2016;183:747–757. doi: 10.1093/aje/kwv287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Jafari Khounigh Ali A., Haghdoost A.A., SalariLak S., Zeinalzadeh A.H., Yousefi-Farkhad R., Mohammadzadeh M., Holakouie-Naieni K. Size Estimation of Most-at-Risk Groups of HIV/AIDS Using Network Scale-up in Tabriz, Iran. J. Clin. Res. Gov. 2014;3:21–26. [Google Scholar]
- 29.Maltiel R., Raftery A.E., McCormick T.H., Baraff A.J. Estimating Population Size Using the Network Scale Up Method. Ann. Appl. Stat. 2015;9:1247–1277. doi: 10.1214/15-AOAS827. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Salganik M.J., Fazito D., Mello M.B. Estimating the Number of Heavy Drug Users in Curitiba, Brazil Using Multiple Methods. Technical Report. UNAIDS; New York, NY, USA: 2010. [Google Scholar]
- 31.Shokoohi M., Baneshi M.R., Haghdoost A.-A. Size Estimation of Groups at High Risk of HIV/AIDS Using Network Scale Up in Kerman, Iran. Int. J. Prev. Med. 2012;3:471–476. [PMC free article] [PubMed] [Google Scholar]
- 32.Wang J., Yang Y., Zhao W., Su H., Zhao Y., Chen Y., Zhang T., Zhang T. Application of Network Scale up Method in the Estimation of Population Size for Men Who Have Sex with Men in Shanghai, China. PLoS ONE. 2015;10:e0143118. doi: 10.1371/journal.pone.0143118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Maghsoudi A., Baneshi M.R., Neydavoodi M., Haghdoost A. Network Scale-Up Correction Factors for Population Size Estimation of People Who Inject Drugs and Female Sex Workers in Iran. PLoS ONE. 2014;9 doi: 10.1371/journal.pone.0110917. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Ahmadi-Gohari M., Zolala F., Iranpour A., Baneshi M.R. Twelve-Hour before Driving Prevalence of Alcohol and Drug Use among Heavy Vehicle Drivers in South East of Iran Using Network Scale Up. Addict Health. 2019;11:256–261. doi: 10.22122/ahj.v11i4.252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Narouee S., Shati M., Nasehi M., Dadgar F. The Size Estimation of Injection Drug Users (IDUs) Using the Network Scale-up Method (NSUM) in Iranshahr, Iran. Med. J. Islam. Repub. Iran. 2019;33:158. doi: 10.34171/mjiri.33.158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Nikfarjam A., Shokoohi M., Shahesmaeili A., Haghdoost A.A., Baneshi M.R., Haji-Maghsoudi S., Rastegari A., Nasehi A.A., Memaryan N., Tarjoman T. National Population Size Estimation of Illicit Drug Users through the Network Scale-up Method in 2013 in Iran. Int. J. Drug Policy. 2016;31:147–152. doi: 10.1016/j.drugpo.2016.01.013. [DOI] [PubMed] [Google Scholar]
- 37.Zahedi R., Noroozi A., Hajebi A., Haghdoost A.A., Baneshi M.R., Sharifi H., Mirzazadeh A. Self-Reported and Network Scale-Up Estimates of Substance Use Prevalence among University Students in Kerman, Iran. J. Res. Health Sci. 2018;18:e00413. [PubMed] [Google Scholar]
- 38.Kadushin C., Killworth P.D., Bernard H.R., Beveridge A.A. Scale-up Methods as Applied to Estimates of Heroin Use. J. Drug Issues. 2006;36:417–440. doi: 10.1177/002204260603600209. [DOI] [Google Scholar]
- 39.Nikfarjam A., Hajimaghsoudi S., Rastegari A., Haghdoost A.A., Nasehi A.A., Memaryan N., Tarjoman T., Baneshi M.R. The Frequency of Alcohol Use in Iranian Urban Population: The Results of a National Network Scale Up Survey. Int. J. Health Policy Manag. 2016;6:97–102. doi: 10.15171/ijhpm.2016.103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Heydari Z., Baneshi M.R., Sharifi H., Zamanian M., Haji-Maghsoudi S., Zolala F. Evaluation of the Treatment Failure Ratio in Individuals Receiving Methadone Maintenance Therapy via the Network Scale up Method. Int. J. Drug Policy. 2019;73:36–41. doi: 10.1016/j.drugpo.2019.07.022. [DOI] [PubMed] [Google Scholar]
- 41.Mohebbi E., Baneshi M.R., Haji-maghsoodi S., Haghdoost A.A. The Application of Network Scale Up Method on Estimating the Prevalence of Some Disabilities in the Southeast of Iran. J. Res. Health Sci. 2014;14:272–275. [PubMed] [Google Scholar]
- 42.Rastegari A., Baneshi M.R., Haji-maghsoudi S., Nakhaee N., Eslami M., Malekafzali H., Haghdoost A.A. Estimating the Annual Incidence of Abortions in Iran Applying a Network Scale-up Approach. Iran. Red Crescent Med. J. 2014;16 doi: 10.5812/ircmj.15765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Zamanian M., Zolala F., Haghdoost A.A., Haji-Maghsoudi S., Heydari Z., Baneshi M.R. Methodological Considerations in Using the Network Scale Up (NSU) for the Estimation of Risky Behaviors of Particular Age-Gender Groups: An Example in the Case of Intentional Abortion. PLoS ONE. 2019;14 doi: 10.1371/journal.pone.0217481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Snidero S., Soriani N., Baldi I., Zobec F., Berchialla P., Gregori D. Scale-up Approach in CATI Surveys for Estimating the Number of Foreign Body Injuries in the Aero-Digestive Tract in Children. Int. J. Environ. Res. Public Health. 2012;9:4056–4067. doi: 10.3390/ijerph9114056. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Carletti G., Soriani N., Mattiazzi M., Gregori D. A Social Network Approach to the Estimation of Perceived Quality of Health Care. Open Nurs. J. 2017;11:219–231. doi: 10.2174/1874434601711010219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Paniotto V., Petrenko T., Kupriyanov O., Pakhok O. Estimating the Size of Populations with High Risk for HIV Using the Network Scale-up Method. Kiev International Institute of SociologyO; Kiev, Ukraine: 2009. [Google Scholar]
- 47.World Health Organization . 2010-Guidelines on Estimating the Size of Populations m.Pdf. WHO; Geneva, Switzerland: 2010. [Google Scholar]
- 48.McCormick T.H., Salganik M.J., Zheng T. How Many People Do You Know?: Efficiently Estimating Personal Network Size. J. Am. Stat. Assoc. 2010;105:59–70. doi: 10.1198/jasa.2009.ap08518. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Haghdoost A., Ahmadi Gohari M., Mirzazadeh A., Zolala F., Baneshi M.R. A Review of Methods to Estimate the Visibility Factor for Bias Correction in Network Scale-up Studies. Epidemiol. Health. 2018;40 doi: 10.4178/epih.e2018041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Sajjadi H., Jorjoran Shushtari Z., Shati M., Salimi Y., Dejman M., Vameghi M., Karimi S., Mahmoodi Z. An Indirect Estimation of the Population Size of Students with High-Risk Behaviors in Select Universities of Medical Sciences: A Network Scale-up Study. PLoS ONE. 2018;13:e0195364. doi: 10.1371/journal.pone.0195364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Habecker P., Dombrowski K., Khan B. Improving the Network Scale-Up Estimator: Incorporating Means of Sums, Recursive Back Estimation, and Sampling Weights. PLoS ONE. 2015;10:e0143406. doi: 10.1371/journal.pone.0143406. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Haghdoost A.A., Baneshi M.R., Haji-Maghsoodi S., Molavi-Vardanjani H., Mohebbi E. Application of a Network Scale-up Method to Estimate the Size of Population of Breast, Ovarian/Cervical, Prostate and Bladder Cancers. Asian Pac. J. Cancer Prev. 2015;16:3273–3277. doi: 10.7314/APJCP.2015.16.8.3273. [DOI] [PubMed] [Google Scholar]
- 53.Kazemzadeh Y., Shokoohi M., Baneshi M.R., Haghdoost A.A. The Frequency of High-Risk Behaviors Among Iranian College Students Using Indirect Methods: Network Scale-Up and Crosswise Model. Int. J. High Risk Behav. Addict. 2016;5 doi: 10.5812/ijhrba.25130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Moradinazar M., Najafi F., Baneshi M.R., Haghdoost A.A. Size Estimation of Under-Reported Suicides and Suicide Attempts Using Network Scale up Method. Bull Emerg. Trauma. 2019;7:99–104. doi: 10.29252/beat-070202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Narouee S., Shatti M., Didevar M., Nasehi M. Estimating Social Network Size Using Network Scale-up Method (NSUM) in Iranshahr, Sistan and Baluchestan Province, Iran. Med. J. Islam. Repub. Iran. 2020;34:35. doi: 10.34171/mjiri.34.35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Shokoohi M., Baneshi M.R., Haghdoost A.A. Estimation of the Active Network Size of Kermanian Males. Addict. Health. 2010;2:81–88. [PMC free article] [PubMed] [Google Scholar]
- 57.Vardanjani H.M., Baneshi M.R., Haghdoost A. Total and Partial Prevalence of Cancer Across Kerman Province, Iran, in 2014, Using an Adapted Generalized Network Scale-Up Method. Asian Pac. J. Cancer Prev. 2015;16:5493–5498. doi: 10.7314/APJCP.2015.16.13.5493. [DOI] [PubMed] [Google Scholar]
- 58.Zamanian M., Baneshi M.R., Haghdoost A.A., Mokhtari-Sorkhani T., Amiri F., Zolala F. Estimating the Size and Age-Gender Distribution of Women’s Active Social Networks. Addict. Health. 2016;8:170–178. [PMC free article] [PubMed] [Google Scholar]
- 59.Zamanian M., Zolala F., Haghdoost A.A., Baneshi M.R. Estimating The Annual Abortion Rate in Kerman, Iran: Comparison of Direct, Network Scale-Up, and Single Sample Count Methods. Int. J. Fertil. Steril. 2019;13:209–214. doi: 10.22074/ijfs.2019.5721. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Gulati A., Pomeranz C., Qamar Z., Thomas S., Frisch D., George G., Summer R., DeSimone J., Sundaram B. A Comprehensive Review of Manifestations of Novel Coronaviruses in the Context of Deadly COVID-19 Global Pandemic. Am. J. Med. Sci. 2020 doi: 10.1016/j.amjms.2020.05.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Istat.It. [(accessed on 16 April 2020)]; Available online: https://www.istat.it/
- 62.Bernard H.R., Johnsen E.C., Killworth P.D., McCarty C., Shelley G.A., Robinson S. Comparing Four Different Methods for Measuring Personal Social Networks. Soc. Netw. 1990;12:179–215. doi: 10.1016/0378-8733(90)90005-T. [DOI] [Google Scholar]
- 63.Killworth P.D., Johnsen E.C., McCarty C., Shelley G.A., Bernard H.R. A Social Network Approach to Estimating Seroprevalence in the United States. Soc. Netw. 1998;20:23–50. doi: 10.1016/S0378-8733(96)00305-X. [DOI] [PubMed] [Google Scholar]
- 64.Maltiel R., Baraff A.J. NSUM: Network Scale Up Method; R package version, 1. [(accessed on 16 april 2020)];2015 Available online: https://cran.r-project.org/web/packages/NSUM/NSUM.pdf.
- 65.Team R.C. R: A Language and Environment for Statistical Computing. [(accessed on 16 april 2020)];2013 Available online: http://softlibre.unizar.es/manuales/aplicaciones/r/fullrefman.pdf.
- 66.Rastegari A., Haji-Maghsoudi S., Haghdoost A., Shatti M., Tarjoman T., Baneshi M.R. The Estimation of Active Social Network Size of the Iranian Population. Glob. J. Health Sci. 2013;5:217–227. doi: 10.5539/gjhs.v5n4p217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Alleva G., Arbia G., Falorsi P.D., Zuliani A. A Sample Approach to the Estimation of the Critical Parameters of the SARS-CoV-2 Epidemics: An Operational Design with a Focus on the Italian Health System. arXiv. 20042004.06068 [Google Scholar]
- 68.Noh J., Danuser G. Estimation of the Fraction of COVID-19 Infected People in U.S. States and Countries Worldwide. PLoS ONE. 2021;16 doi: 10.1371/journal.pone.0246772. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy.

