\documentclass{bmcart}
%\RequirePackage[authoryear]{natbib}% uncomment this for author-year bibliography
%\RequirePackage{hyperref}
\usepackage[utf8]{inputenc} %unicode support
%\usepackage[applemac]{inputenc} %applemac support if unicode package fails
%\usepackage[latin1]{inputenc} %UNIX support if unicode package fails
\usepackage{lineno}
\usepackage{times}
\usepackage{textcomp}
\usepackage{amsmath}
%for including pdfs
\usepackage{pdfpages}
\usepackage{epstopdf}
\usepackage[flushleft]{threeparttable}
\usepackage{graphicx}
\usepackage{algpseudocode,algorithm,algorithmicx}
\usepackage{enumitem}
% definning new commands for algorithms
%\newcommand*\DNA{\textsc{dna}}
%\newcommand*\Let[2]{\State #1 $\gets$ #2}
\algrenewcommand\algorithmicrequire{\textbf{Precondition:}}
\algrenewcommand\algorithmicensure{\textbf{Postcondition:}}
% using foreach
\algnewcommand\algorithmicforeach{\textbf{for each}}
\algdef{S}[FOR]{ForEach}[1]{\algorithmicforeach\ #1\ \algorithmicdo}

\usepackage{nicefrac}
\usepackage{tikz}
\usepackage{lscape}
\usetikzlibrary{shapes, arrows, arrows.meta, shapes.geometric, calc, positioning, backgrounds}

\title{Supplementary Material}
\author{}
\date{\vspace{-12ex}}
\usepackage[utf8]{inputenc}

\begin{document}
\maketitle \\\ 

\section*{\center 1 - Description of different linkage tools} \\ \\\

\textbf{RecLink} is a C++-based record linkage tool implementing probabilistic data linkage methods. Records are blocked based on soundex functions applied over name attributes, and further compared by three different similarity functions. The user can define cut-off points to customize matching decisions. \\\

\noindent \textbf{Febrl} is a Python-based data linkage pipeline implementing data cleansing, de-duplication and pairwise comparison. It has different standards for names, dates, and addresses. Febrl implements seven indexing methods which are customizable by the user. More than 20 different functions are supported for pairwise comparison, as well supervised and unsupervised classifiers are used to perform weight vector classification.\\\

\noindent \textbf{FRIL} is a Java-based tool providing a set of highly customizable functions. Data integration (or reconciliation) is supported through different merging and splitting functions. The user can choose different strategies for pairwise comparison: nested loop joins, for small data sets, or sorted neighborhood or index search, for large data sets. Matching weights are defined based on the expectation-maximization (EM) method. FRIL runs transparently over multi-core architectures. \\\

\noindent \textbf{AtyImo} is a probabilistic data linkage tool, jointly developed by UFBA and CIDACS between 2013 and 2016, that runs distributed over Spark or in parallel over CUDA in hybrid (multicore+multi-GPU) architectures and is freely available on Github. It implements a pipelining comprising data pre-processing (cleansing, standardization, blocking and anonymization), pairwise comparison and accuracy assessment. Blocking is based on different predicates built with five linking attributes (name, mother’s name, date of birth, sex and municipality). \\\

\noindent In AtyImo, anonymization is based on a 128-bit Bloom filter, which guarantees privacy-preserving requisites related to sensitive (identifiable) data, allowing AtyImo to run within less protected environments if needed. AtyImo implements a two-round linkage step in which a mixture of deterministic and probabilistic methods can be used together to generate high accurate data marts (domain specific data). Finally, accuracy assessment can be performed manually, based on gold standards (when existent), to certify small data marts; or automatically, based on supervised machine methods, to certify big data marts.

% Insert Table 3. comparative description of existing linkage tools

\begin{landscape}
\begin{table}[h]

\caption{\centering{Comparative analysis of existing linkage tools}} \\\ \\

\begin{tabular}{p{3cm}p{3cm}p{3cm}p{3cm}p{3cm}p{3cm}p{3cm}}
\hline \\
Feature         & RecLink          & AtyImo     & CIDACS-RL     & FRIL     & Febrl \\   
\hline  \\\
Deterministic   & Pure comparison  & Hybrid approach   & Exact query    & Equality function    & Exact comparison functions \\ \\\ 
Probabilistic   & Character sequence and fuzzy     & Fully probabilistic                                    & Semi-exact and fuzzy queries     & Edit distance, soundex and Q-gram 
                & Approximate comparison functions \\\ \\
Blocking & One step (single attribute) and multi-step (predicates) 
         & Predicates       & TF-IDF indexing              
         & Nested loop join and Sorted neighborhood & Block, sorted and fuzzy (bigram)\\\ \\
Anonymization   & No    & Bloom filter    & No     & No      & No \\\  \\
Manual review of dubious records     & No                
                                     & Second round with adjusted cut-offs 
                                     & Yes      & Yes     & Yes \\\ \\
Automated review of dubious records  & No       & Machine learning-based   & No 
                                     & Yes (expectation maximization)      & Expected \\\ \\
Open source, freely available        & Yes(GPL) & Yes     & Yes    & Yes   & Yes \\\ \\     
\hline
\end{tabular}
\end{table}
\end{landscape}

%\maketitle
\section*{\center 2 Datasets used and construction of gold standard data} 

\begin{table}[ht]
\centering
\caption{Public health datasets (hosted at CIDACS) used in this study}
\label{tbl:datasetsused}
\begin{tabular}{p{2cm} p{7cm} p{2cm}}
\hline \\\ 
 Dataset &Description and data sources  &Coverage\\\ \\ 
\hline \\\ 
Gold Standard & Linked records from mortality (SIM) and live births (SINASC) database. Linkage attributes: name, mother's name, date of birth, municipality and gender, including DN number. It is used to compare the performance of the linkage tools. & \\  \\\  
&SINASC/Live births registrations. Social and demographic data of mother and father, the pregnancy, the delivery and the newborn. Main attribute is DN (unique number identifying the newborn).&2001--2015.\\  \\\
&SIM/Mortality data. Social and demographic information on the deaths and detailed information on the cause of death. For children less than one year old, it also includes the DN number. &2000--2015.\\  \\\

Case Study &Linked records from socioeconomic (CadUnico) and disease notification (SINAN-TB) databases. It is used to compare the performance of CIDSACS-RL & \\ \\\
&CadUnico: main database for social protection programmes storing demographic and socioeconomic data from low-income families. The baseline dataset used for linkage inside CIDACS has 114,008,179 records (individuals) extracted from CadUnico. Linkage attributes are: name, mother‚Äôs name, date of birth, gender, and municipality of residence. &2001--2015.\\  \\\
&SINAN-TB: clinical cases of diseases of compulsory notifications. In this paper, we extracted 1,182,777 tuberculosis notifications. Linkage attributes are: name, mother‚Äôs name, date of birth, gender, and municipality of residence. &2001--2013.\\  \\\

Simulated & Synthetic Dataset generated to assess the scalability of CIDACS-RL. Linkage attributes: name, mother's name, date of birth, municipality and gender & Not Applicable.\\\ \\
\hline
\end{tabular}
\end{table}

\subsection*{\center Creation of the Gold Standard Dataset}\\\ \\

SINASC was implemented in 1990 to collect information on live births throughout the country, based on the "Statement of Live Birth" (DN --- Declara{\c{c}}{\~a}o de Nascido Vivo), a standardized form used to obtain the Birth Certificate. DN is filled out by health professionals and midwives responsible for the delivery or care of newborns. This system records social and demographic data of the mother and father, the pregnancy, the delivery and the newborn. The SINASC database obtained by CIDACS covers the period 2001--2015.\\\

SIM was developed in 1975 to collect data on deaths in the whole country. Data are collected by states and municipalities, through their respective Health Secretariats, based on a Death Certificate, which is a legal, standardized document filled predominantly by physicians. This data source contain social and demographic information on the deaths and detailed information on the cause of death. For children less than one year old, it also includes the DN number. The SIM database obtained by CIDACS covers the  period 2000--2015. \\\

The gold standard dataset was created as follows, also depicted in Figure 1:\\
\begin{enumerate}

\item Only records related to the year 2015 were used in the experiment: 3,013,228 live births and 1,293,219 deaths;\\\

\item Records with missing or invalid DN number, the unique key identifier, were removed from SIM, as they could not be deterministically linked to SINASC. Names with generic or invalid information (e.g. unknown or stillbirth) or with only one name (e.g. Maria) were also removed. We also removed records with void and invalid names from SINASC, as the majority of the babies did not have a name yet;\\\

\item Both SIM and SINASC records of the same child were linked via SINASC registration  (DN) number by exact match, adding up to 3,028 records. Thus, stillbirths who had the same DN number on both systems (SIM and SINASC) were also linked.\\\

\item The SINASC registration (DN) number  was used during the experiment to assess the performance of each tool.\\\

\item A number of non-linked records from SIM and SINASC were added to the final linked dataset in order to simulate a number of matches and non-matches usually found in CIDACS datasets. We added 3,429 records to SIM dataset and 10,017 to SINASC dataset expected to be non-linkable to evaluate linkage performance tools for identifying false matches.\\\

\item Each of the five linkage tools (AtyImo, Febrl, CIDACS\_RL, FRIL and RecLink) produced a data file with the matched pairs and their respective similarity scores. Except from FRIL and RecLink, all other tools produce a resulting data mart of size equals to the smallest dataset linked.

FRIL and RecLink require a threshold to be set prior to execution and they include any pair with a score higher than the threshold in the result, while AtyImo, CIDACS-RL and Febrl keeps only the pair with the highest score found amongst all candidate pairs.\\\ \\

\item When a record from the smallest dataset is matched with two different records on the larger dataset, only the pair with the highest similarity score was considered for metrics calculation. \\\

\item After selecting cut-off points for each record linkage tool, sensitivity, specificity and PPV metrics were calculated.\\\ \\

\end{enumerate} 

\pagebreak

\maketitle
\section*{\center \textbf{ 3 - Comparison Metrics}} \\\ \\
\begin{enumerate}
\item \textbf{Sensitivity or true-positive rate} is the proportion of true matches that have been correctly identified as matches. It is also known as recall and is measured as follows:
	\begin{equation} \text{Sensitivity} = \frac{\text{Number of TP}}{\text{Number of TP + Number of FN}} \end{equation}

\item \textbf{Specificity or true-negative rate} is the proportion of true non-matches that have been correctly identified as non-matches. It is measured as follows:
	\begin{equation}\text{Specificity} = \frac{\text{Number of TN}}{\text{Number of TN + Number of FP}} \end{equation}

This metric is not recommended due to the fact that blocking enables identification of the large number of potential matches leaving a bulk of true non-matches in the comparison space and tending to dominate the calculation.

\item \textbf{Positive predictive value (PPV) or precision} refers to the proportion of all classified matches that are true matches. It is measured as follows:
	\begin{equation} \text{PPV} = \frac{\text{Number of TP}}{\text{Number of TP + Number of FP}} \end{equation} 

Ideally both types of errors (FPs and FNs) need to be minimized but there is always an underlying trade-off between sensitivity and precision (when one is higher, the other is invariably lower).

\item \textbf{F-measure}, which is the harmonic mean of sensitivity and PPV, is thus a way of finding the best compromise between the two metrics. It is calculated as follows:
	\begin{equation}\text{F-measure} = 2*{\frac{\text{PPV*Sensitivity}}{\text{PPV + Sensitivity}}} \end{equation}

\item \textbf{Receiver operating characteristic (ROC) curve} is plotted with the true positive rate (sensitivity) on the vertical axis against the false positive rate (1-specificity) on the horizontal axis for a varying threshold. 

Area under ROC curve (AUC) is a single numerical measure between 0.5 and 1 as the ROC curve is always plotted in the unit square, with a random classifier having an AUC value of 0.5 and larger values indicating better classifier performance. \\\

The AUC has the statistical property of being equivalent to the statistical Wilcoxon-Mann-Whitney U-Statistic %\cite{hajian2013receiver} 
and is also closely related to the Gini coefficient. 
%\cite{hand2001simple}.
While ROC curves are considered robust against skewed class distributions, the problem when using them in data linkage is the number of true negatives, which only appears in the false positive rate. Therefore, this rate will be calculated too low, resulting in too optimistic ROC curves.% \cite{davis2006relationship,christen2007quality}.
\end{enumerate} 
\vfill



\pagebreak
\maketitle 
\section*{\center \textbf{4 - Descriptive Analysis of CADU and SINAN-TB Linkage}}

\begin{table}[ht]
\centering
\caption{\centering Descriptive Analysis of CADU and SINAN-TB Linkage by Demographic Characteristics.}\\\

\label{tbl:cadu-sinantb-linkage-analysis}
\begin{tabular} {l p{1.75cm} p{1.75cm} p{1.25cm} p{1.75cm} p{1.25cm}}
\hline\\\
          & \textbf{All}     & \textbf{Non-linked} &     & \textbf{Linked}  &\\
          & N (\%)           & N & \%      & N & \%  \\\ \\
\hline \\\
\textbf{Total}    & 1,182,777        &680,121 & 57.50  &502,656 &42.50 \\\ \\
\textbf{Race/Skin color} &           &   &        &   &\\
White             & 326,844(27.63)  &185,777 &27.32   &141,067 &28.06 \\
Black             & 129,724(10.97)  &74,238  &10.91   &55,486  &11.04 \\
Yellow            & 10,163(0.86)    &5,830   &0.86    &4,333   &0.86  \\
Brown             & 385,826(32.62)  &223,743 &32.90   &162,083 &32.25 \\
Indigenous        & 11,077(0.94)    &6,541   &0.96    &4,083   &0.81 \\
Missings/invalid  & 319,143(26.98)  &183,992 &27.05   &135,151 &26.89 \\\ \\
\textbf{Sex}      &                  &        &        &         &\\
Male              & 779,325(65.89)  &490,604 &72.13   &288,721 &57.44 \\
Female            & 403,154(34.09)  &189,332 &27.84   &213,822 &42.53 \\
Missings          & 298(0.03)      &185     &0.03    &113     &0.02 \\\ \\
\textbf{Age groups} &                &        &        &        &\\
Below 1 year      &5,479(0.46)    &3,178   &0.47    &2,301   &0.46 \\
 1 - 4 years      &10,734(0.91)    &6,245   &0.92    &4,489   &0.89 \\
 5 - 15 years     &34,196  (2.89)    &19,783  &2.91    &14,413  &2.87 \\
16 - 44 years     &710,395 (60.06)   &408,443 &60.05   &301,952 &60.07 \\
45 - 64 years     &316,423 (26.75)   &181,496 &26.69   &134,927 &26.84 \\
Above 64 years    &104,132 (8.80)    &60,196  &8.85    &43,936  &8.74 \\\ \\
\textbf{Types of entry} &            &        &        &        &\\
New cases         &970,352 (82.04)   &557,677 &82.00   &412,675 &82.10 \\
Others            &212,425 (17.96)   &122,444 &18.00   &89,981  &17.90 \\\ \\ 
\hline
\end{tabular}
\end{table}
\vfill

\pagebreak
\section*{\center 5 - Determination of Threshold in Weight Vector Classification.} \\ \\\

The field comparison step in data linkage involves using several functions to measure the similarity of attributes for each record pair. The choice of the functions is dependent on the content of the field: string comparison functions are used for names and addresses whereas numerical comparison functions are used for fields such as date, age and numerical values [christen, 2012]. Once a vector of numerical similarity values is calculated for each record pair, the candidate record pairs are classified as linked (i.e., candidate pairs that are linked deterministically or probabilistically by the linkage software), non-linked or possibly linked, based on one or more cut-off (threshold) points, in the weight vector classification step as shown in Figure 2 below. \\

\begin{figure}[h!]
\begin{center}
%\includegraphics[width=15cm]{threshold.png}
\includegraphics[width=15cm]{threshold.png}
\end{center}
\caption{Example of a histogram analysis of a dataset produced in a linkage pipeline. Three cut-off points can be chosen according to specific needs. Any pair in between sensitivity and specificity cut-offs is considered a dubious match and thus passed for manual review. When manual review is not possible, a common approach is to choose a cut-off point that averages sensitivity and specificity, maximizing accuracy.}
\label{fig:2}
\end{figure}


\section*{\textbf{Selection of the threshold for the experiment}}\\\

Candidate linking records were ordered by the scores achieved; only the comparison pair with the highest score is retained as a potential link. All remaining candidate records are discarded. If two people received the same candidate as a potential link, we retained only the “best candidate”; as a comparison pair. We removed this candidate as a possible match for all other comparison pair. Then a sample of 2000 pairs stratified in three categories of linkage score (high score – above 0.95, intermediate score – values between 0.90 and 0.95, and low score - below 0.90) is evaluated manually, and the records pairs are classified as likely true pairs or likely false pairs.\\\

Based on the training dataset of 2000, the receiver operating curve (ROC) is built to choose the best cut off point, and calculating the area under the curves (AUC), balancing between sensitivity and specificity values. Records were therefore classified as links or non-links based on a single threshold. We used R software to generate accurate results.



\end{document}