Evaluation of Data on Live Birth Certificates from the Information System on Live Births (SINASC) in Campinas, São Paulo, 2009

Abstract

Objective:

To assess the completeness and reliability of the Information System on Live Births (Sinasc) data.

Methods:

A cross-sectional analysis of the reliability and completeness of Sinasc's data was performed using a sample of Live Birth Certificate (LBC) from 2009, related to births from Campinas, Southeast Brazil. For data analysis, hospitals were grouped according to category of service (Unified National Health System, private or both), 600 LBCs were randomly selected and the data were collected in LBC-copies through mothers and newborns' hospital records and by telephone interviews. The completeness of LBCs was evaluated, calculating the percentage of blank fields, and the LBCs agreement comparing the originals with the copies was evaluated by Kappa and intraclass correlation coefficients.

Results:

The percentage of completeness of LBCs ranged from 99.8%-100%. For the most items, the agreement was excellent. However, the agreement was acceptable for marital status, maternal education and newborn infants' race/color, low for prenatal visits and presence of birth defects, and very low for the number of deceased children.

Conclusion:

The results showed that the municipality Sinasc is reliable for most of the studied variables. Investments in training of the professionals are suggested in an attempt to improve system capacity to support planning and implementation of health activities for the benefit of maternal and child population.

Introduction

One of the objectives of the Brazilian Unified Health System (Sistema Único de Saúde - SUS) is the use of epidemiology to establish priorities, allocate resources, and provide programmatic guidance; thus, data is required to predict changes in the risk patterns and to indicate relevant control measures. The Health Information Systems (Sistemas de Informação em Saúde - SIS) are important tools used to meet these goals.1

Among the several SIS, the Information System on Live Births (Sistema de Informação sobre Nascidos Vivos - SINASC), which aims at obtaining the profile of births nationwide, according to the variables considered important from the epidemiological perspective, is critical.2 The creation of this SIS occurred in 1989, together with the preparation of the official data collection tool, the Certificate of Live Birth (Declaração de Nascido Vivo - DNV). Since 1994, the SINASC has been implemented throughout the national territory, and the data are made available on the website of the Department of SUS within two years, at most, after collection.3 Thus, it is possible to obtain an epidemiological overview of the maternal and child health status. This allows for the development of actions aimed at health promotion, aiming at the development and welfare of the individual in his/her entirety. This recognition also favors the monitoring and evaluation of health programs, particularly in regions with large socioeconomic inequalities, as it occurs in Brazil.4

Since its creation, SINASC has been an important source of data for several types of studies in the country. Some state that the data on DNVs are reliable and complete,5 while others demonstrate, in addition to the lack of completeness, the existence of a significant percentage of underreporting of birth registrations, as observed in cities of Northeast Brazil, where, in a three-year period, 43.6% of infant deaths did not correlate with DNVs.6

Recognizing that data quality is critical for the accurate portrayal of reality and to plan and implement health programs for the maternal and child population, this study was designed to assess the completeness and reliability of data from SINASC in the city of Campinas, state of São Paulo, Brazil.

Method

A cross-sectional study was conducted to analyze the reliability and completeness of data from SINASC, using a sample of DNVs from 2009, related to children born to mothers living in Campinas, whose births occurred in the city's hospitals.

The city of Campinas has approximately one million inhabitants. It has a fully-managed health system, organized according to the principles and guidelines of SUS and divided into five health districts. The system consists of 62 health centers (HCs) responsible for basic care, and several medium- and high-complexity care services, either public, insured, or contracted. The network of health establishments, which performs 99.6% of births, consists of 11 hospitals, of which two are university hospitals, where 13.9% and 17.6% of births occurs, respectively; a maternity hospital, which performs 44.2% of births; and eight private hospitals, with 24%. One of the university hospitals treats only mothers from SUS, while the other provides care for both SUS and private health insurance clients.7

For data analysis, hospitals were grouped into A, B, C, and D, according to the category of care. The selection of DNVs was performed by stratified random sampling. Sample size calculation considered the percentage of discordance observed by Mishima et al 8 for the variable "number of prenatal consultations" (30.7%), with a margin of error of 4% and 95% confidence. The authors calculated n=504 DNVs, plus 20%, considering possible losses, which totaled 600 DNVs.9 Taking into account the distribution of the percentage of births among the hospitals, a sample of 266 DNVs was allocated to Hospital A, 84 to Hospital B, 106 to Hospital C, and 144 to Hospital D.

The DNVs were assigned by drawing lots, which was performed by a neutral investigator, who had no contact with the field team. The original DNVs used in the study were made available by the Municipal Health Secretariat of Campinas, after signing the term of responsibility and confidentiality.

Data collection occurred from June 2011 to May 2012, through completing a blank DNV form identical to the original one, by students trained for the research, using data from hospital records of mothers and their newborns, as well as through telephone interviews with the mothers. Completion was performed according to the instructions from the Manual of Instructions for Completing the Certificate of Live Birth. The phone numbers used in the interviews were recorded on DNVs and records. At least three phone calls were made on different days and times (morning, afternoon, and evening), to exclude only those records whose phone numbers did not exist, or when the mother was not found or refused to participate in the study. When necessary, DNV replacement was performed, following the previously defined randomization criterion.

The gathering of data from the medical records was performed successfully in all mentioned hospitals, excepting one, which did not allow the consultation of medical records, making it impossible to analyze its data, on which the gold standard depended.

When comparing the data, the following were considered as the gold standard: the information obtained by telephone interviews with mothers concerning maternal sociodemographic variables (age, marital status, education, occupation, number of live births, and stillbirths), the number of prenatal consultations, delivery type, gender, and race/ ethnicity of the newborn. As for length of gestation, type of pregnancy, birth weight, Apgar score at 1 and 5 minutes, and the presence of congenital anomalies, the gold standard was obtained by collecting data from the medical records of mothers and newborns.

The variables were analyzed considering the standardized groupings in DNVs, with the exception of "usual occupation or field of activity", which was dichotomized into "paid occupation" (when the mother reported having paid work outside the home) and "no occupation" (for students and unemployed homemakers). Maternal age was calculated from the date of birth of the mother, obtained in a telephone interview. The category "common-law marriage" was added to the variable "marital status", as the classification of this item is based on the legal marital status in DNVs.

Data were processed using the softwares SPSS, release 16.0 (SPSS Inc. - Chicago, Illionois, United States) and Epi-Info release 6.04 (CDC - Atlanta, georgia, United States). The database consisted of information from the original DNVs and those obtained from medical records and telephone interviews. This database was checked for the exclusion of possible typographic errors. For Hospital A, only data from the original DNVs and from the interview were recorded.

The kappa coefficient was used to assess agreement between categorical variables (marital status, mother's education, occupation, number of live births and stillbirths, length of gestation, type of pregnancy, number of prenatal visits, type of birth, gender of the newborn, presence of congenital anomalies, race/ethnicity, and Apgar score at 1 and 5 minutes) with a significance level of 95%. Kappa between 0.81 and 1.00 was considered as almost perfect agreement; excellent between, 0.61 and 0.80; moderate between, 0.41 and 0.60; weak from, 0.21 to 0.40; and very weak, ≤0.20.10

The intraclass correlation coefficient (ICC) was used to measure concordance of quantitative variables (mother's age and birth weight), using the two-way mixed effect model.11

The completeness of data on the DNVs was evaluated by calculating the percentage of incompleteness (blank fields), considering an excellent rate when less than 5% of data were missing; good, 5% to 10%; regular, from 10% to 20%; poor, from 20% to 50%; and very poor, greater than 50%.12 An α error of 0.05 was used in decision-making.

The study was approved by the Research Ethics Committee of Faculdade de Ciências Médicas da Universidade Estadual de Campinas, protocol No. 179/2011. As it was a telephone interview, the need for the informed consent was waived and replaced by verbal consent obtained and recorded during the telephone contacts.

Results

The database used in the reliability analysis included 601 DNVs. Regarding the completeness of variables, there was a low percentage of data left blank or ignored. For maternal education, number of live births and stillbirths, gestation length, number of prenatal consultations, Apgar 1 and 5 minutes, and the presence of congenital anomalies, the percentage of completeness was 99.8%. Other variables showed completeness of 100% (Table 1).

Table 1. Completeness of the variables of Live Birth Certificates in the Live Birth Information System (SINASC), Campinas, São Paulo, 2009.

In the analysis of maternal sociodemographic variables, excellent agreement was observed for mother's occupation and number of living children; moderate, for marital and educational status; and very poor, for number of dead children (Table 2).

Table 2. Agreement between maternal demographic variables of Live Birth Certificates of SINASC, Campinas, São Paulo, 2009.

For the variables related to pregnancy, childbirth, and the newborn, there was almost perfect agreement for type of pregnancy, type of delivery, Apgar at 1 and 5 minutes, and gender of the newborn. For the duration of pregnancy, agreement was excellent, and for race/ethnicity, moderate. However, for the number of prenatal visits and presence of congenital anomalies, agreement ratio was weak (Table 3).

Table 3. Correlation between the variables related to pregnancy, childbirth, and the newborn of the Certificates of Live Births of SINASC, Campinas, São Paulo, 2009.

The measure of agreement for continuous variables through the ICC showed great reliability for the mother's age (ICC=0.9984) and birth weight (ICC=0.994) (Table 4).

Table 4. Agreement between the variables maternal age and birth weight of Certificates of Live Births through the SINASC ICC, Campinas, São Paulo, 2009.

Discussion

Considering that SINASC is a source of data on maternal and child populations of the utmost importance, many studies have been conducted in Brazil and abroad aiming to assess the reliability and completeness of the information contained in declarations of live births.13 ^- 15 This study demonstrated that SINASC coverage in Campinas is excellent since, when comparing the number of live births reported by DNVs with the population younger than a year, estimated by the Brazilian Institute of geography and Statistics (Instituto Brasileiro de geografia e Estatística - IBGE) for 2009, it can be verified that the proportion of SINASC agreement was greater than 100%.16 As for completeness, it was observed that the recording of most items on DNVs showed satisfactory adherence.

Although many studies have assessed completeness by grouping blank and ignored fields, in this study their evaluation was carried out by calculating the percentage of blank fields, as they reflect the lack of importance given to the declaration and its consequent poor completion. As for the ignored fields, they may represent lack of information in the records or ignorance on the part of the companions of the women.12 ^, 17 ^, 18 The excellent degree of completeness observed in Campinas for all variables meets the trend indicated by Barbuscia and Rodrigues-Junior,17 of increase in quality of completion.

Several studies on the subject have also demonstrated improved system coverage in different regions of the country.18 ^- 20 According to Costa and Frias,18 several causes are mentioned for poor completion of DNVs, such as lack of clarity in their instruction manual and the heterogeneity of professionals responsible for this task. Considering the results of the city, it can be assumed that the management and hospital staffs have worked in line with the objectives of SINASC, raising the awareness of health professionals about the potential of this system as a source of information, and the importance of correct and complete certificates.

In addition to great completeness, the agreement was perfect for several variables (mother's age and birth weight), almost perfect (number of living children, type of pregnancy and childbirth, newborn gender, and Apgar) or excellent (mother's occupation and gestation length), showing that it is possible to conduct high reliability analyses based on this information, such as making diagnoses, health surveillance and monitoring of newborns, evaluation of health interventions in maternal and child health, and knowledge of availability of services that perform deliveries, among others.

Regarding the marital status of the mother, there was moderate agreement, similar to other studies, which showed poor agreement for this variable.20 It can be surmised that this distortion arises from the completion options, which do not include the item "common law marriage" but only the marital status that has legal connotation. The current version of DNVs has included this option, which should contribute to the reliable completion of this variable, and thus reflect the real situation of family support. This is important, as the absence of a partner is related to unfavorable outcomes, such as lower adherence to prenatal care for adolescent and adult pregnant women, and low birth weight.21

Maternal education also showed moderate agreement, unlike the results found in Maranhão,19 whose agreement was weak, and those of Rio de Janeiro,20 which showed good reliability. It is possible that the difficulty in completing the certificates determine these discrepant results, as the variable is categorized by completed years of study, and not by school periods, which may cause a bias. The most recent version of the DNV is objective in relation to this variable, by including the fields "schooling" and "last completed grade", making the interpretation easy between what the pregnant woman answered and what should be completed.

It is believed that this new format will help increase the reliability of the data, as studies have shown that maternal education is related to negative outcomes for both maternal and child health, such as higher parity, inadequate prenatal care, higher risk of infant mortality, and low birth weight, among others.22 ^- 24

Another problem emphasized in relation to the correct completion of DNVs concerns the variable race/ethnicity, whose agreement was moderate. Romero and Cunha12 also observed a discrepancy between the proportion of Caucasian and mixed-race children when comparing SINASC to the 2000 Census, and emphasized the lack of specification for the completion of this variable. The study by Buscher et al 25 observed differences between self-reported race and the data on births published in the database in North Carolina and stressed the need for greater clarity in relation to this type of information, especially in populations with high ethnic diversity.

The study by Baumeister et al 26 showed that, despite the limited training of those responsible for completing birth certificates, information on race and ethnicity showed agreement with the self-reported race for groups that were not Native American. This type of evaluation was not performed in this study, but it is believed that the reliability of this variable was compromised by the lack of clarity of the instruction manual. This standardization should follow the instructions of IBGE, which states that the Brazilian population should be classified as white, black, Mixed-race, Asian descendant, and Native Brazilian, based on the subjects' personal perception of their race/ethnicity, and not the interviewer's.27

The agreement of the variable "number of prenatal consultations" was weak. It is known that prenatal care is essential to prevent adverse neonatal outcomes, and the correct completion of this variable may contribute to the identification of vulnerable groups.

Other studies have also shown problems in relation to the completion of this field20 and point to the underutilization of the prenatal medical care records, demonstrating the lack of integration between systems of ambulatory and primary healthcare with the hospitals' information network.8 That would explain the result, even in areas with high rates of prenatal care.28 Moreover, Theme-Filha et al 20 pointed to a difference in this variable agreement when the source of comparison was the medical record or the interview with the mother, showing that the DNV was completed based on verbal information.

Data on the presence of congenital anomalies also obtained poor agreement, as found in two other studies. In the first, which compared data from birth certificates to medical records, there was very poor agreement,15 and in the second, it was observed that more complex and unsuspected abnormalities had lower agreement, while those most evident showed moderate to excellent agreement, as they are more easily identified.29 It was concluded that, because there is no accurate diagnosis of complex abnormalities in the immediate postpartum period, when the DNV is usually completed, there is a great possibility of inadequate recording of that field, thus contributing to lower safety in the use of such data.

The number of dead children was the variable that showed the worst outcome, with very poor agreement. According to a study by Theme-Filha et al, 20 the variables "number of live births" and "number of dead children" showed low completeness, but when these data were reported, they showed excellent reliability. As this information was present in most records, the authors explained the high rate of lack of information due to lack of appreciation of this variable. Because of their excellent completeness in Campinas, it appears that the problem lies in the header "number of previous stillbirths," which creates a doubt for hospitals on the information to be recorded, not specifying whether the total number of earlier fetal losses must be declared, including miscarriages.

Mello Jorge et al,30 in a 1993 study, already demonstrated the poor recording of data related to maternal parity and showed that the denomination of the variable was the cause of its low reliability. Thus, greater clarity of forms and instruction manuals regarding the completion of this variable can clarify doubts and contribute to greater reliability of this field.

The limitation of this study was the incapacity to perform a survey of medical records in one of the hospitals in the city. However, it is believed that this has not affected the results, as most of the variables were collected through telephone interviews.

This study showed excellent completeness and good reliability of SINASC in Campinas. Therefore, this information system can be considered as a source of reliable data on the maternal-child population in the municipality. This study provides support improvement of the system, as well as encourages investment in constant information qualification, so that it remains a reliable tool for the health monitoring of the mother and child population and the planning and implementation of actions aiming at the overall benefit of this group.

Funding Statement

Ministério da Saúde - Process nº 25000.072380/2010-95.