Assessment of the French National Health Insurance Information System as a tool for epidemiological surveillance of malaria

Abstract

Objective: Epidemiological surveillance of malaria in France is based on a hospital laboratory sentinel surveillance network. There is no comprehensive population surveillance. The objective of this study was to assess the ability of the French National Health Insurance Information System to support nationwide malaria surveillance in continental France.

Materials and methods: A case identification algorithm was built in a 2-step process. First, inclusion rules giving priority to sensitivity were defined. Then, based on data description, exclusion rules to increase specificity were applied. To validate our results, we compared them to data from the French National Reference Center for Malaria on case counts, distribution within subgroups, and disease onset date trends.

Results: We built a reusable automatized tool. From July 1, 2013, to June 30, 2014, we identified 4077 incident malaria cases that occurred in continental France. Our algorithm provided data for hospitalized patients, patients treated by private physicians, and outpatients for the entire population. Our results were similar to those of the National Reference Center for Malaria for each of the outcome criteria.

Discussion: We provided a reliable algorithm for implementing epidemiological surveillance of malaria based on the French National Health Insurance Information System. Our method allowed us to work on the entire population living in continental France, including subpopulations poorly covered by existing surveillance methods.

Conclusion: Traditional epidemiological surveillance and the approach presented in this paper are complementary, but a formal validation framework for case identification algorithms is necessary.

BACKGROUND AND SIGNIFICANCE

Malaria is one of the major preventable life-threatening diseases among travelers.^1,2 The number of cases of imported malaria reported in travelers of several Western countries has increased over the past few decades.^2–5

Malaria is not endemic to France. From 2008 to 2014, the estimated annual frequency of imported malaria in continental France was about 3500 to 4600 cases, with 10–20 deaths each year.⁶ After stabilizing in 2011–2012, this estimated frequency increased by 22.0% between 2012 and 2014.

The French National Reference Center for Malaria (NRC) is in charge of malaria surveillance in France. Imported malaria infection is not a mandatory reportable disease. Its surveillance is based on a sentinel network of hospital laboratories, which registers roughly 52–53% of cases.⁶ The NRC does not record data from ambulatory health care providers.

Administrative health databases (AHDBs) are reliable information sources for epidemiological studies^7–12 and surveillance.^13–20 Use of AHDBs for public health purposes is rapidly growing.^21,22 Their appropriateness for the surveillance of malaria has recently been evaluated, but only for specific subpopulations²³ or for hospitalized malaria patients.²⁴

The most comprehensive AHDB in France is the Système national d’information interrégimes de l’assurance maladie (SNIIRAM), the national health insurance database linked to the national hospital discharge summaries database. SNIIRAM provides data on hospital stays, drug reimbursements for patients treated by private physicians and outpatients, and procedures, examinations, and sick leaves for almost the entire population living in France.¹⁶ However, there are almost no diagnosis data in SNIIRAM without hospitalization. Results relating to biological tests and other medical procedures are not recorded. Cases of diseases can only be identified indirectly from health care consumption data. There is currently no malaria case identification algorithm for SNIIRAM.

OBJECTIVE

The objective of the present work was to build such an algorithm in order to assess the value of SNIIRAM as a tool to support nationwide malaria surveillance.

MATERIALS AND METHODS

Data sources

Data were collected from SNIIRAM. The medical indications for reimbursed drugs and procedures are not known, except in the case of long-term disabling disease. Hospital discharge diagnoses are coded using the International Classification of Diseases, 10th revision (ICD-10). Biological procedures are coded using the French Medical Biology Procedure Coding and Pricing System (NABM). Medical procedures are coded using the French Nomenclature of Medical Acts (NGAP) and the French Common Classification of Medical Acts (CCAM). Drug dispensations are coded using the French nomenclature for reimbursed drugs (CIP13). SNIIRAM also collects demographic data about patients, such as age, gender, and details concerning health insurance coverage. The data are anonymized but individually linked, which allows individual longitudinal follow-up. Currently, SNIIRAM data availability cannot exceed 3 years prior to the date of data extraction.²⁵ Nonresident foreigners coming from a country without an agreement with the French National Health Insurance must pay for their health care themselves. Therefore, there is almost no data in SNIIRAM for those patients

Data extraction and processing were performed by the medical department of the French Military Health Insurance Fund (Caisse nationale militaire de sécurité sociale) under authorization granted by ministerial ruling.²⁵

Due to restrictions on use of SNIIRAM, it was impossible to formally validate our results by reviewing patients’ medical records. The NRC data were used as reference data. The NRC database has been declared to the French Data Protection Authority (CNIL), but the delays in obtaining authorization to perform a record linkage between NRC and SNIIRAM data were incompatible with our schedule.

Case selection

Incident malaria cases were counted over one year, from July 1, 2013, to June 30, 2014, for continental France. Since the period of time between the date of care and the date of data upload into SNIIRAM can elapse, data were sought with an upload date up to 6 months from the end of the inclusion period.

Identification of cases was carried out in a 2-step procedure. First, inclusion rules giving priority to sensitivity were defined to include possible cases. Then, based on data descriptions, exclusion rules to increase specificity were applied to keep the likeliest cases.

Identification of possible cases

A possible case of malaria was defined by the presence of at least one of the following events:

• -Hospitalization with at least one malaria ICD-10 diagnosis code;
• -Reimbursement for a malaria-specific curative treatment (treatment with a marketing authorization only as a curative treatment for malaria);
• -Reimbursement for a laboratory malaria diagnosis test or a medical procedure in an emergency department followed within 2 days by reimbursement for a medication within the Anatomical Therapeutic Chemical classification class P01B (antimalarials) whose use is not restricted to curative treatment.

Several antimalarial drugs are used for both curative and prophylactic treatment. To reduce the number of false positive cases, cases for which treatment authorized for curative or prophylactic use was dispensed without prior biology or consultation in an emergency department were not considered as possible cases and were not included.

Biological tests were sought among procedures carried out in private laboratories and public hospital laboratories. Drug dispensing was sought among private pharmacy dispensaries and in-hospital dispensaries for outpatients (Table 1). Corresponding nomenclature codes are detailed in Supplementary Appendix 1. Except in special cases, drugs and procedures dispensed during hospitalization are not recorded in SNIIRAM.

Table 1.

Diagnoses, procedures, and drug codes used to identify care related to malaria cases

ICD-10 diagnosis codes B50.x (Plasmodium falciparum) B51.x (Plasmodium vivax) B52.x (Plasmodium malariae) B53.0 (Plasmodium ovale) B53.1-9 (other Plasmodium) B54.x (unspecified malaria)
Malaria diagnosis procedures (tests that cannot be identified in SNIIRAM were not selected) Biological test: thick and thin blood smears Biological test: serodiagnosis (indirect immunofluorescence or electrosyneresis)
Antimalarial drugs
Malaria-specific curative drugs	Curative or prophylactic drugs
Arthemeter ± lumefantrine Dihydroartemisinin + piperaquine Artesunate Halofantrine Quinine (pharmaceutical forms and presentations with antimalarial indication only) Pyrimethamine + sulfadoxine	Chloroquine Mefloquine Atovaquone + proguanil

Exclusion of cases not corresponding to malaria

We described the raw data in order to identify the characteristics of outlier cases using univariate analysis and multiple component analysis with hierarchical clustering. This method was already published.²⁶ Then, with a panel of experts in SNIIRAM data, malaria, and epidemiology, we analyzed these outliers in order to define exclusion rules to increase specificity. Details relating to the methods used for data description are specified in Supplementary Appendix 2.

Based on the results of this analysis and the experts’ opinions, 7 rules were defined to exclude possible miscoded diagnoses, double counting, and false positive cases with dispensing of a prophylactic therapy:

• -Hospitalizations with a “Z” ICD-10 code as the principal diagnosis (ICD-10 chapter XXIII: Factors influencing health status and contact with health services) not linked to a malaria code.
• -Hospitalizations grouped into a surgical diagnosis-related category.
• -Hospitalizations scheduled for recurrent procedures such as dialysis or chemotherapy (eg, intravenous primaquine).
• -Hospitalizations with malaria as an accompanying diagnosis and more than 12 distinct accompanying diagnoses (including malaria).
• -Hospitalizations corresponding to temporary transfer to another hospital (to avoid double counting of the same case).
• -Dispensing of an antimalarial drug not restricted to curative treatment, with a total dose dispensed at more than twice the therapeutic dose (eg, atovaquone + proguanil dispensed as chemoprophylaxis).
• -Dispensing of an antimalarial drug not restricted to curative treatment, with more than one previous dispensing of the same molecule.

The number of exclusion rules was kept low in order to preserve the robustness of the algorithm.

Automatization of the process

Several SAS^®, Oracle^® SQL, and R scripts were created to automatize the case identification process (flowchart of SNIIRAM data processing in Supplementary Appendix 3).

Outcome criteria

To assess the quality of our algorithm, SNIIRAM and NRC data were compared according to the following criteria:

• -Case count
• -Distribution of case characteristics (mean age, sex ratio, geographic location of the first medical act)
• -Correlation of trends for disease onset dates

Since 1999, studies have been done to implement completeness correction models for NRC data.²⁷ To compare NRC data to our results according to case count, we estimated an overall number of incident cases using the NRC data and the latest completeness correction models.

To accurately compare case characteristics, we used a subset of the SNIIRAM data, including only the cases with at least one care event in a hospital participating in the NRC sentinel network. Mainly in the Paris area, but also in Lyon and Marseille, it was not always possible to link data to a precise hospital. In these areas, hospitals are grouped into single large organizations and the hospital unique identification code could be replaced by a generic group code. Thus, for the geographic analysis and the comparison with the NRC sentinel network data, SNIIRAM records with missing values were excluded.

Statistical analysis

The statistical unit was the malaria case. In case of multiple care events related to malaria for the same patient, events were considered to be part of the same case if the longest time lapse between 2 events was ≤60 days, whatever the type of event. To determine if a case occurred in continental France, we used the location of the first care event.

We had no data about exposures, which did not enable calculation of incidence rates. We used frequency instead. Working on the entire population, we did not estimate confidence intervals for the quantitative variables; means thus represent “true” values and not estimations. Proportions were compared using the Chi-square or Fisher’s exact test, as appropriate. Quantitative variables were compared using the Wilcoxon or Kruskal-Wallis test. The correlation between quantitative variables, such as the distribution of disease onset dates grouped by month, was calculated using Pearson’s correlation coefficient.

Data cleaning, statistical analysis, and analysis of the geographic distribution of cases were performed using R statistical software version 3.1.2.²⁸

RESULTS

Overall, after applying the exclusion rules and longitudinal data linkage, 4248 cases of malaria were identified from July 1, 2013, to June 30, 2014, including 4077 in continental France (Figure 1).

Figure 1.

Flowchart illustrating the SNIIRAM malaria case identification algorithm.

The male-to-female sex ratio was 1.5. The median age was 38.0 years (interquartile range: 26.0–53.0). Among the cases, 3034 (74.4%) had at least one inpatient hospital stay. More than one-third of the cases (n = 1632, 40.0%) were located in the Paris area (map in Supplementary Appendix 4).

For hospitalized cases, a single precise parasitological diagnosis was available for 2369 (78.1%) of the cases. Plasmodium falciparum was reported in 2137 (90.2%) of these cases.

The drug therapies dispensed during hospitalizations were not known, because such dispensing does not lead to reimbursement procedures. For patients treated by private physicians and outpatients, the most frequently reimbursed treatment was quinine, either before or after hospitalization, if the patient was hospitalized (Table 2). The drug types dispensed before and after hospitalization were statistically different for adults (P = .04) and children (P = .05).

Table 2.

Antimalarial drug dispensing reimbursements for patients treated by private physicians and outpatients

Treatment	Before hospitalization		After hospitalization (if patient was hospitalized)
	Adult	Child (<18 years)	Adult	Child (<18 years)
	n = 1000	n = 89	n = 172	n = 26
QN	735 (73.5)	56 (62.9)	110 (64.0)	10 (38.5)
AL	163 (16.3)	9 (10.1)	42 (24.4)	15 (57.7)
CQ	77 (7.7)	23 (25.9)	14 (8.1)	0
DP	14 (1.4)	1 (1.1)	3 (1.7)	0
AP	1 (0.1)	0	0	0
MQ	1 (0.1)	0	0	0
CQ/QN	9 (0.9)	0	2 (1.2)	1 (3.8)
QN/AL	0	0	1 (0.6)	0

Figures represent the number of cases (percentages).

QN = quinine, AL = arthemeter + lumefantrine, CQ = chloroquine, DP = dihydroartemisinine + piperaquine, AP = atovaquone + proguanil, MQ = mefloquine.

Care pathways

The data relating to hospitalization and dispensing of malaria-specific curative treatments enabled most of the cases to be retrieved (Table 3). A biological test or emergency department admission was found in 474 (11.6%) of the cases.

Table 3.

Distribution of care events among identified malaria cases

Health insurance refund				n = 4077
Hospitalizations	Drug dispensing for patients treated by private physicians and outpatients	Laboratory tests	Emergency department admissions
×				2525 (61.93)
	×			944 (23.215)
×		×		302 (7.41)
×	×			134 (3.329)
	×	×		66 (1.62)
×	×	×		46 (1.13)
	×		×	23 (0.656)
×	×		×	13 (0.32)
	×	×	×	10 (0.25)
×	×	×	×	9 (0.22)
×			×	4 (0.10)
×		×	×	1 (0.02)

Figures represent the number of cases (percentages). Note: Column percentages might not add up to 100% due to rounding.

The median duration of the first hospital stay was 3 days (IQ 1–5). The mean duration (3.9) statistically increased in cases of P. falciparum (P < .001) and when the patient was older (Pearson’s correlation coefficient ρ = .11, P < .001).

Comparison with NRC data

During the study period, the NRC recorded 2222 malaria cases through its sentinel network (Paris area: 1210; other regions: 1012). From the results of the 2008 and 2014 NRC completeness studies, the overall number of cases occurring during the inclusion period was estimated between 4131 (+1.3% compared to SNIIRAM) and 4268 (+4.7% compared to SNIIRAM). No confidence interval could be calculated.

The ratio of identified cases between SNIIRAM and NRC data was consistent during our study period (Pearson’s correlation coefficient ρ = .97, P < .001, Figure 2).

Figure 2.

Number of malaria cases captured by the National Reference Center for Malaria and SNIIRAM in continental France, July 2013 to June 2014.

A subset of 1829 SNIIRAM cases was retained for comparison of case characteristics with the NRC data (Paris area: 697; other regions: 1132). A total of 2248 cases were excluded (missing or nonprecise hospital identification code: n = 690; cases without a care event in an NRC hospital: n = 1558).

Excluding the cases identified in the Paris area, where the completeness of data was the lowest, the results were not statistically different between the SNIIRAM and NRC data for age (P = .31), proportion of male cases (P = .07), and geographic distribution among districts (P = .09). After including the cases identified in the Paris area, the results remained not statistically different (age: P = .71; geographic distribution: P = .07), except for the proportion of male cases (SNIIRAM M/F sex ratio: 1.5; NRC: 1.7; P = .04). The SNIIRAM patients with missing data regarding hospital location did not significantly differ from those with complete data in terms of gender and age.

DISCUSSION

We built a reliable algorithm for counting malaria cases among the whole French population. Implementation of an automatized tool to fetch, clean, and analyze data from the SNIIRAM database could allow accurate nationwide prospective malaria surveillance to be set up at relatively low cost.

The original goal of SNIIRAM was to contribute to better management of health insurance and health policies. Consequently, caution should be exercised when using it for epidemiological purposes. Nevertheless, SNIIRAM has already shown its accuracy for epidemiological surveillance,^19,29,30 in particular in cases where traditional passive surveillance induces biases such as underreporting.¹⁰ A major challenge when using an AHDB^9,11,31–35 and especially SNIIRAM³⁶ for public health purposes is in implementing and validating the case identification algorithm. When using an AHDB, several case definitions could be built with different levels of bias and power.¹¹ For malaria, a significant hurdle was the nonsystematic presence of malaria-specific markers. This implied that inclusion and exclusion rules should be defined based on indirect indicators that could reduce algorithm sensitivity or specificity.³⁷

The comparison of our results with the NRC data strongly suggests that our case identification algorithm provides accurate results. Although we found a slightly lower number of malaria cases than the NRC estimation from its sentinel network, results were very close. Since it was not possible to calculate a confidence interval for the number of cases estimated by NRC, we cannot formally assert that the SNIIRAM and NRC counts are different. Moreover, except for the sex ratio comparison using a dataset with many missing values in the Paris area, there was no statistically significant difference between the distributions of case characteristics. Lastly, the correlation between the SNIIRAM and NRC time series could be considered excellent, which provides strong indirect evidence of the accuracy of our results.

The use of SNIIRAM, with a longitudinal linkage between hospital data and data on patients treated by private physicians and outpatients, allowed us to fetch data at a nationwide scale. The NRC surveillance system recorded data only for patients who had contact with a participating hospital laboratory. It excluded an entire segment of the population, mainly corresponding to malaria cases among patients treated by private physicians and outpatients, for which diagnosis was done in private medical laboratories.

The goals of surveillance systems have been extensively described.^38–41 The key objective of a surveillance system is to provide information to guide early public health interventions.⁴² Since it takes more than 4 months to obtain hospital discharge data, our system could not be considered as an alert system. But the data provided by SNIIRAM would help to correctly describe malaria cases. Although we had little information about the clinical status of cases, data on accompanying diagnoses, length of stay, admission and discharge status, and admission to intensive care units are available through SNIIRAM. These data were not analyzed in the present study, but they could be used to obtain complementary indicators. SNIIRAM also provided novel data on drug dispensing outside of hospitals, which complements the NRC data. Moreover, the system’s comprehensive coverage of the population and continuous, long-term data collection could constitute a useful tool to describe the care pathways of patients (eg, the delay between first biological test and first curative treatment or the long-term consequences of a malaria attack) and to evaluate public health actions at a national level.

Despite lengthy delays between care events and their appearance in the database, our work demonstrated the ability of SNIIRAM to support nationwide malaria surveillance.

The present study has several limitations. Frequencies were obtained without information on denominators, an issue inherent to travel-related diseases, where data on exposure are often incomplete and unreliable.^43–46 Consequently, we could not establish any link between exposures and outcomes. It would therefore be difficult to use SNIIRAM data alone to identify specific targets for prevention and other public health actions.

A major challenge of this study was the absence of a previously validated algorithm for malaria case identification. Considering the lack of diagnosis data in SNIIRAM, we had to build our algorithm based on expert opinions, the malaria life cycle, knowledge of drug pharmacodynamics, and national⁴⁷ and international⁴⁸ guidelines for the treatment of malaria. With regard to inclusion rules, this implied that atypical pathways, such as patients possibly being treated with drugs not restricted to curative treatment without any prior biological confirmation, were not included. Similarly, we had no background or clinical information to differentiate between prevalent and incident cases, and this may have led to miscounting.

Case exclusion rules were also based on statistical rules, sometimes without a direct link to a clinical situation, such as the exclusion of hospitalized cases when malaria was not coded as the principal diagnosis and for which there were more than 12 accompanying diagnoses. Although we know that accompanying diagnoses can be less reliable than principal diagnoses in the French national hospital discharge database,¹¹ the use of such rules could yield an algorithm overfitted to the data and thus lead to selection bias.

In addition to its limitations, this work also highlights the need to improve the validation method. The reference method for validating case identification algorithms is the review of medical records as the reference standard,⁴⁹ which allows the sensitivity and specificity of those algorithms to be determined. This method is not applicable to anonymous SNIIRAM data. Anonymous record linkage methods have been described^50,51 and could have allowed a more thorough comparison,⁵² but we did not obtain the mandatory authorizations from the French regulatory authorities to use these methods. Finally, the capture-recapture method could not be applied here because the statistical conditions had not been met.⁵³

In light of the above conclusions, we had to compare our results to the NRC data as the only available reference. The significance of this comparison could be questioned. The NRC data are not exhaustive and might not be considered as the gold standard. Moreover, to compare case characteristics, we had to subset the SNIIRAM data. The analyzed characteristics were not strongly linked to malaria. We could not rule out the possibility that our algorithm includes false positive cases, corresponding to another disease with a comparable population at risk.

The impossibility of providing proof of the method’s validity underlines the need for a gold standard procedure to validate case identification algorithms.^54–56 It would also be interesting to repeat this comparison to evaluate the stability of results over time.

Lastly, the SNIIRAM data are not as detailed as the NRC data, but they are broader in scope and exhaustive for the French population. This underlines that AHDB data make it possible to obtain complementary findings compared to traditional epidemiological surveillance.^57,58 In this particular case, SNIIRAM yields a clearer benefit than the unchained AHDBs already being used for malaria surveillance.^23,24

CONCLUSION

To our knowledge, this study is the first to analyze the performance of a nationwide malaria surveillance system based on administrative data and covering a country’s population as a whole. Our results support the need to create a formal validation method for case identification algorithms. The study also makes it clear that the scope of data collected from AHDBs such as SNIIRAM differs from that of data generated by traditional epidemiological surveillance. SNIIRAM makes it possible to rapidly implement a routine surveillance system. To collect precise and exhaustive data, better performance would probably be achieved by aggregating at least these 2 sources, but also external data about exposure.

Funding

This work was supported by the French Military Health Insurance Fund (Caisse nationale militaire de sécurité sociale), a French national entity financed by public funds.

Competing Interests

The authors have no competing interests to declare. The results and discussions presented in this article are those of the authors and are not the responsibility of the French Military Health Insurance Fund or the French military health service.

Contributors

All the authors participated in this study and concur with the submission and subsequent revisions submitted by the corresponding author.

FD: study conception and design, data collection, data analysis and interpretation, drafting of the paper. AM: data analysis and interpretation. EK: data collection and interpretation. MT, RM, LO: data interpretation. GC: study conception and design, data interpretation. GD: study conception and design, data collection and interpretation.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.