Longitudinal research in Aotearoa New Zealand using the Integrated Data Infrastructure: a review

ABSTRACT

The Integrated Data Infrastructure (IDI) is a collection of deidentified whole population administrative datasets, linked at the individual level, and made available through Stats NZ for ‘public good’ research. This paper reviews longitudinal research that has been undertaken using the IDI, and highlights the strengths, weaknesses, opportunities, and threats of using the IDI for longitudinal research. It is argued that the IDI can and has been used for longitudinal research that would be difficult or impossible to undertake without a resource such as the IDI, including longitudinal research involving small, sometimes marginalised populations, research involving intergenerational research and quasi-experimental family designs, and research investigating residential variations in the natural environment. However, issues regarding ethical governance need addressing. Researchers wishing to use the IDI should familiarise themselves with its limitations, particularly around what service use data capture and what this represents, what is missed by assessing only deficit-focused data, and the variable quality of the data.

Introduction

Aotearoa New Zealand has a long and rich history of longitudinal studies – particularly but not exclusively birth cohort studies – beginning with the Dunedin and Christchurch studies in the 1970s (Silva and Stanton 1996; Fergusson and Horwood 2001), the Auckland Birthweight Collaborative (ABC) in the 1990s (Thompson et al. 2001), and the Pacific Island Families Study (Paterson et al. 2008), Te Hoe Nuku Roa (Fitzgerald and Durie 2000), the Survey of Families, Income and Employment (SOFIE; Carter et al. 2010), and the Growing Up in New Zealand Study in the 2000s (Morton et al. 2013). Each has involved recruitment of participants, intensive assessment at regular intervals as participants age, and an enormous amount of funding, effort, and specialised staff to undertake the assessments, maintain the cohort, and support the day-to-day running of the study.

Using existing data for longitudinal research circumvents the need for researchers to devote time, effort, and resources to data collection. In particular, data collected by administrative agencies as part of their business operations (for example, running a health system, an education sector, etc.) produce on ongoing record of ‘events’ that can be used for longitudinal analysis.

Internationally, linked administrative data have been made available for research in countries such as Sweden, Denmark, Norway, Canada, Taiwan, and Australia (Hamm et al. 2021; Segal et al. 2021). In New Zealand, the development in 2011 of the Integrated Data Infrastructure (IDI) – a collection of whole population administrative datasets linked at the individual level across sectors and made available in de-identified form for research (Milne et al. 2019; Stats NZ 2021a) – has enabled a vast range of possibilities for administrative longitudinal research in Aotearoa New Zealand.

This paper will review longitudinal research that has been undertaken using the IDI with a focus on select examples showcasing different longitudinal uses of the data – it is not the intention to provide a comprehensive review. In the following sections the IDI and methods for the review will be briefly described, longitudinal research using the IDI will be reviewed, and the strengths, weaknesses, opportunities, and threats of using the IDI for longitudinal research will be discussed.

The Integrated Data Infrastructure (IDI)

The IDI is a collection of New Zealand administrative data sources which have been linked at the individual level (Milne et al. 2019; Stats NZ 2021a) and deidentified (i.e. stripped of identifiers such a name and address, and agency identifiers such as IRD and NHI numbers). The IDI is curated and maintained by Stats NZ. Data in the IDI include administrative data from government agencies (for example, Ministry of Health (MOH), Ministry of Social Development (MSD), Ministry of Education), the New Zealand censuses in 2013 and 2018, as well as several social and socio-economic surveys (for example, general social survey, household economic survey). Data are updated (‘refreshed’) up to four times a year with the addition of more recent data and (periodically) new data tables. An up-to-date list of data sources in the IDI can be found here: https://www.stats.govt.nz/integrated-data/integrated-data-infrastructure/data-in-the-idi/.

Data from different agencies are linked to a central concordance table (the ‘spine’), which includes those born in New Zealand since 1920 (from the Department of Internal Affairs’ birth records), those arriving in New Zealand on a long-term visa since 1997 (from Ministry of Business Innovation and Employment’s border movement records), and those working in New Zealand since 1999 (from the Inland Revenue Department’s tax records). The spine forms the base population of the IDI and can be thought of as an ‘ever resident’ New Zealand population (this is currently around 10 million individuals). Linkage is through deterministic and probabilistic methods. Linkage rates vary by agency, though most are >90%, while false positive rates also vary by agency with most ≤1% (Stats NZ 2021b).

Access to the IDI for ‘public good’ research is by application to Stats NZ (https://www.stats.govt.nz/integrated-data/access-microdata-in-the-data-lab/). Stats NZ applies the ‘Five Safes’ framework to statistical disclosure control – Safe People, Safe Projects, Safe Settings, Safe Data and Safe Output (Desai et al. 2016) – and researchers wishing to access the IDI are required to work within this framework.

Methods

Studies were considered for inclusion for review if they used IDI data and focussed on an assessment of individuals over time. A list of IDI papers had been compiled and updated by the author in relation to a previous paper (Milne et al. 2019). This list was augmented by a search of the ‘pubmed’ and ‘scopus’ databases using the term ‘Integrated Data Infrastructure’, conducted in September 2021, which was then restricted to studies focussing on assessments of individuals over time.

Results

The studies reviewed below are organised by the age at which the cohorts investigated were constituted.

Birth cohorts

Berry et al. (2018) investigated long-term health and education outcomes for extremely pre-term babies (as early as 23 weeks gestation). They found that while children born extremely pre-term were more likely to experience health problems and education difficulties, the majority of extremely pre-term children had few developmental concerns, and most took the National Certificate for Educational Achievement (NCEA) Level 1 exam (typically taken in year 11).

Richmond-Rakerd et al. (2021) took a birth cohort approach to define a cohort (all those born in New Zealand from 1928 to 1978), and then followed them as adults and into old age, assessing hospitalisations from 1988 to 2018. They reported that being hospitalised for a mental health condition was associated with double the risk of being hospitalised for a subsequent physical health chronic condition (such as diabetes, cancer, coronary heart disease), and triple the risk for mortality, even after controlling for previous physical health hospitalisations.

Similarly, Paynter et al. (2019) tracked individuals born from 1984 to 1999 and assessed whether they received the meningococcal B vaccination from 2004 to 2008. Those that had were found to have a decreased risk of subsequent hospitalisation for gonorrhea. The vaccine effectiveness against gonorrhea was estimated to be 24%.

Family influences

Two birth cohort investigations using IDI have made use of the fact that children can be linked to parents and wider families through birth records (Milne et al. 2020). Leong et al. (2020) studied children born in 2008–2011 to assess the influence of antibiotic use in pregnancy and during the first two years of children’s life on obesity assessed at age 4. While dose–response associations were found between antibiotic use in pregnancy and in the child’s first two years of life and later obesity, these association were no longer apparent when siblings and twins within the same families were compared, suggesting that the apparent associations may have been due to unmeasured familial confounding.

Slykerman et al. (2020), using a cohort of children born between 1996 and 1998, also compared children within families to assess whether caesarean birth has an influence on educational attainment. Both within-family analyses and covariate-controlled analyses across the whole sample found no evidence that caesarean birth impacts later educational attainment as measured by performance in NCEA level 2 (year 12) examinations.

Influences of the natural environment

Because the IDI covers the full geography of New Zealand it is possible to link geographical-level data to the IDI to assess the impact of factors that vary geographically on outcomes of interest. Donovan et al. (2018) explored this possibility by assessing the association between exposure to surrounding ‘green’ environments and vegetation diversity on childhood asthma for children born in 1998 and followed up until 2016. Both greenness and vegetation diversity of residential areas measured from birth to age 18 predicted lower prevalence of asthma, which the authors postulated might be due to greater and more diverse microbial exposure. In a subsequent paper, Donovan et al. (2019a) reported that both living in rural areas between ages 2 and 18 and vegetation diversity between ages 2 and 18 independently predicted lower prevalence of an attention-deficit hyperactivity disorder (ADHD) diagnosis, though they caution that the rurality finding may reflect healthcare access issues.

‘Natural’ experiments

The availability in the IDI of longitudinal data covering all of New Zealand allow for tests of ‘natural experiments’, whereby natural variation in exposure to interventions and policies can be used to determine whether these interventions and polices are likely to have a causal impact. Vaithianathan et al. (2016) utilised variation by territorial local authority in phasing in of the ‘Family Start Home Visiting programme’ to test the impact of the programme on child outcomes. They report that the programme resulted in fewer neonatal child deaths for the population as a whole, and also specifically for Māori children.

Child cohorts

Charania et al. (2018, 2020) used the IDI to identify migrant status among children in order to assess immunisation uptake and subsequent morbidity. They found that 46% of foreign-born children had an immunisation recorded in the MOH’s National Immunisation Register, and only 3% of refugee children had received all age-appropriate vaccinations (Charania et al. 2018). In a follow-up paper they report greater hospitalisations for vaccine-preventable diseases for both foreign-born children and refugee children (Charania et al. 2020).

Shackleton et al. (2021) utilised data on children aged 0–17 in the Survey of Families, Income and Employment (SOFIE) in the IDI to assess associations between household-level poverty and hospitalisations across the eight waves of the study. They found weak or absent associations between income poverty and hospitalisation, but stronger predictive relationships between measures of individual and area-level deprivation and hospitalisations.

Schluter et al. (2018, 2020) investigated whether data from the Before School Check (B4SC), a nationwide screen of the health, behaviour, and development of four-year-olds (Ministry of Health—Manatū Hauora 2015), could be used to identify children who might need later literacy support (as indicated by a literacy intervention at age 6). The B4SC was found to have poor discriminatory power to detect a later literacy intervention, both for the population as a whole (c-statistic = 0.624; Schluter et al. 2020), and specifically for children of Pacific ethnicity (c-statistic = 0.592; Schluter et al. 2018).

Older age cohorts

Dixon (2015) identified 20-59-year-olds in 2008–2009 and used data provided by MOH, Inland Revenue, and MSD to assess how employment and earnings over a four year follow-up period were impacted by the onset of eight different chronic conditions: stroke, traumatic brain injury, coronary heart disease, diabetes, chronic obstructive pulmonary disease, breast cancer, melanoma, and prostate cancer. Results showed that employment rates and earnings were impacted both one year and four years after disease onset for six of eight conditions (all except melanoma and prostate cancer), with stroke having the largest impact.

Taking a similar approach, Davie and Lilley (2018) compared 45–64-year-old workers who had made an injury claim in 2009 to those who had not. They found that the reduction in income among those injured increased over the three-year follow-up period, to 6.7% in the third year.

Teng et al. (2017) used the IDI to assess health outcomes in those aged 45 and older in relation to exposure to the 2010–2011 earthquakes in Christchurch. Utilising geographic variation in exposure to earthquake damage, they found modest increases in risk for myocardial infarction-related hospital admissions in the year after the earthquakes (risk ratio = 1.25) for those who lived in the most vs the least damaged areas. No increased risk was apparent 2–5 years after the earthquake.

Pierse et al. (2019) worked with a community provider to integrate data on the homeless population aged 15+ years in Hamilton into the IDI and assessed the volume and variety of the services the homeless population accessed in their lifetime. The 390 people in their cohort that were homeless had >200,000 interactions with services, most commonly health, justice, and welfare in their lifetime. This level of service use far exceeded that of the general population.

Building on their previous work with children, Donovan et al. (2019b) explored the impact of exposure to a green environment on recovery from knee and hip surgery arthroplasty for adults who underwent surgery in 2006 or 2007. They found that hip (but not knee) surgery patients exposed to greener environments lived longer and took fewer opioids for pain. The authors suggest that exposure to the natural environment may be beneficial for postsurgical recovery.

Discussion

While the studies reviewed highlight some obvious strengths of the IDI for longitudinal research and suggested opportunities for further research, there are also weaknesses and threats to its ongoing use. These are summarised in Table 1.

Table 1.

Strengths, weaknesses, opportunities, and threats of using the IDI for longitudinal research.

Strengths Whole population Able to study small and difficult to reach populations All life stages able to be studied Tests of cohort effects possible Cross-domain exposures and outcomes	Weaknesses Primarily service use data Primarily deficit focussed Population denominator needs to be inferred Limited follow-up for many datasets Variable data quality and documentation Difficult to determine family and household structures longitudinally Missed and incorrect links
Opportunities Multi-generational investigations Natural experiments Tests of wider environment	Threats Social licence Under-developed ethical governance Inadequate recognition of Māori data sovereignty

Strengths

The IDI contains more or less the whole population, so estimates tend to have high precision, and small and sometimes difficult to reach populations population groups are able to be studied. Life-stages from birth to old age can be followed longitudinally, which greatly increases the range of research questions that can be investigated. Tests for cohort effects are possible (for example, by comparing people of the same age born years apart). The IDI includes data from a wide range of health and social sector agencies, so exposures and outcomes spanning several domains of functioning can be investigated. Some of these data sources (e.g. tax and income data; births, deaths, and marriages data) were not available to researchers before the IDI was established.

Weaknesses

The IDI has the limitations expected of an administrative data resource. Foremost of these is that most data provided by agencies to the IDI capture services use, not experiences had, and these are not the same thing. The IDI does not include data on the health conditions of individuals who do not access services for those conditions. Moreover, only individuals charged or convicted for crimes are included in justice data, and this will be a (biased) subset of individuals committing crimes (JustSpeak 2020). As such, claims about prevalence, trends in prevalence and subgroup differences in prevalence need to be undertaken with caution, and researchers need to be specific about the outcomes for which prevalence is being estimated. For example, an increase in mental health service use (the outcome estimated) does not necessarily reflect an increase in prevalence of mental health problems (a related but different outcome). Comparisons between service use data from the IDI and non-service use data from other sources may help to clarify what prevalence data do and do not mean. For example, Svardal et al. (2022) showed that the ethnic and socio-economic patterning of the prevalence of antidepressant use in pregnancy is completely different from the ethnic and socio-economic patterning of the prevalence of antenatal depression using a diagnostic screen.

Additional limitations include that (i) most IDI data are deficit-focused, as services are often only accessed when an individual is in need; (ii) the denominator for the population at any point in time has to be inferred from the group of people who have used services – this will not be perfect, and it is difficult to estimate prior to 2007 (Zhao et al. 2018; Stats NZ 2021c); (iii) different datasets cover different time-periods (https://www.stats.govt.nz/integrated-data/integrated-data-infrastructure/data-in-the-idi/), which will limit the length and breadth of longitudinal investigations; (iv) data quality and data documentation are variable (Milne et al. 2019; Bycroft et al. 2021); (v) tracking households and families over time is difficult as the records capturing movements in and out of households relies on accurate and timely reporting to social sector agencies; and (vi) some links will be incorrect and others will be missed (Stats NZ 2021b), and methods to account for these false and missed links need to be developed. Note that the final limitation is not shared by some international administrative data resources (particular those in Sweden, Denmark, and Norway), which are able to use national identifiers to track individuals’ interactions across administrative systems so do not rely on record linkage (Hamm et al., 2021).

Opportunities

Opportunities include, first, that natural experiments and exposures which vary by geography can be investigated (see studies by Donovan et al. 2018, 2019a, 2019b; Vaithianathan et al. 2016 reviewed above). For example, investigations of the longitudinal impacts of covid lockdowns may be able to be compared across regions which experienced different durations and intensities of lockdowns. Another opportunity involves greater use of quasi-experimental family designs, such as comparisons of differentially exposed siblings (Leong et al. 2020; Slykerman et al. 2020). Wider environmental influences are also starting to be explored (Roy et al. 2022), and could be explored further.

Threats

There are few threats to the use of the IDI that are specific to longitudinal research, but there are a number of threats to use of the IDI generally about which longitudinal researchers need to be aware. Three are highlighted here: (i) maintaining social licence, (ii) under-developed ethical governance, and (iii) inadequate recognition of Māori data sovereignty.

Social licence

Social licence refers to the permission granted by population groups and stakeholders for businesses and governments to act in certain ways (Ballantyne and Stewart 2019). Regarding permission to use administrative data for research, studies have found general acceptance across population groups so long as there is transparency in what the data include and are being used for, and there is trust in the institutions maintaining the data (Kalkman et al. 2022; Paprica et al. 2019). Concerningly, a recent report found ‘mixed’ levels of trust in Stats NZ’s data stewardship (Nielson 2020), and specific knowledge about the IDI is also low (Gulliver et al. 2018). More needs to be done to education the public about how their data are being used and could be used (e.g. for longitudinal investigations using data from several government agencies) to ensure that there is trust in the IDI and that social licence for the IDI is maintained.

Ethics

Another threat to trust in the IDI is that ethical governance is lacking. There has never been any mandate for non-health IDI projects to undergo ethical review, and the requirement (until mid-2021) for projects involving health data to request an ethical assessment from the Health and Disability Ethics Committee (HDEC) has invariably involved HDEC classifying projects as ‘out of scope’ and not undertaking an ethical assessment. The lack of mandate places the onus on researchers to seek ethical review. However, unequal access to ethical review committees and knowledge of ethical processes among researchers using the IDI – who may belong to government agencies and private research companies as well as universities – makes for uneven ethical governance of IDI research.

A specific ethical issue relating to longitudinal research using the IDI is that individuals have not given informed consent. Lessof (2009) argues that the important question regarding use of administrative data for longitudinal research without informed consent is ‘whether an individual’s health, interests or confidentiality could be affected negatively’ (p42). In the case of the IDI, individuals’ health is unlikely to be negatively affected, and confidentiality is strongly guarded by the Five Safes framework. The remaining issue – whether individual’s interests are likely to be negatively affected – could be addressed through independent and project-specific ethical oversight of IDI projects, which again emphasises the importance of developing a strong ethical governance system for the IDI.

Māori data sovereignty

A final potential threat to trust in the IDI is inadequate recognition of Māori data sovereignty – the idea that ‘Māori data should be subject to Māori governance’ (Te Mana Raraunga 2021). There has been an attempt to incorporate Māori data sovereignty principles in the IDI application process through the Ngā Tikanga Paihere framework (Sporle et al. 2020). The framework provides an assessment tool for IDI projects to ensure project investigators (i) have the appropriate expertise, and relationships with communities; (ii) have public confidence and trust to use data; (iii) use good data standards and practices; (iv) have a clear purpose and action in the proposed research; and (v) have balanced the benefits and risks of conducting the research (Sporle et al. 2020). In addition to Ngā Tikanga Paihere, Māori researchers and communities are working to determine the best ways to use the IDI to suit their communities’ needs (Sporle 2018; Greaves and Milne 2020), while Māori are also at the forefront of initiatives to revise key legislation and processes relating to the IDI (Kukutai and Cormack 2019, 2021; Cormack et al. 2020; Moses 2020; Sporle et al. 2020). Internationally, Indigenous groups are determining what safe use of administrative data looks like for their own communities (Walker et al. 2018; Rowe et al. 2021; Walter et al. 2021).

Good international examples exist which integrate indigenous data sovereignty with ethical processes for accessing administrative data. For example, the Manitoba Multigenerational Cohort requires approval from an ethics board, the Health Information Privacy Committee, and non-health data providers if non-health data are requested; and research projects using linked datasets in which indigenous populations are over-represented also require approvals from the First Nations Health and Social Secretariat of Manitoba and the Manitoba Metis Community Research and Ethics Protocol (Hamad et al. 2021). These processes might be adaptable for the IDI within the New Zealand context.

Concluding remarks

The IDI is a valuable resource for longitudinal research. Some of the research highlighted in this review, such as research involving small, sometimes marginalised populations (Berry et al. 2018; Charania et al. 2018, 2020; Pierse et al. 2019), research involving intergenerational research and quasi-experimental family designs (Leong et al. 2020; Slykerman et al. 2020) and research investigating residential variations in the natural environment (Donovan et al. 2018, 2019a, 2019b) would have been difficult or impossible to undertake without a linked administrative data resource such as the IDI. However, issues regarding ethical governance need addressing. Researchers wishing to use the IDI should familiarise themselves with its limitations, particularly around what service use data capture and what this represents, what is missed by assessing only deficit-focused data, and the variable quality of the data.

Funding Statement

This work was supported by the Ministry of Business, Innovation and Employment (Endeavour Grant, ref 62506 ENDRP; A Better Start National Science Challenge, ref UOAX1511).

Disclosure statement

No potential conflict of interest was reported by the author(s).