Study protocol using informatics to identify and recruit a cohort of older adults in Florida to develop teleneuropsychological norms

Abstract

Introduction

The use of teleneuropsychology or neuropsychological remote assessment increased during and after the COVID-19 pandemic in 2020. Teleneuropsychology facilitates remote assessment for populations that do not have access to neuropsychological services as well as individuals who are vulnerable or have physical restrictions that would otherwise make it difficult for individuals to receive appropriate care. However, there are many instruments that are not validated or lack normative data for the overall population. Therefore, this study aims to develop normative data for a neuropsychological battery administered on telehealth with commonly used tools to identify cognitive performance in older adults.

Methods and analysis

The following study will use a previously informatics-generated list of participants who have a lower risk of developing Alzheimer’s disease and other related dementias. Participants will complete screening surveys related to cognitive and health status. They will also complete questionnaires related to sociodemographic information, depression, functionality and social determinants of health. Participants will undergo a teleneuropsychological battery examination via remote assessment. We estimate recruiting 500 participants to establish normative data.

Ethics and dissemination

The current protocol is approved by the University of Florida’s Institutional Review Board. Results will be analysed and disseminated in a research paper once sample number goals are completed.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• This study is one of the first to use a novel algorithm that measures risk of dementia based on medical charts for the recruitment of participants.

• The teleneuropsychological battery is created to be performed remotely.

• One relevant limitation includes possible complications when recruiting non-white participants of Hispanic ethnicity as well as non-Hispanic black participants.

• One relative limitation of teleneuropsychological battery includes control of the patient environment during evaluations and battery.

Introduction

Worldwide, it is estimated that one in every three individuals over the age of 70 has been diagnosed with some form of mild cognitive impairment or dementia.¹ For older adults, mild cognitive impairment has become one of the highest-risk conditions as it can affect an individual through a variety of different domains, ranging from language, memory, perceptual-motor abilities, attention and executive functions.²

Dementia is one of the leading causes of cognitive impairment, affecting over 50 million people currently worldwide.¹ It is estimated that the global prevalence of dementia continues to double every 5 years, especially in populations between 50 and 80 years of age.³ Alzheimer’s disease (AD) is a leading aetiology of dementia that commonly presents with prominent memory deficits. To date, approximately 6.7 million Americans over the age of 65 are currently living with Alzheimer’s, of which 73% are over the age of 75.⁴ The use of neuropsychological tests in the past 30 years has facilitated AD diagnosis and developed evaluations of more constructive ways to help patients with both memory and executive functioning. Neuropsychological testing, especially in the early detection of CI and AD, is necessary to provide timely treatment in the form of pharmacological and therapeutic interventions.⁴ Early detection not only presents opportunities for intervention but also provides caregivers and patients time to prepare for future care and a proactive approach to diagnosis.⁵ In a study conducted in 2010, around 216 participants with different cognitive impairments underwent different neuropsychological testing for a proper cognitive diagnosis. Of these individuals, most were over the age of 65, and more than 90% of them were diagnosed with a form of Alzheimer’s through different methods of analysis and neurocognitive exams.⁴ In the absence of neuropsychological assessments and the delayed diagnosis of CI, opportunities for treatment to slow cognitive decline are lost, placing a burden on both the patient and the caregiver.⁵ With the ongoing prevalence of cognitive impairment in older individuals, it is crucial to determine and provide cognitive testing to facilitate primary prevention and mitigate exposure to other health risks.

Teleneuropsychology and traditional face-to-face interactions can both produce similar results in diagnostic and treatment outcomes.6,9 A 2014 study on video teleconference-based neuropsychological assessments found that performances on verbal learning, fluency, simple attention and naming were very similar to those of face-to-face assessments.⁶ These findings were reflected in a similar study in 2021, where telemedicine cognitive test results on auditory attention, verbal fluency, confrontational naming and verbal episodic memory did not differ from traditional in-person testing.⁸ In addition, there were no significant differences found between in-person and at-home administration of verbal list learning tasks and auditory tests related to the reliability and validity of the tests.¹⁰ While reliability studies are promising, a relatively small number of the large arsenal of neuropsychological measures have undergone validation studies.¹¹ Further, many of the early validation studies took place in a clinical setting, a more controlled environment with the availability of administrative support.

Since the COVID-19 pandemic, the efficacy and receptiveness of teleneuropsychology have been evident, with existing research supporting its impact relative to that of traditional, in-person testing. The rise in the use of telemedicine after the COVID-19 pandemic has produced a vast literature on the efficacy of teleneuropsychological assessments and telemedicine overall. When comparing the health benefits associated with remote video versus telephone visits among patients with neurocognitive disorders and their caregivers, it was found that patient-caregiver dyads completing treatment through telehealth video conferencing depicted lower levels of cognitive deterioration, better health-related quality of life, decreased caregiver burden and better self-efficacy than their counterparts who completed telephone visits.¹² In a seven-study systematic review on telemedicine for older adults with dementia, telemedicine assessments and interventions have been proven to be helpful tools and viable alternatives to support older adults.⁷ Teleneuropsychology has also received positive feedback from older patients and their caregivers.^{7 12} More recent literature has pointed to benefits from teleneuropsychology such as reducing costs for patients.¹³ Providers have also anticipated some degree of teleneuropsychology to be a permanent part of their clinical practice¹⁴

Despite the observed benefits of using teleneuropsychological assessments in older adults, there is no existing normative data to support clinical interpretations of the results and to improve the diagnostic value in those who have received the remote assessment. Rather, clinicians must rely on normative data gathered from face-to-face evaluations, presuming that test performance is largely similar, despite environmental differences in the test administration. In clinical care, normative data research assists primary care physicians in detailing the clinical history of conditions present within communities and developing a standard of care for these conditions.¹⁵ However, the method by which normative data are collected, whether it be a traditional or remote assessment, can affect the results due to changes in approach, setting and modifications of certain tests to adapt to the platform (eg, Trail A was transformed from a written test to a verbal assessment). Further, the lack of robust normative data for teleneuropsychological assessments has been identified by some neuropsychologists as a barrier to further utilisation and implementation of these services.^{14 16}

Therefore, the aims of this study include the following: (1) establish a pool of cognitively healthy participants based on an electronic health record algorithm that identifies participants with a low risk of developing Alzheimer’s and related dementia (ADRD);¹⁷ (2) conduct a cognitive assessment of identified participants using a telehealth assessment platform; (3) characterize the sample across selected social determinants of health to assess whether they impact cognitive performance and diminish the predictive value of our neuropsychological battery across groups.

Methods and analysis

Study setting

Recruitment

Prospective participants will be recruited from the OneFlorida+Clinical Research Consortium (OneFlorida+). OneFlorida+ is a clinical research network and integrated data repository (IDR) with longitudinal electronic health records for ~16.8 million Floridians across 10 health systems. Longitudinal data collection began in 2012 and includes de-identified patient-level data, such as diagnoses, encounter/procedure codes, medications and vital signs.

Using these data, Li and colleagues retrospectively created an ADRD prediction model.¹⁷ Using the bioinformatic data of known ADRD risk factors, the model showed excellent ability to identify persons who were unlikely to develop ADRD over 5 years (Negative Predictive Value=0.91). Aside from age and sex, key parameters of the model, in order of importance, included history of stroke, heart disease, hypertension, diabetes, depression, elevated body mass index and benzodiazepine use. The parameters of this algorithm will be applied to a smaller cohort of prospective participants whose EHR data are available in the University of Florida (UF) Health IDR (one of the ten health systems within OneFlorida+). Approximately 7000 participants, aged 50+, who are at the lowest risk of developing ADRD over 5 years will be selected for recruitment. Using this procedure of identifying individuals at low risk of ADRD will reduce the likelihood of including participants who are in the prodromal stages of a neurodegenerative condition, creating a robust normative sample.¹⁸ Figure 1 shows the sample selection and study design. Recruitment efforts will occur in a stepwise method with an emphasis on age epochs. First, participants from the ages of 60–69 will be recruited until 55 participants (white non-Hispanic/Latinos=32, Hispanic/Latinos=12 and black=12) complete the full protocol. Next, recruitment will focus on participants aged 70–79, then 80–89 and 50–59. Recruitment of 55 participants per group is done to strike a balance between creating an appropriate number of persons in each cell and the feasibility of completing this project in a reasonable timeframe.

Figure 1. Sample selection and study design. ADRD, Alzheimer’s and related dementia; IDR, integrated data repository; UF Health, University of Florida Health; teleNP, teleneuropsychology; TICS-M=Modified Telephone Interview for Cognitive Status.

All adults aged 50+ are eligible to participate in this project, including participants who had expressed interest in research through Consent2Share. Using an honest broker, members of the research team will have access to limited identifying information (eg, name, address, phone number) for recruitment purposes but not participants’ full medical records. Information about excluded or declined participants will be removed from the database immediately. Recruitment started in January of 2023 and is estimated to continue until August 2025.

Recruitment of study participants will be completed remotely; prospective participants who live in the Northern Central region of Florida will complete all assessments in their homes or other quiet environments. All researchers will complete the assessments at the University of Florida in Gainesville, Florida.

Eligibility criteria

Participants will be included in the study if they meet the following criteria: (1) age 50 and above at the time of enrollment; (2) included in the University of Florida IDR and selected by the computer algorithm as ‘low risk’ of developing ADRD (ie, no dementia, low health comorbidities, one ambulatory outpatient visit in the past 12 months). Participants will be excluded from the study if they have any of the following: (1) a diagnosis of dementia or any other related dementias (ADRD) or other neurological disorders that affect cognitive functioning; (2) history or diagnosis of a severe psychiatric disease (ie, schizophrenia); (3) evidence of cognitive impairment with the Modified Telephone Interview for Cognitive Status (TICS-M <31); (4) no access to reliable internet; (5) no access to a smartphone/tablet/computer with web camera for video conference; (6) people participating in another research study.

Measures

Figure 2 shows the study flow and assessments administered. The main neuropsychological battery used in this study was created by the National Alzheimer’s Coordinating Center (NACC) as part of the Uniform Data Set (UDSv3) that contains longitudinal data collected since 2005 at the NIA-funded Alzheimer’s Disease Research Center (ADRCs).¹⁹ All the measures will be adapted to the Research Electronic Data Capture (RedCap), an online research platform to collect and store data (https://www.project-redcap.org). Additionally, some informant-reported questionnaires are adapted to be responded to by the participants (ie, health history, Functional Activities Questionnaire (FAQ), Geriatric Depression Scale (GDS), demographic information).

Figure 2. Study flow and assessments administered. MINT, Multilingual Naming Test; NACC, National Alzheimer’s Coordinating Center; RAVLT, Rey Auditory Verbal Learning Test; SODH, social determinants of health; TICS-M, Modified Telephone Interview of Cognitive Status; WRAT-4, Wide Range Achievement Test-4th Edition.

Total time of participation is estimated to last approximately 3 hours and 30 min (figure 2). Participants will undergo an initial screening after the informed consent. After verifying eligibility, participants will complete self-report questionnaires through RedCap, and lastly 2 hours of teleneuropsychological testing via Zoom, a Health Insurance Portability and Accountability Act (HIPAA)-compliant video conference platform. All patients will be tested in their preferred language (English/Spanish) and have the option for a 6-month re-evaluation.

Screening measures

1. TICS-M²⁰: a cognitive screening measure used to detect mild cognitive impairment²¹

2. UDSv3 Health History: a questionnaire that measures overall current and past health history. This screening measure explores possible neurodegenerative diseases or other risk factors that might exclude participants from the study.

Questionnaires

1. UDSv3 Demographics: measures used to acquire demographic information of the participant. Additional questions for this study will be included (ie, what is your country of origin? At what age did you relocate/migrate to the USA? Have you lived in other countries not your country of origin or the USA?)

2. GDS²²: measures symptoms of depression in our participants. This questionnaire includes yes/no questions that would be completed by the participants through a RedCap form.

3. Functional Activity Index²³: this questionnaire measures the functionality of the individual on activities of daily living. It also measures levels of independence that can decrease during a neurodegenerative process.

4. Social Determinants of Health: measures household income as well as possible necessities and activities that the participant faces in their day-to-day activities. This questionnaire includes several parts: physical activity, family background (parental or primary caregiver and siblings’ education), occupation, marital status, family income, and selected items from the Protocol for Responding to and Assessing Patient Assets, Risks, and Experiences.²⁴

Teleneuropsychological tests

Uniform Data Set V.3.0 (UDSv3)

We included telehealth adaptations of the UDSv3 Measures: MoCA, Craft Story 21 (immediate and delayed recall), Benson Complex Figure (copy and delay recall), Number Span Test (forward and backward), Category Fluency (vegetables, fruits and animals), Verbal Fluency (F, A, S, L), Oral Trail Making Test (Parts A & B) and Multilingual Naming Test (MINT). For example, PowerPoint will be used for tests that required visual stimuli such as MoCA trails, cube and naming, Benson Complex Figure, and MINT. We would also include the oral version of the Trail Making Test to be completed during this evaluation.

Other measures

1. Wisconsin Card Sorting Test-computer version (WCST-64)²⁵: this test is used to measure executive functions. We will use the computer version of this test which requires sharing the screen with the participant and giving them control of the screen. If the participants are unable to have control over the screen, the evaluator will ask them to say ‘1’ for the red triangle, ‘2’ for the two green stars, ‘3’ for the three yellow crosses and ‘4’ for the four blue circles.

2. Rey Auditory Verbal Learning Test (RAVLT; from UDS telephone battery)²⁶: participants must learn a list of words after continuous repetitions of this list. It also includes a diversion list that is read after the fifth trial (list B), a delayed recall after 20–30 min, and a yes/no recognition trial.

3. Boston Naming Test-15²⁷: an adapted version from the Mayo’s Older Americans Normative Studies (MOANS).

4. Symbol Digit Modalities Test (SDMT)-Oral Version²⁸: a PowerPoint slide will be shared with the participant with symbols and numbers. Participants will be asked to rapidly read out the numbers associated with the symbols within a certain amount of time.

5. Wide Range Achievement Test-4th Edition (WRAT-4)²⁹: to measure premorbid intelligence, we will use the blue form of the reading subtest of this battery. This subtest consists of words in a slide showing that the participant must read out loud.

6. Vegas Odds Test (VOT)30,33: presented via timed PowerPoint presentation, the VOT is a performance validity test to ensure that participants are optimally engaged with cognitive testing. Either two or four ‘target’ playing cards are presented on the screen for either 1 or 2 s. A blank screen is then presented for either 3, 5 or 10 s before the target cards and an equivalent number of foils are presented on the screen. The participant is instructed to pick one of the target cards they previously saw. Initial validation studies showed that, ≤ 28 correct trials out of 30 correctly identified research subjects instructed to feign symptoms of ADHD from control subjects with sensitivity and specificity commensurate with other well-established performance validity measures.

Spanish-speaking only measures

1. Bidimensional Acculturation Scale (BAS)³³: a questionnaire that measures acculturation and language dominance in daily life activities of bilingual participants in both English and Spanish.

2. Wood-Cock Muñoz Language Survey-Third Edition³⁴: designed to measure academic language proficiency. Only the reading subtest was administered in this battery.

Considerations for teleneuropsychological testing

Before starting the neuropsychological testing, the evaluator will read a paragraph of verbal consent for the participants to ensure that they will not be taking notes or screenshots or use any aids to make their performance look better. During required visual stimuli, the evaluator will use a PowerPoint that has the compilation needed for the testing and will share screen as needed during testing. For tests that require drawing, evaluators will ask the participant to show their drawing closer to the camera so the evaluator could take a screenshot of only the drawing. We will ask participants to fold the papers in half and put the documents away once the drawing part is completed. For tests that require timing such as the Oral Trail Making Test or SDMT, the evaluator will read the instructions and ask the participant to say ‘yes’ when ready to start timing the participant. This would help reduce the latency time between the evaluator reading the instructions and the participant receiving the instructions which might add extra seconds that do not happen during in person evaluations. Finally, comments will be made if either the evaluator or participant encounters internet problems to address it as a possible limitation as well as to see how common either might experience this limitation.

Data analysis plan

Initial statistical analysis will be focused on observing the distribution of scores of our participants to detect any outliers and ensure a normal distribution. If outliers are identified, their case will be further explored to make sure that no errors were made in the dataset; if there was a mistake, scores will be corrected or eliminated depending on the availability of the data.

Using methods employed by other well-established normative datasets such as the MOANS.35,37 Age-corrected scaled scores will first be derived using overlapping cell tables³⁸ with distinctive midpoints to maximise the number of participants in each cell (see figure 3). The percentile ranks for raw data of each test will be calculated from the cumulative frequency and scaled to a normal distribution with a mean of 10 and a SD of 3 (ie, scaled score). The normalised test scores will then be entered into a stepwise regression with sex (men, women), education and the four races/ethnicities predominant in the state of Florida (white non-Hispanic, black/African American non-Hispanic, Hispanic/Latino, other). Variables that account for <5% of added variance of the normalised test data will be retained for further regression-based corrections. This process will be repeated for both English and Spanish test batteries.

Figure 3. Overlapping cell tables with distinctive midpoints.

Patient and public involvement

Initial development of the study surged in 2020 when observing how difficult it was to perform neuropsychological evaluations to older adults, since the population was more at risk due to the COVID-19 pandemic. Lack of normative data in teleneuropsychology made it critical to develop a battery that can be administered remotely, at the patient’s home, and that would increase the evaluation of patients remotely without leaving their homes. All participants in the informed consent and initial call will be informed of the design of the study, choice of outcome measures and how the recruitment is made. Initial screening will be done to respond to participants’ questions and inquiries about the study and to respond to any doubts. This study has been approved by the Institutional Review Board of the University of Florida (UF IRB=202001765). The results from this research will be published and presented at both national and international conferences. Once we have completed at least 100 participants, we will communicate the results of the tests to participants. This feedback will consist of an overall description of their measured cognitive capacities using the specifiers ‘below normal’, ‘normal’ and ‘above normal’, compared with other peers of same race/ethnicity, sex and education.

Ethics and dissemination

Foreseeable recruitment challenges include a lack of response from participants contacted through email and postcards, a lack of response from specific minoritised groups (Spanish speakers, black/African Americans, or other ethnicities and races) and a lack of recruitment of Spanish speakers. We foresee these complications in recruiting diverse participants, due to the methods used (ie, email, postcards and electronic medical records). No caller response or prospective participants who have moved from Florida might also be a possible challenge to reach the desired number of participants.

Internet connection might be a possible limitation as well as participants’ knowledge of technology. Unfortunately, unpredictable changes and invalid test results related to internet connectivity will be included in the comments. However, we generally expect participants will have a management of technology that permits them to complete the tests adequately.

For future directions, we expect to include other OneFlorida research centres in Florida to expand our pool of potential participants. Additionally, this can help our aim to recruit more diverse participants as well as Spanish speakers.