Abstract
Background
CDKL5 Deficiency Disorder (CDD) is a severe X-linked developmental and epileptic encephalopathy. Existing developmental outcome measures have floor effects and cannot capture incremental changes in symptoms. We modified the caregiver portion of a CDD clinical severity assessment (CCSA) and assessed content and response-process validity.
Methods
We conducted cognitive interviews with 15 parent caregivers of 1-39-year-old children with CDD. Caregivers discussed their understanding and concerns regarding appropriateness of both questions and answer options. Item wording and questionnaire structure were adjusted iteratively to ensure questions were understood as intended.
Results
The CCSA was refined during three rounds of cognitive interviews into two measures: (1) the CDD Developmental Questionnaire – Caregiver (CDQ-Caregiver) focused on developmental skills, and (2) the CDD Clinical Severity Assessment – Caregiver (CCSA-Caregiver) focused on symptom severity. Branching logic was used to ensure questions were age and skill appropriate. Initial pilot data (n=11) suggested no floor effects.
Conclusions
This study modified the caregiver portion of the initial CCSA and provided evidence for its content and response process validity.
Keywords: CDKL5 deficiency disorder, caregiver-report, validation, outcome measure, severity, developmental attainment
1. Introduction
CDKL5 Deficiency Disorder (CDD) is a severe developmental and epileptic encephalopathy characterized by refractory seizures with onset in the first months of life (Leonard et al., 2022; Moseley et al., 2012). Motor development is impaired with most patients unable to attain independent walking and pincer grasp. Similarly, communication is severely impaired with approximately 20% of patients achieving at least a single word of verbal communication (Fehr et al., 2016; Fehr et al., 2015). In addition to epilepsy and developmental impairment, patients with CDD have a variety of additional health challenges including gastrointestinal problems such as constipation and reflux, dysautonomia, cortical visual impairment, movement disorders, dysregulated sleep, and behavioral challenges (Fehr et al., 2016; Hagebeuk et al., 2013).
Increasing development of novel disease modifying therapeutics necessitates outcome measures that capture symptoms additional to seizure frequency. However, the severity of developmental impairments and broad range of symptoms in CDD (Demarest et al., 2019b; Fehr et al., 2015; Mangatt et al., 2016) reduces the utility of most existing clinical trial outcome measures. Furthermore, different treatments can only be compared if instruments are available to measure the breadth of outcomes that are important to patients and their families. The CDD Clinical Severity Assessment (CCSA) was developed in 2019, through an international multi-stakeholder panel who participated in a modified Delphi process. This was the first attempt to develop a comprehensive scale capturing the diverse challenges of CDD. This initial assessment included clinician- and parent-reported items for functional abilities, comorbidities, and neurologic impairments of CDD (Demarest et al., 2019a). It was then developed further as two separate measures because of differing sources of information (caregiver vs clinician). The CDD Clinical Severity Assessment – Clinician (CCSA-Clinician) has demonstrated content validity for assessing patient characteristics that are directly observable by a clinician within a typical clinical setting (Saldaris et al., 2021). The revised and content-validated CCSA-Clinician includes 29 items measuring dimensions of functional abilities and neurologic impairment and validation is ongoing.
This paper aims to describe the development and modification of the caregiver-reported items in the CCSA, and initial evaluation of content and response process validity and structural characteristics through cognitive interviews. Initial pilot data were collected from participants once the items were finalized as a preliminary assessment of score distribution.
2. Materials and methods
This study was approved by the University of Colorado IRB (COMIRB 18–1598). Consent was obtained remotely during the cognitive interviews using an electronic signature. All cognitive interviews with caregivers were completed through a video interface between June 2020 through March 2021.
2.1. Participants
Participants were parents or guardians of CDD patients, over 18 years old and English speaking. They were recruited from an established cohort of patients at the CDKL5 Center of Excellence at Children’s Hospital Colorado as well as some referrals nationally through the International Foundation for CDKL5 Research (IFCR). We used purposive sampling in recruiting participants aiming for maximum variation regarding age, comorbidities, and developmental skills seen in CDD, evaluated using the 16-point CDD Developmental Score (Brock et al., 2021).
2.2. Preparation of draft instrument
The initial caregiver instrument contained the caregiver questions of the CCSA (Demarest et al., 2019a). This version included 87 items that were organized into the following sections: history of medications and therapy; information about the frequency and intensity of different types of seizures in the past 30 days; cognition, behavior, vision, and speech in the past 30 days; the child’s ability to complete tasks that assess gross and fine motor function and developmental milestones communication, cognition and social functional skills; autonomic functions in the past 30 days; and lastly, overall impressions of changes in the child’s health from caregivers. It was first reviewed by a survey methodologist (SZ) and the study team, and item wording was changed to better conform with common best practices for questionnaire design since the instrument elicited caregiver-reported information (Artino et al., 2018; Gehlbach and Artino, 2018). For example, we revised the language to include lay rather than clinical terms, defined terms where necessary and included time frames for recollection of behaviors or symptoms. We split double-barreled items into component items. The instrument was then programmed into REDCap hosted at the University of Colorado for cognitive interviews (Harris et al., 2019; Harris et al., 2009).
2.3. Validation processes
Cognitive interviews are a qualitative technique essential for establishing content and response process validity (Peterson et al., 2017). All interviews were completed by one interviewer (AM) who had experience with semi-structured interviewing and was specifically trained to conduct cognitive interviews for this study. Cognitive interviews allow researchers to gauge if respondents understand items in the same way they were intended (Willis and Artino, 2013). In addition to neutral probes and follow-up questions after each item, we used the “think-aloud” technique. Respondents were asked to think out loud while they were answering each item which provided information into the various stages of how respondents form their answer to each of the items. Questions asked how participants understand specific terms, how they recalled or retrieved relevant information from memory and how motivated they were to do so, how they used response options to judge which of the retrieved information might be relevant, and if they adjusted the information before they reported it due to social desirability (Ziniel et al., 2019). During the interview, the interviewer audio-recorded and took notes on participant feedback to each item as well as any suggestions on how to improve it. Non-verbal reactions such as hesitating with an answer, taking a long time to answer, or any facial expressions were also noted to help identify the difficulty of items. After each round of cognitive interviews, the study team met to discuss the comments, reactions, and suggestions from the participants and how to address them. These changes were then implemented prior to the subsequent round of cognitive interviews. Recruitment for additional rounds of cognitive interviews was stopped once cognitive interview participants did not offer any further suggestions on how to change the instruments and our interviewer was certain the items were understood in the way they were intended.
Establishing content validity is a critical portion of the outcome measure development processes that ensures that the overall conceptual framework, item content and structure of the outcome measure are relevant, comprehensive, and comprehensible to the population of interest and meet the Food and Drug Administration (FDA) requirements for development of outcome measures (U.S. Department of Health and Human Services et al., 2009). This foundational step precedes quantitative testing of reliability, validity, and responsiveness to change. Content validity had already been established by the expert panel through the modified Delphi process (Demarest et al., 2019a) and we now sought to establish content validity of the two caregiver-reported measures by caregivers with a CDD patient themselves (Magasi et al., 2012). At the end of each section as well as each instrument we asked the caregiver if all aspects of their experience regarding the domain had been covered or if any item included wasn’t relevant to their child. We asked 11 experts in CDD to provide feedback on the final instrument.
2.4. Preliminary pilot testing
Participants were offered the opportunity to complete the instruments again once they had been finalized simulating a real assessment and offer their feedback on the changes that had been made. The data collected from the administration of the finalized CCSA-Caregiver and CDQ-Caregiver was used to assess the general structure of scores produced by the instruments. We used violin plots to illustrate the median, interquartile range, minimum and maximum scores, and the density curves of the distributions of the subdomains and the overall score. Stata version 16.1 was used to compute all scores (Stata Statistical Software, College Station, TX USA) and the violin plots were produced using GraphPad Prism version 9.0.0 (GraphPad Software, San Diego, CA USA).
3. Results
3.1. Participants
We completed three rounds of cognitive interviews with 5 new participants in each round, for a total of 15 cognitive interviews. For one child both caregivers volunteered to participate in a cognitive interview, so the 15 cognitive interviews represent the experience of 15 individual caregivers of 14 children with CDD. Table 1 shows the demographic characteristics of the participants and their children with CDD. Most participants were female (86.7%), white (90%), non-Hispanic (90%), married (70%), and between 31 and 40 years old (60%) at time of child diagnosis. Half had a Bachelor degree (50%). Most of the children with CDD were female (85.7%), white (78.6%), non-Hispanic (85.7%), and had a median age of 10.4 years. The mean CDD Developmental Score was 9 points (interquartile range: 5.75, 10.75, total possible score of 16 points). In addition to participating in a cognitive interview, 11 of the 15 cognitive interview participants (73.3%) volunteered to complete the finalized instruments simulating a real assessment.
Table 1:
Characteristics | Percentage/Median (Interquartile Range) |
---|---|
Cognitive Interview Participant (Caregiver) 1 | |
Gender (n=15) | |
Female | 86.7% |
Male | 13.3% |
Race (n=10) | |
American Indian or Alaska Native | 10.0% |
White | 90.0% |
Ethnicity (n=10) | |
Hispanic or Latino | 10.0% |
Not Hispanic or Latino | 90.0% |
Relationship Status (n=10) | |
Divorced | 20.0% |
Married | 70.0% |
Other | 10.0% |
Number of Dependents in Household (n=10) | |
1 | 40.0% |
2 | 60.0% |
Age at Child’s Diagnosis (n=10) | |
25–30 | 10.0% |
31–40 | 60.0% |
41–50 | 20.0% |
60+ | 10.0% |
Highest Level of Education (n=10) | |
Bachelor’s degree | 50.0% |
High school diploma or equivalent degree | 20.0% |
Post-secondary degree | 30.0% |
Household Income (n=10) | |
$40,000 - $59,999 | 10.0% |
$60,000 - $79,999 | 20.0% |
$80,000 - $99,999 $100,000 - $129,999 $150,000 or greater |
10.0% 40.0% 20.0% |
Child Demographics 2 | |
Gender (n=14) | |
Female | 85.7% |
Male | 14.3% |
Race (n=14) | |
Other | 14.3% |
Unknown | 7.1% |
White | 78.6% |
Ethnicity (n=14) | |
Hispanic or Latino | 7.1% |
Not Hispanic or Latino | 85.7% |
Unknown | 7.1% |
Age at time of parent interview (n=14) | 10.4 (4.2, 15.3) |
Severity Scale3 (16-Point Scale) (n=10) | 9 (5.75, 10.75) |
Demographic characteristics of the parents were collected from the participants that completed the final instrument.
The children’s characteristics were extracted from the medical record with the permission of the parents.
Brock D, Fidell A, Thomas J, Juarez-Colunga E, Benke TA, Demarest S. Cerebral Visual Impairment in CDKL5 Deficiency Disorder Correlates With Developmental Achievement. J Child Neurol. Oct 2021;36(11):974-980.
3.2. Item development
Figure 1 describes the refinement of the initial instrument. The first round of five cognitive interviews revealed that caregivers felt that the survey with 87 items was very long, especially since all items were designed to be answered by all respondents. Caregivers whose children were at an age where certain developmental milestones could not have been attained felt that they should not be asked to answer these questions. Parents reported fatigue as well as frustration to have to answer these questions. We therefore determined for each of the five developmental subdomains of the instrument which questions were appropriate for children’s different developmental age ranges and milestones. If the child was younger and/or had developed fewer skills, then fewer questions would be asked of a caregiver. During this first round, caregivers expressed problems with questions around the different types of seizures and how to count them. Specifically, caregivers were confused by questions about seizures whose definitions showed subtle distinctions, e.g. the definition of a cluster of jerks vs. a series of isolated jerks that should be counted separately.
After the second round of cognitive interviews, we separated items into two caregiver-reported measures: (1) the CDD Clinical Severity Assessment – Caregiver (CCSA-Caregiver) focusing on areas of function and impairment analogous to the CCSA-Clinician but based on caregivers’ observation (Supplemental Material A), and (2) the CDD Developmental Questionnaire (CDQ-Caregiver) (Supplemental Material B). The CDQ-Caregiver seeks to expand on traditional developmental assessments to measure incremental, but important, developmental achievements in this severely impaired population. Respondents seemed to be able to focus better on symptoms on the one hand and developmental accomplishments on the other. Separating the items into two measures might also have an additional advantage allowing different intervals of assessment in a clinical trial, as symptoms may demonstrate change more rapidly than developmental achievements.
During this round of cognitive interviews -just as in the second round, caregivers provided additional comments on the phrasing of some questions and response options, and the think-aloud protocol revealed that some of the questions were still not completely understood as intended. Table 2 illustrates some of the issues that were uncovered because of caregivers thinking aloud while they were answering the questions or from further probing after a caregiver had provided a response. As mentioned before, caregivers had difficulties understanding key terms used to defining different types of seizures. In the original version of the questionnaire each type of seizure was defined in the question that asked about it, causing the respondents to skip around on the page and read the other definitions in an attempt to help them understand how to distinguish different types from each other. Once all definitions were put at the beginning of this section and simplified, respondents seem to have less difficulty categorizing the seizures their child experienced into the different types about which questions were asked. Throughout the first 10 cognitive interviews, caregivers also indicated that response options were often overly complex and not well suited for some of the developmental statements (Table 2). A simplification helped to decrease the cognitive burden when assessing these response options and increased the clarity of the meaning of the items.
Table 2:
Phase of Question-Answering Process | Issue | Quotation | Original Version | New Version |
---|---|---|---|---|
Comprehension of the question | Misunderstanding of objectively defined key terms (lexical understanding) |
“I’m kind of looking down to see what you have – what the Non-convulsive ones are so I can kind of rule those out”
“They’ll be like 2 in ten minutes, so is that considered a cluster, or is that two isolated jerks in ten minutes…maybe it would be helpful for me is you had defined a cluster…” |
Convulsive: Tonic, tonic-clonic, or clusters of drops that are disruptive and bothersome to patient or family. Seizures with multiple phases should be counted as a single seizures with multiple phases should be counted as a single seizure. |
Convulsive: • Tonic, • Tonic-clonic, or • Clusters of drops or jerks or spasms that are disruptive and bothersome to patient or family.Seizures with multiple phases should be counted as a single seizure. (A cluster of seizures would be seizures occurring one right after another without returning to the child’s typical function) All definitions were included at top of each page/question |
Inconsistent understanding of key terms across respondents (inclusion/ exclusion of subgroups) | “Do we need to include Gabapentin for sleep? It is sort of a anti seizure medicine. Right, but we’re not really using it for seizures, so that I’m not quite sure about how to answer.” | How many antiseizure medications is your child currently taking? (Please do not include any rescue medications, Vagal Nerve Stimulator, or ketogenic diet) |
How many antiseizure medications is your child currently taking to control seizures? (Please do not include any rescue medications, Vagal Nerve Stimulator, antiseizure medication used for sleep or ketogenic diet) |
|
Long and overly complex question/response options; logical sentence structure |
“I still, well, what I did was [Pause]… Sometimes well, I’m [Pause]… I would say still working on this one because [Pause]…
I’m not sure what help means? That’s what I was going to ask you that. Able to do this, but sometimes needs help, so I’m like I’m not sure how I’m to help her with doing a long sentence so well, we’re still working on it.” |
1, Always able to do this without help | 2, Able to do this but sometimes needs help | 3, Can do this sometimes but always needs help | 4, Still working on this one | 1, Usually able to do | 2, Often able to do | 3, Sometimes able to do | 4, Not currently able to do | |
Judgement of the information retrieved/preliminary response | Difficulty coming to an answer because of sensitivity of topic | “She doesn’t bathe herself, so I have to do a lot of it -of cleaning her face and she just will push it away or something like that, but I wouldn’t say it’s aggressive, it’s just kind of away, but her nails are sharp. Aggressive, and only because I know what it was like a while back - a while back - where it was real intentional I’m going to do something.” | How often has your child been aggressive in the last 30 days? | How often has your child been aggressive, regardless of intent or purpose, in the last 30 days? |
Communication of the response | Incomplete response options | “So there kind of seems like a big jump there from holding it for 10 seconds to rarely making eye contact for more than a second or two. Because, like for (child name) maybe she would be in-between those two.” | 1, He/she makes direct eye contact for a long time (e.g. for total duration of an interaction) | 2, He/she makes direct eye contact but doesn’t hold it that long (e.g. holds it for at least 10 seconds) | 3, He/she rarely makes eye contact for more than a second or two | 4, He/she makes eye contact but out the corner of his/her eye | 5, He/she doesn’t really seem to look at me | 1, Direct eye contact, for total duration of an interaction | 2, He/she makes direct or indirect eye contact, holds it for at least 10 seconds, but not the total duration of an interaction | 3, He/she makes eye contact but is usually less than 10 seconds | 4, He/she makes eye contact but out the corner of his/her eye | 5, He/she doesn’t really seem to look at me |
Problems matching response to available response option | “So, when the word used, like always, never, that to me limits me, because I’m like no its not never or is not always.” | 1, Always able to do 2, Often able to do 3, Sometimes able to do 4, Never able to do |
1, Usually able to do 2, Often able to do 3, Sometimes able to do 4, Not currently able to do |
We also needed to change questions that were asking about negative behaviors, such as a child being aggressive or self-harming. Caregivers were adamant in their need to point out that the negative behaviors of their children weren’t intentional but more a reflection of the underlying disease and felt that the item needed to reflect that for them to be comfortable to answer the item. We therefore added to these items the phrase “regardless of intent or purpose.” Another common correction we needed to make was adding additional options or rephrasing the current response options because caregivers felt that the response options provided did not reflect the current reality and they could therefore not really answer the question.
The third round of cognitive interviews with caregivers yielded only small wording changes, ultimately discovering no further issues with the two instruments.
After the cognitive interviews with caregivers were complete, 11 experts treating patients with CDD and conducting research in this area provided feedback. We specifically asked them to review the items regarding their relevance and appropriateness for caregivers of children with CDD and comment on if each item as well as the instruments as a whole were consistent with their experience treating CDD patients. The experts confirmed that the items and instruments were relevant and appropriate and had only minor comments regarding potential alternative wording. No major changes were made to the two instruments.
3.3. Instrument scoring and structure
The 11 caregivers completing the final CDQ-Caregiver and CCSA-Caregiver commented on the improvements that had been made to the instruments and had no further suggestions. The final CCSA-Caregiver was divided into three subdomains for scoring purposes: (1) epilepsy symptoms, (2) cognitive, behavioral, and vision symptoms, and (3) autonomic symptoms. The final CDQ-Caregiver was separated into five subdomains: (1) gross motor skills, (2) fine motor skills, (3) receptive communication skills, (4) expressive communication skills, and (5) social and cognitive skills. Because of the branching logic implemented based on symptom severity, the maximum number of items to be administered from the CCSA-Caregiver range from 30 to 50; caregivers of the least symptomatic children only needed to answer 30 questions while caregivers of the most symptomatic children answered 50. The number of items from the CDQ-Caregiver to be answered by the caregiver depends on the age as well as the developmental status of the child with CDD and can range from a minimum of 16 items for 6-month-old patients to 66 items for patients over 60-month-old with more milestones attained. Figure 1 includes the minimum and maximum number of items for each of the subdomains. The response options of each item the respondent is administered are scored between 0 and 100, irrespective of the number of categories, with higher scores indicating less developmental attainment or higher severity. For example, the response options of a question from the severity assessment with four response options receive in ascending severity the scores 0, 33.3, 66.6, and 100. The average score of all eligible items in a subdomain represents the domain score while the average of all domain scores constitutes the overall instrument score. Each instrument can also be summarized by an overall severity score and overall development score, respectively. Figure 2 displays the violin plots for the subdomain and overall score for the CCSA-Caregiver and the CDQ-Caregiver from the pilot testing. None of the distribution within each subdomain show clear floor or ceiling effects suggesting the instruments’ abilities to assess a wide range of symptom severity and developmental milestones.
4. Discussion
This study refined the initial caregiver reporting section of the CCSA (Demarest et al., 2019a) into two separate instruments measuring caregiver-reported observations regarding a child’s CDD: 1) the CDQ-Caregiver that focuses on assessing developmental attainment regarding gross and fine motor skills, receptive and expressive communication skills, as well as social and cognitive skills, and 2) the CCSA-Caregiver that captures the extent of clinical manifestations such as epilepsy symptoms, cognitive, behavioral, and vision symptoms, as well as autonomic symptoms. Both together reflect a comprehensive assessment of a child with CDD and are designed so that scoring can be combined for an overall representation of severity that is anchored to the CDKL5 disease concept model developed through the FDA Patient Focused Drug Development process (U.S. Department of Health and Human Services et al., 2009).
Using cognitive interviews, we revised the instruments and established their content validity from a caregiver perspective consistent with FDA guidance (U.S. Department of Health and Human Services et al., 2009). In addition, we also asked experts in CDD to review the measures for relevance and appropriateness of content. The cognitive interviews with caregivers also allowed us to assess response process validity, ensuring that questions, answer options, and instructions are understood in the same way they were intended by the measure designers. The participants in these interviews were specifically recruited to represent experiences within the spectrum of severity seen in CDD as well as different levels of education. After three rounds of cognitive interviews and iteratively restructuring the measures, deleting, and adding items, as well as changing item wording, we reached two instruments that caregivers felt comfortable to complete. Data from a pilot sample of caregivers showed no clustering of scores at the ends of the scale, a problem previously encountered by other similar measures (Demarest et al., 2019b). More specifically, existing parent-reported measures of developmental functioning and/or delay are not adapted to identify small but clinically important changes regarding the development of gross and fine motor function, and communication, behavioral, and social skills in children with CDD (Sparrow et al., 2016; Squires and Bricker, 2009). These measures have also demonstrated poor performance and significant floor effects in similar populations with developmental and epileptic encephalopathies (Berg et al., 2022). Other, more in-depth, tests that could be more amenable to assess developmental functioning in this population, are mostly designed to be administered by clinicians, nurses, or occupational therapists and focus more on screening for developmental delay in motor function, language, and cognition than on measuring change over time (Bayley and Aylward, 2019; Ellison et al., 1985; Frankenburg et al., 1992).
The separation of the severity and developmental attainment items into two different instruments provides several advantages. For caregivers, the grouping of items in two different instruments seems to bring more cohesion among the question and make more sense since as on the one hand they are reporting developmental attainment and on the other more clinical symptoms. Having two instruments also allows them to be administered in different time intervals since changes to clinical symptoms would be likely to occur more rapidly (e.g. after a change in medication) than changes in development.
A limitation of this study is that the final instruments were only administered in a small sample and therefore conclusions cannot be drawn regarding other aspects of validity. Additional interviews with caregivers whose children are not receiving care at the hospital where the cognitive interviews were conducted might have provided further comments regarding the phrasing of the items. Some domain scores, specifically the Social & Cognitive Skills Domain and the Expressive Communication Domain of the CDQ-Caregiver, also showed skewed distributions. However, given that for both domains the complete range of scores is represented, this could be an effect of the small sample size and the characteristics of the patients regarding these domains. Finally, despite our effort to capture a range of caregiver and patient characteristics, it is possible that our interview samples had some bias. For example, our sample comprised mainly relatively affluent, white, and well-educated caregivers. In addition, we also only included parents as participants but intended these instruments to be applicable to other non-parent primary caregivers of CDD children such as grandparents with custody.
Ultimately, the CCSA-Caregiver and CDQ-Caregiver provide complementary data to the CDD Clinical Severity Assessment that is completed by a clinician (CCSA-Clinician) (Saldaris et al., 2021). The CCSA-Clinician focuses on the assessment of functional abilities and neurologic impairments in the moment from a clinical perspective while the caregiver-reported instruments provide information about CDD symptoms and developmental attainments that necessitate observations over a longer time period.
Future work will need to establish the validity of the CCSA-Caregiver and CDQ-Caregiver in a larger sample to evaluate the structural, reliability, and psychometric, properties of these measures. Any future studies should focus on including participants of lower socioeconomic status, ethnic and racial minorities, and non-parental primary caregivers. Validation of a Spanish translation of these instruments will also help in reaching a wider patient population. This ongoing work will establish the final outcome measures and their readiness for use in future clinical trials.
5. Conclusion
This study refined the caregiver portion of the initial CCSA into two measures, the CCSA-Caregiver and the CDQ-Caregiver, that can be administered independently and create flexibility of administration intervals. Cognitive interviews and additional expert review of the CCSA-Caregiver and CDQ-Caregiver provided satisfactory evidence for both content, and response process validity. Data from a small sample of caregivers showed no clear floor effects on any of the subdomains of the measures. As a next step, both measures will be psychometrically validated to establish readiness for their use in clinical trials for CDKL5 Deficiency Disorder.
Supplementary Material
Acknowledgments:
Study data were collected and managed using REDCap electronic data capture tools hosted at University of Colorado Denver – Anschutz Medical Campus (Harris et al., 2019; Harris et al., 2009). REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing: 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.
Funding:
This work was supported by funding from the International Foundation for CDKL5 Research and the NIH (U01NS114312-01A1) and NIH/NCATS Colorado CTSA (UL1 TR002535). Heather Olson was supported by the National Institute of Neurologic Disorders and Stroke (K23 NS107646-05). None of the funding sources was involved in the design of the study; the collection, analysis and interpretation of data; in the writing and decision to submit the article for publication.
Abbreviations:
- CDD
CDKL5 deficiency disorder
- CCSA
CDD Clinical Severity Assessment
- IRB
Institutional Review Board
- CDQ
CDD Development Questionnaire
- FDA
Food and Drug Administration
Footnotes
Declaration of Competing Interests (COI):
S. Ziniel has no COI.
A. Mackie has no COI.
J. Saldaris has no COI.
H Leonard has consulted with Avexis, Anavex, GW Pharmaceuticals, Newron and Acadia Pharmaceuticals on unrelated subject matter. She has also consulted with Marinus Pharmaceuticals, Ovid Therapeutics and Orion Corporation regarding CDKL5 Deficiency Disorder. H Leonard has previously been funded by a NHMRC Senior Research Fellowship (#1117105). She has received funding from the NIH, International Foundation for CDKL5 Research, the Orphan Disease Centre University of Pennsylvania, Marinus Pharmaceuticals and Orion Corporation in relation to CDKL5 Deficiency Disorder.
P. Jacoby has no COI.
E. Marsh has consulted for Acadia Pharmaceuticals and Medscape. He is also the site PI for Acadia, Marinus, Takeda, Stoke, and Zogenix. He has received grants related to CDD from ICFR, RSRT, IRSF, NIH, and the State of Pennsylvania.
B. Suter has received consulting fees for Ionis Pharmaceuticals, Neurogene and Taysha; all remuneration has been paid to his department. He also acted as investigator for clinical trials with Acadia, Marinus and Newron.
E. Pestana-Knight has received consulting fees from Biomarin Pharmaceuticals, Zogenix, Marinus pharmaceuticals and Biocodex. She is a member of the scientific advisory board and speaker for Marinus Pharmaceuticals.
H. Olson received consulting fees from Takeda Pharmaceuticals, Zogenix, and Ultragenyx regarding clinical trial design, Ovid Therapeutics regarding clinical trial results, Marinus Pharmaceuticals regarding CDKL5 Deficiency Disorder, and has done consulting for the FOXG1 Research Foundation.
D. Price has no COI.
J. Weisenberg has no COI.
R. Rajaraman has consulted for Zogenix and Ultragenyx pharmacueticals. He is on the speaker’s bureau for Marinus Pharmacueticals.
G. VanderVeen has no COI.
T. Benke received research funding from GRIN2B Foundation, the International Foundation for CDKL5 Research, Loulou Foundation, the National Institutes of Health, and Simons Foundation; consultancy for Alcyone, GRIN Therapeutics, the International Rett Syndrome Foundation, Marinus Pharmaceuticals, Neurogene, Takeda Pharmaceutical Company Limited, Ultragenyx and Zogenix/UCB; clinical trials with Acadia Pharmaceuticals Inc., GW Pharmaceuticals, Marinus Pharmaceuticals, Ovid Therapeutics, and Rett Syndrome Research Trust; all remuneration has been made to his department.
J. Downs has consulted for Marinus, Ultragenyx, Orion and Taysha; clinical trials with Anavex; all renumeration has been made to her department. She has received funding from the NIH and International Foundation for CDKL5 Research in relation to CDKL5 Deficiency Disorder.
S. Demarest has consulted for Biomarin, Neurogene, Marinus, Tysha, Ultragenyx, Zogenix and Ovid Therapeutics. He has funding from project 8P and Mila’s Miracle Foundation. He also serves on the advisory board for the non-profit foundations SLC6A1 Connect, Project 8P, Ring14 USA and FamilieSCN2A.
References
- Artino AR Jr., Durning SJ, Sklar DP, 2018. Guidelines for Reporting Survey-Based Research Submitted to Academic Medicine. Acad Med 93, 337–340. [DOI] [PubMed] [Google Scholar]
- Bayley N, Aylward GP, 2019. Bayley Scale of Infant and Toddler Development, Fourth Edition (Bayley™-4). [Google Scholar]
- Berg AT, Kaat AJ, Zelko F, Wilkening G, 2022. Rare diseases - rare outcomes: Assessing communication abilities for the developmental and epileptic encephalopathies. Epilepsy Behav 128, 108586. [DOI] [PubMed] [Google Scholar]
- Brock D, Fidell A, Thomas J, Juarez-Colunga E, Benke TA, Demarest S, 2021. Cerebral Visual Impairment in CDKL5 Deficiency Disorder Correlates With Developmental Achievement. J Child Neurol 36, 974–980. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Demarest S, Pestana-Knight EM, Olson HE, Downs J, Marsh ED, Kaufmann WE, Partridge CA, Leonard H, Gwadry-Sridhar F, Frame KE, Cross JH, Chin RFM, Parikh S, Panzer A, Weisenberg J, Utley K, Jaksha A, Amin S, Khwaja O, Devinsky O, Neul JL, Percy AK, Benke TA, 2019a. Severity Assessment in CDKL5 Deficiency Disorder. Pediatr Neurol 97, 38–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Demarest ST, Olson HE, Moss A, Pestana-Knight E, Zhang X, Parikh S, Swanson LC, Riley KD, Bazin GA, Angione K, Niestroj LM, Lal D, Juarez-Colunga E, Benke TA, 2019b. CDKL5 deficiency disorder: Relationship between genotype, epilepsy, cortical visual impairment, and development. Epilepsia 60, 1733–1742. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ellison PH, Horn JL, Browning CA, 1985. Construction of an Infant Neurological International Battery (Infanib) for the assessment of neurological integrity in infancy. Phys Ther 65, 1326–1331. [DOI] [PubMed] [Google Scholar]
- Fehr S, Downs J, Ho G, de Klerk N, Forbes D, Christodoulou J, Williams S, Leonard H, 2016. Functional abilities in children and adults with the CDKL5 disorder. Am J Med Genet A 170, 2860–2869. [DOI] [PubMed] [Google Scholar]
- Fehr S, Leonard H, Ho G, Williams S, de Klerk N, Forbes D, Christodoulou J, Downs J, 2015. There is variability in the attainment of developmental milestones in the CDKL5 disorder. J Neurodev Disord 7, 2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frankenburg WK, Dodds J, Archer P, Shapiro H, Bresnick B, 1992. The Denver II: a major revision and restandardization of the Denver Developmental Screening Test. Pediatrics 89, 91–97. [PubMed] [Google Scholar]
- Gehlbach H, Artino AR Jr., 2018. The Survey Checklist (Manifesto). Acad Med 93, 360–366. [DOI] [PubMed] [Google Scholar]
- Hagebeuk EE, van den Bossche RA, de Weerd AW, 2013. Respiratory and sleep disorders in female children with atypical Rett syndrome caused by mutations in the CDKL5 gene. Dev Med Child Neurol 55, 480–484. [DOI] [PubMed] [Google Scholar]
- Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, McLeod L, Delacqua G, Delacqua F, Kirby J, Duda SN, Consortium RE, 2019. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 95, 103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG, 2009. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42, 377–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leonard H, Downs J, Benke TA, Swanson L, Olson H, Demarest S, 2022. CDKL5 deficiency disorder: clinical features, diagnosis, and management. Lancet Neurol 21, 563–576. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Magasi S, Ryan G, Revicki D, Lenderking W, Hays RD, Brod M, Snyder C, Boers M, Cella D, 2012. Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Qual Life Res 21, 739–746. [DOI] [PubMed] [Google Scholar]
- Mangatt M, Wong K, Anderson B, Epstein A, Hodgetts S, Leonard H, Downs J, 2016. Prevalence and onset of comorbidities in the CDKL5 disorder differ from Rett syndrome. Orphanet J Rare Dis 11, 39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moseley BD, Dhamija R, Wirrell EC, Nickels KC, 2012. Historic, clinical, and prognostic features of epileptic encephalopathies caused by CDKL5 mutations. Pediatr Neurol 46, 101–105. [DOI] [PubMed] [Google Scholar]
- Peterson CH, Peterson NA, Powell KG, 2017. Cognitive interviewing for item development: Validity evidence based on content and response processes. Measurement and Evaluation in Counseling and Development 50, 217–223. [Google Scholar]
- Saldaris J, Weisenberg J, Pestana-Knight E, Marsh ED, Suter B, Rajaraman R, Heidary G, Olson HE, Devinsky O, Price D, Jacoby P, Leonard H, Benke TA, Demarest S, Downs J, 2021. Content Validation of Clinician-Reported Items for a Severity Measure for CDKL5 Deficiency Disorder. J Child Neurol 36, 998–1006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sparrow SS, Cicchetti DV, Saulnier CA, 2016. Vineland Adaptive Behavior Scales, Third Edition (Vineland-3; ). [Google Scholar]
- Squires J, Bricker D, 2009. Ages & Stages Questionnaire, Third Edition (ASQ-3™). A parent-completed child monitoring system Paul H Brookes Publishing Co, Baltimore, MD. [Google Scholar]
- U.S. Department of Health and Human Services, Food and Drug Administration (FDA), Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiological Health (CDRH), 2009. Guidance for Industry - Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims
- Willis GB, Artino AR Jr., 2013. What Do Our Respondents Think We’re Asking? Using Cognitive Interviewing to Improve Medical Education Surveys. J Grad Med Educ 5, 353–356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ziniel SI, McDaniel CE, Beck J, 2019. Bringing Scientific Rigor to Survey Design in Health Care Research. Hosp Pediatr 9, 743–748. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.