Development of a youth-report measure of DPN symptoms: Conceptualization and content validation

Abstract

Aims

To develop a content valid youth-report measure of diabetic peripheral neuropathy (DPN) symptoms.

Methods

Semi-structured interviews with 5 clinicians and 15 youth aged 8–17 with diabetes were conducted to elicit and clarify youth’s DPN experiences. A systematic review of existing adult-report DPN symptom measures was conducted to identify item concepts representative of each experience. The concepts were transformed into items that were iteratively revised based on cognitive interviews (n = 13 youth aged 8–17) and readability analyses.

Results

Clinician and youth interviews supported a tripartite conceptual framework of youth DPN symptoms: paresthesia, pain, and anesthesia. Forty-eight youth-report items were generated to represent DPN symptoms identified through the semi-structured interviews and a systematic review of 13 symptom questionnaires for adults. Of these, 23 were eliminated and 3 were revised based on cognitive interviews conducted with youth. The remaining 25 items were on average, written at a 3rd grade reading level.

Conclusions

This study is the first to generate a content valid self-report measure of youth’s lived experiences with DPN that uses developmentally appropriate terminology. With further psychometric testing, the measure could be used to advance research on pediatric DPN and enhance clinicians’ capacity to identify the condition in childhood.

Introduction

Diabetic peripheral neuropathy (DPN) is the presence of signs and/or symptoms of peripheral nerve dysfunction among people with diabetes after the exclusion of other causes [1]. Sensory nerve damage produces some of the most troublesome complications of longstanding diabetes including disabling pain or loss of sensation and increased vulnerability to injury and amputation [1], [2]. DPN is well recognized as a major complication of type 1 and type 2 diabetes mellitus in adults, but there is considerable uncertainty as to its incidence, prevalence, diagnosis and prognosis in among youth [2], [3]. Estimates of the prevalence of DPN in childhood range from 7% to 57% [2], [4], [5], [6], [7], [8]. The wide range of prevalence across studies is likely due to the use of different assessment methods and the lack of consensus as to the appropriate diagnostic criteria for pediatric DPN [2], [7].

The American Diabetes Association and the American Academy of Neurology recommend the use of multiple tests to assess the signs and symptoms of DPN [1], [9]. Nerve conduction studies (NCS; e.g., needle electromyography, somatosensory evoked potentials) are considered by many to be the “gold standard” DPN measure, but only a few studies have been performed on NCS in youth with diabetes and the procedures are invasive, painful, and difficult to perform [6], [7]. Moreover, NCS selectively examines large fiber dysfunction and fails to detect small fiber neuropathy, which may be more prevalent in youth [7], [10]. Compared to NCS, neurological tests (e.g., vibration sensation, tactile perception, thermal discrimination) are more feasible to administer in everyday clinical practice, but less sensitive to detecting DPN in youth [6], [11].

Patient-reported outcomes (PRO) measures (e.g., questionnaires, screening instruments) are sometimes used to assess DPN symptoms and associated functional impacts [12]. PRO measures provide information about patients’ subjective experiences of a health condition and functional and quality of life impacts. Prior research indicates that in early stages of diabetes, patients’ subjective symptoms of paresthesia, numbness, and pain correlate poorly with conduction velocity and other clinical findings [6], [7], [13]. Thus, failure to assess patient-reported DPN symptoms may result in the under-identification of patients at risk for worsening symptoms and related complications. Unfortunately, most studies that assess youth-reported DPN symptoms fail to use standardized measures, which greatly limits measurement validity and prohibits the synthesis of research and comparisons across studies [6], [13]. Moreover, all existing PRO measures of DPN symptoms were developed for adults. The appropriateness, meaningfulness, and psychometric properties of the measures have not been evaluated for youth.

Measurement challenges pose significant barriers to research on the natural history of DPN in childhood. In particular, sensory nerve dysfunction and DPN symptoms associated with shorter-duration diabetes in youth, as well as their relationship with more serious neuropathies in adulthood are not well understood [2], [7]. Imprecise measurement may also impede the early detection of DPN, which is important to reduce of risk of future amputation, disability, and impaired quality of life [2], [7]. Thus, there is a critical need for a developmentally appropriate, reliable, accurate, sensitive, and feasible youth-report measure of DPN symptoms [5], [6]. Such a measure would strengthen research on lifespan development of DPN and enhance providers’ capacity to identify and address youths’ DPN symptoms. In accordance with multiple PRO measure development standards [14], [15], [16], the purpose of this study was to develop a content valid youth-report measure of diabetic peripheral neuropathy (DPN) symptoms. For PRO measures, content validity is the degree to which items reflect how patients from a target population understand, experience, and discuss the outcome [17], [18].

Methods

The youth-report DPN symptom screener was developed following best-practice standards [14], [19], [20], [21]. The methods are summarized in Fig. 1. Study procedures that involved human subjects were approved by the Institutional Review Board at the Children’s Hospital of Philadelphia.

Fig. 1.

Measure development process. ^*Facets not represented in the final item pool.

Clinician semi-structured interviews

A preliminary conceptual framework was generated based on prior pediatric DPN research [2], [3], [4], [7], [8], [13]. Thereafter, we conducted semi-structured concept elicitation interviews with 5 clinicians to maximize the framework’s coverage of DPN concepts that are clinically meaningful [17], [18], [22]. Clinicians (2 pediatric endocrinologists, 1 adult endocrinologist, 1 pediatric neurologist, 1 pediatric diabetes nurse practitioner) provided feedback on the framework’s component labels, definitions, and organization. We revised and expanded the DPN framework based on expert feedback.

Youth semi-structured interviews

We conducted semi-structured interviews with 15 youth to elicit DPN experiences that are relevant and meaningful to patients themselves [17], [18], [22]. Youth with type 1 diabetes were recruited from the Diabetes Center at a large children’s hospital if in the context of clinical care, they reported experiencing altered sensations consistent with DPN. They were asked to describe burning sensations, loss of feeling, pain, and irritation in their feet and hands, as well as the consequences of those experiences using their own words. DPN experiences were derived through thematic analysis of youth interview transcripts until saturation was achieved (i.e., when new participants failed to provide novel information) [23]. We revised and expanded the DPN framework based on youth interview findings.

Systematic review of DPN measures

Existing DPN PRO measures were identified through a systematic literature search in MEDLINE [24]. Search terms captured DPN experiences (e.g., numbness), self-report instruments (e.g., questionnaire), and measurement (e.g. reliability). We collected the DPN PRO measures described in the identified articles from published works. Three investigators reviewed all DPN item concepts and using a binning and winnowing process, sorted them into existing conceptual framework categories, generated new categories as needed, and eliminated redundant or vague item concepts [25].

Item generation

We transformed DPN concepts into item expressions that met the following criteria: (1) item independence: items could stand alone without reference to any other item, (2) recall period: the context of all items was “In the past 4 weeks”, (3) item wording: items were worded to inquire about the frequency of a DPN experience, (4) response options: responses were standardized and measured the frequency of symptoms (never-rarely-sometimes-often-always), and (5) clarity: items were as concise and simply worded as possible.

Cognitive interviews

We conducted cognitive interviews with 13 youth with Type 1 diabetes aged 8–17 years to evaluate item understandability and content validity [17], [26]. Youth were recruited from the Diabetes Center at a large children’s hospital. They completed a paper-and-pencil questionnaire that included about 20 items. Then, using standardized probes, interviewers asked youth to read each question aloud, state the item’s meaning in their own words, and explain their response. Youths’ understanding of each item was coded on a 3-point scale (1 = poor, 2 = partial, 3 = full understanding). Each item was tested with at least 5 youth. Items with average ratings of less than 2 were removed or revised and re-tested.

Reading level analysis

Items were reviewed for readability by calculating the Flesch-Kincaid Grade Level equivalent using Microsoft Word software. This test assigns a U.S. school grade-level to text based on sentence length and the average number of syllables per word [27].

Results

As shown in Fig. 1, The original DPN conceptual framework (version 1) was composed of paresthesia, pain, and anesthesia domains and 9 smaller conceptually-distinct categories called facets. Paresthesia was defined as abnormal sensation of tingling, itchiness, prickling, or crawling of the skin. Paresthesia facets were tingling/pins & needles, “funny” feelings, creeping/crawling sensations, and itchiness. Pain was defined as a bodily sensation characterized by physical discomfort ranging from mild sensitivity to unbearable agony. It included sharp/stabbing, shooting, and burning facets. Lastly, anesthesia was defined as a total or partial loss of tactile sensation. Anesthesia facets were numbness and loss of proprioception.

Clinician semi-structured interviews

We conducted semi-structured concept elicitation interviews with 5 clinicians to ensure that the conceptual framework included clinically meaningful DPN symptoms [17], [18], [22]. Clinicians (2 pediatric In general, clinicians agreed with the framework’s tripartite structure and domain definitions. They recognized that very little is known about how youth experience and talk about DPN symptoms and thus, confirmed the need for a developmentally appropriate child-report DPN symptom measure. Clinicians identified three DPN experiences that were missing from the preliminary conceptual framework. These included two functional consequences of DPN-related anesthesia: immobility (especially inability to move one’s hands or feet) and loss of balance (unsteadiness while standing or walking). Clinicians also noted that tactile hyperaesthesia is an important DPN symptom that was missing from the pain domain. Tactile hyperaesthesia is sensitivity to touch such that even mild physical contact can elicit significant discomfort. We generated version 2 of the DPN conceptual framework based on clinical feedback. We added two anesthesia facets (immobility, loss of balance) and expanded the anesthesia domain definition to reflect these experiences. We expanded the pain domain to include a tactile hyperaesthesia facet. Final DPN domain definitions are shown in Table 1.

Table 1.

Final DPN domain definitions.

Paresthesia	abnormal sensation of tingling, itchiness, prickling, or crawling of the skin
Pain	bodily sensation characterized by physical discomfort ranging from mild sensitivity to unbearable agony
Anesthesia	total or partial loss of tactile sensation that may result in loss of balance or mobility

Youth semi-structured interviews

We conducted semi-structured interviews with 15 youth to ensure that the conceptual framework reflected patient-identified DPN symptoms. Youth semi-structured interviewees (n = 15) were primarily male (n = 11, 73%) and Caucasian (n = 14, 93%). They ranged in age from 8 to 17 years (M = 13.2, SD = 4.6). Youths’ average duration of diabetes was 6.7 years (range: 0.8–17, SD = 5.4) and they had an average HbA1c of 8.0% (range 6.1%-11.3%) at the time of the interview. Youth identified 11 unique DPN experiences, 4 of which were not included in version 2 of the DPN framework. Within the paresthesia domain, youth described experiencing tingling, “funny” feelings, and creeping/crawling sensations, but did not reference itchiness. Many youth used creative and idiosyncratic language to describe their paresthesia symptoms. Most commonly, these descriptions referenced insects (e.g., “ants crawling on feet,” “mosquitos buzzing around feet,” “bees stinging feet”) or electricity (e.g., “electric shocks going through feet,” “static in feet,” “sparks hitting toes”). Youth identified three unique paresthesia experiences: squeezing/pressure, vibrating, and contact with rough surfaces. They reported experiencing all previously identified pain subdomains (burning, radiating/shooting, sharp/stabbing, tactile hyperaesthesia) and a few youths described throbbing sensations. Youth identified the anesthesia experiences of numbness and immobility/weakness, but they did not describe loss of proprioception or balance. Based on the youth interviews, we added four new facets to the DPN framework (version 3): squeezing/pressure, vibrating, rough, and throbbing.

Existing DPN PRO measures

The literature search yielded 115 citations from articles that described the development and/or application of 13 unique DPN PRO measures. All measures were intended for adults. A total of 145 item concepts were derived from 13 PRO measures. Of these, 91 items (63%) assessed DPN symptoms. DPN item concepts assessed 12 of the 16 subdomains included in version 3 of the DPN framework. There were no or very few (≤2) item concepts that represented creepy/crawling, vibrating, and rough sensations, loss of proprioception, itchiness, throbbing, and loss of balance. We generated new item concepts for these facets. Other facets were represented by a large number or redundant item concepts. For example, 15 item concepts described numbness and 10 described tingling/pins & needles sensations. A total of 17 item concepts measured three pain experiences that had not been identified by clinicians and youth: aching, cramping, and tenderness. We added these facets to the DPN framework (version 4).

Item pool development

The final DPN conceptual framework (version 4) included 3 subdomains and 19 facets. Table 2 shows the DPN domains and facets and indicates whether they were derived from clinician interviews, youth interviews, or existing PRO measures. After eliminating redundancy, the facets were represented by 48 unique item concepts. We transformed the item concepts into item expressions with standardized recall periods and response categories.

Table 2.

DPN domains and facets, item sources, and number of items derived from adult measures, tested in cognitive interview, and retained.

Domain/Facet	Sourcesb	# of items
		Derived from adult measures	Testing in cognitive interview	Retained after cognitive interview
Paresthesia
Tingling/pins & needlesa	CL,CH,L	10	8	5
“Funny” feelingsa	CL,CH,L	1	3	2
Creeping/crawlinga	CL,CH	0	1	1
Itchinessa	CL,L	1	1	1
Squeezing/pressure/tight	CH,L	4	3	1
Vibrating	CH	0	3	2
Roughc	CH	0	2	0
Total paresthesia items		16	21	12
Pain
Burninga	CL,CH,L	7	4	2
Radiating/shootinga	CL,CH,L	9	4	2
Sharp/stabbinga	CL,CH,L	7	3	3
Tactile hyperaesthesia	CL,CH,L	8	2	1
Throbbing	CH,L	2	1	1
Achingc	L	9	2	0
Crampingc	L	5	1	0
Tenderc	L	3	1	0
Total pain items		50	18	9
Anesthesia
Numba	CL,CH,L	15	4	3
Loss of proprioceptiona,c	CL	0	1	0
Loss of balancec	CL,L	2	1	0
Immobility/weakness	CL,CH,L	8	3	1
Total anesthesia items		25	9	4
Total (all items)		91	48	25

^aConcepts included in initial conceptual framework.

^bConcepts confirmed or elicited from: CL = clinician semi-structured interviews; CH = child semi-structured interviews; L = systematic literature review of DPN symptom questionnaires (for adults).

^cFacets not represented in final item pool.

Cognitive interviews

Of the 13 cognitive interviewees, 7 (54%) were male and 12 (92%) were Caucasian. They ranged in age from 8 to 17 years (M = 11.8, SD = 4.3). Youths’ average duration of diabetes was 8.1 years (range: 1–17, SD = 4.8) and they had an average HbA1c of 8.4% (range 6.5–10.7%) at the time of the interview. Of the 48 items tested in cognitive interviews, 23 (48%) were removed because they were poorly understood, 22 (46%) were retained without revision, and 3 (6%) were revised, retested, and ultimately retained. Some items were eliminated because they contained specific terms or phrases that were poorly understood (e.g., tender, sensitive, dull pain). A few of these items were revised to include more illustrative language that described the target DPN concept in ways that children could better understand. For example, many children were unfamiliar with the phrase “sensitive to touch,” but most understood the concept when is was described as: “my feet or toes were bothered by something touching them lightly.” Other items were eliminated because they were understood in ways that were unrelated to DPN. For example, children interpreted “cramping” to mean abdominal pain or a sports-related injury. Several items that described DPN symptoms using extreme and metaphorical language were eliminated for poor understandability (e.g., feeling that one’s feet or toes were ‘on fire’ or ‘dead’). Lastly, cognitive interviews demonstrated that with a few exceptions, items that reflected youths’ distinctive and creative descriptions of DPN (e.g., “ants crawling on my feet or toes,” “electric shocks going through my feet or toes”) resonated with youth and were well-understood.

Item readability

On average, items were written at a third grade level (M = 3.0, SD = 1.1) and ranged in estimated grade level equivalent from 1.2 to 4.8. Grade-level equivalents for each item are shown in Table 3.

Table 3.

Child-report DPN symptom screener items and estimated grade-level equivalent.

Domain/Itema	Grade levelb
Paresthesia
I felt pins and needles in my feet or toes.	1.2
I felt like there was static in my feet or toes.	1.5
my feet or toes felt like they were asleep.	3.7
I had a prickly feeling in my feet or toes.	3.3
I had tingling feelings in my feet or toes.	3.7
my feet or toes felt weird.	3.5
my feet or toes felt funny.	3.5
I felt like ants were crawling on my feet or toes.	3.5
my feet or toes felt itchy.	3.5
my feet or toes felt tight.	3.5
I felt like my feet or toes were vibrating.	3.7
I felt buzzing inside my feet or toes.	3.7
Pain
my feet or toes felt like they were burning.	3.8
my feet or toes felt hot.	3.1
I felt like there were electric shocks going through my feet or toes.	4.3
I had shooting pain in my feet or toes.	3.3
I had stabbing pain in my feet or toes.	3.3
I had sharp pain in my feet or toes.	1.5
I felt like my feet or toes were being poked.	1.5
my feet or toes were bothered by something touching them lightly.	4.8
I had throbbing pain in my feet or toes.	1.5
Anesthesia
my feet or toes were numb.	1.5
I could not feel my feet or toes.	1.5
I could not tell the difference between hot and cold in my feet or toes.	4.4
I felt like I could not move my feet or toes.	1.2

^aAll items start with “In the past 4 weeks…;” Response categories for all items are Never – Rarely – Sometimes – Often – Always.

^bFlesch-Kincaid Grade Level equivalent.

Discussion

DPN is a major complication of diabetes, but there is little known about how DPN symptoms manifest in childhood and develop across the lifespan [2], [7]. Limitations in the tools and strategies used to identify and diagnose DPN symptoms pose barriers to research on the condition’s natural history and limits its identification and treatment in clinical contexts [7], [28], [29]. The purpose of this study was to generate a developmentally appropriate, conceptually grounded, and content valid youth-report measure of DPN symptoms for use in research and clinical care.

The youth-report DPN measure is grounded in a tripartite conceptual framework that characterizes youths’ paresthesia, pain, and anesthesia symptoms. The framework was informed by clinician and youth input and a systematic review of existing adult-report DPN symptom measures. Items that reflect DPN symptoms were iteratively refined based on cognitive interviews to maximize their understandability and meaningfulness to youth.

This study demonstrates that youth may have a unique DPN symptom profile characterized by amplified paresthesia symptoms and relatively fewer or less intense pain experiences. Youth were most likely to report symptoms of paresthesia, which they described as “funny” or “weird,” tingling, creeping/crawling, squeezing/pressure, and vibrating sensations in their feet and toes. In cognitive interviews, several youth critiqued items that described painful sensations (e.g., stinging) for being “too severe” and suggested item revisions that produced items that better characterize paresthesia symptoms. For example, one child described his sensory experience as “not as painful as a bee sting” and “more like a dull bee sting or a bee sting without the poison stingers.”

Existing adult-report DPN measures are likely to underestimate youths’ symptoms because they disproportionately target pain and contain fewer items that assess paresthesia. Of the 91 unique concepts derived from existing adult-report measures, 51% measure pain symptoms and only 8% assess paresthesia symptoms. Consistent with the DPN experiences described by youth in this study, 48% of the youth-report items retained after cognitive interviews assess paresthesia symptoms and a smaller proportion of items measure pain (36%).

In addition to revealing potential development differences in DPN symptoms, this study indicates that youth are capable of self-reporting DPN symptoms when items contain language that they understand and find relevant and meaningful. Youth in this study experienced DPN symptoms, but communicated about their symptoms using terms and phrases that differ from those used in adult-report DPN measures. In particular, youth semi-structured interviewees used distinct, creative language to describe their DPN symptoms. Their descriptions of the symptoms reflected themes of smallness, vibration, motion, static, and electricity.

Caution is needed, though, in balancing youth-friendly language with specific idiosyncratic descriptions. For example, items that described very specific experiences like a “mosquito buzzing” or a “bee stinging” were too narrowly focused and were not well-understood by many youth. In contrast, an item that included the more general descriptive term “buzzing,” was understood by youth to connote motion and vibration, which commonly characterized their altered sensations.

Limitations and future directions

This study has several limitations that should be addressed in future research. The semi-structured interview sample was predominantly Caucasian and male and all study participants received care at a dedicated diabetes clinic within a large children’s hospital. Further qualitative studies are needed to validate the DPN conceptual framework and to evaluate the items’ content validity among more socio-demographically diverse youth with a broader range of healthcare experiences. In its current version, the youth-report DPN measure assesses symptoms experienced in the feet or toes. We chose to focus on the lower extremities because prior research indicates that longer nerves in the lower limbs are affected by DPN first, followed by those in the upper arms [7]. Still, DPN symptoms experienced in the hands and fingers should not be ignored. Lastly, this study was conducted to generate a meaningful, relevant, and understandable youth-report measure of DPN symptoms. Our attention to the instrument’s content validity is a critical, but often over-looked component of best-practice PRO measure development procedures [17], [18]. Nonetheless, the measure should undergo a formal psychometric evaluation, including assessments of the instrument’s dimensionality, reliability, validity, and sensitivity to change in DPN symptoms [14], [16]. Psychometric analyses should result in the elimination of some items to reduce conceptual redundancy and enhance the measure’s clinical utility. Future work should also establish cut-points that can be used to identify children with clinically meaningful levels of DPN symptoms.

Conclusion

Self-report DPN symptom measures developed for adults have poor content validity among youth and if applied in pediatric populations, may lead to “false-negative” classifications of youth who are actually experiencing DPN symptoms. This study highlights the need for improved non-invasive approaches for assessing DPN symptoms among youth with diabetes. This study is the first to generate a content valid self-report measure of youth’s lived experiences with DPN that uses developmentally appropriate terminology. With further psychometric testing, the measure is expected to advance research on pediatric DPN by facilitating identification of children who may be eligible for DPN prevention or treatment trials and as a DPN symptom outcome measure. In clinical contexts, the tool may enhance clinicians’ capacity to identify the condition in childhood and gauge treatment effects. A youth-report DPN symptom measure should be considered for inclusion in annual pediatric DPN screening protocol. If adopted, clinicians’ compliance with DPN screening recommendations could be considered an indicator of healthcare system performance.