Intra- and Interrater Reliability and Concurrent Validity of a New Tool for Assessment of Breast Cancere—Related Lymphedema of the Upper Extremity

Abstract

Objective:

The goal of this study was to develop and assess intra- and interrater reliability and validity of a clinical evaluation tool for breast cancer–related lymphedema, for use in the context of outcome evaluation in clinical trials.

Design:

Blinded repeated measures observational study.

Setting:

Outpatient research laboratory.

Participants:

Breast cancer survivors with and without lymphedema (N = 71).

Interventions:

Not applicable.

Main Outcome Measure:

The assessment of intraclass correlation coefficients (ICCs) for the Breast Cancer–Related Lymphedema of the Upper Extremity (CLUE) standardized clinical evaluation tool.

Results:

Intrarater reliability for the CLUE tool was ICC: 0.88 (95% confidence interval [95% CI], 0.71-0.96). Interrater reliability for the CLUE tool was ICC: 0.90 (95% CI, 0.79-0.95). Concurrent validity of the CLUE score (Pearson r) was 0.79 with perometric interlimb difference and 0.53 with the Norman lymphedema overall score.

Conclusions:

The CLUE tool shows excellent inter- and intrarater reliability. The overall CLUE score for the upper extremity also shows moderately strong concurrent validity with objective and subjective measures. This newly developed clinical, physical assessment of upper extremity lymphedema provides standardization and a single score that accounts for multiple constructs. Next steps include evaluation of sensitivity to change, which would establish usefulness to evaluate intervention efficacy.

Breast cancer–related lymphedema (BCRL) is an abnormal accumulation of interstitial fluid containing proteins and cellular debris that can affect the upper extremity, breast, and/or torso. It has been reported to affect anywhere from 5% to 54% of breast cancer survivors, depending on the population, their breast cancer treatment, measurement methods, and length of follow-up.¹ BCRL is associated with reduced health-related quality of life.^2–5 If untreated, this condition may progress, leading to infections, hospitalizations, and increased health care costs.^6,7 This highlights the need for valid and reliable measures of BCRL, which are essential for evaluating prevention and treatment strategies.⁸

The Coverage and Analysis Group at Medicare and Medicaid Services requested a technology assessment from the Agency for Healthcare Research and Quality that included examination of the performance of diagnostic tests for secondary lymphedema. Results of this review, published in 2010, found that “there does not appear to be a gold standard to formally grade or measure the severity of lymphedema.”^9(p.3) It was noted that the lack of valid and reliable measurements hindered establishing the evidence base necessary for Medicare coverage of lymphedema treatments. Methods that have been used to measure upper extremity lymphedema include bioimpedance spectroscopy (BIS), water displacement, circumferential tape measurements, perometry, and self-report.^10,11

BIS can be viewed as a direct measure of edema. BIS measures the impedance to flow of an electrical current and is able to detect changes in extracellular fluid.^12,13 It has been shown to be effective in early detection of BCRL.¹⁴ However, BIS is costly and concerns exist for measuring late-stage lymphedema due to soft-tissue changes that may occur.¹⁰ Most of the research studies investigating BCRL use tools such as tape measure, volumeter, or perometer to assess limb size, which are indirect measurements of edema that fail to distinguish the interstitial space from other tissue compartments which are typically spared.¹⁵ Although tape measurement is a clinically feasible method to determine the size of the limb, there is no universal protocol for performing measurements.¹⁶ Measurements may be taken at anatomical landmarks or at fixed points along the limb.¹⁶ Although water displacement using a volumeter has been long used to assess limb size, there are a number of limitations. Skin breakdown is a contraindication for water displacement in patients with BCRL due to concerns for cross-contamination and infection.¹¹ Complete immersion of the arm may not be possible for many individuals due to the size of volumeters; therefore, it may not be able to assess BCRL of the upper arm.¹¹ In addition, many individuals have concerns with using a volumeter due to time constraints and costs associated with set up and measurement.¹¹ Finally, perometers are not readily available in clinical practice likely due to their size and cost of equipment.¹¹

A limitation of limb size measures (eg, water displacement, circumferential tape measurements, perometry) is the reliance on diverse criteria to define the presence of lymphedema. Criteria reported in the literature include >2 cm difference between arms, >10% volume difference between arms, >10% relative volume difference, >10% relative volume change, or >10% weight-adjusted volume change.^17,18 This is problematic because any limb size criteria could result in a missed lymphedema diagnosis because lymphedema may be present but fails to meet the defined threshold. In the absence of a criterion standard or agreed-upon diagnostic criterion, a thorough history and physical examination are essential for diagnosing and assessing BCRL.^19,20 A number of lymphedema staging or classification systems recognize that assessment of lymphedema is multidimensional (table 1).^21–24 Physical assessment of lymphedema may include visual inspection, examination for pitting or nonpitting edema, and skin examination for tissue texture changes.¹¹ These characteristics provide important information beyond the volumetrically based outcomes that are commonly used in research and may be clinically helpful for staging along with providing important clinical indicators of improvement or worsening of lymphedema.¹¹

Table 1.

Lymphedema staging or classification systems

	Stage
	0	1	2	3
CTCv4 lymphedema11		Tissue thickening or faint discoloration	Marked discoloration; leathery skin tissue; papillary formation; limiting instrumental ADL	Severe symptoms; limiting self-care ADL
CTCv4 edema11		5%-10% interlimb discrepancy in volume or circumference at point of greatest visible difference; swelling or obscuration of anatomic architecture on close inspection	>10%-30% interlimb discrepancy in volume or circumference at point of greatest visible difference; readily apparent obscuration of anatomic architecture; obliteration of skin folds; readily apparent deviation from normal anatomic contour; limiting instrumental ADL	>30% interlimb discrepancy in volume; gross deviation from normal anatomic contour; limiting self-care ADL
Foldi classification12,13	Latency stage; subclinical lymphedema; altered transport capacity and a reduced functional reserve	Reversible lymphedema; accumulation of high-protein edema, focal fibrosclerotic tissue alterations are present, swelling is soft and pitting, elevation reduces swelling	Spontaneously irreversible; extensive fibrosclerosis and fat deposition; tissue is hard, pitting is no longer present; elevation does not reduce swelling	Elephantiasis is characterized by extensive fibrosclerosis and fat deposition like stage 2; invalidism
International Society of Lymphology	Latent or subclinical condition; swelling is not yet evident; the individual reports changes in subjective symptoms.	Early accumulation of fluid, which decreases with limb elevation	Pitting is manifest; limb elevation seldom reduces swelling; Late in stage 2 tissue may or may not pit due to fat and fibrosis.	Lymphostatic elephantiasis; pitting can be absent; trophic skin changes such as acanthosis, further deposition of fat and fibrosis, and warty overgrowths

Abbreviations: ADL, activities of daily living; CTCv4, common toxicity criteria advers events version 4.

The Breast Cancer–related Lymphedema of the Upper Extremity (CLUE) is a standardized clinical, physical assessment tool for lymphedema. The CLUE tool was developed based on the Common Terminology Criteria for Adverse Events (CTCAE) v3.0, along with clinically relevant BCRL domains identified by lymphedema experts and reinforced through consensus processes among investigators. CLUE was designed to be usable by a number of health professionals (ie, surgeons, oncologists, radiation oncologists, nurses, physical therapists, occupational therapists) involved in the care of patients with history of breast cancer treatment. A standardized physical assessment for BCRL that is reliable and valid would advance the field by standardizing clinical discourse and offering a uniform single score criterion to estimate treatment efficacy in clinical trials. Therefore, the purpose of this study was to develop a new evaluation tool for BCRL and determine the intra- and interrater reliability and concurrent validity of the newly developed CLUE tool.

Methods

Study design

This was a blinded repeated measures observational study.

Participants

Women with a history of breast cancer between the age of 31 and 81 years were recruited from the Philadelphia area in 2014 through 2015 to participate in the study. Recruitment was accomplished by alerting certified lymphedema therapists in the area to the study and requesting referrals to the study. A small number of participants were self-referred into the study based on word-of-mouth referral. Women who were ineligible for other ongoing studies in our laboratory were informed of the study and invited to participate. All participants provided informed consent prior to participating in any study activities and the rights of all participants were protected. The University of Pennsylvania Institutional Review Board reviewed and approved all study activities. Women with and without a prior diagnosis of BCRL were eligible to participate. Exclusion criteria were as follows: (1) self-reported medical conditions that would negatively affect measurement validity or reliability (eg, heart failure or chronic kidney disease); (2) pregnancy; (3) unstable BCRL, defined as having needed medical treatment of a lymphedema exacerbation or cellulitis in the past 3 months; (4) currently receiving lymphedema treatment; and (5) history of alcohol or substance abuse within the past 12 months, including at-risk drinking. These exclusion criteria were established via self-report in a telephone screening to only include women with a low risk for fluctuation of lymphedema over a 21-day period.

Perometry

The interlimb volume difference was derived from perometer measures. The limb dimensions and volumes of bilateral upper extremities were assessed using the Optoelectronic Perometer.^a,25–27 This device uses infrared light beams to scan limb length and circumference to calculate the volume of the limb. The percent differences between limbs was then calculated using the volumes obtained from perometry. Our protocol for perometry measurements was standardized in that we assessed limb length (length at which to halt the measurement of limb volume on repeated measurements). We also specified limb and hand positioning (fisted hand). We used a motorized table for the perometer to allow adjusting the height of the perometer, which could ensure the extended arm parallel to the floor during the measurement. All measures were taken in a climate-controlled setting, between 11 am and 3 pm.

Norman lymphedema survey

Self-report of lymphedema symptoms was assessed using the Norman lymphedema survey,²⁸ which asks women “during the past 3 months, did your right and left [hands/lower arms/upper arms] seem to you to be different sizes from each other?”^(p.1195) If women answered yes, women were asked: “During the past 3 months, would you say that, on average, the difference in the size of your [hands/lower arms/upper arms] was 1) very slight, you are the only person who would notice this; 2) noticeable to people who know you well but not to strangers, or 3) very noticeable?” The original Norman lymphedema survey has been shown to have a specificity of 0.90 and sensitivity ranging from 0.86 to 0.92 for diagnosing women with BCRL.²⁸ The Norman lymphedema survey also includes a section that asks women to rate the frequency and severity of 14 lymphedema symptoms.

Breast Cancer-related Lymphedema of the Upper Extremity

CLUE involves assessing obscuration of anatomical architecture (eg, bony prominences, tendons), deviation from normal anatomical contour, change of soft-tissue texture, and the presence of pitting or nonpitting edema. These aspects of lymphedema evaluation were included because they are integral to clinical evaluations of BCRL. Measures included in the CLUE tool were completed in approximately 10 minutes, in the order noted below.

Obscuration of anatomical architecture

Assessment of obscuration of anatomical architecture involved clinicians visually inspecting for loss of symmetry and visibility of anatomical landmarks compared to the unaffected side. The following anatomical structures of the upper extremity were assessed: metacarpal phalangeal joints (knuckles), extensor tendons on the dorsal aspect of the hand, flexor tendons at the wrist, ulnar styloid process, and olecranon process. Obscuration of anatomical architecture for each structure was rated as either none, close inspection, or readily apparent. Operational definitions for rating each structure can be found in table 2.

Table 2.

Operational definitions for obscuration of anatomical architecture

Anatomical Structure	None	Close Inspection	Readily Apparent
MCP joint (2-4)	Symmetrical convexity of MCP joints, and symmetrical concavity between second and third, third and fourth, and fourth and fifth MCP joints	Loss of convexity or concavity but still able to visualize with full-digit flexion compared to unaffected side	Complete loss of convexity or concavity; unable to visualize MCP joints with full-digit flexion
Extensor tendons	Symmetrical appearance of extensor tendons at dorsal hand	Extensor tendons not as prominent compared to unaffected side with full-active digit extension and abduction	Unable to visualize extensor tendons
Flexor tendons	Symmetrical appearance of flexor tendons at ventral wrist	Flexor tendons not as prominent with active wrist and finger flexion and thumb opposition compared to unaffected side	Unable to visualize flexor tendons
Ulnar styloid	Symmetrical appearance of ulnar styloid	Ulnar styloid less visible compared to unaffected side; loss of convexity	Unable to visualize ulnar styloid
Olecranon process	Symmetrical appearance of olecranon process with elbow flexed	Olecranon process less prominent compared to unaffected side	Unable to visualize olecranon process

Abbreviation: MCP, metacarpal phalangeal.

Anatomical contour

Deviation from normal anatomical contour was assessed at the hand, wrist-forearm, and elbow-upper arm, and rated as none, readily apparent, or gross deviation from normal anatomical contour. Operation definitions can be found in table 3.

Table 3.

Operational definitions for deviation from normal anatomical architecture

Anatomical Region	Normal	Readily Apparent	Gross Deviation
Hand	Symmetrical appearance of hand, relatively flat dorsal hand with a smooth transition between the hand and digits	Hump on the dorsal aspect of the hand (raised<1cm)	Hump on the dorsal aspect of the hand (raised>1cm)
Wrist-forearm	Symmetrical appearance of wrist-forearm; forearm circumference should be larger than the wrist.	Decreased forearm-to-wrist circumference ratio causing a cylinder-shaped appearance (less than width of hand); increased forearm-to-wrist circumference ratio (forearm ≈ 2 × size of wrist)	Cylinder-shaped appearance (= />width of hand) Forearm: wrist circumference ratio (forearm >2× size of wrist)
Elbow-upper arm	Symmetrical appearance of elbow and upper arm	Increased posterior arm convexity (<5cm compared to unaffected side)	Increased posterior arm convexity (>5cm compared to unaffected side)

Tissue texture assessment

Soft-tissue texture, frequently altered in BCRL due to subdermal fibrosis, was assessed at the digits, dorsal hand, wrist, forearm, elbow, and upper arm, and rated as normal, spongy, firm, or hard.

Pitting or nonpitting edema assessment

Pitting edema was defined by a visible indentation that remains in the skin after applying gentle pressure.²⁹ Pressure was applied by the examiner’s finger or thumb for 5 seconds. The traditional assessment of pitting has used a 1+ to 4+ grading system.³⁰ However, an adaptation of this scoring system in which pit depth was visually estimated showed low-to-moderate agreement among examiners (kappa statistics ranged from −0.01 to 0.68).³¹ Therefore, we choose to rate edema as either none, nonpitting, or pitting. Edema assessment was performed at the following locations: digits, dorsal hand, wrist, forearm, elbow, and upper arm (table 4).

Table 4.

Assessment of pitting and tissue texture

Location of Pitting Assessment
Dorsal hand	Dorsal aspect of hand around midpoint of second and third metacarpals
Wrist	Dorsal and ventral aspects of wrist at midpoint between ulnar styloid process and radial styloid process
Forearm	Dorsal and ventral aspects of forearm approximately 5 cm distal to elbow
Elbow	Posteromedial to medial epicondyle and lateral epicondyle
Upper arm	Medial and lateral aspects of upper arm midway between axillary fold and elbow
Assessment of Tissue Texture
Normal	No abnormality of tissue texture; tissue felt similar to uninvolved side
Spongy	Tissue felt squishy or boggy
Firm	Tissue felt solid; not soft
Hard	Tissue felt very solid or firm

CLUE scoring

Regional subscores were derived for the hand, wrist or forearm, and elbow or upper arm region for obscuration of anatomical architecture, deviation from normal anatomical contour and edema or tissue texture. Description of the scoring system can be found in table 5. The total CLUE score was the sum of all regional subscores and ranges from 0 to 72 where 0 indicates no lymphedema. The following CLUE subscores were derived: edema subscore (sum of all edema scores for hand, wrist or forearm, elbow or upper arm), tissue texture subscore (sum of all tissue texture scores for the hand, wrist or forearm, elbow or upper arm), and anatomical architecture-anatomical contour subscore (sum of all anatomical architecture and anatomical contour scores for the hand, wrist or forearm, elbow or upper arm). The choice to combine anatomic architecture and contour was made based on the opinion of the lymphedema specialists that these constructs addressed complementary aspects combined with the observation that they contributed equally to improving the reliability and validity of the tool. Multiple scoring approaches were attempted. The one presented herein was optimal quantitatively (eg, maximized intraclass correlation coefficients [ICCs] for reliability and validity) and simultaneously viewed as defensible when considering lymph physiology or pathophysiology.

Table 5.

Scoring system for CLUE scale

Upper Extremity Subscores (Range)	Anatomical Regions
	Digits or Hand	Wrist or Forearm	Elbow or Upper Arm
Obscuration of anatomical architecture (0-18)	Maximum between knuckles and extensor tendons	Maximum between flexor tendons and ulnar styloid	Olecranon process
	None = 0	None = 0	None = 0
	Close inspection = 3	Close inspection = 3	Close inspection = 3
	Readily apparent = 6	Readily apparent = 6	Readily apparent = 6
Deviation from normal anatomical contour (0-18)	Maximum from dorsal hand	Maximum between wrist and forearm	Maximum between elbow and upper arm
	None = 0	None = 0	None = 0
	Readily apparent = 3	Readily apparent = 3	Readily apparent = 3
	Gross deviation = 6	Gross deviation = 6	Gross deviation = 6
Tissue texture (0-18)	Maximum between fingers and dorsum	Maximum between wrist and forearm	Maximum between elbow and upper arm
	Normal = 0	Normal = 0	Normal = 0
	Spongy = 2	Spongy = 2	Spongy = 2
	Firm = 4	Firm = 4	Firm = 4
	Hard = 6	Hard = 6	Hard = 6
Edema (0-18)	Maximum between fingers and dorsum	Maximum between wrist and forearm	Maximum between elbow and upper arm
	None = 0	None = 0	None = 0
	Nonpitting = 3	Nonpitting = 3	Nonpitting = 3
	Pitting = 6	Pitting = 6	Pitting = 6

Procedures

Flow of participants is described in fig 1. During assessment 1, all participants completed the Norman lymphedema survey, perometry, and underwent all of the physical assessments included in the CLUE tool.Participants were classified as having stage 0, 1, 2, or 3 lymphedema according to CTCAE (see table 1) by rater 1 and then returned within 21 days for assessment 2, performed by rater 1.

Fig 1.

Flow of participants.

Raters

Assessments were completed independently by 2 certified lymphedema therapists (rater 1 and rater 2), each of whom have over 10 years of clinical experience in evaluating and managing BCRL. Both raters hold a Lymphedema Association of North America certification. All assessments included within the CLUE tool are standard clinical evaluations used by certified lymphedema therapists. As such, there was no specific training provided to the raters to complete the CLUE evaluation beyond a short (5 min) discussion to ensure they understood how to record their findings on the form provided. Raters were blinded to each other’s assessments and participant’s responses on the Norman lymphedema survey.

Data analysis

Descriptive data were derived for perometry, Norman lymphedema scale, and CLUE scores and were reported within stage of lymphedema. ICCs were calculated to determine both intra- and interrater reliability³² for perometry scores (affected limb, unaffected limb, interlimb difference), overall CLUE total scores, and for all corresponding CLUE subscores. The 95% confidence intervals were determined for each ICC by taking 1000 bootstrap samples. The sampling unit is the eligible pair (2 measures by same rater for intrarater, 2 measures 2 raters for interrater), and all sampling was done with replacement. For the interrater reliability, we always compared rater 1 first measurement session to rater 2 measurements. Note that if a negative ICC occurs (mathematically possible with smaller numbers of pairs), this was dealt with by replacing the negative values with zero; the interpretation is the absence of variance between the pairs. This follows advice in the literature that although negative ICCs are mathematically possible, they are not theoretically possible.³³ Pearson product-moment correlation coefficients were used to quantify concurrent validity between CLUE (total and sub) scores with perometry interlimb difference and the Norman lymphedema survey (overall score, number of symptoms, symptom severity). For concurrent validity, we always used values from rater 1’s first measurement session. This study was ancillary to a larger trial; therefore, recruitment goals were based on feasibility. We aimed to recruit 24 women in each of the 3 lymphedema stages (Stages 0, 1, and 2-3). This sample size supported a detectable correlation coefficient of 0.65 or greater, at 85% power. We interpret ICC values according to published guidelines: <0.40 is poor, 0.40-0.59 is fair, 0.60-0.74 is good, and 0.75-1.0 is excellent.³⁴ Finally, we developed scatterplots to show graphically the correlations described statistically (supplemental fig S1, available online only at http://www.archives-pmr.org/) (see table 1). SAS version 9.4^b was used for all analyses.

Results

Description of study sample

A total of 71 women participated in the study (mean age ± SD: 58.7±9.0y, mean body mass index ± SD: 31.5±5.7kg/m²). Most of the participants had completed college. The racial make-up of the sample was as follows: 42.3% of participants were black and 57.7% were white. Most of the participants had received both chemotherapy and radiation treatment. Descriptive data can be found in table 6.

Table 6.

Description of participants (N = 71)

	Mean ± SD or n (%)
Variable	Overall (N = 71)	Stage 0 (n = 38)	Stage 1 (n = 19)	Stages 2 and 3 (n = 14)	P Value
Age (y)	58.7±9.0	59.7±9.1	55.5±8.8	60.1±8.6	.21*
Marital status					.55†
Never married	16 (24.2)	11 (28.9)	3 (15.8)	2 (14.3)
Married	29 (43.9)	15 (39.5)	9 (47.4)	5 (35.7)
Divorced/separated	18 (27.3)	9 (23.7)	6 (31.6)	3 (21.4)
Widowed	3 (4.5)	1 (2.6)	0 (0.0)	2 (14.3)
Unknown (did not answer)	5 (7.0)	2 (5.3)	1 (5.3)	2 (14.3)
No. of people living in the home‡	2.3±1.4	1.9±0.9	2.4±1.5	2.9±2.0	.06*
Race					.14†
Black	30 (42.3)	20 (52.6)	5 (26.3)	5 (35.7)
White	41 (57.7)	18 (47.4)	14 (73.7)	9 (64.3)
Education level					.06†
Less than high school	2 (2.9)	0 (0.0)	0 (0.0)	2 (14.3)
High school	6 (8.7)	3 (7.9)	1 (5.3)	2 (14.3)
Some college	21 (30.4)	14 (36.8)	3 (15.8)	4 (28.6)
College or more	40 (58.0)	21 (55.3)	13 (68.4)	6 (42.9)
Unknown (did not answer)	2 (2.9)	0 (0.0)	2 (10.5)	0 (0.0)
Weekly hours paid work outside the home§	24.3±21.8	24.6±24.4	21.2±18.6	26.9±18.5	.77*
BMI (kg/m2)	31.5±5.7	31.6±5.4	30.2±5.8	33.3±6.4	.31*
No. of nodes removed‖	9.2±8.6	6.4±6.0	11.0±9.8	14.3±10.4	.009*
Radiation (yes, n (%))¶	51 (72.9)	25 (67.6)	13 (68.4)	13 (92.9)	.18†
Radiation location	#	#	#	#	NA
Partial	4 (7.8)	3 (12.0)	0 (0.0)	1 (7.7)
Full	30 (58.8)	14 (56.0)	8 (61.5)	8 (61.5)
Axilla (armpit)	14 (27.5)	9 (36.0)	3 (23.1)	2 (15.4)
Neck	4 (7.8)	2 (8.0)	1 (7.7)	1 (7.7)
Unknown	14 (27.5)	5 (20.0)	5 (38.5)	4 (30.8)
Chemotherapy (yes, n (%))	55 (77.5)	28 (73.7)	14 (73.7)	13 (92.9)	.36†
History of physical therapy after breast cancer (yes, n(%))**	41 (62.1)	18 (48.6)	12 (75.0)	11 (84.6)	.042†

^*Compares mean differences across the groups using an F test (generalized linear model).

^†Compares the association between the 3 groups using Fisher exact test.

^‡There are 2 missing values for this variable (both stage 1).

^§There are 3 missing values for this variable (all stage 1).

^‖There are 4 missing values for this variable (3 stage 0 and 1 stages 2 and 3).

^¶There is 1 missing value for this variable (stage 0).

^#Can be multiple areas, sum to over 100%.

^**There are 5 missing values for this variable (1 stage 0, 3 stage 1, 1 stages 2 and 3).

Establishing the scoring system for the CLUE tool

Stemmer’s sign (the adherence of skin to the underlying soft tissue) and obliteration of skin folds were also assessed, but not included in the final CLUE score. Stemmers sign was only observed in 1 of 71 women. As such, it did not add meaningfully to differentiating between women with more versus less severe lymphedema. Further, obliteration of skinfolds was seen to be highly correlated with anatomic architecture. Inclusion of obliteration of skinfolds and anatomic architecture within the score would have resulted in a higher weighting of these constructs than made sense regarding face validity (eg, consideration of lymph physiology or pathophysiology). Finally, we also considered several approaches to scoring (0, 1, 2, 3 vs 0, 3, 6, 9) for each attributed evaluated. Decisions as to which scoring to use were made based after considering underlying physiology or pathophysiology and effect of sensitivity analyses to examine the effects of scoring on ICCs and concurrent validity evaluation.

Table 7 provides data regarding limb volume, Norman lymphedema survey values, and CLUE values for all 71 participants, within stages of previously diagnosed lymphedema severity, as defined by CTCAE criteria. On average, lower CLUE scores were found in participants with no lymphedema (mean ± SD, 1.53±6.77) compared to participants with stage 1 lymphedema (mean ± SD,14.11±10.30). Participants with stage 1 lymphedema had lower CLUE scores on average than participants with stage 2 and 3 lymphedema (mean ± SD, 30.07±17.59).

Table 7.

Description of participants’ volume discrepancies, lymphedema symptoms, and clinical lymphedema evaluation results (mean ± SD)

	Lymphedema Stage
Variable	Stage 0 (No Lymphedema) (n = 38)	Stage 1 (n = 19)	Stages 2 and 3 (n = 14)
Perometry
Affected (mL)	3070.9±575.0	2899.0±636.5	3468.2±955.5
Unaffected (mL)	3102.4±608.4	2740.0±572.3	3007.2±924.3
% interlimb difference	−0.7%±6.7%	+5.7%±7.0%	+19.4%±29.7%
Norman lymphedema survey
Overall score	1.79±2.36	3.68±3.06	4.93±3.81
No. of symptoms endorsed	2.79±2.76	3.95±2.68	5.86±4.17
Average severity for endorsed symptoms	1.16±0.88	1.49±0.73	1.56±1.12
Clinical evaluation (CLUE form)
Overall score
Upper extremity 72 (0-72 range)	1.53±6.77	14.11±10.30	30.07±17.59
Subscores
Obscuration of anatomical architecture + Anatomical contour change (0-36 range)	0.79±3.53	3.95±4.70	14.79±8.84
Tissue texture (0-18 range)	0.26±1.16	4.00±2.98	5.00±3.57
Edema (0-18 range)	0.47±2.15	6.16±3.67	10.29±5.97

Intrarater reliability

As shown in table 8, for intrarater reliability, ICCs were determined for interlimb difference assessed by the same therapist, with a range of 7-21 days (mean = 10) between assessments. The ICC for intrarater reliability using perometry was 0.93 (95% CI, 0.72-0.99). The ICC for CLUE was 0.88 (95% CI, 0.71-0.96). When divided into subcategories, it can be seen that the intrarater correlation coefficient varies from 0.76 for tissue texture to 0.90 for obscuration of anatomic architecture.

Table 8.

Intrarater reliability for lymphedema measures compared with 2 weeks between assessments

Variables	Measure 1 (B.A.S., Visit 1) (Mean ± SD)	Measure 2 (B.A.S., Visit 2) (Mean ± SD)	ICC (95% CI)
Perometry
Affected limb (n = 43)	3046.2±682.6	3150.6±666.8	0.85 (0.75-0.91)
Unaffected limb (n = 43)	2980.2±738.5	3086.4±655.9	0.84 (0.68-0.92)
Interlimb difference (n = 43)	+3.5%±14.5%	+2.9±13.7%	0.93 (0.72-0.99)
Clinical evaluation (CLUE form)
Overall score
Upper extremity 72 (0-72 range) (n = 49)	8.67±12.73	6.92±10.80	0.88 (0.71-0.96)
Subscores
Obscuration of anatomical architecture + Anatomical contour change (0-36 range) (n = 49)	3.61±6.74	2.63±5.73	0.90 (0.71-0.97)
Tissue texture (0-18 range) (n = 49)	1.76±2.30	1.59±2.34	0.76 (0.49-0.91)
Edema (0-18 range) (n = 49)	3.31±4.34	2.69±3.79	0.80 (0.62-0.92)

Interrater reliability

In table 9, for interrater reliability, ICCs for inter-limb difference assessed by 2 different therapists, on the same day, using perometry, was 0.98 (95% CI, 0.97-0.99). The ICC for CLUE, comparing 2 raters, was 0.90 (95% CI, 0.79-0.95). When divided into subcategories, the range of intrarater correlations extended from 0.77 for tissue texture to 0.89 for obscuration of anatomic architecture.

Table 9.

Interrater reliability for lymphedema measures comparing assessments by 2 certified lymphatic therapists on 1 day

Variables	Rater 1 (B.A.S., Visit 1) (Mean ± SD)	Rater 2 (J.C., Visit 1) (Mean ± SD)	ICC (95% CI)
Perometry
Affected limb (n = 47)	3068.1±689.3	3077.4±697.1	0.98 (0.95-0.99)
Unaffected limb (n = 47)	2972.6±735.2	2973.0±696.1	0.98 (0.95-0.99)
Interlimb difference (n = 47)	+4.8%±17.4%	+4.8±17.6%	0.98 (0.97-0.99)
Clinical evaluation (CLUE form)
Overall score
Upper extremity 72 (0-72 range) (n = 51)	8.35±13.31	9.96±15.46	0.90 (0.79-0.95)
Subscores
Obscuration of anatomical architecture + Anatomical contour change (0-36 range) (n = 52)	3.92±7.34	5.13±8.82	0.89 (0.74-0.95)
Tissue texture (0-18 range) (n = 51)	1.65±2.36	2.20±2.92	0.77 (0.57-0.87)
Edema (0-18 range) (n = 52)	3.35±4.74	3.17±4.80	0.87 (0.70-0.94)

Concurrent validity

Pearson product-moment correlation coefficient between the CLUE and perometry was 0.79. Pearson product-moment correlation coefficients between the CLUE and Norman survey—overall score, Norman survey—number of symptoms, and Norman survey—symptom severity were 0.53, 0.50, and 0.40, respectively. Pearson product-moment correlation coefficients for each of the CLUE subscores and perometry ranged from 0.71 to 0.79 (table 10). Correlations between each CLUE subscores and Norman survey—overall score, number of symptoms, and symptom severity—ranged from 0.34 to 0.54 (see table 10).

Table 10.

Concurrent validity of the CLUE form with perometry and Norman lymphedema survey

CLUE Score	Perometry Interlimb Difference (95% CI*) (N = 71)	Norman Survey Overall Score (95% CI*) (N = 71)†	Norman No. of Symptoms (95% CI*) (N = 71)	Norman Symptom Severity (95% CI*) (N = 71)
Clinical evaluation (CLUE form)
Overall score
Upper extremity 72 (0-72 range) (N = 71)	0.79 (0.67-0.86)	0.53 (0.34-0.68)	0.50 (0.30-0.65)	0.40 (0.18-0.58)
Subscores
Obscuration of anatomical architecture + Anatomical contour change (0-36 range) (N = 71)	0.79 (0.68-0.86)	0.50 (0.29-0.65)	0.51 (0.31-0.66)	0.39 (0.17-0.57)
Tissue texture (0-18 range) (N = 71)	0.71 (0.56-0.80)	0.51 (0.31-0.66)	0.42 (0.20-0.59)	0.34 (0.11-0.53)
Edema (0-18 range) (N = 71)	0.73 (0.59-0.82)	0.54 (0.34-0.68)	0.48 (0.27-0.64)	0.39 (0.17-0.57)

^*The 95% CIs are calculated by taking Fisher z transformation.

^†For upper extremity–related scores and subscores, only check against the applicable (first 3 of 4, 0-9).

Scatterplots

In supplemental online data (fig 2), we provide graphic representation of the intra- and interrater correlation coefficients, as well as concurrent validity.

Discussion

The Oncology Section of the American Physical Therapy Association’s clinical practice guideline on diagnostic methods for upper-quadrant cancer-related lymphedema was not able to provide strong recommendations for the clinical examination of lymphedema due to the lack of studies investigating reliability and diagnostic accuracy.¹¹ In addition, the choice of a single primary outcome for lymphedema clinical trials is challenging, given the multidimensionality of the condition. The development of a single score (eg, the CLUE tool) that embodies this multidimensionality could be of value to the field. The results of this study suggest that CLUE is a reliable and valid physical assessment. We deem CLUE to have excellent reliability and validity given published guidance regarding interpretation of ICC values,³⁴ as well as comparison to evaluation of the most broadly accepted clinical evaluation of BCRL, arm volume measurements using perometry. CLUE differs from other diagnostic methods of BCRL by potentially providing information on the type or stage of BCRL through the assessment of pitting and tissue texture. Majority of objective measures used in clinical practice provide information on limb size, which is only one characteristic of BCRL. In addition, CLUE does not rely on diagnostic criteria (ie, 200 mL increase) that may lead to a missed diagnosis of lymphedema. The concept behind the CLUE tool is relatively simple: Take the standard clinical physical assessment of upper extremity lymphedema and ensure that the examiner completes and records results for each element. Also, use a standardized scoring algorithm for all elements of the assessment. The key difference between the CLUE tool and a standard BCRL lymphedema assessment form used in the clinical setting is that the CLUE tool requests that each construct to be assessed and recorded. Further, scoring is standardized. In contrast, for standard BCRL assessments, it is common for therapists to only record notable findings. Further, scoring varies across clinics and sometimes between therapists. The habit in most clinical settings is to use an evaluation form to record notable findings, leaving questions as to whether other elements were evaluated.

The CLUE tool demonstrated excellent overall intrarater (ICC=0.88) and interrater reliability (ICC=0.90), which is consistent with other clinical measures of lymphedema such as water displacement and tape measurement. Water displacement and tape measurement have been found to have excellent intrarater and interrater reliability with reliability values ranging from 0.94 to 0.99.^35–38 It should be noted that the second measurement in studies reporting intrarater reliability for water displacement or tape measurement were either not described or performed on the same day as the first measurement while in our study women returned 7-21 days later.^35–38

Strong concurrent validity of the CLUE (r=0.79 with perometry) is in alignment with previously published studies that have shown excellent agreement between objective measures of lymphedema.^16,35,36,39 Czerniec et al⁴⁰ found an excellent correlation between BIS and total limb volume (r=0.89-0.92). The CLUE tool was moderately correlated (r=0.40-0.53) with the Norman lymphedema survey (overall score, number of lymphedema symptoms, severity of lymphedema symptoms). This is consistent with the findings of Czerniec who found that there was only a moderate agreement (r=0.65-0.66) between objective measures of lymphedema (perometry, circumferential measurements) and self-report.⁴⁰ It is possible that the constructs assessed by self-report, which would assess sensation, are distinct from the physical measures, which primarily assess swelling.

There was variability in the intra- and interrater reliability coefficients across specific subsections of the CLUE assessment. A similar pattern of variability was also noted in the assessment of concurrent validity. Generally speaking, assessment of obscuration of anatomic architecture and anatomic contour change was more reliable and valid than assessments of tissue texture or edema. This may reflect the extent to which the latter measures are more qualitative or subjective.

To our knowledge, CLUE is the first standardized clinical assessment of BCRL that provides a single score intended to reflect the multidimensionality of lymphedema. We have demonstrated the CLUE tool to be reliable and valid. The results of this study have the potential to directly affect rehabilitation through creation of a multidimensional single score outcome measure that can be used in lymphedema clinical trials. Characteristics of a high-quality outcome measure include clarity in the concept to be measured, reliability, validity, and minimized burden for administration.^41–43 We have addressed each of these elements. In particular, determining the reliability and validity of CLUE was the first step for providing clinicians with a physical assessment of lymphedema to be used in clinical trials. Future research is needed to address additional characteristics of a high-quality outcome measure, including ease in assigning meaning to the score, responsiveness to change, and diagnostic accuracy of CLUE.

Strengths and limitations

A primary strength of this study was the use of 2 certified lymphedema therapists who each had over a decade of experience with the clinical examination and treatment of BCRL. This is also a limitation, in that the interrater reliability is likely better between the 2 therapists who participated in this study that it would be in other settings, given their expertise and that they had worked together for more than a decade. Also, the intrarater estimates were based on 1 rater, a seasoned lymphedema therapist. In addition, we were unable to recruit 24 patients in each stage category, as planned. This is, perhaps, not surprising, given the prevalence of stage 0, stage 1, and stages 2 and3 lymphedema in the population of breast cancer survivors. We were able to accomplish our study goals with the recruited sample.

There could be concern as to whether the reduction of lymphedema severity to a single score is of value for the clinic or research. Similar concerns could be voiced for the single scores used to describe other conditions that manifest multidimensionally, for example, rheumatoid arthritis.⁴⁴ The goal in creating tools that produce a single score is to provide outcomes that can be used in the context of clinical trials to standardize measures of improvement in clinical trials across the multiple constructs that underlie complex conditions. This might be preferable to relying on a single construct of such conditions, as often happens. For example, in lymphedema, there is over reliance on the interlimb differences in clinical trials, despite broad acceptance on the part of researchers and clinicians that this does not reflect the overall experience of lymphedema. The CLUE tool is offered as a possible outcome measure for evaluation of efficacy of lymphedema treatments that includes multiple signs and symptoms of the condition. Given that a strength of the CLUE tool is its comprehensiveness, the possibility that it could be applied to patients with bilateral lymphedema is worth exploration.⁴⁵

Conclusion

Development of the CLUE tool provides clinicians and researchers with a standardized assessment for BCRL. Additional testing is needed to discern whether this new tool is capable of detecting subclinical and or new onset lymphedema and sensitivity to change.

List of abbreviations:

BCRL

breast cancer–related lymphedema

BIS

bioimpedance spectroscopy 95% CI 95% confidence interval

CLUE

Breast Cancer–related Lymphedema of the Upper Extremity

CTCAE

Common Terminology Criteria for Adverse Events

ICC

intraclass correlation coefficient