Case for Validated Instead of Standard Cut-Offs for SARC-CalF

Dear Editor

We read with interest the study by Bahat et. al. which concluded that addition of calf circumference item to SARC-F (SARC-CalF) improved the specificity and diagnostic accuracy but did not improve sensitivity in a community-dwelling Turkish older adult population sample (1). These findings run counter to the findings of earlier studies which consistently demonstrated that the SARC-CalF improved the sensitivity and overall diagnostic accuracy of SARC-F albeit with a slight tradeoff in specificity (2-5). We propose that these discrepant findings go beyond differences in sarcopenia prevalence to reflect, inter alia, differences in participant characteristics and variation in study methods which may have implication on the appropriate choice of cut-offs for SARC-CalF.

The study of Bahat et al. was conducted in the setting of a geriatrics outpatient clinic. Not surprisingly, the study population involved a higher age group (mean age: males=75.4 years, females=74.2 years) with a high proportion of co-morbidities such as diabetes mellitus (35.3%), hypertension (75.4%), heart disease (25.6%) and falls (43.5%). Interestingly, the prevalence of sarcopenia as denoted by international criteria was low compared with earlier studies (range: 0.7 – 10.4% vs 8.8–26.9%) (2, 3, 4). The lower prevalence can either reflect the good physical health of the study population despite the older age and co-morbidity, as evidenced by the good grip strength (mean: 32.7kg for males; 21.1kg for females) and gait speed (mean: 0.98m/s). Alternatively, it could reflect differences in study methods such as the modality used for determination of appendicular skeletal mass (ASM) and the choice of cut-off for calf circumference.

To ascertain how participant characteristics and study methods of Bahat et al. (2018) compare with other SARC-CalF cohort studies (2, 3, 4), we tabulated data regarding demographics, co-morbidities, study methods, sarcopenia prevalence, and diagnostic performance of SARC-F and SARC-CalF using the European Working Group on Sarcopenia in Older Adult (EWGSOP) and Asian Working Group for Sarcopenia (AWGS) criteria (6, 7) (Table 1). For SARC-F and SARC-CalF, we compared sensitivity, specificity and area under curve (AUC) using standard cut-offs, whereby a total score of >4 and >11 respectively indicates positive screening for sarcopenia. To further explore the impact of cutoffs on diagnostic performance of SARC-CalF, we performed receiver operating characteristic (ROC) curve analysis using data from the “Longitudinal Assessment of Biomarkers for characterization of early Sarcopenia and predicting frailty and functional decline in community-dwelling Asian older adults Study” (GERI-LABS) (4) to derive optimal cut-offs as determined by the Youden method. This study comprised fairly robust communitydwelling older adults who were cognitively intact and independent in instrumental activities of daily living (8).

Table 1.

Comparison between Bahat et al. (2018) and other SARC-CalF validation studies

	Bahat 2018	Lim 2018	Yang 2018	Barbosa-Silva 2016
Study Setting	Geriatric outpatient clinic Turkey N=207	Community cohort Singapore N=193	Community cohort Chendgu, China N=384	Community cohort Pelotas, Brazil N=179
Age, years	Male: 75.4 (5.9) Female: 74.2 (7.1)	67.9 (7.9)	71.5 (5.8)	60-69: 103 (57.5%) 70-79: 56 (31.3%) >80: 20 (11.2%)
Female gender, N(%)	140 (67.6)	133 (68.9)	224 (58.3)	110 (61.4)
Chronic disease, N(%)
Diabetes Mellitus	73 (35.3)	38 (19.7)	36 (9.4)	53 (29.6)
Hypertension	156 (75.4)	92 (47.7)	116 (30.2)	n.a.
Heart disease	53 (25.6)	3 (1.6)	36 (9.4)	58 (32.6)
Stroke	n.a.	4 (2.1)	36 (9.4)	n.a.
ASM measurement	BIA	DXA	BIA	DXA
CC cutoff	31cm (standard)	Male: 34cm	Male: 34cm	Male: 34cm
33cm (population)	Female: 33 cm	Female: 33cm	Female: 33cm
Sarcopenia prevalence, %	3.8 (EWGSOP)	24.9 (AWGS)	15.9 (AWGS)	8.8 (EWGSOP)
		26.9 (EWGSOP)	11.7 (EWGSOP)
SARC-F
Total score	1.3 (2.2)	0.5 (0.8)	0 (2.0)	n.a.
Sarcopenia (≥4), N(%)	39 (18.8)	4 (2.1)	47 (12.2)	n.a.
SARC-CalF
Total score	Male: 2.21 (3.96)	Male: 4.32 (4.96)	Male: 10.0 (11.0)	n.a.
	Female: 3.52 (4.62)	Female: 3.90 (4.85)	Female: 10.0 (2.0)
Sarcopenia (≥11), N(%)	21 (10.1)a / 5 (2.4)b	24 (12.4)c	99 (25.8)	n.a.
EWGSOP criteria
1. SARC-F (≥4)
Sensitivity, %	25.0	7.7	20.0	33.3
Specificity, %	81.4	100.0	95.6	84.2
AUC	0.522	0.522	0.81	0.592
2. SARC-CalF (≥11)
Sensitivity, %	25.0a / 25.0b	21.2	48.9	66.7
Specificity, %	90.0a / 98.0b	90.8	90.6	82.9
AUC	0.746a,c / 0.590b	0.677c	0.85c	0.736c
3. SARC-CalF (≥3)
Sensitivity, %	n.a.	63.5	n.a.	n.a.
Specificity, %	n.a.	73.0	n.a.	n.a.
AUC	n.a.	0.677c	n.a.	n.a.
AWGS criteria
1. SARC-F (≥4)
Sensitivity, %	n.a.	14.0	29.5	n.a.
Specificity, %	n.a.	92.0	98.1	n.a.
AUC	n.a.	0.522	0.89	n.a.
2. SARC-CalF (≥11)
Sensitivity, %	n.a.	22.9	60.7	n.a.
Specificity, %	n.a.	91.0	94.7	n.a.
AUC	n.a.	0.691c	0.92c	n.a.
3. SARC-CalF (≥3)
Sensitivity, %	n.a.	64.6	n.a.	n.a.
Specificity, %	n.a.	73.1	n.a.	n.a.
AUC	n.a.	0.691c	n.a.	n.a.

ASM: Appendicular Skeletal Mass; AUC: Area under receiver operating characteristic curve AWGS: Asian Working Group for Sarcopenia; BIA: Bioelectrical impedance analysis; CC: calf circumference; DXA: dual-energy X-ray absorptiometry; EWGSOP: European Working Group on Sarcopenia in Older People; Mean (SD) unless otherwise indicated; a. Using population cut-off for calf circumference (≥33cm) b. Using standard cut-off for calf circumference (≥31cm); c. P<0.01, comparison with AUC for SARC-F.

Unlike the Bahat et al. study, the other three studies involved population-based community samples in Pelotas (COCONUT), Chengdu and Singapore (GERILABS) respectively (2, 3, 4). These community cohorts were predominantly female, generally younger, and had lower co-morbidity; for instance, the prevalence of heart disease in the Chengdu and GERILABS studies were 9.4% and 2.1% respectively, compared with 25.6% in the Turkish study.

Key differences were noted in the study methods. The COCONUT and GERILABS studies used dual-energy X-ray absorptiometry for ASM measurement, as opposed to bioelectrical impedance analysis (BIA) in the Chengdu and Turkish studies. It is well-established that ASM estimates can vary depending on the choice of BIA instrument and reference population (9). Bahat et al. did not provide details of the BIA instrument model or the cut-offs employed. In addition, the gender-specific cut-offs for calf circumference (male: 34cm; female 33cm) employed in the cohort studies were higher than the standard (31cm) and population (33cm) cut-offs of the Turkish study, which were not gender-adjusted. Earlier studies which examined calf circumference as a screening instrument for ASM demonstrated gender differences in optimal cut-offs, being higher in males compared with females (10, 11). Thus, the choice of lower cut-offs for calf circumference without gender adjustment in the Bahat et al. study is likely to have resulted in the paradoxically lower prevalence of positive screening using standard cut-offs (≥11) of SARC-CalF (2.4-10.1% vs 12.4-25.8% in cohort studies) despite the much higher prevalence of positive cases using standard cut-offs (≥4) of SARC-F (18.8% vs 2.1-12.2% in cohort studies). Of note, in the Bahat et al. study, SARC-CalF33 which utilizes the higher population-derived national cut-off (33cm) had better diagnostic accuracy compared with SARC-CalF31 which uses standard cut-off (31cm), alluding to the possibility that the diagnostic performance of SARC-CalF would be improved if higher gender-specific cut-offs were employed. Looking at diagnostic performance, the SARC-F had low sensitivity and high specificity in all four studies, with AUC lowest in the Bahat et al. and GERILABS studies. An interesting dichotomy emerged when we examine the diagnostic performance of SARC-CalF using standard cut-offs (≥11); unlike the Chengdu and COCONUT studies, the SARC-CalF had improved specificity and diagnostic accuracy but not sensitivity compared with SARC-F in the Bahat et al. and GERILABS studies. AUCs were also comparatively lower in the latter studies (0.590 – 0.746). In contrast, using ROC-derived cut-off (>3) in the GERILABS study, the sensitivity of SARC-CalF improved albeit with a slight tradeoff in specificity, which is similar to the results reported in the Chengdu and COCONUT studies.

The results of our study suggest that similar to the SARC-F, contextual factors of patient characteristics and study methods can modulate the diagnostic performance of the SARC-CalF (8). In particular, in physically more robust populations where the expected prevalence of sarcopenia would be lower, it is important to use gender-specific population cut-offs for calf circumference so as not to affect the diagnostic performance of SARC-CalF. It is also timely to review cut-offs for the SARCCalF. The basis for the standard cut-off (≥11), as proposed in the COCONUT study (2), was to provide equal weightage to SARC-F (surrogate of muscle function, 10 points) and calf circumference (surrogate of muscle mass, 10 points) in the 20-point SARC-CalF scale. As demonstrated, the cutoff of SARC-CalF is not invariant and should be locally validated using ROC-adjusted cutoffs. The optimal cut-off (>3) in the GERILABS study represents a shift away from the equal emphasis on muscle mass and muscle function, and is consistent with the current emphasis on muscle function in lieu of muscle mass as represented in the recently released EWGSOP2 guidelines (12). Taken together, we affirm the premise of the original validation studies that adding calf circumference to SARC-F (SARC-CalF) improves diagnostic performance by increasing sensitivity (and not specificity), and that SARC-CalF cut-offs should be locally validated if adopted in contexts which are different from those reported in the original validation studies (2, 3).