Schizophrenia Quality of Life Scale and Schizophrenia Quality of Life Scale Revision 4: A Systematic Review of Measurement Properties

Abstract

Background and Hypothesis

Schizophrenia is a severe mental disorder that has a significant impact on quality of life (QOL). Measuring QOL can offer insights into treatment efficacy and areas of intervention, highlighting the importance of valid tools assessing QOL in people with schizophrenia.

Study Design

We employed the COSMIN systematic review guideline to assess the psychometric properties of the schizophrenia quality of life scale (SQLS) and its 4th revision, the schizophrenia quality of life scale revision 4 (SQLS-R4), as patient-reported outcome measures (PROMs).

Study Results

The search yielded 455 papers, 16 were included, 7 for the SQLS and 9 for the SQLS-R4. Both scales demonstrated good results in risk of bias assessment for internal consistency and convergent validity, the SQLS-R4 additionally for known-groups validity. For the SQLS, PROM development, structural validity, and reliability were suboptimal. The SQLS-R4 showed suboptimality regarding structural validity and reliability and inadequacy for cross-cultural validity and responsiveness. The updated criteria for good measurement properties indicated good convergent validity for the SQLS and good internal consistency, reliability, and convergent validity for the SQLS-R4. The SQLS showed suboptimal results for reliability and known-groups validity, while the SQLS-R4 demonstrated suboptimality in structural validity and known-groups validity. The SQLS had indeterminate structural validity and internal consistency; the SQLS-R4 showed indeterminate responsiveness, and insufficient cross-cultural validity. When using the updated GRADE approach of the COSMIN group, both scales received a very low grade.

Conclusions

The SQLS and SQLS-R4 hold the potential for recommendation in rating QOL. Identified weaknesses necessitate further validations.

Introduction

Schizophrenia as a chronic and severe mental disorder can have a significant impact on a person’s quality of life (QOL).¹ Measuring QoL in patients with schizophrenia can offer clinicians and researchers valuable insights into the disorder’s impact beyond clinical symptomatology, facilitating the identification of intervention areas, treatment efficacy assessment, and understanding of QoL determinants.²

The Schizophrenia Quality of Life Scale (SQLS) is a 30-item self-reported measure of schizophrenia-related QOL with 3 subscales (psychosocial; motivation and energy; symptoms; and side effects), each with a range of 0 (best possible health state) to 100 (worst possible health state).³ The SQLS scale has undergone further development and resulted in the Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4), comprising 33 items with 2 subscales (psychosocial feelings; cognition, and vitality).⁴ Both scales are assessed on a 5-point Likert scale and take 5–10 minutes to complete. They are intended to be used in clinical trials and clinical intervention evaluation.^3,4

To this point, there has not been an attempt to systematically review the psychometric properties of the SQLS and SQLS-4. Therefore, the current review aims to scrutinize the measurement properties of the SQLS and SQLS-R4 using the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines and Risk of Bias Checklist.^5–7

Methods

The COSMIN methodology, described in detail by Prinsen et al.,⁶ Mokkink et al,⁵ and Terwee et al,⁷ was developed to provide a standardized and transparent approach to evaluate the quality of measurement instruments in healthcare, mainly including patient-reported outcome measures (PROMs), which the SQLS und SQLS-R4 represent. The evaluation can help healthcare professionals and researchers select the most appropriate instruments for their needs. In brief, the methodology comprises 3 steps: The first part is to conduct a literature search. The main part is the evaluation of the individual measurement properties: (1) Risk of Bias assessment, (2) “updated criteria,” and (3) modified GRADE approach. Lastly, recommendations are formulated. As for a detailed description of the steps see the COSMIN checklist and user manual^5–7 and papers of Wehr et al⁸ and Weigel et al.⁹ Similarities in wording may have occurred as the underlying methodology is the same.

Protocol Registration

The study protocol has been registered with OSF: osf.io/r5bx9.

Scale Selection

In order to select appropriate scales for our systematic review, we relied on the investigation by van Krugten et al,¹⁰ who examined the extent to which existing QOL scales cover the dimensions identified by Connell et al.^11,12 Connell et al.^11,12 identified several domains that are considered important for individuals with mental health issues in the concept of QOL. Apart from the SQLS and SQLS-R4, we chose other scales covering these dimensions at least partially (Heinrichs-Carpenter QoL Scale, SQoL-41/-18, Lehman Quality of Life Interview, Lancashire Quality of Life Profile, Manchester Short Assessment of Quality of Life, Wisconsin Quality of Life Index, and Quality of Life in Schizophrenia questionnaire). The examination of the remaining measurement properties of these scales will be reported in separate papers (manuscript in preparation).

Literature Search

We searched PubMed and EMBASE for English-language articles published between January 2000 and November 2022. Our search terms included “Schizophrenia Quality of Life Scale,” “SQLS,” and “S-QLS” in the “Title/Abstract” fields. Two reviewers (MW, MZ) independently assessed the titles and abstracts of the articles to determine whether original data were reported on a minimum of one of the psychometric measurement properties described in the COSMIN Risk of Bias checklist. Disagreement was resolved until a consensus was reached, and if necessary, resorting to third reviewers (SL, SW, and LW).

Assessment of the Measurement Properties

The evaluation of the measurement properties was conducted independently by 2 reviewers (MZ and MW) for all 3 steps. Disagreement was resolved by discussion between the reviewers until a consensus was reached. If no consensus could be attained, third reviewers of expert-level (SL, SW, and LW) were resorted to.

Evaluating the Risk of Bias

The Risk of Bias (RoB) checklist^5–7 aims to assess the methodological quality of validation studies of an instrument. The rating system includes 4 levels of quality: “very good,” “adequate,” “doubtful,” and “inadequate.” Additionally, there is the option of a “not applicable” rating. The checklist operates on the principle that the “worst score counts” (for methodological details see the COSMIN Risk of Bias Checklist^5–7). The standards for good methodological quality are arranged by criteria in 10 boxes: PROM development, content validity, structural validity, internal consistency, cross-cultural validity/measurement invariance, reliability, measurement error, criterion validity, hypotheses testing for construct validity, and responsiveness. Boxes 1 and 2 of the Risk of Bias checklist, “PROM development” and “content validity,” which assess the content validity of an instrument, are only relevant to the initial papers that document the development of the scales. Using the COSMIN Risk of Bias Checklist, measurement properties relevant to each study were rated, followed by an overall assessment of the methodological quality of each measurement property.

Evaluating the Updated Criteria for Good Measurement Properties

In the next step, the COSMIN methodology intends to evaluate the results of each individual validation study against the “updated criteria for good measurement properties”.^5–7 The evaluation of measurement properties involves giving an overall score to each property, which is subsequently categorized as either “sufficient” (+), “insufficient” (−), or “indeterminate” (?). This rating depends on various factors such as the study design, methods, and outcomes of the validation study. The rating of “indeterminate” is assigned when the necessary information required to form a conclusive result is either missing or unclearly defined.

Criteria exist for the following 8 measurement properties: structural validity (namely, the instrument validly reflects the construct being measured by using classical test theory [CTT] or IRT/Rasch), internal consistency (determined by Cronbach’s alpha with the prerequisite of sufficient structural validity), reliability (including test-retest reliability, measured by intraclass correlation coefficient [ICC] or weighted Kappa), measurement error (defining smallest detectable change [SDC] or limits of agreement [LoA] against minimal important change [MIC]), hypothesis testing for construct validity (ie, evaluating if precise hypotheses were formulated and verified), cross-cultural validity/measurement invariance (measurement invariance between groups on a translated or culturally adapted version of the PROM), criterion validity (correlation with a gold standard) and responsiveness (detecting change over time in the construct measured).

Grading the Quality of Evidence

We used the modified GRADE approach by the COSMIN group^5–7 to assess the quality of evidence of all studies included in the analysis, which determines the level of the results’ trustworthiness. When evaluating the measurement properties of PROMs in systematic reviews, 4 factors are considered: (1) the risk of bias in the studies (ie, the methodological quality of the studies), (2) inconsistency (unexplained differences in results across studies), (3) imprecision caused by the sample size of available studies, and (4) indirectness (evidence comes from populations that differ from the population of interest in the review). The GRADE approach is employed to reduce the quality of evidence when there are apprehensions concerning the quality of the studies (see table 1). Initially, the overall result is deemed high quality. If the criteria are regarded as insufficient, the quality of evidence may be downgraded by up to 3 levels per factor.

Table 1.

Definitions of GRADE Following COSMIN^5–7

Quality of Evidence	Lower If
High = We are very confident that the true measurement property lies close to that of the estimate of the measurement property	Risk of bias -1 Serious -2 Very serious -3 Extremely serious Inconsistency -1 Serious -2 Very serious Imprecision (n = sample size) -1 total n = 50–100 -2 total n < 50 Indirectness -1 Serious -2 Very serious
Moderate = We are moderately confident in the measurement property estimate: the true measurement property is likely to be close to the estimate of the measurement property, but there is a possibility that it is substantially different
Low = Our confidence in the measurement property estimate is limited: the true measurement property may be substantially different from the estimate of the measurement property
Very low = We have very little confidence in the measurement property estimate: the true measurement property is likely to be substantially different from the estimate of the measurement property

1. Risk of Bias^5–7

In applying the GRADE approach’s Risk of Bias (RoB) assessment, we evaluated each RoB box using the criteria outlined in table 2. Using the COSMIN “worst score counts” approach, a single box with an extremely serious risk led to a maximum 3-point downgrade. Downgrading of confidence in evidence-based on RoB items only occurred when the item had a definite result (obtained a “+” or “−” rating, rather than a “?”) in Step 2: Updated criteria of good measurement.

Table 2.

GRADE Downgrading Criteria for Risk of Bias^5–7

Risk of Bias	Downgrading for Risk of Bias
No	There are multiple studies of at least adequate quality, or there is 1 study of very good quality available
Serious	There are multiple studies of doubtful quality available, or there is only 1 study of adequate quality
Very serious	There are multiple studies of inadequate quality, or there is only 1 study of doubtful quality available
Extremely serious	There is only 1 study of inadequate quality available

1. Inconsistency

Given our qualitative approach without quantitative pooling (meta-analysis), we established criteria for downgrading as follows: if little to no inconsistency was identified with a valid explanation, we did not downgrade the scale. However, we downgraded the scale by 1 point (serious) if little inconsistency was found with no explanation or moderate to high inconsistency was detected with a valid explanation. If moderate to high inconsistency was found without having a reasonable explanation, we downgraded the scale by 2 points (very serious).

1. Imprecision

Imprecision assesses the total sample size of all studies analyzed. A sample size ranging from n = 50–100 led to a one-point downgrade, and a sample size less than n = 50 resulted in a 2-point downgrade (following COSMIN^5–7)

1. Indirectness

The scale was downgraded for indirectness if patients in the studies were not representative of the population of interest, however, the sample groups for this review included only schizophrenia or schizophrenia spectrum disorder patients. No downgrade occurred if the comparison group was of a different disease or healthy controls.

Results

The PubMed search retrieved 322 articles, of which 17 were identified based on title and abstract, and 14 were included in the systematic review. Additionally, the EMBASE search yielded 133 papers, with 2 more articles included (see table 3). Figure 1 shows that 7 articles concerned the validation of the SQLS scale, while the remaining 9 articles pertained to the SQLS-R4 scale. For exclusion criteria see figure 1.

Table 3.

Characteristics of Included Study Population^3,4,13–26

	Language	Country	Population	Mean (SD) Age	Gender Female (%)	Number of Patients
SQLS
Wilkinson et al3	English	GB	In- and outpatients with schizophrenia	Item reduction study: 43 (11.3); Construct validity study: 40 (11.9)	Item reduction study: 35	Item generation study: 20; Pilot study: 20; Item reduction study: 161; Construct validity study: 78
Kaneda et al13	Japanese	Japan	Inpatients with schizophrenia	46.6 (11.3)	31	55
Luo et al18	English, Chinese	Singapore	Outpatients with schizophrenia	Total: 37.8 (10.2); English: 37.4 (10.1); Chinese: 38.8 (10.4)	Total: 52.0; English: 53.6; Chinese: 48.4	Total: 202 (140 English-speaking); Reliability: 32 English-speaking, 17 Chinese-speaking
Wang et al19	Chinese	China	Inpatients with schizophrenia	NR	53.9	128
Wang et al20	NR	China	Patients with schizophrenia	NR	NR	30
Domenech et al17	NR	AR, BR, CA, FR, DE, IT, ES, GB	Outpatients with schizophrenia	42.1 (11.50)	29.44	1374
Franklin et al., 201926	English	GB	Outpatients with schizophrenia	42 (NR)	22	304
SQLS-R4
Martin and Allan4	English	GB	In- and outpatients with schizophrenia	40.70 (11.43)	26	100
Kuo et al24	Chinese	Taiwan	In- and outpatients with schizophrenia	36.9 (8.82)	25	100
Kuo et al14	Chinese	Taiwan	In- and outpatients with schizophrenia	36.9 (8.82)	25	100
Taha et al25	Malay, English	Malaysia	Outpatients with schizophrenia	37.73 (10.13)	50.5	222
Chou et al22	Taiwanese	Taiwan	Inpatients with schizophrenia	Patients: 37.9 (7.36); Symptom-remission: 38.2 (8.2); Non-symptom-remission: 38.1 (7.7)	Total: 29.3; Symptom-remission: 43.3; Non-symptom-remission: 40	Total: 41; Known-groups validity: 30 (Symptom-remission), 30 (Non-Symptom-remission)
Rofail et al15	NR	BR, FR, DE, HU, JP, MX, PL, RU, US	NR	39.9 (10.1)	35.9	312
Isjanovski et al23	Macedonian	Macedonia	Outpatients with schizophrenia	47.4 (8.9)	31.1	61
Su et al16	Chinese	Taiwan	Inpatients with schizophrenia	49.2 (7.8)	34	100
Arraras et al21	Spanish	Spain	Outpatients with schizophrenia and schizoaffective disorder	39.6 (11.5)	47.6	168

Note: NR, not reported.

Fig. 1.

Flowchart of Study Selection Process (PubMed/EMBASE).

Risk of Bias of the SQLS and SQLS-R4

See table 4.

Table 4.

COSMIN Risk of Bias and Updated Criteria Rating^5–7

Measurement Property (No. of Studies Assessing Measurement Property)	COSMIN Risk of Bias				Updated Criteria of Good Measurement
	Very Good	Adequate	Doubtful	Inadequate	+	−	?
SQLS
PROM development (n = 1)	0	0	1	0
Content validity (n = 0)
Structural validity (n = 2)	0	0	0	2	0	0	2
Internal consistency (n = 4)	4	0	0	0	0	0	4
Reliability (n = 1)	0	0	1	0	0	1	0
Hypotheses testing for construct validity  - convergent validity—totala (n = 1)	1	0	0	0	3	1	0
Hypotheses testing for construct validity  - convergent validity—same subscoresa (n = 3)	0	3	0	0	7	1	0
Hypotheses testing for construct validity  - known-groups validity (n = 1)	1	0	0	0	0	1	0
SQLS-R4
PROM development (n = 0)
Content validity (n = 0)
Structural validity (n = 4)	1	0	0	3	1	2	1
Internal consistency (n = 7)	7	0	0	0	1	0	6
Cross-cultural validity (n = 1)	0	0	0	1	0	1	0
Reliability (n = 3)	0	1	2	0	3	0	0
Hypotheses testing for construct validity  - convergent validity—totala (n = 3)	3	0	0	0	4	1	0
Hypotheses testing for construct validity  - convergent validity—same subscoresa (n = 5)	3	2	0	0	12	0	0
Hypotheses testing for construct validity  - known-groups validity (n = 2)	2	0	0	0	1	1	0
Responsiveness (n = 1)	0	0	0	1	0	0	1

^aIn convergent validity, n represents studies examining SQLS/SQLS-R4 convergent validity, equating to the number of RoB ratings. The count of confirmed hypotheses is detailed under the “Updated Criteria of Good Measurement” section and deviates from the RoB rating count.

Content Validity

PROM Development

The box PROM development relies on the satisfaction of general design prerequisites and evaluating comprehensibility and comprehensiveness during pilot testing. However, PROM development is not regarded as a measurement property.

One article elucidated the development of the SQLS and received a “doubtful” rating regarding PROM development³ as the developers did not apply to the rather strict prerequisites formulated by the COSMIN manual to receive a “very good” rating.^5–7 Interviews with schizophrenia patients generated items refined into a piloted and shortened questionnaire, resulting in the SQLS.

No PROM development paper has been published for the SQLS-R4.

Content Validity

Content validity refers to whether a PROM’s content accurately reflects the construct being measured. It assesses relevance, comprehensibility, and comprehensiveness for both patients and experts.

None of the studies in our analysis reported on the SQLS or SQLS-R4’s content validity.

Internal Structure

Structural Validity

Structural validity is relevant for scales based on a reflective model, such as the SQLS and SQLS-R4, as it measures the degree to which scores accurately reflect the evaluated construct. In these models, items are expected to correlate with each other, and the underlying construct being measured, in this case, the QOL in individuals with schizophrenia.

The structural validity of the SQLS was examined and reported on in 2^3,13 of the 7 included studies. Both studies were found to have an “inadequate” rating due to the small sample size utilized in their factor analyses.

Regarding the SQLS-R4, 4^4,14–16 of the 9 included studies reported on structural validity. Three studies obtained an “inadequate” rating^4,14,16 resulting from a low sample size and 1 was deemed “very good,”¹⁵ attributed to confirmatory factor analysis.

Internal Consistency

Internal consistency was evaluated in 6 of the studies included in the analysis of SQLS.^3,13,17–20 Five studies^3,13,18–20 obtained a “very good” rating for their calculation of Cronbach’s alpha (CA) for each subscale. The 3 subscales of SQLS—“psychosocial,” “motivation and energy,” and “symptoms and side effects”—were initially identified via factor analysis in the original paper.³ One study¹⁷ was rated “inadequate” for only calculating CA for the total score.

Seven of the studies included on SQLS-R4^4,15,21–25 reported on internal consistency and received a “very good” rating for computing CA for each individual subscale. Two^4,15 of the articles investigating the internal consistency of the SQLS-R4 also conducted a factor analysis of the SQLS-R4. In both cases, a 2-factor solution emerged—“psychosocial feelings,” and “cognition and vitality.” CA was calculated for these factors in all 7 studies.

Cross-cultural Validity

The Updated version of the COSMIN Risk of Bias Checklist^5–7 requires direct comparison data from at least 2 distinct groups, such as different language groups, to assess the cross-cultural validity of an instrument’s adaptation. However, the quality of translation is not assessed.

One study¹⁵ analyzed the cross-cultural validity of the SQLS-R4 but was evaluated as “inadequate” due to insufficient sample size within each group.

Remaining Measurement Properties

Reliability

Reliability assesses the consistency of scores obtained from patients who have not changed over time in repeated measurements, such as test-retest reliability.

One study¹⁸ evaluated test-retest reliability for the SQLS and received a “doubtful” reporting quality due to a test-retest time interval ranging between 13 and 112 days, whereas the COSMIN methodology considers a time interval of about 2 weeks appropriate.^5–7

Three studies^15,21,24 investigated the test-retest reliability of the SQLS-R4. Among these, 1 study²⁴ received an “adequate” rating, whereas 2 studies^15,21 were rated as “doubtful” due to an 8-week interval and the lack of ICC calculation, respectively.

Hypotheses Testing for Construct Validity

The construct validity of a PROM refers to how well its scores align with theoretical expectations regarding relationships with other measures and differences between relevant groups, assuming it is a valid measure of the intended construct. The “Hypotheses Testing for Construct Validity” section of the Updated COSMIN Risk of Bias Checklist^5–7 is divided into 2 parts. The first part, convergent validity, measures the reporting quality of studies comparing the PROM with other outcome measurement instruments. The second part measures known-groups validity, which evaluates differences in scores among anticipated subgroups.

Convergent Validity.

No uniform definition of QOL exists.¹ (Sub-)scales of Health-related QOL (HRQoL) measuring constructs also covered by the SQLS and SQLS-R4 were considered similar. Constructs like anxiety and depression were deemed conceptually related. Constructs like self-esteem were considered dissimilar, but related constructs.

Four papers^3,13,17,18 reported on convergent validity of the SQLS, 1 received a “very good” rating,¹⁷ and 3^3,13,18 an “adequate” rating.

Six articles documented the convergent validity of the SQLS-R4,^4,21–25 4 received a “very good”^4,21,22,25 reporting quality rating, and 2 an “adequate.”^23,24

Known-Groups Validity.

One study²⁶ reported on known-groups validity of the SQLS receiving a “very good” reporting quality rating. The article compared patients scoring differently on the subscales of various instruments (PANSS, CGI, NSA-4, and HoNOS-PbR) regarding their scoring on the SQLS psychosocial, motivation, and symptoms subscales.

Two articles^21,22 reported on known-groups validity of the SQLS-R4 receiving a “very good” rating. Chou et al,²² compared healthy people, and patients with schizophrenia symptom-remission and non-symptom-remission, respectively, regarding their SQLS-R4 score. They hypothesized that the score of the healthy people would be highest, followed by patients with schizophrenia with symptom-remission and finally, by non-symptom-remission. Arraras et al,²¹ compared patients of different groups regarding their age, gender, employment status, time since diagnosis, number of acute episodes, negative symptoms, and depression.

Responsiveness

Responsiveness evaluates an instrument’s capacity to identify changes over time in the construct being measured. The criteria for responsiveness are comparable to those for construct validity.

One article²⁴ reported on the responsiveness of the SQLS-R4 and received an “inadequate” rating as they used Guyatt’s responsiveness ratio which is considered an unsuitable measure according to the COSMIN manual.^5–7

Updated Criteria for Good Measurement Properties SQLS and SQLS-R4

Structural Validity.

Two studies^3,13 evaluated the structural validity of the SQLS using principal component analysis³ and exploratory factor analysis.¹³ Both received an indeterminate “?” rating as they did not report the results of CTT, Rasch, or IRT analyses which is required by the COSMIN methodology.^5–7 Hence, no conclusion can be made regarding the structural validity of the SQLS. Wilkinson et al,³ found a 3-factor structure consisting of “psychosocial,” “motivation and energy,” and “symptoms and side effects.” Kaneda et al,¹³ confirmed the same structure but identified some items that did not load onto expected factors, possibly due to cultural differences or the smaller sample size of the Japanese study.

Four studies evaluated the structural validity of the SQLS-R4 scale,^4,14–16 with 1 study⁴ receiving a satisfactory rating of “+” for meeting the criteria for structural validity. This study found that the best model fit was achieved by a 2-correlated-factors solution consisting of “psychosocial feelings” and “cognition and vitality.” Two studies^15,16 received an insufficient rating of “−” for their 2-correlated-factors model, as they did not meet the required thresholds for acceptable comparative fit index (CFI) and Tucker-Lewis Index (TFI) > 0.95 or root-mean-square error of approximation (RMSEA) < 0.06. Another study¹⁴ received an indeterminate “?” rating due to the use of exploratory factor analysis, which extracted 7 factors classified into 3 dimensions (“psychosocial,” “physical condition,” and “vitality”). Cultural differences and different sample characteristics may explain the inconsistency of results among studies with determinate results (“+” and “−” ratings). For instance, the studies were conducted in different countries such as the United Kingdom⁴ and Taiwan,¹⁶ and 1 study¹⁵ focused on patients with prominent negative symptoms of schizophrenia.

Internal Consistency.

Six studies^3,13,17–20 calculated CA to evaluate the internal consistency of the SQLS scale. The values ranged from 0.61 to 0.93, but all 6 studies received an indeterminate rating due to insufficient evidence of structural validity, precluding a definitive statement on the internal consistency of the SQLS scale according to the COSMIN criteria.^5–7

Seven studies^4,15,21–25 assessed internal consistency of the SQLS-R4, with all but one⁴ receiving a “?” due to the absence of the prerequisite “at least low evidence for sufficient structural validity.” CA values ranged from 0.76 to 0.95. One study⁴ received a “+” rating for the psychosocial feelings subscale (CA = 0.96) and cognition and vitality subscale (CA = 0.82). Therefore, the internal consistency of the SQLS-R4 scale can be deemed sufficient based on Martin’s⁴ findings of Cronbach’s alpha values above .70.

Cross-cultural Validity.

One¹⁵ study evaluated the cross-cultural validity of the SQLS-R4 scale by examining the effect sizes (ES) of the differential item functioning (DIF) across various cultural groups (see table 3 for the included groups). The results indicated moderate to high DIF-ES in both SQLS-R4 subscales, particularly among Uralic, Slavic, and Japanese cultural subgroups. The study did not explore DIF among other subgroups, such as sex or age. According to this study, the SQLS-R4 received a “−” rating, indicating that the SQLS-R4 scale is insufficient in terms of cross-cultural validity.

Reliability.

The test-retest reliability of the SQLS was evaluated in 1 study,¹⁸ of which received a “−” rating due to mostly low ICC values, ranging from 0.62 to 0.69 for the English version and 0.46–0.82 for the Chinese version. Variations in the linguistic versions used may explain the inconsistency observed between Chinese- and English-speaking patients.

Three studies^15,21,24 examining the test-retest reliability of the SQLS-R4 received a “+” rating, with ICC values ranging from 0.74 to 0.84, indicating consistent results. Based on these results, the SQLS-R4’s reliability is considered sufficient.

Hypotheses Testing for Construct Validity

Convergent Validity.

We employed 4 hypotheses to assess Convergent validity, as outlined in the Cosmin Risk of Bias Manual.^5–7 Satisfaction of these hypotheses is necessary to achieve a sufficient rating.

1. Correlations with instruments measuring similar or conceptually related constructs should be ≥.50.

2. Correlations with instruments measuring related, but dissimilar constructs should be lower, that is, .30 to .50.

3. Correlations with instruments measuring unrelated constructs should be <.30.

4. Correlations defined under 1, 2, and 3 should differ by a minimum of .10.

We concentrated on comparing the total scores of SQLS or SQLS-R4 with those of reference scales, and on assessing the correlation between specific subscores of SQLS or SQLS-R4 and corresponding subscores of similar scales. This choice was guided by the absence of a logical foundation for formulating hypotheses concerning correlations between subscores and total scores, or correlations between subscores of unrelated constructs. When 75% of the hypotheses regarding correlations between total scores or subscores were confirmed, the total scale and subscales, respectively, received an overall “+” (based on^5–7).

Convergent Validity of the SQLS.

Four studies evaluated the convergent validity of the SQLS scale.^3,13,17,18

Three studies^3,13,18 compared 2 of the 3 SQLS subscales—“motivation and energy” and “psychosocial”—with analogous subscales of the SF-36. In these comparisons, the “motivation and energy” subscale of the SQLS demonstrated correlations mostly exceeding .5 (ranging from .59 to .72), with 1 exception (r = .38 in Kaneda’s study), possibly attributable to cultural variations. The SQLS “psychosocial” subscale exhibited a strong correlation (ranging from .64 to .69) with the SF-36 “mental health” subscale, which measures depression and anxiety. With 7 hypotheses confirmed and 1 not confirmed, the SQLS subscales “motivation and energy” and “psychosocial” received 7 “+” and 1 “−,” resulting in an overall “+” for the 2 subscales, given that over 75% of the hypotheses were confirmed.

The SQLS total score displayed a robust correlation (r = .75) with the mental component score of the SF-36,¹⁷ comprising the dimensions “vitality,” “social functioning,” “role-emotional,” and “mental health.” However, it did not exceed r = .5 when correlated with the SF-36 physical component score (PCS; r = .40), which includes “physical functioning,” “role physical,” “bodily pain,” and “general health,” dimensions not fully covered by the SQLS, thus explaining the correlation below .5. Furthermore, the SQLS total score demonstrated a correlation of r = .64 with the EQ-5D Tariff score, summarizing health-related QOL across mobility, self-care, usual activities, pain, and anxiety/depression. It also exhibited a correlation of r = .60 with the EQ-5D VAS score, a vertical visual analogue scale for assessing self-rated health. Given that 3 hypotheses were confirmed, while 1 was not, the SQLS total score obtained 3 “+” and 1 “−,” resulting in an overall “+” assessment, signifying a 75% confirmation rate for the hypotheses.

In summary, the SQLS total scale and 2 of its 3 subscales demonstrates convergent validity when compared to similar constructs. Discrepancies in results could be attributed to cultural adaptations and disparities in dimensions between the SQLS and the comparator instrument. No statement can be made about the “symptoms and side effects” scale of the SQLS.

Convergent Validity of the SQLS-R4.

Convergent validity of the SQLS-R4 scale was evaluated in 6 studies.^4,21–25

Two articles compared the SQLS-R4 subscales with analogous subscales of the SF-36.^23,24 The correlation between SQLS-R4’s “vitality/cognition” and SF-36’s “energy/vitality” consistently exceeded .5 (r = .63 and .65). Likewise, the correlation between SQLS-R4’s “psychosocial” and SF-36’s “mental health” consistently surpassed .5 (r = .52 and .67).

In a similar vein, 2 studies compared SQLS-R4 subscales with analogous subscales of the EQ-5D.^21,25 The correlation between SQLS-R4’s “vitality/cognition” and EQ-5D’s “anxiety/depression” consistently exceeded .5 (r = .51 and .62). Correspondingly, the correlation between SQLS-R4’s “psychosocial” and EQ-5D’s “anxiety/depression” consistently surpassed .5 (r = .57 and .67).

Furthermore, 2 studies compared SQLS-R4 subscales with analogous subscales of the WHOQoL-BREF.^21,22 The correlation between SQLS-R4’s “vitality/cognition” and WHOQoL-BREF’s “physical” consistently exceeded .5 (r = .76 and .62). Similarly, the correlation between SQLS-R4’s “psychosocial” and WHOQoL-BREF’s “psychological” consistently surpassed .5 (r = .52 and .65).

With the confirmation of twelve hypotheses concerning the 2 SQLS-R4 subscales, the SQLS-R4 obtained 12 “+” ratings for its 2 subscales, yielding an overall “+” rating for the 2 SQLS-R4 subscales.

In a distinct study,²¹ the SQLS-R4 total score was correlated with the EQ-5D value/tariff score (r = .62) and the EQ-5D VAS score (r = .60), both exceeding .5. Given the intrinsic conceptual overlap between “anxiety” and “depression” with QOL, a correlation r > .5 was anticipated. Indeed, a study⁴ revealed a robust correlation between the SQLS-R4 total score and both the HADS-A (r = .89) and HADS-D (r = .70).

The assessment of self-esteem using the Rosenberg Self-Esteem Scale (RSES), initially considered a related but dissimilar construct and was therefore expected to correlate r = .3 to .5, unexpectedly demonstrated a high correlation of r = .68.²² This finding suggests a potential strong connection between self-esteem and QOL, though caution is warranted as this correlation may be population-specific.

The SQLS-R4 total scale received an overall “+” rating, with 4 hypotheses regarding the SQLS-R4 confirmed and 1 not, leading to over 75% confirmation.

To summarize, the SQLS-R4 and its subscales exhibit convergent validity when compared to analogous or conceptually related constructs.

Known-Groups Validity.

Hypotheses testing for known-groups validity evaluates the consistency between a PROM’s scores and the expected differences between relevant groups. Ratings of “+” and “−” are given if the results are consistent or inconsistent with the hypotheses, respectively.

One study evaluated the known-groups validity of the SQLS and received a “−” rating.²⁶ Two studies evaluated the known-groups validity of the SQLS-R4, 1 received a “+” rating,²² the other a “−” rating.²¹ The results showed that the SQLS-R4 can differentiate between certain characteristics of known groups, but no reliable conclusion can be drawn due to the variation in study populations.

Responsiveness.

One²⁴ study evaluated the responsiveness of the SQLS-R4 and received a “?” rating as the authors of the study calculated a responsiveness score (RC), on the basis of which no hypotheses could be made. Consequently, the quality of the responsiveness of the SQLS-R4 cannot be validated.

Modified Grade Approach to SQLS and SQLS-R4

SQLS

1. For structural validity, internal consistency, and hypotheses testing, no downgrade for RoB was given. For reliability, there was only 1 study of doubtful quality. Therefore, the scale was downgraded to −2 for RoB. This results in a low level of evidence grade for the SQLS.

2. Except for hypotheses testing, none of the measurement properties received a determinate rating and was assessed by more than 1 study. As there was moderate inconsistency regarding the results of the SF-36 with a valid explanation, a downgrading of −1 was proposed for hypotheses testing. The explanation is described in detail in the “Updated Criteria of Good Measurement” section. This results in a very low level of evidence grade for the SQLS. At this point, one could stop to downgrade the evidence because we already reached the maximum possible downgrade. For informative aspects, we report on the following last 2 factors as well.

3. With an overall sample size of n = 2.372 as for the SQLS, no further downgrade would have been given.

4. As all the validation studies of the SQLS were performed in in- or outpatients with schizophrenia or schizophrenia spectrum disorder, there is no indirectness.

The overall quality of the evidence is considered as “very low” as for the SQLS scale.

This implies that the degree of confidence we have in the estimation of the measurement property is limited, as the actual measurement property is expected to deviate significantly from the estimated value.^5–7

SQLS-R4

1. Structural validity, internal consistency, hypotheses testing, and responsiveness did not receive a downgrading for RoB. For cross-cultural validity, there was only 1 study of inadequate quality, therefore, the scale was downgraded to −3. For reliability, the scale was also downgraded to −1, as there was only 1 study of adequate reporting quality and 2 doubtful studies. Therefore, the scale was downgraded to the maximum possible of −3 for RoB. This results in a “very low” level of evidence grade for the SQLS-R4. One could stop at this point with the GRADE approach as no further downgrading below “very low” is possible. As for the SQLS, we describe the other factors for informative purposes.

2. Inconsistency was observed in the structural validity. Structural validity produced inconsistent factor-model-solutions among the determinate results (“+” and “−” ratings), which may be attributed to variations in the characteristics of the study samples and also to different cultural contexts. Had the RoB not already warranted a downgrade of −3, we would have considered a downgrade of −1 appropriate.

3. With an overall sample size of n = 1.104 for the SQLS-R4, no further downgrade would have been given.

4. As all the validation studies of the SQLS-R4 were performed in in- or outpatients with schizophrenia or schizophrenia spectrum disorder, there is no indirectness.

The overall quality of evidence of the SQLS-R4 is rated as “very low,” signifying limited confidence in the measurement property estimation, with a high probability of significant deviation of the actual measurement property from the estimated value.^5–7

Discussion

For the first time, the present review evaluates the SQLS and its modified version, the SQLS-R4, in accordance with the updated COSMIN methodology.^5–7 We found 7 validation studies for the SQLS and 9 validation studies for the SQLS-R4.

The SQLS demonstrated favorable convergent validity but displayed limitations in terms of structural validity, internal consistency, reliability, and known-groups validity. The SQLS-R4 showed promising results concerning its reliability and convergent validity but presented weaknesses in structural validity, internal consistency, cross-cultural validity, known-groups validity, and responsiveness. Currently, there is no research verifying the content validity of these scales. However, during the PROM development study for the SQLS, patients were interviewed multiple times to inform the content, ensuring it reflected their views on important QOL aspects.³

These ratings permit an overall evaluation of the degree to which the SQLS and the SQLS-R4 can be recommended. The COSMIN framework classifies PROMs into 3 distinct categories^5–7:

• (A) PROMs with evidence for sufficient content validity (any level) AND at least low-quality evidence for sufficient internal consistency.

• (B) PROMs categorized not in A or C.

• (C) PROMs with high-quality evidence for an insufficient measurement property.

PROMs classified as “A” are deemed fit for use and their outcomes are considered as being reliable. The PROMs categorized as “B” hold potential for recommendation but necessitate further investigation to assess their quality. PROMs classified as “C” should not be recommended for employment.

The SQLS and the SQLS-R4 fall into category B, as there is no evidence for sufficient content validity and there is no high-quality evidence for an insufficient measurement property. Therefore, the SQLS and SQLS-R4 need further investigation to be fully recommendable.

To be classified in category “A,” it is crucial to demonstrate the content validity of the SQLS and SQLS-R4. Nevertheless, retrospective assessment of content validity can be performed via means such as focus groups, which evaluate comprehensiveness, comprehensibility, and relevance in accordance with the COSMIN manual guidelines.^5–7 Successful completion of this process could substantiate content validity and potentially elevate the scales to category A, indicating official eligibility for usage.

The SQLS and SQLS-R4 encounter structural validity issues due to an insufficient number of studies receiving determinate ratings. To assess structural validity conclusively, the COSMIN manual^5–7 requires reporting CFI, TLI, or a comparable measure >0.95, OR RMSEA < 0.06, or SRMR < 0.082 in CTT applications. For Rasch or IRT analyses, adherence to unidimensionality without violation and reporting a model fit are essential.

Despite excellent alpha coefficients in all 4 studies for SQLS and 7 studies for SQLS-R4 pertaining to individual subscales, internal consistency results are indeterminate. This is attributed to the lack, at the very least, of low evidence for structural validity. This limitation hampers confidence in the overall internal consistency of both scales. Consequently, we suggest conducting additional confirmatory factor analyses for the SQLS and SQLS-R4 scales to enhance confidence in their internal structure.

Other than that, known-groups validity has been evaluated in only 1 article²⁶ for the SQLS showing an insufficient result, and 2 articles for the SQLS-R4 showing 1 sufficient²² and 1 insufficient²¹ result; consequently, additional validation is recommended.

The SQLS lacks confidence regarding its reliability with only 1 study of doubtful methodological quality¹⁸ receiving an insufficient rating due to low ICC values. Therefore, additional studies with improved methodological quality are essential to establish the reliability of the SQLS.

Only 1 study¹⁵ has assessed the cross-cultural validity of the SQLS-R4, resulting in a significant downgrade due to a small sample size. The COSMIN manual is highly rigorous in its evaluation, and thus, a maximum downgrading of −3 was performed in the grade approach. Future studies should investigate cross-cultural validity using more extensive sample sizes to rectify the limitations of the Rofail et al¹⁵ study. Additionally, examining Differential Item Functioning (DIF) across relevant subgroups (eg, sex, age) is recommended to comprehensively evaluate the psychometric properties of the SQLS and SQLS-R4, improving understanding of demographic influences on item interpretation and scale validity and reliability across populations.

Moreover, only 1 study assessed the responsiveness of the SQLS-R4²⁴ resulting in an indeterminate rating, as the study authors calculated a responsiveness score based on which no hypotheses could be generated. Future studies should evaluate the responsiveness of the SQLS and SQLS-R4 according to the COSMIN guidelines,^5–7 clearly stating hypotheses about expected treatment effects or between groups. This will help to determine if the scales can accurately detect significant changes in patient’s QOL, demonstrating their sensitivity to change.

Furthermore, high rates of missing responses may signal a questionnaire’s relevance issues. Among 7 studies on the SQLS, 2^18,26 reported missing data, each less than 1%. For the SQLS-R4, 3^15,21,25 of 9 studies noted missing data, ranging from 0.5% to 13.1%, indicating generally high acceptability. However, according to the COSMIN group, the absence of reporting on missing data does not necessarily indicate biased study outcomes.^5–7

Our work has certain limitations as well. First, we did not perform a meta-analysis due to heterogeneous data, and because much of the data were not reported in the studies. Instead, we relied on qualitative data evaluation to identify the direction of results and provide recommendations. Second, our review relies solely on papers published in English. Given that it is the primary language of scientific communication, this approach captured a substantial amount of relevant literature in our systematic review. Third, there is no consensus on a unique definition of QOL. We relied on the van Krugten¹⁰ paper to include only scales that capture at least partly important domains of QOL.

In conclusion, validation studies of suboptimal methodological quality, particularly regarding the original version, diminish our confidence in the results of the validation studies, thus preventing us from making definitive recommendations regarding the usefulness of the SQLS and SQLS-R4. Future investigations should address the identified limitations, particularly the lack of a content validity study, and adhere to the criteria of the COSMIN manual^5–7 to gain greater confidence in the study results. It is crucial to note that the COSMIN methodology is still a relatively new tool, and its criteria are often very strict.

However, in the field of QoL research, the notion that scale selection for assessment should depend on sample characteristics and study design, rather than a universal instrument, is gaining acceptance.² PROMs are appropriate for patients with sufficient insight into their illness for measuring QOL. As the SQLS and the SQLS-R4 require only 5–10 minutes for completion, following the resolution of their limitations, these instruments may serve as suitable self-reported measures for patients possessing disease insight in routine care settings.

Author Contributions

The authors substantiate their contributions to the article in the following manner: study conception and design: M.Z., M.W., S.W, L.W., and S.L.; data collection: M.Z. (first rater) and M.W. (second rater); analysis and interpretation of results: M.Z. (first rater) and M.W. (second rater), S.W., L.W. and S.L. (third raters); draft manuscript preparation: M.Z.; revision for important intellectual content: M.W., S.W., L.W., S.L.; general advice: S.W, L.W. The work will be part of the doctoral thesis of M.Z. All authors reviewed and approved the final manuscript. They commit to personal accountability for their contributions and pledge to address and document any concerns about the accuracy or integrity of the work.