Considerations for the Development and Implementation of Brief Screening Tools in the Identification of Early Psychosis

The concept of prevention of mental health concerns has a long history. Preventive developmental psychopathology has identified biopsychosocial risk and protective factors, the elucidation of which has led to a variety of effective, disorder-specific, intervention strategies for mitigating symptoms and promoting wellness (e.g., child behavior issues, adolescent depression, anxiety and traumatic response). As the benefits of early intervention for those experiencing impairing attenuated psychosis symptoms (some of whom on an apparent trajectory towards psychosis-spectrum disorders) continues to mount (Okuzawa et al., 2014; Stafford et al., 2013), accurate early identification has become a priority. Semi-structured interviews such as the SIPS (Structured Interview for Prodromal Syndromes) and CAARMS (Comprehensive Assessment of the At-risk Mental States) have improved the identification of attenuated symptoms and, in some cases, the prodromal stage of illness, allowing for the possibility of effective early intervention. With accurate prediction continuing to evolve (Cannon et al., 2016), brief psychosis-risk screeners have emerged as an essential part of the prediction equation. Self-report measures can provide objective thresholds to determine if further evaluation is warranted, particularly valuable for those whose emergent symptoms may go unnoticed due to insidious onset, as well as for self-referrals who are more likely to be false-positive for risk (Fusar-Poli et al., 2015; 2016b). Several self-report measures strongly align with clinician-rated psychosis-risk interviews, as well as with future conversion (Kline et al., 2012a; 2015). These brief, low cost tools may help refine referral pathways, enrich recruitment strategies (reduce false-positives, increased true-positives of clinical interviews), reduce duration of untreated psychosis, and help clients find appropriate specialty care.

The two most commonly used screening tools for attenuated symptom risk are the Prime Screen-Revised (Miller et al., 2004) and the Prodromal Questionnaire (which has two short forms, the PQ-B and the PQ-16; Loewy, 2005) (Kline & Schiffman, 2014). The 12 item Prime Screen-Revised was developed at the Yale PRIME clinic, originally designed as a complement to the SIPS. Supported by a relatively larger body of studies, items from the PQ and its shorter versions are primarily taken from the Schizotypal Personality Questionnaire (Raine, 1991), a pre-existing measure designed for assessment of schizotypal personality disorder, as well as from the SIPS. Despite strengths of both the Prime Screen-Revised and the PQ, comprehensive validation is still on-going. Notably, item generation for both measures was derived directly from the gold-standard interview; therefore, assuming that the gold-standard is imperfect, there is risk for over-optimistic prediction whereby the screeners align more with the interview than the true construct itself.

In their paper “The Early Psychosis Screener (EPS): Quantitative Validation against the SIPS Using Machine Learning,” Brody and colleagues (2018) share compelling preliminary evidence in support of their newly created Early Psychosis Screener (EPS). Creation of the EPS differed relative to the other more established measures, as an expert panel was employed for item development, modification, and selection, and cognitive interviewing techniques were implemented before field testing to promote stronger construct validity, with the ultimate goal of improving accuracy of true risk prediction. The EPS also benefited from having North America’s largest CHR research network in NAPLS, as well as COPE, as its validation sample. Additionally, a machine learning analytic approach leveraged cutting edge statistics to optimize the number and type of items included in the measure to improve efficiency.

Despite the promise of screeners such as the EPS, PRIME-Screen, and PQ as complements to clinician interviews, care must be taken to ensure that screening is conducted with appropriate sensitivity to, and inclusivity of, a wide range of factors. Brody and colleagues did not consider basic demographic information (age, gender, race, education, employment) in their model. Future studies are still needed that incorporate larger samples to provide insight into the impact of demographic features, as well as a host of other factors that could impact screening. For instance, evidence suggests that living in a high-crime neighborhood can account for high endorsements of suspiciousness (Wilson et al., 2016); that screener accuracy can vary by race (Millman et al., under review) and age (Rakhshan et al., in preparation); that parent input can complement self-report (Kline et al., 2013); and that symptom distress is important for determining the implications of item endorsement (Kline et al., 2014a). Still other factors need further elucidation, including the role of substance use, trauma exposure (Kline et al., 2016a), and other comorbidities. The possible impact of stigma, as well as strategies for disclosure of results that can be highly stigmatizing and distressing for families, remain largely unstudied; and only a few studies have looked at the accuracy of screening over time (Kline et al., 2015). Additionally, the role of sampling in the understanding of screening tools cannot be overstated, as mounting evidence suggests that enriched samples such as those referred to CHR research studies are far more likely to provide optimistic estimates of screener performance relative to the general population (Fusar-Poli et al., 2016a). Prediction studies within broader help-seeking samples are required to establish specificity of risk for psychosis versus pluripotent risk for mental health concerns more generally (Millman & Schiffman, in press). Nonetheless, given that the majority of those who develop psychosis do not engage with mental health services until after illness onset, improved screening in non-clinical samples, possibly implemented on-line for broader outreach, is needed.

An important set of questions for psychosis-risk screening research is to determine what screener to use, when, with whom, and how. The EPS should now be added to the conversation. To their credit, the authors reported a direct statistical comparison of the EPS to the well-studied PQ-B. Unfortunately, however, they did not analyze the PQ-B with its distress scale, so the comparison cannot provide true insight into the relative value of each. Future work should continue to study the relative benefits and limitations of different screeners for different purposes (Kline et al., 2012b). One strategy that is not often considered in screening is the use of larger, more comprehensive checklists that provide a broader mental health perspective beyond psychosis risk. For instance, we found that the “Atypicality Scale,” embedded in the BASC-2 (Reynolds & Kamphaus, 2004) comprehensive behavior checklist, predicted risk similarly well to the PQ-B and the Prime-Screen (Thompson et al., 2013). Although longer to administer, the BASC-2 provides a range of mental health concerns and strengths, facilitating a well-rounded, and potentially less stigmatizing, conversation about a person’s functioning. The BASC-2, as well as our modified version of the PRIME-Screen for parents, also provide multi-informant information useful for triangulating on risk (Thompson et al., 2014). On the other hand, if brevity is a priority, the PRIME-Screen’s 12 items take under a minute, making it useful not just for baseline screening, but ongoing clinical monitoring (Kline et al., 2016b). The PQ’s relatively well validated distress scale also provides a unique clinical advantage. Finally, if distinguishing between risk and diagnosable psychosis is a priority, it should be noted that no screener has yet to demonstrate a strong ability to discriminate between the two.

A final consideration is the uptake of screeners by community providers. Brody and colleagues provided three helpful predictive scenarios, offering community groups who might implement the screener a glimpse as to how the EPS might operate given different clinical priorities (more or less sensitivity relative to specificity). Ignoring barriers such as the need to educate providers on the screeners and risk more generally (Jacobs et al., 2011; Jacobs et al., 2012; Kline et al., 2014b), as well as the availability of specialty clinics to help work with those identified as at risk, many clinicians protest that even a 12 item screener is too long. Although anecdotal, our clinical partners have lobbied for a model whereby clinical judgment can be used as a trigger for a four item measure. Empirical work should continue to hone measurement, and consider examining systems that incorporate clinical judgment as the gatekeeper for screening that can then cue an interview if indicated. Irrespective of the accuracy, consideration of real-world uptake is critical for widespread dissemination.

Regardless of the screener used, replication in this field is imperative (Szucs & Ioannidis, 2017). Although still in its infancy, the CHR prediction literature is suffering from overoptimistic predictive accuracy estimates derived from very small samples. The current body of work can serve as a place from which to start, but not from which to generalize.