The Psychosocial Assessment Tool (PAT2.0): Psychometric Properties of a Screener for Psychosocial Distress in Families of Children Newly Diagnosed with Cancer

Abstract

Purpose

Psychometric properties of the Psychosocial Assessment Tool 2.0 (PAT2.0), a brief screener for psychosocial risk in families of children with cancer, are presented.

Methods

Female (N =132) and male (N =72) caregivers of 141 children newly diagnosed with cancer completed the PAT2.0 and measures of child behavior symptoms, anxiety, acute stress, and family functioning to establish validity. Internal consistency and test–retest reliability of the PAT2.0 were also examined.

Results

Internal consistency and two-week test–retest for the PAT2.0 Total score was strong. Validity for the PAT2.0 was supported by significant correlations between the PAT2.0 subscales and measures of corresponding constructs. PAT2.0 Total scores were correlated with acute stress and child behavior symptoms for both mothers and fathers. Receiver-Operating Characteristic curves provided preliminary support for the proposed cutoffs.

Conclusion

The PAT2.0 Total score is a useful screening tool for family psychosocial risk in the pediatric oncology population.

The treatment of pediatric cancer involves a demanding medical regimen in which families are confronted with multiple and pervasive stressors including significant medical side effects (Bryant, 2003), considerable changes in daily activities (Woodgate, Degner, & Yanofsky, 2003), disruption of social and family roles (Kazak, Simms, & Rourke, 2002), the burdens of adhering to complicated and often very intense treatment regimens (Crist & Kun, 1991), and the threat of death. Recognizing the impact of such stressors, national and international recommendations for comprehensive cancer care include the provision of psychosocial services to families of children with cancer (American Academy of Pediatrics, 2004).

Unfortunately, clear guidelines have not been established with regard to the delivery of these services. Systematic approaches for assessing psychosocial need and formulating types and levels of intervention for particular patients and families are not available. Although, collectively, the majority of children with cancer and their families are resilient in the face of cancer diagnosis and treatment (Kazak, 2006), subgroups of children and their families are at risk for or evidence of clinically significant distress and impaired coping (Patenaude & Kupst, 2005) and warrant more consistent evidence-based care (Kazak, 2005).

The Pediatric Psychosocial Preventative Health Model (PPPHM; Kazak, 2006; See Fig. 1) may be helpful as a framework for conceptualizing psychosocial risk¹ and formulating the type and level of interventions required. This model, adapted from the National Institute of Mental Health prevention framework, describes the pediatric health population by conceptualizing families in terms of psychosocial risk (National Institute of Mental Health, 1998). Presented as a pyramid, PPPHM estimates that the majority of families, represented in the base of the pyramid, are transiently and understandably distressed, but resilient (Universal). Another smaller set of families, represented in the middle tier of the pyramid, experience acute distress and the presence of some psychosocial risk factors (Targeted). The remaining set of families, smallest in number, are at the apex of the pyramid, with multiple risk factors indicating intense, persistent and/or escalating distress (Clinical; Kazak et al., 2001, 2003). An evidence-based assessment approach that could reliably classify families by level of psychosocial risk could streamline the delivery and increase specificity in the provision of psychosocial services in health care environments, which have limited resources to meet the needs of children diagnosed with cancer and their families.

Figure 1. The pediatric psychology preventative health model (Kazak, 2006).

Note. Based on PAT2.0 score cutoffs, 55% of mothers, and 67% fathers scores fell in the Universal category, 32% of mothers and 32% of fathers scores fell in the Targeted category and 13% of mothers and 1% of fathers scores fell in the Clinical category.

The Psychosocial Assessment Tool (PAT) was a screening instrument designed to assess psychosocial risk in families of children newly diagnosed with cancer. The original PAT was a 20-item screening questionnaire that assessed a constellation of risk and resource factors including family structure, family resources, social support, child knowledge, school attendance, child emotional, and behavioral concerns, child maturity for age, marital/family problems, family beliefs, and other stressors (Kazak et al., 2001). In a prospective study of 125 families of children newly diagnosed with cancer, preliminary reliability and validity data for the PAT was established (Kazak et al., 2001, 2003). Higher PAT scores were associated with higher levels of psychosocial risk (Kazak et al., 2003), and PAT scores at time of diagnosis were also significantly related to PAT scores 3–6 months later (p < .01). Although, the PAT was a viable instrument, testing revealed that some items were difficult for respondents to understand and some open ended questions failed to elicit detailed information about child and family psychological symptoms.

Therefore, a data-driven revision of the original PAT was undertaken. The resulting PAT2.0 was modified to improve the clarity of questions, reformatted to be more appealing and user friendly, and expanded in item content based on new knowledge and data from the original PAT study. The PAT2.0 is a two-page self-report measure consisting of 15 item sets. The response format for the items was designed to be brief and simple (e.g., yes/no, categorical responses, Likert-type scales). Completion of the PAT2.0 takes approximately 10 min.

The purpose of the current study is to evaluate the psychometric properties of PAT2.0. First, we evaluate the internal consistency and test–retest reliability of the PAT2.0. In addition, the PAT2.0 scores for mothers and fathers are compared to identify whether there are differences between reporters on the PAT2.0. Next, we examine the content validity of the PAT2.0 subscales, correlating subscale scores with scores on standardized measures designed to measure similar constructs. We hypothesized that higher scores on PAT2.0 subscales would be significantly associated with higher scores on measures that correspond to the content the PAT2.0 subscale is intended to measure (e.g., PAT2.0 Child Behavior Problems subscale and the BASC-2 Behavioral Symptom Index). To assess criterion-related validity we test whether PAT2.0 Total Scores are significantly related to more adverse psychosocial outcomes typically associated with family psychosocial risk. We predicted that higher PAT2.0 Total scores would be significantly associated with higher scores on psychosocial outcomes including acute stress, child behavior symptoms, state anxiety, and family conflict and lower scores on family cohesion. The PPPHM categories were also examined to assess criterion-related validity. We predicted that families that score in the Clinical category would score higher on measures of psychological distress (i.e., PTSS and anxiety symptoms for parents and behavioral symptoms for children and family conflict) than families that fall in the Targeted or Universal categories. In addition, families in the Targeted category on their PAT2.0 score would score higher on measures of psychological distress and family functioning than those in the Universal category.

Evidence for convergent validity is sought through investigation of correlations between PAT2.0 scores and a staff version of the PAT completed by physicians and nurses. Finally, the PAT2.0 does not merely reflect characteristics of the cancer diagnosis or treatment, but rather to capture characteristics and patterns of functioning within the family that put them at risk when confronting cancer diagnosis and treatment. Previous study findings have been mixed, with some showing a relationship between psychological outcomes and medical indicators (Phipps, Long, Hudson, & Rai, 2005) and others not (Alderfer, Cnaan, Annunziato, & Kazak, 2005). Therefore, no relationship between the PAT2.0 Total score and treatment intensity are anticipated (discriminative validity). Finally, we predicted that the PAT2.0 scores would discriminate between families that score high on the clinical outcomes [i.e., BASC-2 Behavioral Symptom Index and the Acute Stress Disorder Scale (ASDS)] and those who do not.

Method

Participants

Female (N =132) and male (N =72) caregivers of 141 children newly diagnosed with cancer participated. Inclusion criteria were a confirmed diagnosis of a pediatric malignancy in a child under the age of 18 years without prior chronic or life threatening illness, and fluency in English or Spanish. Six parents completed the PAT2.0 in Spanish (two mothers and four fathers). The majority of families (83.0%, N =117) were two-caregiver households and parents’ average age was in the late 30s and early 40s (M_{mother age} = 38.09, SD =7.25; M_{father age} = 41.11, SD =7.03). Educationally, the sample was diverse, with 23.4% of mothers (N =31) and 16.2% of fathers (N =12) having a high school education or less, 58.3% (N =77) of mothers and 56.7% (N =42) of fathers having college courses or a college degree and 18.2% (N =34) of mothers and 27.0% (N =20) of fathers having some postgraduate education.

The patients ranged in age from 5 weeks to 18 years (M =8.2 years, SD =5.6 years). Sixty percent were male (N =84). Ethnic background was as follows: Caucasian (N =111, 78.7%), African-American (N =13, 9.2%), Hispanic (N =7, 5.0%), Asian (N =3, 2.1%), and bi-racial (N =7, 5%). Cancer diagnoses were: leukemias (ALL, AML, CML; N =54, 38.3%), brain tumors (N =31, 22.0%), and solid tumors (lymphoma, neuroblastoma, sarcomas, Hodgkin’s Disease, germ cell tumors, Wilm’s tumor, and carcinoma; N =56, 39.7%).

Twenty-seven oncologists and 46 nurses were asked to complete the Staff PAT for families in the study. Staff PATs were completed by either the nurse or oncologist for 135 families (95.7% of the total sample) and were completed by both the nurse and oncologist for 80 families. Oncologists were either the attending physician or a fellow assigned to the patient. On average the oncologists had 6.6 (SD =8.9) years experience and spent an average of 11 days (SD =12.7 days) on service with each family prior to completing the measure. Most nurses were the assigned primary nurse. In 27% of the cases, we were unable to identify the primary nurse, and the bedside nurse with most contact with the patient was asked to participate. On average, nurses had 4.4 (SD =4.5) years experience. They spent, on average, five shifts (SD =4.3 shifts) with each family prior to completing the measure.

Procedures

Study Procedures

This study was approved by the Committees for the Protection of Human Subjects of the Institutional Review Board at the hospital where the study was conducted. Newly diagnosed patients were identified when they completed their initial family diagnostic meetings during which a diagnosis of cancer was discussed with parents. Families were approached for participation during the child’s first inpatient hospitalization or during an out-patient appointment (if the child was not admitted). One hundred and fifty-eight families were approached for participation over a 16-month period between January 2005 and April 2006. Of those, 141 families agreed to participate (89% participation rate) and provided consent. Reasons families gave for declining participation included: feeling overwhelmed (N =12), not interested in participating (N =3), already participating in another research study (N =1), or not wanting to consent to use of data from the medical record (N =1). Subsequent to providing written informed consent, parents completed self-report measures. Parents completed study measures typically within 2 weeks of the date of diagnosis (Mothers: median =4.0 days, M =7.1, SD =7.2; Fathers: median =5 days, M =7.4, SD =6.8). The majority of staff completed their measures within the first month of diagnosis (Nurses: median =22.0 days, M =28.3, SD =20.9; Physicians: median =11.0 days, M =20.8, SD =22.5). To assess test–retest reliability the first 25 families enrolled in the study with even numbered participant numbers were asked to complete the PAT2.0 a second time two weeks after baseline. A total of 25 mothers and 20 fathers completed the second PAT2.0 an average of 13.5 days (SD =7 days) after baseline.

Measure Development Procedures: The Psychosocial Assessment Tool 2.0 (PAT2.0)

Prior to enrolling participants for this study, the original PAT measure was revised. Eighteen domains of psychosocial risk were identified based on previous literature and the original PAT study to guide PAT2.0 development. They were as follows: Family Conflict, Family Resources, Family Structure, Social Support, Stress Reactions, Family Substance Use, Family Psychological Problems, Child Internalizing Problems, Child Externalizing Problems, Child Cognitive Problems, Child Social Problems, Child School Enrollment, Child Educational Placement, Patient’s Medical Status, Child Knowledge of Cancer, Family Beliefs, Family Medical Problems, and Sibling Problems. As an initial step in the validation of the PAT2.0, 84 clinical experts in pediatric oncology (21 each—oncologists, nurses, social workers, and psychologists), outside our institution, rated the 18 domains with regard to its level of psychosocial risk using four categories—“no risk”, “low risk”, “medium risk”, or “high risk”. In general, there was strong agreement that the domains were tapping risk factors for ongoing distress. All 18 domains were rated as medium or high risk by at least 65% of the expert raters. Items on the PAT2.0 were developed to correspond with these 18 domains.

The product of this development process was a screening measure for assessing family psychosocial risk in families with a child newly diagnosed with cancer (Fig. 2) which was then administered to caregivers of newly diagnosed patients. A PAT2.0 Total score and seven subscale scores were formed based on theoretical and empirical grounds (Kazak et al., 2001, 2003) and to ensure acceptable preliminary internal consistency (i.e., α ≥ .60; Ware et al., 1980). The scales are theoretically based on a social ecological perspective on child health (Kazak, 1989; Kazak et al., 2002), reflecting the importance of parent and family functioning, psychosocial resources, social support and illness factors in child outcomes. The family structure and family resources domains were combined to form the Family Structure and Resources Subscale. Likewise, the family problems, family substance use problems, family conflict, and family medical problems domains were collapsed into one subscale with acceptable reliability called Family Problems. The child internalizing, externalizing, cognitive, and social problems domains were collapsed into a Child Problems subscale that mirrors the Sibling Problems subscale. Consistent with the original domains, Social Support, Stress Reactions, Sibling Problems, and Family Beliefs subscales were also developed. There were five items representing four domains (child knowledge of cancer, child school enrollment, child educational placement, and patient’s medical status) that were not included in the PAT2.0 Total score because they were represented by too few items to conduct reliability analyses. They are included on the PAT2.0 because these items assess valuable information for clinicians.

Figure 2.

The psychosocial assessment tool 2.0.

Ultimately, the PAT2.0 was comprised of the following subscales: Family Structure and Resources, Family Social Support, Family Problems, Parent Stress Reactions, Family Beliefs, Child Problems, and Sibling Problems. Each item was dichotomously scored (i.e., risk/ no risk), based on the literature and the consensus of our multidisciplinary research group (e.g., respondent indicates that an adult in the child’s home has experienced prolonged sadness or depression). Ranges of possible scores for each of the subscales are in Table I. To derive the PAT2.0 Total scores, adjusted PAT2.0 subscale scores were calculated by dividing the number of high-risk items endorsed in each domain by the total number of questions in the respective domain. Using this calculation, the adjusted score for each subscale could range from 0.00 to 1.00. These adjusted subscale scores were then summed to create a PAT2.0 Total score for each respondent. Possible PAT2.0 Total scores ranged from 0 to 7.

Table I.

Descriptive Statistics for PAT2.0 for Female and Male Caregivers

			Female caregivers N =132			Male caregivers N =73
PAT2.0 scale (Items)	Scale range	Internal consistency	M	SD	Range	M	SD	Range
Total	0–7	.81	1.11	.81	0–3.87	.85	.53	0–2.58
Structure/resources (1, education, 5, 7, 8)	0–8	.62	1.15	1.37	0–6	.89	1.02	0–4
Family problems (13a–j)	0–10	.72	1.42	1.68	0–8	1.16	1.44	0–6
Social support (4 a–d)	0–4	.69	.12	.48	0–4	.04	.20	0–1
Stress reaction (14a–c)	0–3	.64	.61	.91	0–3	.37	.79	0–3
Family beliefs (15a, c, f, h)	0–4	.59	1.33	1.24	0–4	1.18	1.09	0–4
Child problems (11a–n, p)	0–15	.81	2.74	3.01	0–13	2.70	2.88	0–13
Sibling problems (12a–n, p)	0–15	.73	1.27	2.10	0–12	1.89*	2.48	0–7

Note: Mean comparisons between mothers and fathers on PAT2.0 Total and subscale scores (N =64).

^*Significant difference between mothers and fathers (p < .05). Internal consistency calculated using Kuder–Richardson-20 and conducted on data provided by the primary caregiver (whether identified as mother or father).

Validation Measures

Demographic Information

Parents provided information regarding their age, relationship status (married/not married), highest level of education completed, and child’s age and race/ethnicity. Patients were categorized as either Caucasian or nonCaucasian, based on parent-identified race of the child. Parent report of years of education was used to estimate socioeconomic status (SES; Cirino et al., 2002).

Behavioral Assessment Scale for Children – Second Edition, Parent Rating Scales (BASC-2)

The BASC-2 (Reynolds & Kamphaus, 2004) is comprised of 134–160 items (depending on the age of the child), 4-point Likert-type rating scale assessing parental report of child psychosocial competence, standardized for ages 2.5–18 years. Only parents of children over age 2.5 completed the BASC-2. Internal consistencies for the Behavioral Symptoms Index on the current sample ranged from.80 to.94. The Behavioral Symptoms Index was used as a measure of child behavior symptoms.

Acute Stress Disorder Scale

The ASDS (Bryant, Moulds, & Guthrie, 2000) is a 19-item inventory, rated on a 5-point Likert-like scale, designed to serve as a screening instrument to identify acutely traumatized individuals and predict posttraumatic stress disorder. In addition to the Total score, the ASDS consists of four subscales/clusters, Dissociation, Re-experiencing, Avoidance, and Arousal. In the normative sample, internal consistency of the total score was high (α =.96). Two seven-day test–retest correlation coefficients have shown to be strong for the total score (.94; Bryant et al., 2000). The ASDS Total score was used in the current study and the internal consistency for ASDS Total score in our sample was excellent (mothers =.91, fathers =.90).

Family Environment Scale (FES) – Conflict and Cohesion Scales

The FES (Moos & Moos, 1974) is a well-established self-report measure of family functioning. The Conflict and Cohesion subscales are 9-item scales using a True–False format. Higher scores indicate greater conflict and cohesion. Adequate internal consistency has been previously demonstrated for the Conflict (α =.75) and the Cohesion scales (α =.78; Moos & Moos, 1974) along with adequate two- and four-month test–retest reliability (Conflict: r’s =.85 and .66, respectively; Cohesion: r’s =.86 and .72, respectively; Auerbach et al., 2005). Within our sample, alpha was marginal for both the Cohesion (mothers =.55, fathers =.43) and Conflict (mothers =.62, fathers =.63) scales.

State-Trait Anxiety Inventory (STAI-Y)-State Scale

The STAI-Y (Spielberger, 1983) is a 40-item self-report questionnaire, rated on a 4-point Likert-like scale (1 =not at all, 2 =somewhat, 3 =moderately so, 4 =very much so) that assesses symptoms of current anxiety (state anxiety) and general anxiety (trait anxiety). Only State anxiety was reported in this study as we were interested in measuring current anxiety symptoms rather than dispositional characteristics. Internal consistencies for the State Anxiety scale have ranged from .89 to .96 (Spielberger, 1983). Within our sample, the internal consistency for the state anxiety scale was excellent for both mothers (α =.93) and fathers (α =.94).

Staff Report

Staff PAT

The Staff PAT is a 17-item Likert type rating scale developed by our team that provides a parallel assessment of psychosocial risk from the perspective of the patient’s oncologist (attending or fellow) and nurse (bedside or primary). The items correspond to the PAT2.0 and ask whether a particular risk factor is an area of concern for the family (four point scale ranging from definitely no [0] to definitely yes [3]). A sum score is derived, ranging from 0 to 51. Internal consistency for the Staff PAT in the current sample was α =.88 for oncologists and α =.82 for nurses.

The Intensity of Treatment Rating scale (ITR-2)

The ITR-2 provides a categorization of the intensity of pediatric cancer treatment from least intensive (Level 1) through most intensive (Level 4). Ratings were based on treatment modality (radiation, chemotherapy, surgery) and stage/risk level for the patient. Ratings were made by a pediatric oncologist (A.T.R.), blind to patient identity, based on disease and treatment data extracted by chart review. The ITR has been used in prior studies (Hobbie et al., 2002; Kazak, Boeving, Alderfer, Hwang, & Reilly, 2005; Kazak et al., 2003) but has been revised recently (Werba et al., 2007). Content validity for the ITR-2 was demonstrated on previous samples by agreement between internal criterion raters and external criterion raters on all items (r =.95; Werba et al., 2007). Inter-rater reliability on the current sample was evaluated by calculating inter-rater reliabilities on randomly selected proportion of the sample (45%) and was .89.

Overview of Statistical Analyses

Analyses were conducted in four stages. First, descriptive statistics were calculated for all study measures. In the second stage, reliability analyses were conducted for the PAT2.0 Total and Subscale scores including internal consistency, test–retest reliability, and reliability across respondents. Because items were dichotomously scored (i.e., risk/no risk), internal consistencies were calculated using Kuder–Richardson 20 coefficients. Where the internal consistency of the scale was ≥.60 no items were removed as internal consistency coefficients of ≥.60 are considered adequate for newly developed scales (Ware et al., 1980). For scales where the internal consistencies were inadequate (<.60) individual items from each scale were examined and removed one at a time until the internal consistency for each scale reached >.60. Test–retest reliability was also calculated for the PAT2.0 Total score. Finally, differences between mothers’ and fathers’ responses on the PAT2.0 total and subscales were compared using paired t-tests.

Second, validation analyses were conducted using Pearson Product Moment Correlation Coefficients. Correlations were calculated between PAT2.0 Total and subscale scores and measures of parent acute distress and state anxiety, family functioning, child psychological symptoms, and treatment intensity. In cases where there was a clear construct that corresponded to the PAT2.0 subscales and a well-validated measure to test that construct, the content validity was assessed by examining correlations between the PAT2.0 subscales and measures of the corresponding construct. Specifically, the Family Problems, Family Structure and Resources, and Sibling Problems subscales were correlated with the FES conflict and cohesion scales; the Stress Reaction and Family Beliefs subscales were correlated with the ASDS; the Stress Reaction subscale was correlated with the STAI-State and the Child Problems were correlated with the BASC-2. Due to the marginal reliability of the FES scales, corrected correlations were calculated for all variables correlated with the FES for both mothers and fathers and are reported in Tables III and IV (Spearman, 1904).

Table III.

Correlations of PAT2.0 Subscale Scores and Validation Instruments: Construct Validity

Construct related validity
			Correlations
Construct	PAT2.0 subscale	Validation instrument	Mothers	Fathers
Family functioning	Structure/resources	FES-cohesion	−.27**	−.27*
		FES-conflict	.15	.17
	Family problems	FES-cohesion	−.41***	−.12
		FES-conflict	.27**	.29*
	Sibling problems	FES-cohesion	−.12	−.02
		FES-conflict	.30***	.09
Acute stress	Stress reaction	ASDS	.59***	.46***
	Family beliefs	ASDS	.18*	.09
Anxiety	Stress reaction	STAI-state	.48***	.44***
Child behavior	Child problems	BASC-2	.71***	.60***

Disattenuated correlations were calculated for variables correlated with the FES-cohesion and FES-conflict scales and are as follows: FES-cohesion and Structure and Resources (r_mothers =−.47, r_fathers =−.52); FES-cohesion and Family Problems (r_mothers =−.65, r_fathers =−.22); FES-cohesion and Sibling Problems (r_mothers =−.19, r_fathers =−.04); FES-conflict and Structure and Resources (r_mothers = .24, r_fathers = .27); FES-conflict and Family Problems (r_mothers = .40, r_fathers = .43); and FES-conflict and Sibling Problems (r_mothers = .45, r_fathers = .13).

^*Note: p < .05,

^**p < .01,

^***p < .001.

Table IV.

Correlations Between PAT2.0 Total Score and Validation Instruments: Criterion-Related Validity

Criterion related validity
			Correlations
Construct	PAT2.0 subscale	Validation instrument	Mothers	Fathers
Family psychosocial risk	Total score	ASDS	.57***	.30*
		STAI-state	.41**	.28*
		BASC-2	.65***	.51***
		FES-conflict	.40***	.31**

Disattenuated correlations were calculated for the correlation between the FES-conflict scale and the PAT2.0 Total score (r_mothers = .56, r_fathers = .44).

^*Note: p < .05,

^**p < .01,

^***p < .001.

Criterion-related validity, the ability of a measure to predict a particular outcome, was examined by correlating the PAT2.0 Total score with measures of parent acute distress and anxiety, family functioning, and child psychological symptoms and treatment intensity. Criterion-related validity was also tested by using PAT2.0 scores to categorize patients according to the PPPHM model and comparing PPPHM groups on scores for the outcome variables. The Staff PAT scores were correlated with the PAT2.0 to examine convergent validity. Finally, discriminant validity was examined using two methods. Physician rated treatment intensity were correlated with PAT2.0 scores and Receiver Operating Characteristic Curves (ROC) were examined to determine the ability of the PAT2.0 to discriminate those parents that concurrently report clinical levels of child or parent distress.

Results

Descriptive Statistics

Descriptive statistics were calculated for the PAT 2.0 as well as the validating measures. PAT2.0 Total and subscale statistics are reported in Table I. Descriptive statistics for the validating measures are in Table II.

Table II.

Descriptive Statistics for Validation Measures for Female and Male Caregivers

	Female caregivers			Male caregivers
Measure	N	M	SD	N	M	SD
ASDS total	128	50.33	16.06	70	46.14	14.84
STAI-state	128	50.57	11.90	70	48.46	12.87
BASC-2	100	47.55	9.39	50	46.69	7.26
FES-cohesion	128	8.08	1.24	70	8.28	1.02
FES-conflict	128	2.30	1.74	70	1.76	1.71

Note: The N refers to the number of valid cases for a particular measure. N’s vary due to missing data.

Reliability

Internal consistency for the Total PAT2.0 score was strong (α =.81, Table I). For six of the seven PAT2.0 subscales an alpha coefficient of .60 or above was successfully obtained through removal of items. Due to the multidimensional nature of the Family Beliefs subscale, only four items were analyzed. The internal consistency of these four items were α =.59 but was retained in analyses for theoretical reasons (Kazak et al., 2004).

These items were chosen because they represent a subset of cancer-related beliefs, competence (See Fig. 2; items 15a, 15f, and 15h) and positive growth (item 15c), theoretically measured similar beliefs and demonstrated adequate internal consistency. Pearson Product Moment correlations indicated very good test–retest reliability for the PAT2.0 Total score for mothers (r =.78, p < .001) and fathers (r =.87, p < . 001).

Mothers’ and fathers’ scores were compared using paired sample t-tests on the PAT2.0 Subscale and Total scores. There were no significant differences, with one exception. Mothers reported significantly fewer sibling problems than fathers [t (1, 63) =−3.74, p < 01].

Content Validity

The content validity of the PAT2.0 subscales was examined by correlating PAT2.0 subscale scores with measures that theoretically assess the same content. All correlations were conducted independently for mothers and fathers (Table III). Specifically, the Family Structure and Resources, Family Problems, and Sibling Problems subscales were correlated with the FES Cohesion and Conflict Scales; the Family Beliefs and Stress Reaction subscales were correlated with the ASDS; and the Child Problems subscale was correlated with the BASC-2. For mothers, correlations were in the expected directions between the Structure and Resources subscale and FES-Cohesion (p <.05); Family Problems Subscale and the FES-Cohesion (p < .05) and FES-Conflict Scales (p < .05); the Stress Reaction and the ASDS and State anxiety scores, the Family Beliefs subscale and the ASDS (p’s < .05); the Sibling Problems and FES-Conflict (p < .05); and between the Child Problems Subscale and the BASC-2 (p’s < .001). For fathers, significant correlations were observed between the Family Structure and Resources and FES-Cohesion (p < .05), between Family Problems and FES-Conflict (p < .05); the Stress Reaction subscale and the ASDS and State anxiety scale and between Child Problems and the BASC-2.

Criterion-Related Validity

To assess criterion-related validity, PAT2.0 Total scores were correlated with outcome variables indicative of, or associated with, psychosocial risk. Maternal PAT2.0 scores were significantly correlated in the predicted directions with maternal ASDS, State anxiety, and FES-Conflict and the BASC-2 (all p’s < .01). Likewise, higher father PAT2.0 scores were significantly correlated with higher paternal ASDS, FES-Conflict, and the BASC-2, as well as lower FES-Cohesion (Table IV). The ITR-2 was not significantly associated with PAT2.0 total scores.

Next, PAT2.0 Total score cutoffs were established to classify families into PPPHM categories. Score cutoffs were determined a priori based on the PPPHM theory (Kazak, 2006) and our previous empirical evidence using the original PAT (Kazak et al., 2001). Then the cutoffs were examined to determine where they fell in the distribution of the sample scores. PAT2.0 Total Scores of <1SD above the mean were placed in the Universal category, scores between 1 and 2SD above the mean were classified in the Targeted category and scores >2SD above the mean were classified in the Clinical category. Consistent with what would be predicted by the PPPHM, based upon maternal reports, 55% of the families fell into the Universal category, 32% fell into Targeted, and 13% fell into the Clinical category. For fathers’ reports, 67% of the families fell into Universal, 32% fell into Targeted, and 1% fell into the Clinical range.

To further validate these cutoffs, scores on the ASDS, BASC-2, FES-Conflict scale, and the STAI-Y State scales were compared among the PPPHM categories. Analyses were conducted separately for mothers and fathers. For mothers, omnibus ANOVAs were significant for all measures (p’s < .05). Follow-up comparisons were conducted using Bonferroni corrections to determine which levels of the PPPHM model differed significantly from one another. Mean differences and effect sizes for each of the comparisons are listed in Table V. ASDS scores were significantly lower for mothers in the Universal versus the Targeted and Clinical groups (p’s < .05) and those in the Targeted group were significantly lower than the Clinical group. With regard to BASC-2 scores, the Universal and Targeted groups defined by mothers did not differ significantly; however both of these groups had more favorable BASC-2 scores than the Clinical group. Finally, the STAI-Y State scale was significantly lower for mothers in the Universal group compared to mothers in the Targeted or the Clinical groups. Mothers in the Targeted and Clinical groups did not differ in anxiety scores (p >.05). For fathers, comparisons were only conducted between the Universal and Targeted groups. The Clinical group only included one family, therefore precluding post hoc analyses with this group. No significant differences were observed between the Universal and Targeted groups on the ASDS, BASC-2, FES-Conflict, or the STAI-Y State for fathers.

Table V.

Mean Differences on Outcome Measures Between PPPHM Risk Categories for Mothers and Fathers

		Mothers		Fathers
Variable	PPPHM categories compared	Mean Δ	d	Mean Δ	d
ASDS		F(2,127) =25.68, p < .001		F(1, 69) = 2.92, p =.06
	Universal vs. Targeted	6.66*	.49	7.06	.47
	Universal vs. Clinical	26.86***	1.41	–	–
	Targeted vs. Clinical	20.00***	2.11	–	–
BASC-2		F(2, 99) =20.09, p < .001		F(1, 48) = 4.73, p =.01
	Universal vs. Targeted	2.99	.02	3.23	.29
	Universal vs. Clinical	15.02***	1.25	–	–
	Targeted vs. Clinical	12.02***	1.43	–	–
FES-Conflict		F(2, 125) = 9.33, p < .001		F(1, 69) = 2.70, p =.07
	Universal vs. Targeted	.46	.31	.55	.48
	Universal vs. Clinical	2.00***	.90	–	–
	Targeted vs. Clinical	1.54**	1.15	–	–
State Anxiety		F(2, 127) = 14.87, p < .001		F(1, 69) = 1.86, p =.16
	Universal vs. Targeted	8.07**	.74	5.62	.45
	Universal vs. Clinical	14.16***	.53	–	–
	Targeted vs. Clinical	6.09	1.41	–	–

Note: Cohen’s d was used as a measure of effect size. Effect sizes were not calculated between the Universal and Clinical and the Targeted and Clinical for fathers because only one father was in the Clinical category based on his PAT2.0 Total Score. Number of participants varies depending on the valid data available for each measure.

^*p < .05,

^**p < .01,

^***p < .001.

Convergent Validity

Convergent validity, the relationship between two measures purported to measure the same domain, was assessed by calculating correlations between PAT2.0 total scores and Staff PAT scores from nurses and physicians. Maternal PAT2.0 scores were significantly associated in the expected directions with both physician (r =.45, p < .01) and nurse (r =.38, p < .01) staff PAT reports, respectively. For fathers, PAT2.0 scores were significantly correlated with nurse reported Staff PAT (r =.36, p < .05) but not physician reports (r =.17, p > .05).

Discriminant Validity

As predicted, the PAT2.0 was not correlated with physician rated treatment intensity for mothers (r = −.10; p > .05) or fathers (r =−.05; p > .05). The sensitivity and specificity of the PAT2.0 to detect clinically significant outcomes was also examined using ROC Curves (McFall & Treat, 1999; Zweig & Campbell, 1993). The ASDS and the BASC-2 were chosen for the ROC analyses as both measures have established clinical or “at risk” cutoffs, (for the ASDS a Total score of 56 or greater and for the BASC-2 a Behavioral Symptom Index a T-score of 60 or greater). Here, the ROC curves resulted in areas under the curve (AUC) significantly better than .50 (the value when diagnostic performance of a measure is equal to chance) for mothers on both the BASC-2 and the ASDS and for fathers on the BASC-2. For mothers, the AUC for the BASC-2 was .94 (p < .001) and for the ASDS the AUC was .80 (p < .001). Using a PAT2.0 Total score of 1.0 for the cutoff, the PAT2.0 correctly classified 35/47(75%) of the mothers with scores above the clinical cutoff on the ASDS (sensitivity) and 60/81(74%) of the mothers who did not have ASDS scores above the clinical cutoff (specificity). Maternal PAT2.0 scores of 1.0 or above correctly classified 8/8 (100%) of the participants with BASC-2 scores over the “at-risk” cutoff and 55/92 (60%) of those scoring below the “at-risk” range. We then examined whether a PAT2.0 Total score of 2.0 or greater discriminated those that scored in the “clinically significant” range on the BASC-2 (score > 70). Indeed, PAT2.0 scores above 2.0 identified 3/3 children with BASC-2 scores above 70 and correctly classified 83/95 (87%) children that did not score in the “clinically significant” range.

For fathers the AUC for the BASC-2 was significant (AUC =.96, p < .001) but not for the ASDS (AUC =.60, p > .05). Further analyses of paternal data on the BASC-2 indicated that a PAT2.0 Total score of 1.0 or greater correctly classified 5/5 (100%) of the participants with BASC-2 scores over the “at-risk” cutoff and 30/45 (67%) of those under the cutoff. There was insufficient data to examine whether a PAT2.0 Total score of 2.0 or greater discriminated children in the “clinically significant” range on the BASC-2 (no fathers reported their child as in the BASC-2 clinically significant range).

Discussion

Despite longstanding recognition of the stressors associated with having a child diagnosed with cancer, evidence-based assessment approaches for identifying families at risk have not been available. The PAT2.0 is a standardized, reliable and valid method for medical and allied health providers to objectively and efficiently assess a family’s risk for experiencing clinically significant distress during treatment. The PAT2.0 is a second generation measure that offers a user-friendly screening approach that takes less than 10 min to complete.

Good internal consistency and test–retest reliability were demonstrated for the PAT2.0 Total Scale. Reliability for the Subscale scores were acceptable for a newly developed instrument and suggest that the items within the subscales of the PAT2.0 are relatively homogeneous and measuring aspects of the same construct without being overly redundant. Initial construct and criterion related validity for the PAT2.0 were also demonstrated. Overall, the subscales were associated with corresponding constructs in the expected directions. The PAT2.0 Total Score was also significantly associated with critical child behavioral outcomes in the expected directions. Notably, the PAT2.0 Total score was associated with parent acute distress, child behavior problems, and family conflict for both mothers and fathers. These findings support the PAT2.0 as a useful screening tool for newly diagnosed pediatric oncology patients and their families.

The PAT2.0 is linked to the PPPHM, a model that can be used to guide the development and evaluation of interventions. The data in this study support the use of the PAT2.0 to categorize families based on their level of risk where higher scores are indicative of greater levels of risk. Overall, mothers in the Universal and Targeted groups reported significantly less acute stress symptoms, fewer child problems and less family conflict than their counterparts in the Clinical PPPHM risk group. Mothers in the Universal group also reported less current anxiety than those in the Targeted and Clinical groups. Although trends were in the expected directions for fathers, PPPHM risk group did not distinguish paternal scores on key outcomes. However, sensitivity and specificity analyses support the use of a PAT2.0 Total score of 1.0 to indicate that a particular family may need some type of increased psychosocial services. Due to the limited number of participants with PAT2.0 Total scores higher than 2.0 and BASC-2 scores in the clinical range, future research with larger sample sizes will be needed to determine if the Clinical category cutoff (PAT2.0 Total score of 2.0) is sensitive to differences in degree of clinical need of families assessed with the PAT2.0. Conceivably, this could be due to a lack of power to detect differences between the PPPHM groups as the current sample of fathers was significantly smaller than that of mothers. Alternatively, fathers may respond differently to the PAT2.0 and thus may require the establishment of different cutoffs and norms. Overall, these findings provide preliminary validity for using the PAT2.0 to identify families of newly diagnosed patients most in need of more intensive levels of psychosocial intervention.

Despite the favorable psychometric properties reported in this article, further research is necessary to refine the measure further. First, the PAT2.0 subscales were theoretically derived and empirically tested on a single sample. Although parents in the sample were asked to complete the measures independently we have no way of directly verifying this. A larger multisite sample would increase the diversity if the sample thereby increase the generalizability of the instrument possibly increasing the range of observed scores. Additional higher-level analyses are needed to empirically examine the factor structure of the instrument to determine whether these subscales accurately capture the underlying structure of the measure. Going forward, the sensitivity and specificity of the PAT2.0 needs to be evaluated to fully understand its clinical utility in predicting distress. Such an evaluation should include the examination of both short-term (patients still on treatment) and long-term (patients that are posttreatment and long-term survivors) outcomes for patients and families. Although the FES subscales had low internal consistencies, the results held even after correcting for the reliability of the measures. In addition, the sample size for fathers was relatively small, limiting the conclusions that can be drawn about the data from fathers. Some of the constructs likely related to psychosocial risk, such as family beliefs, are difficult to measure and may not necessarily relate consistently to other constructs. For example, items assessing beliefs related to treatment-related suffering and death from the Family Illness Beliefs Inventory (Kazak et al., 2004) included on the PAT2.0 form were not included in the score due to poor reliability, yet these beliefs may be informative about the psychosocial risk level of the family and may be important in interventions.

Although families are under added acute strain at the time of diagnosis, psychosocial risk may be thought of as a moderately stable factor over the course of treatment. That is, pre-existing family stressors (e.g., child behavioral concerns, martial discord) are likely to persist across treatment. Although family structure and resources may shift, in most cases they are not likely to change dramatically. Future research is needed to determine if the PAT2.0 could identify specific areas of risk and directly inform the multidisciplinary treatment teams on targeted psychosocial interventions and care. Eventually, using the PAT2.0 immediately after the diagnostic meeting could allow for quick identification of areas in need of further assessment and subsequently could guide the forms of assistance offered to families such as financial and social services or consultations to prevent child or family difficulties during treatment.

Understanding the ability of the PAT2.0 to predict the “costs” associated with elevated risk is essential. That is, those families at the Clinical and Targeted levels are most likely to have ongoing difficulties that may contribute to a more difficult treatment course (e.g., personal distress of patient and or family members, nonadherence to treatment, difficult relationships with the treatment team, difficulty attending outpatient clinic visits). The previous version of the PAT showed associations between scores on the screener and subsequent utilization of social work services during treatment (Kazak et al., 2003). We expect this to be the case with this revision but will await further data to test the ability of the PAT2.0 to predict the nature and intensity of services provided by social workers, child life specialists and psychologists, as well as the added care provided by oncologists and nurses in addressing concerns related to psychosocial risk.

The PPPHM asserts that the majority of families with a child with cancer are competent and adaptively organized. However, a proportion of families have pre-existing vulnerabilities or difficulties that may be exacerbated by the diagnosis of cancer and that could result in diminished functioning and increased risk for clinical-levels of distress. In the future, the PPPHM could provide a model to guide interpretation of PAT2.0 scores and subsequently service delivery for all families of children entering the pediatric healthcare system. For example for families in the Universal category services could consist of a family centered care approach by the entire oncology team as well as standard programming that all families have the option to receive. Interventions in the Targeted category would focus on specific or acute problems that the child or family may have (e.g., interventions addressing procedural distress). Finally, interventions for families in the Clinical category would be much more resource intensive and focused, reflecting realistic goals of assuring that medical care is provided safely and that acute distress is reduced, without attempting to “cure” more severe and chronic family problems that may contribute to the presenting problem. For a more detailed description of the intervention suggestions for specific levels of the PPPHM model see Kazak and colleagues (in press). The PAT2.0 may be one tool that can be used to test this model as it provides a standardized and reliable method to identify families that may be at risk. In this way the PAT2.0 could be an essential tool for providing good preventative patient care, as well as an efficient and cost-effective approach to the allocation of psychosocial resources within the health care environment.