Correlation and Crosswalks between Patient-Reported Functional Outcomes and PROMIS Physical Function among Medically Ill Patients

Abstract

Objectives:

To develop scoring crosswalks between a new multi-domain patient-reported outcome measure, the Functional Assessment in Acute Care (FAMCAT) with a PROMIS measure of physical function and examine correlations with existing legacy instruments.

Design:

Cross-sectional, single-group design study

Setting:

Large Midwestern academic teaching hospital

Participants:

A sample of 1,885 patients (53% men, average age 62 (SD 16)) hospitalized on the general medical services between May 2016 – June 2017.

Intervention:

Not applicable.

Main Outcome Measures:

Scores from the FAMCAT administered via computerized adaptive testing were compared to scores on the 8-item PROMIS Physical Function short form.

Results:

Correlations with the FAMCAT and the PROMIS PF were strong for initial scores (MCAT_Mobilty r=0.78, p<.0001; MCAT_DailyAct r=0.81, p<.0001). The Applied Cognition scale did not demonstrate adequate correlations, thus was not a candidate for crosswalk scores. While the MCAT_Mobility scale could be initially linked, subsequent analysis did not support a valid crosswalk. Linking criteria were applied with the Daily Activity scale to developing a final concordance table.

Conclusions:

The FAMCAT Daily Activity scale yielded robust correlations to develop crosswalk scores with the PROMIS PF. The resulting crosswalk conversion metric may be useful to compare outcomes across these constructs for assessing functional abilities among patients on general medical services. The Applied Cognition and Basic Mobility scales did not meet criteria; therefore, alternate legacy instruments are needed to develop additional crosswalks.

Introduction

Preserving functional mobility among hospitalized older adults is an essential component to preventing hospital-acquired disability and functional decline.^1,2 Given the increased demand for rehabilitation services among older adults upon admission, there is a continued need for systematic, efficiently administered measures to appropriately assess physical function among acutely hospitalized, medically ill patients.³ Such measures are paramount in providing data driven, skilled recommendations to direct mobility preservation programs within acute care hospital settings.

Physical function is one of the most common constructs assessed by rehabilitation clinicians in hospital settings.⁴ Physical function can be measured in a variety of ways using several measurement techniques. With the proliferation of patient-reported outcome measures (PROM) in rehabilitation and health outcomes assessment, it is important to investigate the extent to which newly developed tools can link with previously established legacy instruments. However, the challenge then becomes how to directly compare the scores for the various assessments given the range of variability in item content, response options, and scoring rules. Researchers have developed a new PROM—the Functional Assessment in Acute Care MCAT (FAMCAT)—as a means by which to efficiently obtain data-driven, standardized assessment of a patient’s rehabilitative needs.^3,5–7

The FAMCAT is an item response theory (IRT) based tool implemented through computerized adaptive testing (CAT) that generates multidimensional scores for Basic Mobility, Daily Activities, and Applied Cognition. The FAMCAT aims to expand the breadth, depth, and precision in measuring functional limitations among patients hospitalized with medical diagnoses in acute care settings. The goal of this body of work is to integrate the FAMCAT into electronic health records in order to optimize triage and recommendations for appropriate mobility preservation programs in acute care settings. As a computerized assessment embedded in the electronic health records, the scores from the FAMCAT can be considered along with existing data to optimize and improve patient care and evaluate outcomes compared to other clinical criteria and assessments. For example, the widely adopted self-reported instrument available in the public domain is the Patient-Reported Outcomes Measurement System (PROMIS®) Physical Function item bank (PROMIS PF), and its derivative fixed short forms and CAT. The PROMIS PF item banks include the IRT calibrations on the items offering a useful infrastructure upon which to begin to compare scores with this existing legacy instrument to the newly developed tools.⁸

One advantage to developing IRT-based measures such as the FAMCAT, is that the underlying methodology allows empirical linkages across instruments yielding a common metric upon which to standardize and compare scores. The score comparisons are represented by a crosswalk table that provides score equivalencies for each instrument that is successfully linked. This crosswalk provides opportunities for clinicians and researchers alike to make efficient comparisons of outcomes of similar constructs (e.g. Physical Function) across existing clinical populations and studies.

Using an analytic process called “linking” the goal of the present study is to examine the feasibility of developing a common metric by which scores from the FAMCAT can be directly compared to scores on the PROMIS PF instrument using the PROMIS SF V1.2 – Physical Function 8B: PROMIS Physical Function 8-item short form. Additionally, construct validity is examined between two additional legacy measures to explore the extent to which the FAMCAT may provide useful information to characterize function among patients hospitalized with medical diagnoses and complications of surgery.

Methods

The following sections provide a brief description of the design, participants, sampling, recruitment, and data collection related to the FAMCAT development, calibration, and validation. Further detailed measurement methods and development information is reported in complementary articles within this supplement.^{3, 5–7}

Design and Setting

This study implemented a single-group design linking study.⁹

Participants

The overall sample for this study was obtained from the FAMCAT calibration study.³ The linking sample was a subset of 1885 individuals who were part of the FAMCAT calibration sample. Each participant took the FAMCAT in computerized adaptive testing format and subsequently the PROMIS SF V1.2 - Physical Function short form 8B. See Supplemental Table 1 for the PROMIS PF items administered for this study. A full description of the calibration sample characteristics can be found in several complementary manuscripts describing the FAMCAT instrument development and validation.^{3,5–7,16,17}

Data Collection

The present study used data collected as part of the primary FAMCAT development and calibration. Study participants were recruited from a large mid-Western teaching hospital. Eligibility criteria included admission to a medical services unit or having complex postoperative cases with readmission to the general medical service. In addition, all subjects needed to understand English as all the FAMCAT items were developed using English only to date. Purposive sampling was used to ensure adequate representation of demographic and clinical subgroups. All subjects had at least one chronic condition. Eligibility was established via medical record review to exclude subjects who may be unable to provide informed consent due to medical complications or compromised cognitive status. Detailed procedures of enrollment and consent are described elsewhere in complementary manuscripts in this supplement.^3,5–7

Data for all measures, except the FAMCAT, were collected via interview using Qualtrics software^a for item administration. The FAMCAT was delivered as a multidimensional IRT-based CAT using the FastTest Testing System (Assessment Systems Corporation, 2015^d). Demographic and clinical information was extracted from the electronic medical record system. All study procedures were approved by an academic teaching hospital ethics review board prior to study implementation.

Measures

FAMCAT

The FAMCAT is a newly developed IRT-based, multidimensional CAT that assesses three key constructs related to physical function among hospitalized populations—Basic Mobility, Daily Activities, and Applied Cognition. The item bank consists of 324 items. Items use a 4-point polytomous response scale reflecting the amount of difficulty patients report for a given task or activity for each domain, and are indexed with values from “1: unable” to “4: none” for use in IRT calibration with a multidimensional graded response model. The IRT-based FAMCAT scores for this study were converted to T-scores whereby the scores are standardized to a mean of 50 for a general medical population and SD of 10. Scores higher than the mean indicate better functioning while scores lower than the mean indicate worse functioning. In FAMCAT, each item loads on only one dimension, but the correlation between dimensions is used to set priors for scoring. Specially, we used Bayesian maximum a posteriori (MAP) to estimate examinee trait levels. A multivariate normal prior with a mean zero vector and a covariance matrix of $10\times \Sigma$ was used in MAP, where $\text{Σ}=\left(\begin{array}{ccc}1& 0.6& 0.46\\ 0.6& 1& 0.83\\ 0.46& 0.83& 1\end{array}\right)$ was obtained from the multidimensional IRT item calibration. Further details of the development, calibration, and validation of the FAMCAT are reported elsewhere in this supplement.^{3,5–7, 16–17}

PROMIS PF (8-item short form): PROMIS SF V1.2 – Physical Function 8B

We refer to the PROMIS Physical Function short form (PROMIS PF) as the reference or “legacy” measure upon which to develop the crosswalk metric. The PROMIS PF v1.0 item bank consists of 124 items that assess mobility (lower extremity), dexterity (upper extremity), axial or central (neck and back function), and complicated actions that cover multiple domains (e.g., daily living activities). Of the 124 items in the bank, we used the validated 8-item short form comprised of the most highly discriminating items in the bank that cover the entire trait range.¹⁰ The PROMIS PF 8-item short form assesses components of mobility and daily living activities (see Supplemental Table 1 for a list of items used). Because PROMIS items are not scored as sums, rather on a standardized T-score metric using IRT, scores obtained from different item subsets are readily comparable. Specifically, for the present study we used a score transformation method by which calculation of a sum score is transformed to an “IRT score metric” through a transformation table. The metric used for PROMIS is the T score, standardized with respect to mean (50) and SD (10) and centered around the U.S. general population, matching the marginal distributions of gender, age, race, and education in the 2000 U.S. Census.¹¹

Activity Measure for Post-Acute Care (AM-PAC™) “6-Clicks”

This AM-PAC short form instrument has 6 questions evaluating a person’s need for assistance in completing distinct functional mobility activities.^12,13 The 6-clicks instrument is often used to guide acute care discharge disposition using its function multidimensional function outcome scores representing basic mobility and daily activity domains. Based on clinician judgment, each question is scored on a 4-point ordinal scale, where a score of 1 indicates that the person is unable to complete the task and 4 indicates that the person is independent in completing that activity.

Johns Hopkins – Highest Level of Mobility Scale (JH-HLM)

The JH-HLM evaluates general mobility, over a fixed observation period.¹⁴ The JH-HLM is a performance measure that characterizes a patient’s highest level of mobility achieved during the assessment. Scoring is based on a person’s observed activity as a 1-item scale with 8 ordinal response options: 1 = only lying, 2 = bed activities, 3 = sitting at edge of bed, 4 = transferring to chair, 5 = standing for greater than or equal to 1 minute, 6 = walking 10 or more steps, 7 = walking approximately 7.5 m or more (25 ft or more), and 8 = walking approximately 75 m or more (250 ft or more).¹⁵

Analytic Approach

Descriptive statistics (mean, standard deviation, frequency distributions) were calculated to describe the characteristics of the sample, including age, sex, discharge disposition, and scores for the measures used. All FAMCAT scores were converted to the T-scale to facilitate comparisons across the legacy measures. It should be noted that the PROMIS-PF is standardized to a mean of 50 (SD 10) based on the US general population whereas the FAMCAT mean of 50 (SD 10) is not standardized to the US population but within the clinical population used for calibration.

To have successfully linking, there must be a strong relationship between linked instruments both conceptually and empirically. Item content between PROMIS PF and FAMCAT were examined to assess concept similarity. Additionally, score distributions and scatterplots of the scores were generated to examine score distributions of the various instruments. The empirical strength of the relationship between two linked instruments was assessed by calculating the Pearson correlation of PROMIS PF and FAMCAT scores.¹⁸ We then calculated the root expected mean square difference (REMSD) to examine the population invariance.¹⁹ The REMSD is the difference of the standardized difference of subpopulations (e.g., age, or gender) weighted by the sample size differences in subpopulations. The standardized difference within subpopulation should be similar across the instruments. The distributions of REMSDs based on 500 bootstrap samples were calculated to estimate the 95% confidence intervals of REMSD. The 95% confidence intervals were generated by calculating the 2.5 and 97.5 percentiles of this distribution.

For scales that did not meet the criteria of a REMSD less than 0.08, further analysis was conducted to examine the impact of violation of population invariance for subgroups. Linking was then created for each subgroup separately based on Kernel=based equipercentile linking methods. Differences in linking scores across subgroups were calculated and the bootstrap method was used to generate the 95% confidence intervals (CIs) of the score differences. Statistically significant score differences were identified as the 95% CIs not including 0.

To link the FAMCAT and PROMIS PF scales the method implemented was the Equipercentile method, creating a linkage at the score level rather than the item level. This was chosen as the optimal method, provided the data for the FAMCAT was collected using CAT (rather than the full item banks in which an IRT-linking method could be used). Previous work has shown method equivalence when linking PROMIS PF metrics between IRT-based linking and Equipercentile methods.⁸ The percentile ranks from FAMCAT and PROMIS PF scores were calculated, then the scores from each scale with same percentile ranking were aligned. For example, the FAMCAT score of 40 was linked with PROMIS PF score 33 because both have the same percentile ranking (about 25) within each scale.

We conducted four different equipercentile linking methods: the equipercentile linking without post-smooth (EQP no SM), with cubic-spline post-smoothing at 0.3 and 1 (EQP SM=0.3, EQP SM=1.0). Post-smoothing is the process of applying cubic splines to the raw equipercentile equating function to minimize the sampling error, the degree of smoothing is controlled by the smoothing parameter SM (SM=0.3 less smooth, SM=1.0 more smoothing).²⁰ The analyses were performed in LEGS program.²¹ Another method is the Kernel equipercentile (EQP Kernel) linking. First, the polynomial log-liner model (power of 3) is used to fit the proportion of the raw data to generate the score distribution and reduce the random error (pre-smoothing). Second, the score probabilities are calculated from the estimated score distribution, and Gaussian Kernel is used to convert the discrete cumulative distribution function (CDF) to a continuous CDF. Finally, equipercentile linking is used to create the linkage between scores. The different linking methods were compared by calculating the standard error of equating scores, and the bias and root mean squared error (RMSE) between the PROMIS PF score and converted PROMIS PF scores from FAMCAT. The standard errors of equating scores were estimated by calculating the standard deviation of equating scores from 500 bootstrap samples. Upon evaluation of these techniques, the optimal linking method with the lowest standard error of equating, bias, and RMSE was chosen to develop the final crosswalk conversion table. All FAMCAT scores were then converted onto the PROMIS PF score metric.

To examine initial validity of the linking method, we looked at the descriptive statistics and examined the mean score difference of PROMIS PF score and converted score. Score agreement was examined by calculating the intraclass correlation coefficient (ICC3,1). Cumulative score distributions were generated and then the Kolmogorov-Smirnov Two-Sample Test was applied to examine the difference in score distributions.^22,23 Additionally, Bland-Altman plots were generated to examine the variation of the score difference between the original and converted PROMIS PF scores at each average score level. Analyses were completed using SAS statistical software^b, and the Kernel-based Equipercentile linking method utilized analyses using “kequate” (R program^c).²⁴

Lastly, complementary analyses were conducted to further examine convergent validity of the FAMCAT with additional legacy measures. Pearson correlations were calculated comparing the FAMCAT with the 6-Clicks and JH-HLM Scale instruments. Large correlation coefficients (>.60) were interpreted as evidence of convergent validity.²⁵

Results

The overall sample included 1,885 participants who had complete data for both measures being linked (FAMCAT and PROMIS PF). The mean age of the sample was 67.8 years of age, 54% male, and the majority of the subjects (81.3%) were discharged to home with self-care (See Table 1). The average score of participants in the sample using the PROMIS PF was 36.0 (SD 9.0), indicating that the sample is approximately 1.5 SD below the US average in physical function. For the FAMCAT measures, the sample demonstrated similar functional levels for each scale (see Table 1).

Table 1.

Sample Characteristics (N=1885):

Characteristic	Categories		n (%)
Age; mean (SD)			61.78 (16.09)
Sex	Male		1014 (53.79)
	Female		871 (46.21)
Discharge disposition	Acute Care Hospital		2 (0.11%)
	Critical Access Hospital		1 (0.05%)
	Expired		10 (0.53%)
	Home with Self-Care w/ Planned Readmission		1 (0.05%)
	Home with Self-Care		1533 (81.33%)
	Home-Health Care Svc		106 (5.62%)
	Hospice - Home		15 (0.8%)
	Hospice - Medical Facility		5 (0.27%)
	Left Against Medical Advice/Discontinued Care		1 (0.05%)
	Long Term Care Hospital		1 (0.05%)
	Psychiatric Hospital		1 (0.05%)
	Rehab Facility		16 (0.85%)
	Rehab Facility w/ Planned Readmission		1 (0.05%)
	Skilled Nursing Facility		174 (9.23%)
	Transitional Care Unit		17 (0.9%)
	UNKNOWN		1 (0.05%)
Measure	Mean	Standard Deviation	Range (% Floor; % Ceiling)
PROMIS PF*	36.04	9.02	20~60(Floor: 7.59%,ceiling: 2.76%)
FAMCAT** Applied Cognitive	49.27	12.79	10~90(Floor: 0%,ceiling: 0.53%)
FAMCAT** Basic Mobility	45.97	11.39	10~90(Floor: 0%,ceiling: 0%)
FAMCAT** Daily Activity	47.52	10.32	10~90(Floor: 0%,ceiling: 0%)

Table note: PROMIS PF is calibrated using a t-score metric with mean = 50, SD 10 relative to the US population. To calculate the floor and ceiling, we used 8-item physical function short form, the lowest and highest T score in the score transformation table are 20 and 60, so we defined the floor and ceiling as T score at 20 or 60. FAMCAT scores are calibrated using a t-score metric with a mean = 50, SD 10 using the study calibration sample. For calculating FAMCAT floor and ceiling, we defined the T score below the lowest (<10) or above the highest (>86 Applied cognitive, or >90 for Basic Mobility and Daily Activity) possible scores from MCAT algorithm as floor or ceiling.

The initial review of the item content by clinical research team members yielded consensus that items from the FAMCAT Basic Mobility and Daily Activity scales conceptually aligned with the construct being measured in the PROMIS PF tool. The content in the FAMCAT Applied Cognition appeared beyond the scope of the physical function construct. These conceptual linkages were supported by the correlation results (Applied Cognition r = 0.28; Basic Mobility r = 0.81 and Daily Activity r = 0.78; all correlations were significant at the alpha 0.05 level). The Basic Mobility and Daily Activity scales met the correlation criteria (> 0.70)¹⁸ for linking and comparing the scores at the group level, but did not meet the criteria of correlation >0.866 for linking with the same constructs^26–28; therefore, further concordance table linking analysis was developed for crosswalk scores from FAMCAT to PROMIS PF.

The REMSDs of PROMIS PF and FAMCAT Basic Mobility for age-related difference (age variable was dichotomized to subpopulations based on median age 64) and for sex-related difference were Age: 10.6% (95%CI: 7.8%~13.5%) and Sex: 0.82% (95%CI: 0.05%~3.8%). REMSDs of PROMIS PF and FAMCAT Daily Activity for age-related difference and for sex-related difference were Age: 6.5% (95%CI: 3.9%~9.3%) and Sex: 4.6% (95%CI: 1.9%~7.5%). REMSD values of FAMCAT Daily Activity were less than 8%, which supports the population invariance assumption.⁹ However, the REMSD for FAMCAT Basic Mobility/PROMIS PF in age categories was 10.6%, which is greater than the 8% criterion.^9,19,26 The standardized mean difference between two age subgroups (<64, >=64) in the FAMCAT Basic Mobility score was 0.35, but this value in PROMIS PF score was 0.13. To address this finding, further examination included generating linking scores from FAMCAT Basic Mobility to PROMIS PF in two age groups separately to confirm that the linking score differences were statistically significant. Based on these results, the Basic Mobility domain did not meet the population invariance assumption for further linking steps (see Figure 1). All final linking procedures involved only the FAMCAT Daily Activity scale with the PROMIS PF scale.

Figure 1.

The x-axis is the FAMCAT mobility T score, y-axis is the difference of converted PROMIS PF T-scores generated in age<64 and age>=64 groups, the pink area is the 95% confidence band of the converted score difference.

Results of the FAMCAT Daily Activity scale presented little difference in linking PROMIS PF scores across different methods for most of the score range (Figure 2). However, larger differences emerged at the upper and lower bounds of the score range. Additionally, findings provided evidence for the EQP Kernel method as having the lowest standard error of equating in most of the score levels compared with other methods—little bias of the converted scores (less than 0.1), and RMSE was less than 6 score points. (See Supplemental Figures S1 & S2). Based upon these results, the Kernel Equipercentile linking method was applied to validate the linking.

Figure 2.

The cumulative score distributions of PROMIS and converted PROMIS scores

Results examining initial validity of the linking process supported development of crosswalk scores between the FAMCAT Daily Activity scale and the PROMIS PF short form (See Supplemental Table 2 Crosswalk Table Conversions). Figure 2 illustrates the cumulative score distributions of the PROMIS PF and converted PROMIS PF scores. The mean, standard deviation, and the range of converted PROMIS PF scores (36.06, SD 8.93, 18.81~60.67), are matched with the observed PROMIS PF score (36.04, SD 9.02, 20~60). The difference in mean score is −0.02 and there was no statistically significant difference in mean of the original and converted scores (Wilcoxon signed-rank test: p=0.96). The ICC3,1 = 0.82 (95% CI = 0.79–0.84) indicated good score agreement. The results do indicate the presence of some degree of statistical difference in observed and converted PROMIS PF score distributions (Kolmogorov-Smirnov Two-Sample Test (Asymptotic), p=0.0037); however, the cumulative distribution functions of the scores were aligned well with each other. Additionally, as seen in Figure 3, Bland-Altman plots, 53% of the score difference between original and converted scores were less than 5 (half of the standard deviation of the score) with less variation in score difference at the ends of the scale as compared to the middle of the scale.

Figure 3.

Note: The score difference between PROMIS PF and converted PROMIS PF scores at Y axis, and the average of original and converted scores at X axis. Both metrics use a t-score. The solid line is the mean score difference between converted and observed scores, the dash lines are the upper and lower limits of agreement (LOA) (mean score difference +/−1.96 standard deviations (−10.7~10.67)). The size of the circle represents the sample size at that score point

In terms of examination of the relationship of the FAMCAT scales to additional legacy measures, results indicated low to moderate correlations with the 6-Clicks and JH-HLM Scale instruments (See Table 2). The FAMCAT Basic Mobility scores presented the highest correlations among the scales (r = .57; r=.36 respectively). This finding supports similar construct validity with these instruments as we hypothesized only moderate correlations at most given the breadth and depth of content included in the FAMCAT as compared to these more narrowly defined legacy measures.

Table 2:

Correlations of FAMCAT to 6-clicks and JH HLM scale

FAMCAT Scale	6-Clicks		JHHLM Scale
	r	n	r	n
Applied Cognition	0.23	956	0.16	682
Daily Activity	0.47	956	0.28	682
Mobility	0.57	956	0.36	682

Note: All correlations significant at the <.0001 level

Discussion

The goals of this research were to (1) assess the feasibly of developing crosswalk scores between the newly developed FAMCAT tool with the PROMIS Physical Function short form; and (2) explore further construct validity using analogous legacy measures. Results from this study demonstrate the feasibility and validity of developing the concordance table for the FAMCAT Daily Activity scale with the PROMIS PF short form physical function assessment. The scores generated from the FAMCAT Daily Activity scale can be converted to PROMIS PF equivalents, thus expanding opportunities to evaluate treatment outcomes from the patient perspective as well as help direct care in recommending optimal mobility preservation programs. By using the crosswalk condordance table, these results will allow clinicians and investigators to compare outcomes across clinical and research applications in which the FAMCAT Daily Activity scale and PROMIS PF instruments are used.

While creating the concordance table of the FAMCAT Daily Activity scale was successful, the crosswalk analysis was not able to be performed for the Mobility and Applied Cognition scales. Findings indicated that the Basic Mobility and Applied Cognition scales were limited in the extent to which these scales measure similar underlying constructs as well as limited in meeting underlying assumptions required for empirically linking health outcomes measures. The Applied Cognition scale did not meet the first linking assumption of having adequate correlations between the scores. This result was expected given conceptual differences in the target constructs of the FAMCAT Applied Cognition and the PROMIS PF measures. While the correlation of the FAMCAT Basic Mobility scale was sufficient to proceed with initial linking procedures, the subsequent analysis testing population invariance did not support further development of a valid crosswalk; further research is need to create a concordance table for each sub-demographic group.

In closer examination of the item content within the PROMIS PF short form and FAMCAT instrument, we found that the PROMIS PF content aligns more with content representing functional abilities in completing activities of daily living and instrumental activities of daily living versus that of basic mobility or applied cognition. Specifically, only two of the short form items are more narrowly defined mobility items referencing walking or climbing stairs, whereas the remainder of the items are situated in the context of performing a more complex task such as lifting and carrying groceries, doing yard work, heavy work, or other chores.

Furthermore, upon investigating the relationship between the FAMCAT and the 6-Clicks Basic Mobility tool and JH-HLM Scale, we found that while the FAMCAT measures similar constructs as these legacy tools, it offers unique content and expands upon what is currently available. The correlations between these instruments were low to only moderately correlated. When examining the item content of these legacy instruments, the 6-Clicks and JH-HLM Scale tend to assess function at a much lower threshold (i.e. bed mobility, basic sit to stand transfers, walking in hospital room) than that of many of the items in the FAMCAT. Most of the participants in the present study did not endorse these low-function items; therefore, the data were limited in the extent to which valid linking scores could be estimated for the FAMCAT scales.

This study’s findings suggest a potential benefit of expanding the breadth of the FAMCAT item banks at the upper and lower ends of the scales to increase the robustness of the crosswalks with commonly used legacy tools as well as identify effective sampling techniques to include more patients with higher degrees of functional limitations. Such future analytic and replenishment work is needed to increase the ability to crosswalk and facilitate reconciliation of measured constructs such as Applied Cognition and Basic Mobility across existing as well as newly developed instruments.

Study Limitations

The approach used to crosswalk the FAMCAT Daily Activity scores with PROMIS PF scores in this study has several strengths. Most notably its approach in using a single-sample design with sequential analysis of the linking validity for each scale—however, some limitations should be noted. The score correlation between FAMCAT Daily Activity scale and PROMIS PF was 0.78, which is not strong enough (>0.866) for individual level score comparison; nevertheless, this correlation is sufficient for group level comparison and for developing the concordance table to crosswalk score from different instruments. Recent literature has suggested that correlations of (0.75–0.80) are appropriate for health outcomes measures linking as the original linking criteria were developed and implemented in a different context including high-stakes educational fields.^26,29 Additionally, literature suggests that when using single-sample design linking approaches the results may be sensitive to population differences.^19,22 For this study, the same sample was used to both develop and validate the linking. Future work is needed to evaluate the robustness and validation of the linking relationships on additional independent samples—especially populations that have higher concentration of scores at the lower and upper bounds of the FAMCAT scale’s continuum. A further limitation is the absence of methods to limit or estimate the “fatigue effects” that members of the study cohort, being infirm and hospitalized, may have experienced in responding to the set of items. Such response set effects may have potential for impacting the responses to those measures that are presented first or later in the data collection process.

Conclusion

This study demonstrated successful creation of score linkages between the FAMCAT Daily Activity scale and PROMIS PF short form scale. The single-sample design approach allowed rigorous evaluation of the validity of developing crosswalk scores between the two instruments. Practical products of this work include a large item bank of item parameters anchored to the PROMIS PF T-score metric whereby summary scores from legacy instruments can be cross-walked to the PROMIS PF scores, allowing researchers and clinicians to aggregate and compare group level scores across studies or clinical populations using the FAMCAT tool.

Suppliers.

1. Qualtrics (2019) Provo, Utah, USA Available at: https://www.qualtrics.com

2. SAS Institute. SAS users guide, version 9.4. Cary, NC: SAS Institute, Inc.; 2013

3. R Core Team (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

4. Assessment Systems Corporation (2015), Stillwater MN. https://www.assess.com

Abbreviations:

PROM

Patient-Reported Outcome Measure

FAMCAT

Functional Assessment in Acute Care

MCAT IRT

Item response theory

PROMIS PF

Patient-Reported Outcome Measure Information System Physical Function

JH-HLM

Johns Hopkins – Highest Level of Mobility Scale

REMSD

Root Expected Mean Square Difference Root Mean Squared Error (RMSE)

EQP

Equipercentile method

EQP SM

Equipercentile smoothing method

EQP Kernel

Equipercentile kernel method

ICC

Intraclass correlation coefficient