Abstract
The focus of these analyses was to examine the psychometric properties of the Lichtenberg Financial Decision Screening Scale (LDFSS). The purpose of the screen was to evaluate the decisional abilities and vulnerability to exploitation of older adults.
Methods
Adults ages 60 and over were interviewed by social, legal, financial or health services professionals who underwent in-person training on the administration and scoring of the scale. Professionals provided a rating of the decision making abilities of the older adult.
Measure
Ten screening items assessing inability to make financial decisions were included in the measure.
Sample
The analytic sample included 213 individuals with an average age of 76.9 (s.d.=10.1). The majority (57%) were female.
Analyses
Data were analyzed using item response theory (IRT) methodology; several forms of the measure were evaluated.
Results
The results supported the unidimensionality of the item set. Several IRT models were tested. Ten ordinal and binary items evidenced a slightly higher reliability estimate (0.85), than the other versions, and there was better coverage in terms of the range of reliable measurement across the continuum of financial incapacity for this model as compared to other models. The most informative items in this model were “What is the purpose of your decision?”, and “What is the primary financial goal?” The least informative items in the same model were: “Who benefits most from this financial decision?”; “Does this decision change previous planned gifts or bequests to family, friends, or organizations?”; and “Was this your idea or did someone suggest it or accompany you?” The shorter and/or binary versions of the instrument may be adequate, depending in the context.
Conclusions
The LDFSS performed well in terms of IRT reliability and information provided, and can be recommended as a short screen for financial decision making capability and vulnerability to exploitation.
Keywords: financial decision screening, financial abuse, competency screening, item response theory, reliability, information
INTRODUCTION
The dearth of screening instruments to assess financial exploitation was noted in a review (Ernst et al., 2014) of evidence based practices in Adult Protective Services (APS). Although screening tools are needed, the challenge of introducing new assessment instruments into APS practice is related to two barriers. First, although more APS cases are being referred, there is a tremendous national shortage of APS workers to investigate the cases (Ernst et al., 2014). Second, APS workers are not used to administering structured assessments and changing practice is often difficult. One such measure, The Lichtenberg Financial Decision Screening Scale (LFDSS; Lichtenberg et al., 2015) is unique in that it examines financial exploitation from a person-centered perspective, evaluating both the decisional abilities of the older adult and the older adult’s vulnerability to exploitation. APS worker ratings of financial transactional capacity for specific decisions using the scale were highly related to substantiation of financial exploitation.
Conceptual Underpinnings of the Lichtenberg Financial Decision Screening Scale
Development of the LFDSS was guided by two conceptual frameworks: person-centeredness and decisional abilities. These frameworks affirm the importance of assessing the older adult’s understanding of the financial decision in question, with the requirement that the older adult be able to communicate four important elements of his or her decision: choice, understanding, appreciation, and reasoning.
A Person-Centered Approach to Financial Decision Making
In working with older adults who suffer from neurocognitive disorders, the person-centered approach seeks to support autonomy by building on the individual’s strengths and honoring his or her values, choices, and preferences (Fazio, 2013). Mast (2011) describes a new approach to the assessment of persons with neurocognitive impairment, the Whole Person Dementia Assessment, which seeks to integrate person-centered principles with standardized assessment techniques. Some of Mast’s underlying assumptions are that (a) people are more than the sum of their cognitive abilities, and (b) traditional approaches overemphasize deficits and underemphasize strengths. We used these guiding principles to assess actual financial decisions or transactions that an older adult was making or wanting to make.
Decisional Abilities Framework
Appelbaum and Grisso (1988) examined the legal standards used by states to determine incapacity and identified the abilities or intellectual factors necessary to make informed decisions: choice, understanding, appreciation, and reasoning. These have since been reiterated as fundamental aspects of decisional abilities (American Bar Association [ABA] Commission on Law and Aging & American Psychological Association [APA], 2005). Indeed, the ABA/APA’s Assessing Diminished Capacity in Older Adults: A Handbook for Attorneys (2005) urges attorneys to assess the older adult’s underlying decision-making abilities whenever diminished financial judgment is suspected.
According to the decisional abilities framework, an older adult must be able to communicate choice, understanding, appreciation, and reasoning as they relate to the choice. An individual must be able to communicate his or her choice and understand the nature of the proposed decision and its risks and benefits. Appreciation is the ability to grasp the situation and its potential consequences—which may affect not only the older adult, but family members and others as well. In this vein, Appelbaum and Grisso (1988) contend that the most common causes of impaired appreciation are lack of awareness of deficits and/or delusions or distortions. Reasoning includes the ability to compare options—for instance, treatment alternatives in medical decision making—and provide a rationale for the decision or explain the communicated choice.
We aimed to build on the conceptual model of decision-making abilities described by Appelbaum and Grisso (1988) and incorporate the Whole Person Dementia Assessment approach by using both person-centered principles and standardized assessment methods. Person-centered principles allow for the fact that even in the context of dementia or other mental or functional impairments, the individual may still possess important areas of reserve or strength, such as financial judgment. The value of standardization is that it allows a domain to be assessed across time and practitioners, with the assurance that the same areas will be evaluated.
AIMS OF THE ANALYSES
The focus of these analyses was to examine further the psychometric properties of the LDFSS financial decision screening scale using item response theory (IRT) methodology. The motivation for performing IRT analyses is to determine which items are most informative. Such items are often those selected in computerized adaptive tests (CATs). Additionally, when shorter forms are developed this information is used to select the more informative items for inclusion in a short-form version. Finally, it is possible with IRT to examine reliability at different points along the latent continuum rather than to have only an omnibus statistic to describe an entire measure, given that reliability can vary depending on the location of individuals along a trait continuum.
METHODS
Sample
The analytic sample included 213 individuals. Fifty seven percent were female, the mean age was 76.9 years (s.d. = 10.1) and the average highest grade of education was 13.7 (s.d. = 2.9) with 38% of subjects who finished high school and 10% less than high school.
Item set
The measure consisted of 10 screening items assessing inability to make financial decisions. Based on their responses, individuals were categorized by the interviewer into those for whom there are major concerns, some concerns, or no concern. The response, ‘don’t know/inaccurate’ was coded in the direction of inability. Original item responses were recoded in order to facilitate the IRT analysis. In the first analysis, all items were dichotomized such that the response, ‘don’t know or inaccurate’ was contrasted with all other response categories. The general method of coding used a mixed response format, with six out of the ten items recoded into ordinal responses from no risk/impact/problem for items where the financial decision could e.g., not negatively affect the respondent, to high risk/impact/inability to make financial decisions. Regardless of how the response codes were modified, the underlying construct measured the degree of difficulty making financial decisions or financial decision incapacity.
Analyses
Tests of IRT Model Assumptions
Unidimensionality of the underlying construct, an assumption of the IRT model, was examined by merged exploratory factor analysis (EFA) and confirmatory factor analysis (Asparouhov & Muthén, 2009) with polychoric correlations using MPlus (Muthén & Muthén, 2011). Additionally, model fit was evaluated using the Comparative Fit Index (CFI) and the root mean square error of approximation (RMSEA; Bentler, 1990).
The assumption of local dependency (LD) was examined using the generalized, standardized local dependency chi-square statistics (Chen & Thissen, 1997) provided in Item Response Theory for Patient Reported Outcomes (IRTPRO), version 2.1 (Cai, Thissen, & du Toit, 2011).
IRT Model Fit
Model fit for the IRT models was examined using the root mean square error of approximation from IRTPRO (Cai et al., 2011) software.
Reliability and Information
IRT-based reliability measures were examined at selected points along the underlying latent continuum. IRT-based information functions were also calculated to determine which items were most informative at different levels of theta (Cheng, Liu, & Behrens, 2015; Teresi, Kleinman, & Ocepek-Welikson, 2000). Because the peak item information is related to the values of the discrimination (‘a’) parameters, items that are more discriminative provide more information.
IRT Models
The graded response model (Samejima, 1969) was used for estimation of item parameters. The item characteristic curve (ICC) that relates the probability of an item response to the underlying attribute (denoted θ), measured by the item set is characterized by two parameters: a discrimination parameter, proportional to the slope of the curve (denoted a), and the item location (severity) parameters (denoted b). The two-parameter logistic IRT model was used for binary items. For this model, the parameter ‘b’ defines the point on the underlying construct (theta) where the probability of endorsing an item response reaches 50%. The discrimination parameters ‘a’, describe how well items differentiate participants into impaired or not impaired, and, similar to factor loadings, how well the item relates to the construct measured.
RESULTS
Model Assumptions
The results supported the unidimensionality of the item set. The first eigenvalue (7.958) for the dichotomized item set explained 80% of the variance and the ratio of the first eigenvalue to the second was 10.0. The CFI estimate was 0.994 and the RMSEA, 0.05 indicating good unidimensional model fit. When the set of six ordinal and four binary items was analyzed, the first eigenvalue (6.785) explained 68% of variance and the ratio of the first to second eigenvalue was 7.3. The CFI and RMSEA statistics were somewhat lower 0.978 and 0.085.
LD statistics were in the acceptable range. None of the values were over 10.0, a suggested cutoff value for further investigation, for the 10 binary item set. Examining the mixed response format of four binary and six ordinal items, the highest LD statistic observed (19.7) was for the item pair, “Who will benefit most from this financial decision?” and “What is the purpose of your decision?”
IRT Model Fit
Several IRT models were tested with differing item subsets, including modified item sets: models with ten, nine, and seven dichotomized (binary) items; six ordinal and one dichotomized item; and six ordinal and four dichotomized items. RMSEA statistics ranged from 0.03 (for the model with seven binary items) to 0.10 (for the model with seven ordinal items). The RMSEA statistic was 0.08 for two models: one with ten binary items and one with ten mixed ordinal and binary items (see Table 1).
Table 1.
Item response theory (IRT) parameter estimates by model (IRTPRO) (n=213)
| Item | Ten Binary Items |
Nine Binary Items |
Seven Binary Items |
Seven Ordinal Items | Ten Items: Six Ordinal and Four Binary |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| a | b | a | b | a | b | a | b1 | b2 | b3 | a | b1 | b2 | b3 | |
| What is the financial decision you are making/have made? | 2.74 (0.60) | 1.29 (0.15) | 2.43 (0.54) | 1.33 (0.18) | Item not included | 2.18 (0.43) | 1.37 (0.20) | |||||||
| Was this your idea or did someone suggest it or accompany you? | 4.54 (1.19) | 1.32 (0.13) | 3.41 (0.84) | 1.38 (0.16) | 2.62 (0.64) | 1.49 (0.19) | 1.46 (0.24) | 0.37 (0.17) | 1.91 (0.28) | 1.86 (0.29) | 0.31 (0.14) | 1.68 (0.24) | ||
| What is the purpose of your decision? | 20.44 (1.43) | 1.03 (0.08) | Item not included | 4.36 (1.23) | 1.08 (0.15) | |||||||||
| What is the primary financial goal? | 4.28 (1.03) | 0.96 (0.11) | 4.24 (1.09) | 0.95 (0.12) | Item not included | 4.09 (1.03) | 0.89 (0.16) | |||||||
| How will this decision impact you now & over time? | 2.73 (0.56) | 1.05 (0.13) | 2.92 (0.62) | 1.02 (0.14) | 3.09 (0.69) | 1.00 (0.13) | 1.88 (0.29) | −0.81 (0.18) | 0.61 (0.16) | 1.18 (0.18) | 1.89 (0.28) | −0.81 (0.15) | 0.61 (0.15) | 1.17 (0.20) |
| How much risk to your financial well-being is involved? | 2.78 (0.57) | 1.04 (0.13) | 3.10 (0.67) | 1.00 (0.14) | 3.58 (0.84) | 0.97 (0.12) | 2.46 (0.46) | 0.38 (0.14) | 0.65 (0.14) | 1.06 (0.15) | 2.41 (0.38) | 0.38 (0.13) | 0.65 (0.14) | 1.06 (0.16) |
| How may someone else be negatively affected? | 3.20 (0.71) | 1.21 (0.14) | 3.54 (0.84) | 1.18 (0.14) | 3.16 (0.76) | 1.21 (0.15) | 2.77 (0.58) | 0.22 (0.14) | 1.21 (0.14) | 2.20 (0.34) | 0.23 (0.13) | 1.31 (0.18) | ||
| Who benefits most from this financial decision? | 5.32 (1.45) | 1.22 (0.12) | 4.84 (1.37) | 1.22 (0.14) | 5.45 (1.78) | 1.21 (0.13) | 1.53 (0.25) | −0.79 (0.19) | 1.73 (0.25) | 1.43 (0.23) | −0.83 (0.17) | 1.78 (0.27) | ||
| Does this decision change previous planned gifts or bequests to family, friends, or organizations? | 3.68 (0.91) | 1.34 (0.14) | 4.14 (1.10) | 1.30 (0.15) | 4.22 (1.20) | 1.30 (0.15) | 1.73 (0.31) | 0.46 (0.15) | 1.67 (0.22) | 1.47 (0.24) | 0.50 (0.16) | 1.81 (0.28) | ||
| To what extent did you talk with anyone regarding this decision? | 3.15 (0.75) | 1.42 (0.16) | 3.17 (0.77) | 1.41 (0.17) | 3.07 (0.77) | 1.42 (0.17) | 2.66 (1.01) | 1.50 (0.17) | 2.71 (0.95) | 1.49 (0.16) | ||||
| RMSEA | 0.08 | 0.06 | 0.03 | 0.10 | 0.08 | |||||||||
IRT Parameter Estimates
Estimated IRT parameters are summarized in Table 1. Discrimination parameters were higher overall for all three models with only binary items, compared with models including ordinal items. One item, “What is the purpose of your decision?” evidenced an unusually high ‘a’ parameter (20.44), possibly indicative of model assumption violation. As stated above, a higher LD statistic was observed in the pairing of this item as binary with the ordinal item, “Who will benefit most from this financial decision?” However, the local dependency estimates were all in the desirable range (below 10) in the model with all binary items; thus, there was no obvious reason for the high ‘a’ parameter. Models tested excluding the item resulted in mixed results overall. While the seven binary item model was considerably improved in terms of model fit, the seven ordinal item model evidenced the worst fit among all models tested.
Reasonable estimates for the discrimination parameters were observed for most items; the most discriminating item (with estimates ranging from 4.84 to 5.45) in the models with binary items was: “Who benefits most from this financial decision;” followed by “What is the primary financial goal” (4.28 and 4.4). The latter item remained the second most discriminating (4.09) in the mixed ten ordinal and binary item set. The most discriminating item in this mixed model was the binary item, “What is the purpose of your decision”, with an ‘a’ value estimate of 4.36. The item, “Who benefits most from this financial decision” did not discriminate as well as in the ordinal version.
Examining the item set coded as all binary, the ‘b’ parameters were concentrated around theta = 1, ranging from 0.95 for the item, “What is the primary financial goal?” to 1.49 for the item “Was this your idea or did someone suggest it or accompany you?” In the models with seven ordinal items or ten ordinal and binary items, the ‘b’s covered a wider range of the theta continuum starting at −0.81 for the lowest response category and ranging to 1.91 for the highest response category.
IRT-Estimated Reliability
Reliability estimates based on IRT using IRTPRO software (Cai et. al., 2011) are summarized in Table 2. The reliability estimates were limited to theta levels for which there were respondents. The overall average reliabilities ranged from 0.78 for the model with seven binary items to 0.85 for the model with ten ordinal and binary items. Across models, the item sets were most reliable in the theta ranges from 0.8 to 1.6 and from 0.90 to 0.98, reaching the highest value for the ten binary item model at theta = 1.2.
Table 2.
Item response theory (IRT) reliability estimates at varying levels of the attribute (theta) estimate based on results of the IRTPRO analysis for total sample
| Financial Decisions Incapacity (Theta) |
IRT Reliability | ||||
|---|---|---|---|---|---|
| Ten Binary Items |
Nine Binary Items |
Seven Binary Items |
Seven Ordinal Items |
Ten Ordinal and Binary Items |
|
| −1.6 | N/A | N/A | N/A | 0.69 | 0.69 |
| −1.2 | N/A | N/A | N/A | 0.74 | 0.75 |
| −0.8 | 0.54 | 0.54 | 0.53 | 0.79 | 0.79 |
| −0.4 | 0.60 | 0.60 | 0.58 | 0.84 | 0.84 |
| 0.0 | 0.74 | 0.74 | 0.70 | 0.87 | 0.87 |
| 0.4 | 0.87 | 0.88 | 0.85 | 0.89 | 0.91 |
| 0.8 | 0.95 | 0.95 | 0.93 | 0.90 | 0.94 |
| 1.2 | 0.98 | 0.97 | 0.96 | 0.90 | 0.95 |
| 1.6 | 0.95 | 0.95 | 0.94 | 0.89 | 0.92 |
| 2.0 | 0.88 | 0.88 | 0.86 | 0.85 | 0.87 |
| 2.4 | 0.74 | 0.74 | 0.72 | 0.79 | 0.81 |
| Overall (Average) | 0.81 | 0.80 | 0.78 | 0.83 | 0.85 |
Note: Reliability estimates were calculated for theta levels for which there were respondents
Information
Figures 1 to 5 depict the test and item information functions for the individual models. The test and item information for the models with binary items was limited to the theta range from about 0 to 2.4. A wider range of −2.8 to 2.8 was covered for the models with ordinal and mixed ordinal and binary items. The model with seven ordinal items was the least informative test, with the peak information = 9.11. In models with only binary items, the items “What is the purpose of your decision?” and “Who benefits most from this financial decision?” were the most informative. In the model with ordinal and binary items, the most informative items were: “What is the purpose of your decision?” and “What is the primary financial goal?” Both of these items were included as binary items. The item “Who benefits most from this financial decision?” was included as an ordinal item in this model, and provided less information in this form.
Figure 1.
Test and item information functions – Ten dichotomized items
Figure 5.
Test and item information functions – Ten ordinal and dichotomized items
The least informative items in the models with ten or nine binary items were: “What is the financial decision you are making/having made?”; “How will this decision impact you now & over time?”; and “How much risk to your financial well-being is involved?” In the model with seven binary items the least information was provided by the item, “Was this your idea or did someone suggest it or accompany you?” In the models with seven ordinal or ten ordinal and binary items, the least informative items were: “Who benefits most from this financial decision?”; “Does this decision change previous planned gifts or bequests to family, friends, or organizations?”; and “Was this your idea or did someone suggest it or accompany you?”
DISCUSSION
The results of IRT analyses supported the performance of the items in this set. The reliability estimates were high, and the information provided by most items was adequate. Reliability estimates were higher in the more impaired (financial incapacity) tail of the distribution. Examining the ten items in a binary form indicated that all items performed well and provided information. The ten ordinal and binary item version evidenced a slightly higher reliability estimate than the binary version (0.85 vs. 0.81). While the difference is not of a magnitude to recommend this version over the binary version, there is better coverage in terms of the range of reliable measurement across the continuum of financial incapacity using the mixed ordinal and binary version.
Limitations
These analyses should be viewed as preliminary because the sample size (213) although adequate, was small for latent variable modeling, and did not permit examination of measurement equivalence across subgroups such as education and race/ethnicity. The one item with a high discrimination parameter may require further investigation; however, there is no obvious violation of model assumptions.
Summary
The results presented here support the use of a brief screening measure of financial incapacity. The performance of the ten item version with both binary and ordinal items appears to perform with slightly greater precision than the other versions; however, the slight gain in reliability should be considered in the context of respondent burden. The shorter binary versions may be adequate, depending on the context. Additionally, even shorter versions may be of use. Future analyses could use these IRT parameter estimates and information functions to select items for shorter-form measures. For example, the most informative four or five items could be selected if a shorter-form measure is desirable for screening. In summary, the LDFSS screen performs well, and can be recommended as a brief screen for financial incapacity.
Figure 2.
Test and item information functions – Nine dichotomized items
Figure 3.
Test and item information functions – Seven dichotomized items
Figure 4.
Test and item information functions – Seven ordinal items
Acknowledgments
Partial funding for this study was provided by the National Institute of Justice MU-CX-0001, and the National Institutes of Health P30 AG015281, Michigan Center for Urban African American Aging Research
References
- American Bar Association Commission on Law and Aging and the American Psychological Association. Assessment of older adults with diminished capacity: A handbook for lawyers. Washington, DC: American Bar Association and American Psychological Association; 2005. [Google Scholar]
- Appelbaum PS, Grisso T. Assessing patients’ capacities to consent to treatment. New England Journal of Medicine. 1988;319:1635–1638. doi: 10.1056/NEJM198812223192504. [DOI] [PubMed] [Google Scholar]
- Asparouhov T, Muthén B. Exploratory structural equation modeling. Structural Equation Modeling. 2009;16:397–438. doi: 10.1080/10705510903008204. [DOI] [Google Scholar]
- Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107(2):238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
- Cai L, Thissen D, du Toit SHC. IRTPRO: Flexible, multidimensional, multiple categorical IRT Modeling [Computer software] Chicago, IL: Scientific Software International, Inc.; 2011. [Google Scholar]
- Chen WH, Thissen D. Local dependence indices for item pairs using item response theory. Journal of Educational and Behavioral Statistics. 1997;22:265–289. doi: 10.2307/1165285. [DOI] [Google Scholar]
- Cheng Y, Liu C, Behrens J. Standard error of reliability estimates and the classification accuracy and consistency of binary decisions. Psychometrika. 2015;80:645–664. doi: 10.1007/s11336-014-9407-z. [DOI] [PubMed] [Google Scholar]
- Ernst JS, Ramsey-Klawsnik H, Schillerstrom J, Dayton C, Mixson P, Counihan M. Informing evidence-based practice: A review analyzing adult protective services (APS) data. Journal of Elder Abuse and Neglect. 2014;26:458–494. doi: 10.1080/08946566.2013.832605. [DOI] [PubMed] [Google Scholar]
- Fazio S. The individual is the core—and key—to person-centered care. Generations. 2013;37(3):16–22. [Google Scholar]
- Lichtenberg PA, Stoltman J, Ficker LJ, Iris M, Mast BT. A person-centered approach to financial capacity assessment: Preliminary development of a new rating scale. Clinical Gerontologist. 2015;38:49–67. doi: 10.1080/07317115.2014.970318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mast BT. Whole person dementia assessment. Baltimore, MD: Health Professions Press, Inc.; 2011. [Google Scholar]
- Muthén LK, Muthén BO. M-PLUS users guide. 6. Los Angeles, California: Muthén and Muthén; 2011. [Google Scholar]
- Samejima F. Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement. 1969;34:100–114. doi: 10.1007/BF02290599. [DOI] [Google Scholar]
- Teresi JA, Kleinman M, Ocepek-Welikson K. Modern psychometric methods for detection of differential item functioning: Application to cognitive assessment measures. Statistics in Medicine. 2000;19:1651–1683. doi: 10.1002/(SICI)1097-0258(20000615/30)19:11/12<1651::AID-SIM453>3.0.CO;2-H. [DOI] [PubMed] [Google Scholar]