Assessing the valuing process in Acceptance and Commitment Therapy: Experts’ review of the current status and recommendations for future measure development

Abstract

Within Acceptance and Commitment Therapy (ACT), personal values provide the foundational framework of the therapeutic process and are considered necessary to facilitate targeted behavioral movement and a more vital, meaningful life. Considering the proposed nature of values as a core mechanism of change in this way, a thorough understanding of the therapeutic valuing process through which targeted changes occur is essential to evaluate the true efficacy of the ACT model empirically and implement it most effectively. However, to date, development of measurement tools for this purpose is limited and those that do exist are often inconsistent in their targeted constructs. The current study collected in-depth, descriptive data from ACT experts to critically examine how the valuing process in ACT is currently defined and measured and make recommendations for future measure development. 11 experts participated in semi-structured interviews and responded to topic guided questions. Thematic analyses of experts’ responses were then conducted, and eight core themes were identified. Findings denoted that experts’ definitions of the valuing process and its core components are largely consistent with theoretical conceptualization of ACT but that no measurement tool to date provides a complete and adequate assessment of this process. Experts expressed the need for measures capable of assessing essential contextual aspects of the valuing process and gaining a more proximal evaluation of values-consistent behaviors and experiential momentary awareness. The current study provides insight into critical areas for improvement and provides recommendations for future measure development for the empirical assessment of the valuing process in ACT which is crucial for evaluating the role of values work as a therapeutic mechanism of action.

1. Introduction

Acceptance and Commitment Therapy (ACT) is a well-established third wave intervention which posits the use of six interrelated processes (i.e. acceptance; cognitive defusion; present moment awareness; self as context; values; and committed action) to target an increase in psychological flexibility (i.e. the ability to contact the present moment fully and consciously, and change or persist in behavior(s) in the service of one’s chosen values; (Hayes, Luoma, Bond, Masuda, & Lillis, 2006). While a complete review of the ACT model and each core process is outside the scope of the current paper, several can be found within previous literature (e.g. Hayes, 2004; Hayes et al., 2006). What is important to emphasize however, is that each therapeutic process is interrelated with the others and all are employed to establish the common goals of increasing one’s psychological flexibility and creating an alternative context where behavior and decision making are consistently done in alignment with one’s chosen values (Hayes et al., 2006).

1.1. The valuing process in ACT

Although valuing is considered an essential process within the ACT model, it has not yet been well articulated or evaluated empirically throughout ACT literature. Within ACT, values are defined as “freely chosen, verbally constructed consequences of ongoing, dynamic, evolving patterns of activity” (Dahl, 2015). The therapeutic process is aimed at helping clients identify and clearly define what is important to them (i.e. their chosen values), and provide a context in which one is more willing to experience difficult thoughts or feelings and make specific behavioral changes that move them toward a more values consistent life (Dahl, Plumb, Stewart, & Lundgren, 2009; Lundgren, Luoma, Dahl, Strosahl, & Melin, 2012).

Empirical evidence supports values consistent living as a meaningful therapeutic target generally (Bardi & Schwartz, 2003; Schwartz, 1999; Verplanken & Holland, 2002) as well as within the context of ACT intervention studies (e.g. Lundgren, Dahl, & Hayes, 2008; Vowles & McCracken, 2008). While the valuing process is theoretically considered to be a core therapeutic mechanism of change within ACT (Hayes, 2004; Verplanken & Holland, 2002), the exact mechanism(s) driving targeted increases in values consistent living are less empirically understood or widely agreed upon. Across various domains of research, several different mechanisms have been hypothesized such as priming or conscious awareness (Bardi & Schwartz, 2003; Maio, Pakizeh, Cheung, & Rees, 2009), the centrality of a value to one’s self concept (Verplanken & Holland, 2002), or socially normative pressures (Bardi & Schwartz, 2003; Ryan & Deci, 2000). Evidence to date has suggested that each of these mechanisms may uniquely contribute to the influence of values on one’s behavior –but whether additional mechanisms exist and how each specific mechanism individually contributes relative to others specifically within ACT interventions remains unclear. Identifying which, if any, of these mechanisms are contributing as active components when conducting values-based therapy work within ACT interventions promoting an increase in values consistent living would therefore allow for more targeted training, intervention, and assessment of the valuing process within ACT and promote more effective therapeutic change.

In addition to identifying the active components within the valuing process, ways to accurately assess this process at multiple timepoints during therapy are necessary to empirically evaluate the efficacy of ACT-based interventions and assess client progress over time. However, despite this importance, very few measures currently exist that assess even one of the hypothesized components of the valuing process in ACT, and even fewer exist attempting to assess the valuing process as a whole. Among the empirical research that has attempted to examine the valuing process and/or any of its components, most measures to date have sought to assess levels of value-consistent behavior primarily as a treatment outcome (e.g. the Valued Living Questionnaire (VLQ); Wilson, Sandoz, Kitchens, & Roberts, 2010). However, Wilson et al. (2010) correctly denote that the development of measures capable of effectively evaluating valuing as a process consistently throughout ACT interventions would not only inform research, practice, and, ultimately, the validity of the ACT model but could also assist in demonstrating the potential importance of valuing processes in relation to positive outcomes of psychotherapy more generally.

1.2. Current study

The current study sought expert feedback to critically examine the current empirical approaches for measuring the valuing process in ACT and make recommendations for future measure use and/or development. Experts’ definitions of the essential components making up the valuing process, their opinions on how efficiently and accurately this process is currently being measured, and/or ways in which they feel the field can work to improve this measurement in the future were collected through a semi-structured interview process.

2. Methods

A series of semi-structured interviews were conducted with ACT experts to 1) establish how experts conceptualize the valuing process, 2) evaluate whether current measurement approaches adequately assess the valuing process, and 3) obtain their recommendations for future measurement of this process when conducting ACT empirical work.

2.1. Participants

Individuals were considered “ACT experts” and eligible to participate if they were: a) a masters or doctoral level clinician specifically trained in the implementation of ACT and currently utilizing ACT as the primary method of therapeutic treatment with at least one individual and/or group of patients; and/or 2) had published at least two peer reviewed journal articles relating to the use of ACT with a clinical population, valuing, or values measurement in ACT within the past 5 years.

2.2. Sampling procedures

Approval for the current study was obtained by Drexel University’s Institutional Review Board. Clinicians and researchers were recruited between December 2016 and April 2017 from psychological clinics, universities, private practices, and advertisements for ACT clinical trainings. The study was also advertised through the professional mailing listserv of the Association for Contextual Behavioral Science’s (ACBS) requesting participants. Most participants however, were recruited through individual email contact by the first author requesting participation. Individuals were specifically contacted if a) they had published a recent book or peer reviewed article related to the role of values in ACT; b) were the author of a commonly used ACT values measure; c) were a practicing clinician stating they used ACT, or d) if they were identified as a peer reviewed ACT trainer and were able to speak fluent English. Practicing clinicians and ACT trainers were randomly selected from ACBS’s ‘find an ACT therapist’ and ‘find an ACT trainer’ links. Many individuals met more than one eligibility criterion and were only contacted once if their contact information was identified in multiple search processes. Considering the use of qualitative thematic analyses within this study, advertisements and individual contacts continued until data saturation was achieved. In total, five authors of commonly utilized ACT values measures, 20 denoted ACT trainers (all of whom were also listed as ACT clinicians and/or had recently published a peer reviewed article related to the role of values in ACT), and 15 ACT clinicians were contacted individually. Two participants were recruited through the ACBS advertisement and wo measure authors, six trainers, and one clinician agreed to participate following individual contacts prior to saturation being reached.

2.3. Data collection

Data collection and analyses followed the COREQ guidelines for reporting qualitative research (Tong, Sainsbury, & Craig, 2007). Individual semi-structured phone interviews were all conducted by the first author using topic-guided questions to ensure discussions addressed the overarching research aims of the current study. Individuals provided written consent to participate prior to the phone interview and were then reminded that the interview would be audio recorded and asked to provided verbal consent again once again at the start of each interview. Interviews lasted 60–90 min and were audio-recorded with permission by interviewees. Target questions and probes were developed to facilitate conversation around the experts’ conceptualization of the valuing process in ACT and implementation of this process within a clinical context. Further, each expert was asked about his or her thoughts and opinions regarding measurement of the valuing process currently and how it may be improved in the future. Experts were asked to respond to questions utilizing both a clinical and empirical perspective when relevant to provide pertinent information for future measurement tool development in both domains. Data saturation, or “the point in data collection and analysis when new information produces little or no change in emerging themes” (Guest, Bunce, & Johnson, 2006) was used to determine when to cease recruitment and the individual interview process.

Questions regarding values measurement were broken into three sections: 1) opinions about the current state of measures available to assess the valuing process, 2) thoughts on ideal measurement designs for assessing the valuing process, and 3) specific thoughts and comments relevant to commonly utilized values measures in ACT: the Valued Living Questionnaire II (VLQ-II: Wilson, 2009) and the Bulls-Eye Values Survey (Lundgren et al., 2012). The VLQ-II was chosen for reference during interviews rather than the original VLQ despite the latter being utilized far more frequently in ACT empirical work due to its elaborated assessment including several additional aspects of valuing (e.g. level of satisfaction with one’s recent action within a valued domain; level of concern that one will not make progress within that domain) while still retaining all of the assessment items within the original version. Considering the overall purpose of interviews being to garner feedback from experts broadly on measurement of the valuing process, the widened scope of the VLQ-II was considered more useful to depict a wider array of possible assessment domains. Additionally, the semi-structured nature of interviews allowed experts to provide commentary on their experiences using the original VLQ as opposed to the VLQ-II when pertinent to the discussion. Experts were sent a copy of the VLQ-II and the BEVS 24 h prior to their phone interview and asked to have each available during this portion of the discussion.

2.4. Qualitative data analysis

All interviews were transcribed verbatim by the first author, a trained masters level research assistant or a trained undergraduate research assistant. Thematic analyses were then conducted to synthesize the qualitative data through the identification and description of major patterns across responses into identifying themes (Braun & Clarke, 2006). Thematic analyses began with the independent coding of each transcript by the first author and a trained master’s level research assistant. Codes were then reviewed independently and grouped into initial themes by each coder. Themes were defined as significant ideas capturing something important about the data in relation to the overarching research question(s) driving the study and/or representing consistent patterns across individuals’ responses. All identified themes were then reviewed by the first author for discrepancies or inconsistencies in categorization requiring clarification. Inconsistencies were discussed collaboratively between the two coders and a doctoral level psychologist with expertise in ACT who was not interviewed for the current project. Major themes and sub-themes were finalized and agreed upon by the research team to illicit an overall depiction of the qualitative data set. Core themes are identified within the results section below, some containing a unique set of sub-themes which were then classified at a further level of description.

3. Results

A total of 11 ACT experts (five females, six males) were interviewed for the current study. While inclusion criteria were intentionally broad due to concerns that recruitment of this population may be difficult due to rigorous and often fluctuating schedules, our final sample of experts had an extensive knowledge of the theoretical, empirical, and/or clinical underpinnings of ACT. Experts were all doctoral-level clinical psychologists and represented three different countries. Ten were licensed psychologists and nine were actively seeing at least one patient at the time of their interview. Most experts (n = 8) were currently working within an academic/research setting; two within a solely clinical setting; and one split her professional hours 50/50 between research and clinical work. Nine indicated having a secondary profession of either a private practice or in a training role (e.g. directing a training clinic; facilitating ACT based trainings), however all nine indicated that at the time of the interview, they more often had the opportunity to use ACT empirically in their own professions. All 11 experts stated that they had been using ACT as their primary therapeutic modality for 6–20 + years, with five stating that it was the sole modality they implemented.

Eight core themes were identified across interview responses from experts, categorized among the three research aims of the current study (outlined in Table 1 below).

Table 1.

Core themes identified within feedback from ACT experts.

Interview target	Themes
1. Understanding the valuing process	1. Valuing is a multifaceted process
	a. Component 1: Values Identification/Construction
	b. Component 2: Identifying Values-Consistent Behaviors
	c. Engagement in a Values-Consistent Behavior
	2. ACT processes are interrelated and interdependent which may limit the efficacy or utility in adequately assessing each independently.
2. Current measurement of the valuing process in ACT	3. Current measurement tools oversimplify the valuing process.
	4. Current measures do not evaluate one’s level of awareness of an inherently rewarding experience when actively engaging in values-consistent behaviors.
	5. Current measures do not recognize or evaluate the individualized nature of the valuing process.
3. Recommendations for future measures of the valuing process	6. Future measures should be more comprehensive.
	7. Future measures of the valuing process should assess the momentary and experiential aspects of valuing.
	8. Future valuing measures should be capable of assessing the individualized nature of one’s valuing process.

3.1. Understanding the valuing process

Prior to evaluating current measurement of the valuing process, experts’ conceptualization of what this process includes were gathered. Particular focus was given to whether there were any discrepancies between experts’ conceptualization of this process/the essential components for promoting targeted therapeutic change and the theoretically distinguished components of the valuing process within current ACT literature. Overall, two major themes emerged: (1) Valuing is a multifaceted process and (2) ACT processes are interrelated and interdependent which may limit the efficacy or utility in adequately assessing each independently, that were largely consistent with current ACT theory. They are discussed briefly below.

Theme 1.

Valuing is a Multifaceted Process. All experts agreed that valuing in ACT is a multifaceted therapeutic process that includes several essential components to be implemented effectively. Three components emerged repeatedly as essential: (1) Values Identification/Construction; (2) Identifying Values-Consistent Behaviors; and (3) Active Engagement in and Awareness of Values-Consistent Behaviors. These components are largely consistent with the theoretical conceptualization of the valuing process across ACT literature (Dahl, 2015; Wilson & Murrell, 2004), and are defined in Table 2 below.

Table 2.

The core components of the valuing process in ACT as defined by experts.

Component	Definition
Values Identification/Construction	Assisting clients in identifying what they care about most within their life and then working with them to construct a mental and verbal understanding of these things (i.e. one’s values) such that they are consistent with an ACT definition of ‘values’ (i.e. they are personal meaningful and generate long-term behavioral direction).
Identifying Values-Consistent Behaviors	Working with clients to determine what behaviors or patterns of action they can engage in that will truly evoke a sense of personal meaningfulness due to movement toward an identified valued direction.
Engagement in Values-Consistent Behaviors	Purposeful and consistent translation of identified values-consistent behaviors into action within their everyday lives. Individuals should regularly engage in these behaviors and seeks out opportunities to integrate them into daily routines/habits, while practicing present moment awareness of the values-consistent functions and how they elicit a sense of personal meaning.

While the overarching components of the valuing process identified were consistent with ACT theory, experts highlighted several fundamental qualities of these components they felt are often overlooked in the empirical assessment of this process. Regarding values identification/construction, it was widely agreed that the specific verbal content of the chosen value should not be the primary focus of the assessment. Rather, the functional quality of the verbally stated value should be evaluated for its appetitive, personally meaningful quality that establishes a behavioral pattern which is ‘inherently reinforcing’ to the individual. Additionally, one’s present moment awareness of the value while actively engaging in values-consistent behaviors was identified consistently as important in order to fully experience the value’s appetitive quality. Experts therefore stated that one’s level of awareness of his or her value(s) in these moments may also be beneficial for empirical assessment.

Consistent with this functional aspect of values, experts emphasized that it is essential that behaviors defined as “values-consistent” truly elicit a sense of vitality and personal meaningfulness for the individual simply because they move one in a valued direction. Therefore, when assessing this aspect of the valuing process, experts stated that it was important to extend beyond a list of behaviors the individual states are consistent with his or her values and assess why they are to determine that they are actually values-consistent behaviors from an ACT perspective. For example, Expert 9 stated that questions such as “Does [engaging in the behavior] make you feel more alive?” or, for younger clients, “Does it feel like it’s giving you life?” are profoundly useful when evaluating this clinically, and therefore suggested similar questions within empirical assessment may be useful.

Experts also highlighted two additional qualities fundamental to the identification of values-consistent behaviors they felt are often underrepresented in empirical measurement: (1) variety and flexibility in one’s values-consistent behaviors and (2) navigating perceived barriers to identified behaviors. Six discussed the importance of generating variety among identified values-consistent behaviors to ensure that no one behavior becomes too rigid and narrowly defined and to help clients identify values-consistent options within a variety of contexts. Five also denoted the significance of identifying client’s perceived barriers to engaging in identified values-consistent behaviors and determining whether they are true logistical barriers (i.e. financial barriers; medical barriers) or if they are actually facilitated by avoidance and then navigating them accordingly. While these qualities are once again with current theoretical and clinical literature regarding the valuing process in ACT, experts stated that when assessed empirically, they are often inadequately addressed.

Finally, while there were some experts (n = 2) who differed from others in the belief that increasing engagement in values-consistent actions was included within the valuing process of ACT (these experts considering it a fundamentally different therapeutic process (i.e. committed action)), all experts agreed that increasing one’s actual engagement in values-consistent behavior is both conceptually and functionally interrelated to valuing and is the therapeutic target of the additional two components discussed. Given this, most experts framed engagement in values-consistent behavior within the valuing process itself, and all agreed that when measuring the valuing process, consideration of this component in relation to the others is imperative. Further, while experts agreed that an overall increase in frequency of values-consistent behaviors is often important, many also emphasized the necessity of generating present moment awareness of the implicitly rewarding experience inherent during values-consistent behavior for clients. For example:

Expert 1: “I think there’s a piece that tends to be missed…how to link action to values especially when the client has an opposition to the action, and when he is engaging in the action. Engaging in the process of values, and valued action, is not just a matter of removing barriers, it’s also a matter of bringing that symbolic reinforcement into the action. The action is going to be reinforcing only if you make the connection between the action and the value. It’s not enough to identify the ‘what’ and then do it, you also have to connect, you know, as you are doing the action, think ‘Yeah, I’m doing that because it’s a part of my value.’”.

Overall, awareness of the connection between one’s values and behaviors in the moment was considered crucial to elicit the rewarding and reinforcing qualities inherent within the action and produce targeted therapeutic changes posited by the ACT model. Experts largely agreed that assessment of this awareness is frequently missed when measuring the valuing process in ACT and is discussed more thoroughly below.

Theme 2.

ACT processes are interrelated and interdependent which may limit the efficacy or utility in adequately assessing each independently. Most experts (n = 6) also discussed the interrelated nature of the six core therapeutic processes in ACT as a reason why it may not be possible or beneficial to distill the model down and assess each process individually. For example, Expert 10 stated:

“Operationalizing any of those six core processes, you know it’s tough…I think that it’s sort of a question of can you have one or two of the domains in action without having the others? So then, can you assess for one or two of them on their own without taking the others into consideration? And I would say…that I’m not sure we can do that. Because I think that they’re integrated and that’s what the hexiflex model really shows. That these processes are all working together. And so, should we be measuring experiential avoidance, should we be measuring acceptance? Or, should we be measuring psychological flexibility, behavioral flexibility, quality of life?”

Several others similarly addressed the implications the highly interrelated nature of ACT’s core processes may have for the assessment of the valuing process independently from the others. Each discussed how, although at any point in time, emphasis may be placed on one individual process, clients are often directly engaging in one or more of the other processes as well. Many described the necessity of defusion and acceptance at times when clients are identifying values-consistent behaviors and perceived barriers to such behaviors. For example, if a client is fused with rigid thoughts surrounding his or her values and what behaving consistently to those values looks like (e.g. “I must value tradition to honor my family”; “If I don’t always host family meals every week I have failed at valuing tradition”), defusion and acceptance strategies may be implemented to shift this rigidity and re-construct his or her valued directions. Considering this innate overlap in the core processes, experts questioned the ability to accurately implement, evaluate, and assess valuing as an independent process without also considering the other core processes.

3.2. Current measurement of the valuing process in ACT

All experts agreed that sufficient measures to assess the valuing process have not yet been developed. Three major themes emerged across experts’ discussions of the limitations and difficulties in measuring this process to date: (3) Current measures over-simplify the valuing process; (4) Current measures do not evaluate one’s level of awareness of an inherently rewarding experience when actively engaging in values-consistent behaviors; and (5) Current measures do not recognize or evaluate the individualized nature of the valuing process.

Theme 3.

Current measures over-simplify the valuing process. Experts stated that current measures of the valuing process are overly simplistic and fail to adequately portray one’s movement across the complete process. More explicitly, while the valuing process itself can be broken down into individual components (as described above), it is a complex therapeutic process in which each component is reliant on the others. Experts iterated that each of these components cannot and should not be separated from each other, especially when approached from a process evaluation perspective, and agreed that current measures are therefore inefficient as most only assess one or two essential components. For example:

Expert 2: “My feeling is that some of what we have that we call values measures are just measuring aspects of it versus all the elements of that. So, I think if we want to more accurately tap into, and it could even be a paper and pencil thing, but that has all of the multiple pieces of it.”

This feeling was echoed by several others who stated that future measures may require a multifactorial or multimodal evaluation to effectively target all relevant components.

Eight experts also discussed problems specific to how values and values consistent behavior are defined within current measurement tools. Current measures utilized to assess one’s values and engagement in values-consistent behavior often face difficulty in clearly defining ‘values’ in an ACT consistent way, particularly prior to an intervention. However, as previously emphasized, defining and constructing values that are personally meaningful and inherently reinforcing is crucial within the therapeutic context of ACT. Yet, experts highlighted that to date, measures tend to rely solely on the assumption that all individuals have interpreted broad terms or statements similarly and that the indicated personal values they provide are truly ACT consistent in this way. Expert 2 highlighted the VLQ as just one example of this:

“I think the VLQ gets at what’s important to [the individual] and how well they’re doing it. But there’s an assumption that the first part is done right. That the things they’re saying are important to them are honestly important to them they’re honestly motivated. Like that’s not really confirmed. So that part may be invalid.”

While several experts described this problem as an overarching limitation of current measurement tools empirically, all stated that many of these tools have strong clinical utility. They noted that although current measures do not generate a complete picture of the valuing process ideal for empirically evaluations, they can be highly beneficial in providing clinicians with a foundational basis of clients’ valued domains upon which further information can be built during therapy sessions.

Overall, oversimplification of the valuing process was consistently reiterated as a limitation to its current measurement within ACT. Experts stated that without a more comprehensive consideration of all essential components, and a proper understanding of how individual respondents have defined their values and values-consistent behaviors within their response, we cannot truly assess the efficacy of this therapeutic process empirically.

Theme 4.

Current measures do not evaluate one’s level of awareness of an inherently rewarding experience when actively engaging in values-consistent behaviors. All experts agreed that one of the primary limitations of current valuing process measures is that they do not evaluate an individual’s awareness of his or her values and experiences of personal meaningfulness or vitality while engaging in values-consistent behavior in the moment. Additionally, they stated that current measures do not assess the specific appetitive qualities or reinforcing factors relevant to the valued action making the individual define the behavior as such. It is therefore not implicit through one’s responses whether these behaviors are truly ‘values-consistent behaviors’ as conceptualized by ACT, or if one is engaging in a set of behaviors they perceive to be values-consistent based on a socially constructed idea of their stated value regardless of their own personal experiences. For example, Expert 10 stated:

Expert 10: “If they say ‘I have a value around health and yeah, I work out 4 days a week, and I eat a vegetarian diet, and I see my doctor regularly, and I do all these things right. Those are all great measures of like, ‘oh good, check; check; check, you know you must really value your health’. But, I mean are they happy? Or maybe, is that meaningful? Or do they feel like they’re just following a bunch of rules?”

Thus, according to experts, failure to adequately measure the contextual aspect of experiential momentary awareness when engaging in values-consistent behavior is a major limitation to the depth of valuing process measurement to date.

Experts posed several factors they felt contribute to this limitation. Six specifically commented that self-report measures asking clients to report their behaviors retrospectively long after they’ve occurred are problematic because they may frequently result in incorrect and/or biased answers. Nine experts also stated that current measures focus too heavily on the specific verbal content of identified values and/or values-consistent behaviors. These measures often ask individuals to identify or select from a pre-identified list the values which are most important to them. They indicated that this is not necessarily problematic and can be helpful in facilitating values oriented discussions, however, these measures then fail to move beyond initial identification to assess the other essential qualities of values and values-consistent behavior as they are conceptualized and function within the valuing process. This poses problems empirically when conversations within therapy cannot be used to supplement limited information and clarify any assumptions being made.

Several experts identified the BEVS as one example of a measure with this limitation. The BEVS asks individuals to write out their values within four defined areas of life but then does not seek additional information as to why the individual feels these values are important or meaningful to them. Additionally, while the measure then asks individuals to denote how closely they are living a life that is in accordance to their values, the measure does not ask what specific behaviors they perceive to be consistent with these values and why they feel this way. Without this insight, it is not possible to ascertain whether the values and values consistent behaviors identified by any individual respondent are truly motivated by appetitive and personally meaningful qualities, and thus truly ACT consistent.

Overall, the necessity of assessing contextual aspects of the valuing process including, perhaps most importantly, one’s present moment awareness of their value(s) and experience of personal meaningfulness when engaging in values-consistent behavior was consistently reiterated. According to experts, a more proximal assessment focused on the experiential qualities related to the valuing process may provide such accuracy when measuring individual progress and the therapeutic efficacy of this process empirically.

Theme 5.

Current measures do not recognize or evaluate the individualized nature of the valuing process. Experts consistently highlighted how current measures fail to adequately assess the individualized nature of valuing both in terms of (1) individually defined meaning and (2) culturally and developmentally relevant factors. While the shortcomings in assessing individually defined meaning(s) of values and values-consistent behavior are discussed in previous sections, the limitations pertaining to culturally and developmentally relevant factors are discussed briefly below.

Experts noted that a major limitation of current measures is the possibility that the pre-identified values or valued domains many measures provide and/or how they are defined (most often through a primarily westernized and adult-focused lens) may not resonate with all individuals. Differences in cultural contexts, age, and developmental level were noted as just some of the factors that may influence one’s definition and experience of specific values and valued living. For example, multiple experts discussed how what feels meaningful and reinforcing to child clients may not parallel the reinforcing values of geriatric clients and each may not perceive the others’ values as personally (or possibly even generally) meaningful. However, if what each individual stated as his or her personal value possessed appetitive, meaningful qualities consistent with an ACT conceptualization of values and could be utilized to reinforce identified values-consistent behaviors that elicit this meaning experientially, then both would be considered functional personal values from an ACT perspective and equally beneficial within the therapeutic valuing process.

Some experts also discussed likely variation in values between individuals who reside within collectivist cultures and those who reside in highly individualistic cultures. For example, Expert 2 stated:

“I think for some people the concept of what I care about can never be separated from a group as much as they would like to say yeah well I really like this. That they can maybe never see it in that way like, “this is what I care about here, and this is what my group cares about.” they must always be a part of that group and there’s no way to separate it. And when people become more westernized they tend recognize they’re being pulled from the group and can have that conversation. But I think that if you caught someone in a collectivist culture with tell me about what’s important to you, you would ultimately get answers that are important to my group.”

Elaborating, experts denoted that it was important to consider that not all individuals may identify with or prioritize the same values as individuals in highly individualistic or westernized cultures where many of the current measures for the valuing process have been developed. Therefore, depending on what items are included in preidentified lists of items, where the list was originally developed, and how the items are defined and interpreted by respondents may greatly influence how various individuals from differing cultures or age groups perceive their values to be represented within the assessment. In these circumstances, individuals who feel that their personal values are not represented or for whom pre-identified values do not resonate with may feel as though their responses on the measure are not an adequate or true depiction of their personal values. Therefore, although several experts noted that providing lists of possible values can be useful to provide guidance and a frame of reference when engaging in the therapeutic valuing process, measuring values in this way poses significant limitations to generalizability.

3.3. Recommendations for future measures of the valuing process

After discussing the limitations of current measures, experts were asked what they would like to see in future attempts to assess the valuing process. Each provided their thoughts relevant to the ideal design and method for evaluating the valuing process empirically based on their own clinical and empirical ACT experience to date. Three themes emerged consistently across experts’ recommendations for future measures of the valuing process: (6) Future measures should be more comprehensive; (7) Future measures should assess the momentary and experiential qualities of valuing; and (8) Future measures should assess the individualized nature of one’s valuing process.

Theme 6.

Future measures should be more comprehensive. All experts stated they hoped to see more comprehensive and multimodal measures to assess the valuing process developed. Due to the highly complex nature of the process and the necessity to consider each of its interrelated components, four experts specifically emphasized the likelihood that a multi-factorial measure is necessary to assess the process properly. Similarly, these experts suggested that it would likely be beneficial, if not necessary, to utilize multiple methodologies in tandem to garner a more adequate assessment. For example,

Expert 5: “I think that this is consistent with contextual behavioral science [CBS] you know. The way that I interpret the CBS strategy is that we should have multiple methods-that we should have diversity in methodology and that our scientific strategy involves multiple iterations in this sort of reticulated development. So, the sort of pieces over here from this methodological approach sort of standing with this when it fits, and then different methodology that can prescribe different kinds of validity and have different kinds of you know problems but is more nuanced in terms of being ‘rich’ -like qualitative approaches.”

Additionally, while all experts agreed that the valuing process is capable of empirical assessment, they were also in agreement that, due to the metacognitive and experiential components involved within the therapeutic process itself, it is likely to require more complex and intricate assessment methods than the existing self-report methods used to date. Experts noted that measure development in this regard is likely to be an iterative learning process and one that will require continuous building upon previous attempts, but stated that it is essential that it continue to be pursued and refined.

Theme 7.

Future measures of the valuing process should assess the momentary and experiential aspects of valuing. Perhaps the most consistently reiterated recommendation by all experts was the need for more direct momentary assessment of one’s experiences as they are engaging in values-consistent behaviors. For example:

Expert 11: “I think that the trick is to get as close to the behaviors as possible… and to try to evaluate, are you doing what is important to you right now? Is this in line with what you think of what you value. And to get a measure of “are you totally engaged in what is important to you at the moment”.

All experts voiced that future measures of the valuing process will only be more useful than those currently available if we are able to tap into this experiential component of the process. These measures should seek to evaluate whether one is aware of the direct connection between his or her current behaviors and how they move him or her toward identified valued directions. Further, the appetitive, or reinforcing experiences that are innately tied to this connection (as theorized by the ACT model) should be evaluated in these moments, and whether the individual feels they are present. Without this information, we miss assessing the fundamental qualities of the valuing process indicating whether valued-living has successfully been achieved, and whether the valuing process is truly what has reinforced the behaviors an individual engages in.

Eight experts suggested ecological momentary assessment (EMA) and daily diaries as possible ways to assess the more experiential aspects of engagement in values consistent behavior in-the-moment.

Expert 7: “I suspect that going forward, we’re going to see more kind of diary stuff and momentary kinds of event sampling you know? Where you’re going to be able to do more idiographic kinds of things you know, like we see what you care about and then we ping you on your iPhone a few times a day and we can just you know touch down and report on it. I think those kinds of technologies are almost certainly going to be the way forward.”

Use of such alternative methods was suggested due to the increasing prevalence and administration ease associated with these measures. Multiple experts identified current projects ongoing within their own research implementing novel assessments designed in this way. For example, one expert discussed an EMA platform through which they prompt user’s multiple times a day to respond to questions about the behavior they are actively engaged in at that time, and whether or not they perceive it to be meaningful. Continued development and evaluation of such assessments was strongly supported by experts and denoted as an essential next step in improving current assessments of the valuing process and accurately validating its therapeutic efficacy.

Theme 8.

Future valuing measures should be capable of assessing the individualized nature of one’s valuing process. Lastly, considering the wide variation in what an individual could identify as a value, and the varying individually defined behavioral representations of movement toward that valued direction, experts recommended that future assessments utilize more individualized response options. While experts denoted that open-ended responses often do not elicit ideal, straightforward psychometrics, these modalities were discussed as likely necessary to facilitate the level of individualization needed to appropriately assess the valuing process.

4. Discussion

The current study sought to utilize expert feedback to critically evaluate how the valuing process in ACT is currently being defined and measured empirically, and to make recommendations for future measure development. Experts identified three essential components of the valuing process which were largely consistent with the ACT theoretical conceptualization of values throughout literature to date. However, experts also discussed the shortcomings and limitations of current measures in assessing these components and highlighted the need for continued effort toward the development of tools capable of assessing the more contextual aspects salient within the valuing process.

4.1. Summary of results and current status

The three components identified as essential to the valuing process by experts (i.e. Values identification/construction; Knowledge of values-consistent behaviors; and Engagement in values-consistent behaviors) largely paralleled the theoretical and functional utility of values presented across ACT literature to date. Perhaps the most significant variation in this regard was experts’ statements suggesting that purposeful and increased engagement in values-consistent behaviors is an essential component contained within the therapeutic valuing process of ACT, as opposed to a separate but related process (i.e. committed action) as it has been traditionally conceptualized within ACT literature. From a clinical standpoint, whether these two processes are distinct is somewhat arbitrary and whatever framework is best understood and functional for the client is recommended. However, when attempting to measure the valuing process empirically, the results of the current study suggest that inclusion of one’s actual engagement in values-consistent behavior is essential to evaluate in tandem with the identification and construction of one’s values and value-congruent behaviors to achieve a complete picture. As several experts denoted, these components are all interdependent, and therefore necessary to determine if one is truly engaged in ‘valuing’ as conceptualized by ACT.

Experts also discussed several shortcomings relevant to existent measures used to assess the ACT valuing process. These shortcomings suggested that current measures universally tend to generate incomplete and potentially ACT-inconsistent responses depending on the contexts within which they are used. Two of the most commonly utilized measurement tools in ACT empirical work to date, the BEVS (Lundgren et al., 2012) and the VLQ (Wilson et al., 2010) were frequently referenced and utilized as examples of measurement tools that posed these limitations. While each of these tools was stated to have high clinical utility, experts denoted that without the ability to expand on and clarify answers provided by respondents, both the BEVS and VLQ do not provide a complete assessment of the valuing process.

4.2. Recommendations for future measure development

The results of the current study suggest that more thorough, comprehensive measures that adequately target all three core components of the valuing process are necessary. Several experts suggested multifactorial measure designs as one potentially ideal approach for such comprehensive assessments. Multifactorial measures would provide the opportunity to evaluate each of the essential components within the valuing process independently, while also generating a ‘complete’ or ‘global’ assessment. This more comprehensive assessment may benefit both the clinical and empirical utility of measures, providing a more nuanced depiction of an individuals’ movement throughout the valuing process across time. Measures designed in this way would also allow for the possibility to conduct network analyses and additional statistical approaches for examining the specific interactions between individual components of the valuing process at both the individual and group level. These methods can also be expanded to additional measures evaluating other core ACT processes, advancing the field’s empirical knowledge of the interrelations between each process and the therapeutic efficacy of the theoretical notions of each process posited by the ACT model.

To facilitate the development of such measures, ACT clinicians and researchers should work together to generate a clear and consistently utilized conceptualization of the valuing process and its core components. Further, ways to operationally define each of these constructs and empirically evaluate the essential qualities of each component to detect appropriate, ACT consistent responses is essential. To generate the most efficient measurement tools for empirical use, consideration toward defining these components such that respondents can understand and consistently interpret them without prior knowledge of ACT or extensive guidance is also needed. The current lack of clarity and consistency in this regard has led to a wide array of terminology and varying levels of descriptive features being utilized to target and assess the valuing process and related constructs. Streamlining terminology and the operational definitions of pertinent constructs would allow for more widespread generalizability and interpretability of results when assessing the valuing process in ACT.

Perhaps most importantly, and most consistently emphasized by experts, is the need for measures that more adequately assess one’s experience and awareness of vitality/personal meaningfulness while directly engaging in the valuing process. Measurement of this intrinsic experience and one’s awareness of it as it is occurring is crucial to adequately evaluate the valuing process in ACT. However, further exploration and clarification as a field as to what this experience comprises is needed. Experts posed many suggestions, such as exploring biological and neurological changes that may be occurring while one is engaging in self-reported values-consistent behavior as ways to evaluate this experience. However, significant caution should be taken to prevent future explorations from only examining biological indicators of pleasant or positive feelings. This is critical to denote due to the viewpoint within the ACT model that values, and the valuing process broadly, may at times be linked to difficult internal experiences and throughout life, experience of these uncomfortable internal states is inevitable. However, the willingness to experience difficult thoughts, feelings, and sensations in the service of one’s values is, according to ACT theory, ultimately reinforcing and meaningful when one makes this connection explicitly. Thus, although multi-modal assessments and the use of biological indicators may add a great deal of insight to the measurement of the therapeutic valuing process, the indicators evaluated that may be associated to the experience of vitality and meaningfulness should not be limited to traditional positive domains.

One of many reasons the identified limitations to current measurement tools exist is likely the fact that all measurement tools assessing valuing and its related components to date rely solely on paper-and-pencil self-report methodology. Future exploration of alternative measure designs utilizing innovative technologies or qualitative formats such as ecological momentary assessment (EMA) or web platforms; daily diaries; or structured interviews may also facilitate more comprehensive and detailed assessments of the valuing process. The use of EMA or daily diaries presented through online web platforms or thorough smartphone applications may generate measurement tools that are more widely accessible and capable of frequent and quick completion allowing for the more momentary depiction of one’s valuing experience that is ideally desired. As such measures are created, of primary importance should be assessing one’s in the moment experiences and awareness of the connection (or lack thereof) between their behavior within that moment and his or her values. Measures of quality of life and motivation may both provide insight as to previous ways in which one’s personal experience of appetitive and meaningful life have been targeted. For example, brief EMA assessments of one’s awareness of his or her values throughout their day as well as their appetitive experiences and motivations for engaging in their current behavior(s) are potential ways to targeting the valuing process in this way. Alternatively, future assessments utilizing a structured interview format could also be beneficial as participants could be asked to describe their thoughts and feelings while engaged in a specific activity or behavior that they have identified as values consistent. Responses could then be coded for various indicators of awareness related to his or her values, and a sense of vitality present in the thoughts and feelings he or she describes –similar to coding processes derived for open ended responses to diagnostic interview questions such as the Objective Binge Assessment contained in the Eating Disorder Examination (Fairburn, Cooper, & O’Connor, 2014). It is also be possible that utilization of such assessments in and of themselves may facilitate regular increases in one’s awareness and promote positive behavioral change, as evident in previous literature regarding priming effects (e.g. Maio et al., 2009). Thus consideration of potential measurement effects should also be considered as future measure development is explored in this way.

Lastly, future measures should seek to provide a more comprehensive assessment of the six ACT therapeutic process broadly. As experts denoted, the theoretical conceptualization of the ACT model is based on the interrelated nature of all six core processes which together facilitate psychological flexibility. Achieving a values-consistent life, and thus, attaining psychological flexibility, according to the ACT experts interviewed for the current study, is facilitated by one’s ability to implement all six core therapeutic processes in tandem. This conceptualization of the core ACT processes is consistent with the overarching theory of the ACT model (Hayes et al., 2006), however empirical process evaluation of each components unique contributions in relation to the others is still needed. Due to the posited interwoven nature of each process however, it is likely that a singular assessment measuring only one or two of the core ACT processes may not only be an inadequate representation of what is truly occurring but may also be impossible without also capturing the relative influence of the others. Therefore, significant consideration of the potential benefits of focusing more explicitly on developing and refining thorough and comprehensive measures of general psychological flexibility, rather than attempting to develop single-process measures in the future should also be given.

4.3. Limitations

One limitation of the current study is the low threshold set for inclusion as an “ACT expert”. No previous guidelines to classify an individual’s level of expertise in the field exist, so it is possible that the criteria developed for the current study may include individuals whose knowledge and training in ACT is insufficient to differentiate true expertise. However, despite the low threshold set for inclusion, the qualifications and background of all individuals who participated in the study far exceeded the minimal necessary credentials for inclusion, making it more likely that they represent a sufficient pool of experts. However, the high caliber of expertise in the current study was in spite of the inclusion criterion set, as opposed to a result of them.

4.4. Conclusion

The current study used feedback from ACT experts to facilitate a thorough evaluation of measurement tools and approaches for evaluating the values work process in ACT utilized to date. To our knowledge, the current study is the first study to utilize feedback provided by experts in both clinical and research orientated ACT professions to critically evaluate the status of empirically measuring the valuing process to date and make recommendations for the future. Results of this study provide a clear depiction of the current shortcomings in valuing process measures for empirical use and provides a foundation for a more concrete and thorough operationalization and assessment of the valuing process.

While this study offers a great deal of insight into how the field can begin to improve empirical assessment of the valuing process in ACT, it is not without its limitations. The majority of experts interviewed were currently working within an academic psychology setting, which may have implications for how they conceptualize and utilize the ACT model. It is possible that the viewpoints of the experts included may be biased toward a more academic or research-oriented understanding of the ACT model as opposed to an applied clinical perspective. Additionally, the study focused primarily on the empirical evaluation of the valuing process in ACT and it should be noted that the proposed methodologies and functions of ideal measures provided by experts for this purpose may not parallel ideal assessment tools for use by practicing clinicians during the therapeutic process. Experts frequently mentioned that many current measures possess strong clinical utility, suggesting that these tools may actually be better suited for use in clinical settings than research settings. However, additional study with a more clinician-rich sample would be needed to establish the utility of these measurement tools in a clinical context. Of note, because interviews were also conducted on an individual basis, we did not provide the opportunity for a dynamic dialogue between experts in both fields to allow each to respond to the others’ perspectives and generate an overarching consensus. Thus, future collaboration among both academic and clinically focused professionals working with the ACT model is essential to developing beneficial and accurate instruments in both settings.

This study highlights the fact that, to date measures for empirically assessing one’s values and values-consistent behavior within ACT research have been limited in scope clarity and have failed to provide a complete depiction of the valuing process as a whole. These shortcomings illustrate the need for continued improvement in comprehensiveness and clarity of such measures, to accurately evaluate the efficacy of the valuing process as a therapeutic mechanism of change in ACT. The recommendations made are hoped to guide future directions for the continued development of valuing process measures in this re-gard.