Lessons Learned in Digital Health Promotion: The Promise and Challenge of Contextual Behavioral Science Consistent Methodology in Valuing Intervention Research

1. Introduction

College students in the United States face significant health risks, including problematic substance use, poor nutrition, low sleep quality, and low rates of exercise (Lawrence, 2017; University of Minnesota, 2015). Given the many behavior patterns in college students that can lead to unwanted outcomes (e.g., Association of American Colleges & Universities, 2021; Becker et al., 2018; Gray & Squeglia, 2017; Lederer & Oswalt, 2017; Reilly & Kelly, 2010; Simmonds et al., 2015; van der Zee et al., 2019), it is essential to develop accessible resources and interventions for health in the college population.

Many prevention and intervention approaches have been developed to address such health risks, with varying effectiveness. Public health approaches often use informational interventions, which include free resources such as health screenings and community health literacy, as low health literacy has been shown to correlate with poorer health outcomes (Fan et al., 2021). Yet, interventions disseminating health information typically do not result in significant behavior change unless given in one-on-one clinical settings (Kahwati et al., 2016; Ross & Melzer, 2016). In fact, recent systematic reviews show that more effective informational interventions are more likely to intervene using multiple simultaneous strategies and individualized intervention aspects such as behavioral counseling, self- reflection on motivation, and self-affirmation (Sheeran et al., 2017; Stormacq et al., 2020). The benefits of an individualized approach are also demonstrated in exercise science research exploring motivation, such that higher levels of behavioral engagement and long-term change are correlated with intrinsic health motivation and personally chosen behavioral intentions (e.g., Kane et al., 2004; Teixeira et al., 2012; Webb & Sheeran, 2006). Such interventions may include motivational interviewing and programs or smartphone applications (apps) with tailored feedback (Lustria et al., 2013; Morton et al., 2014). Recent trends in health research are exploring apps with personalized feedback and practitioner-led personalized medicine to increase preventative health behaviors, further individualizing interventions (Buford et al., 2013; Ghanvatkar et al., 2019).

Studies and meta-analysis also demonstrate significant health behavior change from values-centered studies, indicating the efficacy of personalized interventions focusing on motivation (Epton et al., 2015; Levin et al., 2021). Additionally, meaning in life has been shown to correlate with physical activity, sleep quality, and lower rates of alcohol and drug use (Csabonyi & Phillips, 2017; Hooker & Masters, 2014; Hooker et al., 2020; Kim et al., 2015). Despite these relationships, there has been limited attention given to values- based health interventions for the college population. One useful framework for development of interventions to increase healthy behaviors and reduce unhealthy behaviors is Acceptance and Commitment Therapy (ACT).

1.1. ACT and Valuing

ACT is a third wave behavioral therapy, the central aim of which is not to reduce human suffering, as suffering is assumed to be a typical aspect of the human experience (Fung, 2014). Alternatively, ACT-based approaches seek to change a person’s relationship with suffering and self (Wilson & Murrell, 2004) and to imbue life with meaning through mindful actions in line with one’s values (Harris, 2019; Michelson et al., 2011). The overarching model for this change has been coined psychological flexibility (PF; Hayes et al., 2006), which entails acceptance and awareness of one’s own thoughts, feelings, and emotions, and working toward living in line with values even in the presence of aversive contingencies (Hayes, 2004; Hayes et al., 2012). The six core processes of PF are contacting the present moment, defusion, acceptance, self-as-context, values, and committed action (Harris, 2019). Values provide meaning in life and direction in the use of the other processes, while committed action refers to patterns of successful behavior associated with personal values (Hayes et al., 2012).

In ACT, values are theorized to be “freely chosen, verbally constructed consequences of ongoing, dynamic, evolving patterns of activity, which establish predominant reinforcement for that activity that are intrinsic in engagement in the valued behavioral pattern itself (Wilson & DuFrene, 2009, p. 66).” Unlike goals, which can be completed, valuing is an ongoing process (Chase et al., 2013; Hayes et al., 2012). Values are constructed on a moment-to- moment basis and change over time, as engagement with valued action will influence one’s future construction of valuing in the same area. These valued actions are reinforcing in and of themselves, with behaviors under appetitive instead of aversive control (Smout et al., 2014). Engagement in valued behavior also broadens one’s behavioral repertoire, leading to expanding patterns of behavior which allow for increasing engagement in even more valued actions (Louisiana Contextual Science Research Group, 2022).

1.2. Valuing and Rule-Governed Behavior

From a behavior analytic approach, valuing is an instance of rule-governed behavior (RGB), which is controlled by verbally mediated consequences, as opposed to behavior shaped by environmental contingencies with which we come into direct contact (Törneke et al., 2008). One type of RGB is pliance, defined as behavior controlled by socially mediated consequences, wherein another person both establishes the rule and delivers the consequence, directly connecting the rule and the behavior that will result in those consequences (Kissi et al., 2017). Pliance may also be directed by self-established rules with socially mediated consequences such as social approval (Törneke et al., 2008). For example, a person may go to the gym because they receive social acceptance and praise by doing so. Inherent in this conceptualization is motivation by aversive control, which is not congruent with the CBS conceptualization of valuing (Plumb et al., 2009).

1.3. Valuing in Research

Several recent studies collecting longitudinal ecological momentary assessment (EMA) data on valuing each demonstrates the dynamic nature of valuing and its role in both day-to-day and longer-term health outcomes. Grégoire et al. (2021) found wellbeing and distress were more accurately predicted by day-to-day valued actions than averaged valued living, and that higher daily variability predicted increasing distress over time. Finkelstein-Fox et al. (2019) and Berghoff et al. (2018) found that different psychological flexibility processes discriminatively predicted daily valued living, with the former showing day-to-day within-person variance in valued action as an indicator of daily psychological health. Together, these studies show that small, consistent behaviors in line with one’s values can increase wellbeing and health most effectively over time. However, valued actions also have immediate benefits and other aspects of psychological flexibility influence our ability to engage in these actions in the moment. Many other ACT interventions also show that valuing may be the mechanism through which multiple predictors result in favorable outcomes (Michelson et al., 2011; Vowles & McCracken, 2008; Wallin et al., 2018; Wersebe et al., 2017).

Two recent studies show promise in more personalized research regarding health values-consistent behavior. Stapleton and colleagues (2020) used health specific values to predict health outcomes in college students rather than using general valued living. General values consistent living and psychological flexibility were measured using the Valued Living Questionnaire and CompACT, respectively. Outcomes included survey measures for physical activity (International Physical Activity Questionnaire; Craig et al., 2003), dietary quality (Diet Quality Tool; O’Reilly & McCann, 2012), sleep quality (Pittsburgh Sleep Quality Index; Buysse et al., 1989), alcohol consumption (Quick Drinking Screen; Sobell et al., 2003), and cigarette consumption (adapted from the National Health Interview Survey; CDC, 2016), all which areas are identified by the Centers for Disease Control and Prevention (CDC, 2019) as key health components. Health values consistent living predicted higher sleep quality but no other outcomes. General values consistent living predicted more physical activity and better sleep quality but not diet quality or cigarette and alcohol consumption. The authors concluded that abstract values were not enough to predict behavioral outcomes, and that value measurement and behavioral outcomes should match in specificity.

Another recent study (Jackson et al., 2016) sought to ensure a functional value- behavior relationship using a specific, pre-set target behavior participants’ chose themselves (a university cycling class) while allowing them to state their own personal motivation for said behavior. Participants rated a list of 24 common health and aesthetic related reasons why one might engage in physical activity and then constructed their own personal statement expressing what motivates them to exercise. Their own statement and six highest rated statements from each category were used to create an Implicit Relational Assessment Procedure (IRAP; Barnes-Holmes et al., 2006) unique to each participant. The IRAP is a computerized procedure based on Relational Frame Theory’s (RFT) derived relational responding principle which allows for identification of implicit relational preferences, and results were used to validate the selection of values statements used in the experimental condition. Participants attended 10 cycling exercise classes and were given either a statement they had indicated was in line with their motivation for health (e.g., “to fit into my skinny jeans”) or a statement containing an instructional message (e.g., “push through your heels”) with assignment utilizing an alternating treatment design. They found that when participants had a statement they rated as motivating presented to them their heart rate was significantly higher compared to presentation of an instructional statement and to baseline. These findings reflect the concept of valuing, as verbal establishing operations influenced behavior by connecting it to consequences. Further, motivational statements differed by individual, and the reinforcing consequences linked to these statements were often temporally distal while cycling (e.g., “to fit in my skinny jeans”).

1.4. Implications for Novel Research Methods

Jackson et al. (2016) and Stapleton et al. (2020)’s methods have exciting implications for valuing studies, as survey measures assessing broad, abstract values do not steadily predict engagement in target behaviors (Barrett et al., 2019; Dahl, 2015). One reason for this disconnect may be that researcher-generated target behaviors may not hold the same evocative effect and connection to that value for each or every participant (Hayes, 2004). Additionally, one feature of valuing as conceptualized in ACT is that the reinforcer for valued behaviors is self-selected for its inherently reinforcing nature to an individual, rather than socially mediated through compliance with researcher expectations (Bond, 2004; Dahl, 2015). Researcher-decided target behaviors and valued domains may induce participants to perform due to pliance (Kissi et al., 2017; Zettle & Hayes, 1982), and researchers should seek to mitigate some of this effect with design elements.

An individualized approach such as in Jackson et al. (2016) can navigate some of these difficulties by having participants create a motivational statement that is more likely to be intrinsically reinforcing and encouraging them to connect target behaviors to their valuing in a functional manner. Most research on valuing to date has utilized traditional group level design, such as self-report questionnaires and an intervention applied to a group, despite the ideographic nature of valuing. Stapleton et al. (2020) and Jackson et al. (2016)’s studies establish techniques to make group level design more congruent with the nature of valuing by including individualized intervention elements and connecting valuing to one specific domain (i.e., health).

There are multiple ways to individualize participants' values and target behaviors in group designs. One of these is the implementation of a front-end intervention to guide participants’ engagement with values as freely chosen and not as socially desirable (e.g., fusion with socially desirable values). A second method is narrowing down the valued domains to one domain associated with target behaviors (e.g., health) and keeping data from participants who rate this as highly valued (Stapleton et al., 2020). A third method is to broadly assess multiple target behaviors within a valued domain to explore participants’ unique pattern of behavior change. Fourth, and most importantly, allowing participants to construct their own personal valuing statement around this domain instead of choosing from a researcher-generated list will assist in encouraging a more direct relationship between valuing and target behaviors. These methods allow this analogue study for short-term behavior change to examine the mechanism of action involved in such change and the utility of valuing interventions to potentially develop long-term intervention to disseminate to the general college population.

1.5. Current Study

This current study explored the influence of a previously validated valuing intervention on health behaviors in a college population. This was done through replication and extension of studies exploring health valuing with innovative methods (Jackson et al., 2016; Stapleton et al., 2020). Specifically, this study replicated the multiple domain-specific outcomes demonstrated by Stapleton et al. (2020) and Jackson et al. (2016)’s procedure for generating ideographic motivational statements. In addition, this study extended the use of valuing interventions (Harris, 2008) to health-specific domains. Synthesis of these methods in this analogue study has the potential to inform research developing practical interventions in college populations to encourage increased health behaviors, including physical activity, dietary quality, sleep quality, and alcohol consumption.

The current study had two main hypotheses. First, we hypothesized that participant selected outcomes of most personal importance would predict higher pre-post change in the related outcome for those in the intervention condition. The intervention condition would show greater standardized improvements in selected health outcome of most personal importance relative to pre-post changes in the control condition. Second, we hypothesized that changes in values awareness and engagement would mediate the relation between intervention condition and the standardized change score in selected health outcome of most personal importance, with participants in the intervention condition experiencing improvements in their chosen domain via the indirect effect of the valuing process. This analysis was primarily used to assess valuing as the mechanism of change involved in behavior change following the valuing intervention. It additionally provided information on engagement with the intervention. Because ACT interventions elicit change through learning by direct experiences (such as the valuing intervention) and not psychoeducation, changes in valuing itself are more congruent with the ACT model than using attention checks to assess engagement.

Exploratory analyses were conducted to examine the difference between behavior change in a specific, personally important domain and behavior change across domains at the group level. It was expected that some domains of behavior would result in more change than others at the group level, or that some domains would not result in any change without consideration of personal importance. Results allowed for comparison in participant selected change targets in line with the ACT conceptualization of valuing versus researcher-generated change targets; information which can inform future research and public interventions in specific health domains (i.e., physical activity, dietary quality, sleep quality, alcohol use, and cigarette consumption). Additionally, the impact of the values intervention and control intervention across these health domains was explored to determine if the values intervention leads to greater general improvements in health domains.

2. Materials and Methods

2.1. Participants

Data were collected from a sample of students enrolled at a university in the midwestern United States. Inclusion criteria for survey participation required participants be age 18 or older and have English language proficiency.

2.2. Procedure

All procedures were approved by the university’s Institutional Review Board (Protocol 22-030) and funding was obtained through a university grant via the Murray State University Office of Research and Creative Activity (ORCA Grant MS #1129) prior to data collection. Additionally, this study was preregistered on Open Science Framework (Center for Open Science; https://doi.org/10.17605/OSF.IO/V3QXD) There were two recruitment strategies used: the first was recruitment through undergraduate psychology classes participating in the SONA participant pool across two semesters. The second recruited students from campus through emails to official campus organizations. The samples were pooled due to similarity in demographics.

After providing informed consent to participate, participants followed a link leading them to eight psychometric battery measures, which took approximately 30-minutes to complete. Participants then continued to the intervention portion of the study. Random assignment to an intervention condition (values-based or informational) was automated in the online open-source Lime survey platform (LimeSurvey GmbH, n.d.). After intervention and control procedures, all participants selected the health domain of most importance to their own health from a list of the domains assessed in this study.

Participants were then invited to sign up for ecological momentary interventions (EMI) over text. During the 10-day intervention window, participants assigned to the intervention condition received a text daily at 10:00 am that contained their personally crafted motivational statement. Participants assigned to the informational control condition received texts each morning with information regarding a domain of health being assessed. At the end of the 10-day intervention window participants in both conditions were presented with the same battery of questionnaires given at baseline, excluding the demographic survey. They then completed a program evaluation to assess the perceived benefits and usability of the intervention.

2.3. Conditions

2.3.1. Intervention Condition

Participants assigned to the values condition listened to a 4-minute audio recording during the initial study timepoint guiding them in the process of valuing as conceptualized in ACT, differentiating values from goals, and giving an example of valuing. Examples from the recording include “values are directions we keep moving in, whereas goals are what we want to achieve along the way” and “they are leading principles that can guide us and motivate us as we move through life.”

The audio recording was adapted from two widely used clinical tools created by ACT practitioners (Harris, 2008; Wilson & Sandoz, 2010), the Values Worksheet and the Sweet Spot exercise. As per the Values Worksheet, a focus of the recording was valuing as a dynamic, evolving process (e.g., “Values are directions we keep moving in, whereas goals are what we want to achieve along the way.”). It stressed valuing as freely chosen and differing from person to person (e.g., “not everyone has the same values, and this is not a test to see whether you have the "correct" values.”). The script was further tailored to those who value health by giving an example of possible health values and by including questions from the Values Worksheet and the Sweet Spot exercise that engage participants in actively thinking about what might personally motivate them to engage in healthy behaviors (e.g., “How do you want to look after your health, with regard to sleep, diet, exercise, smoking, alcohol, etc.? Why is this important?”). As per the format of The Values Worksheet and Stapleton et al. (2020), this recording was followed by a list of values to aid in their consideration; all specifically related to health (e.g., fitness, empowerment, accomplishment).

Next, participants were given the textual prompt “What are your values related to maintaining your physical well-being? How do you want to look after your health, regarding sleep, diet, exercise, smoking, alcohol, etc.? Please list your own personal reasons for choosing healthy behaviors. Be specific.” A free response text box was provided. This item is consistent with the methods used in Jackson et al. (2016) and Harris (2008) and supplied an ideographic (i.e., specific to that individual) statement on values surrounding physical health for each participant.

2.3.2. Informational Control Condition

To give the control condition equivalent time spent in the study, participants randomized to the control group listened to an approximately 4-minute audio recording. The content included statements from credible health organizations detailing recommendations for ideal dietary quality, sleep quality, physical activity, cigarette consumption, and healthy alcohol consumption levels. The health psychoeducation procedure is in line with Jackson et al. (2016)’s informational exercise statements in their control condition and provides data on how individualized motivational statements affect health behaviors above and beyond health literacy. Time equivalence was ensured through word count compared to the intervention condition (i.e., approximately 500 words).

2.4. Daily Interventions

Starting approximately one week after completion of baseline measures and the intervention or control recordings, values intervention participants received a morning text containing the motivational statement they constructed. Participants began the texting phase of the intervention between one and six days following the initial study timepoint. Informational control condition participants received a morning text containing instructional, actionable statements from respected organizations on how to improve that domain of health (e.g., “Look for opportunities to reduce sedentary time and to increase active time. For example, instead of watching TV, take a walk after dinner. – CDC.”). These informational texts were designed to be consistent with the instructional statements seen in Jackson et al. (2016). All texts were sent through CallFire (CallFire Inc., 2024), a secure, professional texting service.

2.5. Measures

Demographics Questionnaire. A short questionnaire gathered self-report data on demographics including age, ethnicity, gender and sexual identity, and college class.

Valued Living Questionnaire (VLQ; Wilson et al., 2010). The VLQ assesses participants’ values and how they are living in accordance with them across 10 common life domains (e.g., family, career). Participants were asked to rate the importance of each domain and how consistently they have acted in accordance with their values in the past week on a 10-point scale, with higher scores indicating a greater importance in an area and greater belief one is living in line with one’s values (α = 0.86 in the current sample). Psychometric evaluation has found acceptable internal consistency (α = 0.67 – 0.79) test-retest reliability (r = 0.74 - 0.76), and convergent and discriminant validity (Cotter, 2011).

Valuing Questionnaire (VQ; Smout et al., 2014). The VQ is a two-factor, self-report measure that assesses progress and obstruction to valued living. Progress is defined as engagement with valued living, awareness of values, and perseverance. Obstruction is defined as lack of engagement with valued behaviors due to avoidance or inattention to values. The VQ shows good convergent and discriminatory validity and good internal reliability (α = 0.79 - 0.81; Smout et al., 2014). Higher scores in the two separate progress and obstruction subscales indicate higher engagement with or barriers to valued living, respectively (α = 0.80; α = 0.81 in the current sample).

Nicotine Consumption. Nicotine consumption was assessed using one item taken from the National Health Interview Survey (CDC; 2016). Participants reported frequency of current cigarette and e-cigarette smoking behaviors on a scale of 0 – Not at all to 2 – Every day. No Cronbach’s alpha was calculated for cigarette consumption because it contains a single item.

Diet Quality Tool (DQT; O’Reilly & McCann, 2012). The DQT is a 13-item self-report measure that assesses the intake of important dietary nutrients by combining information on the number of servings of four food groups (e.g., fruit) and the quality of seven food groups (e.g., grain). Criterion and construct validity were acceptable for use, with overall DQT scores correlating with fat, fiber, and omega-3 scores (r = -0.50, r = 0.55, r = 0.32 respectively). A higher score indicates eating habits more in line with nutritional guidelines (α = 0.77 in the current sample). Wording has been replaced with American English where necessary to ensure comprehension by the sample (e.g., “biscuits” changed to “cookies”).

Pittsburgh Sleep Quality Index (PSQI; Buysse et al., 1989). Sleep quality was evaluated with the PSQI. Participants rated seven components of sleep quality and disturbances (e.g., duration, daytime dysfunction) in 19 items. The component subscales are summed for a total score, with lower scores indicating higher sleep quality (α = 0.76 in the current sample). The PSQI demonstrates good internal consistency reliability (α = 0.70 - 0.80) and good convergent and discriminant validity (Buysse et al., 1989; Carpenter & Andrykowski, 1998).

Quick Drinking Screen (QDS; Sobell et al., 2003). The QDS assesses unhealthy alcohol consumption in five areas: average number of days drinking per week, average drinks per week, average number of drinks when drinking, frequency of binge drinking occurrences, and the highest number of drinks consumed on one occasion. The QDS was originally developed to estimate these factors over the last 90 days. While no test-retest reliability is available, interclass correlations with a previously validated standard drinking measure were all significant (p < .001) across all domains. The QDS was also found to be more reliable than another commonly used measure, the Alcohol Use Disorders Identification Test - Concise (see Letourneau et al., 2017; Sobell et al., 2003). No Cronbach’s alpha was calculated due to only being a single item.

International Physical Activity Questionnaire - Short Version (IPAQ; Craig et al., 2003). The IPAQ was designed to evaluate physical activity. Participants rate nine items that collect information on intensity of physical activities (slow, moderate, vigorous, and walking) and how often they were sedentary in the past seven days. It is a psychometrically sound measure, with good inter-method concurrent validity with the IPAQ long form (r = average of 0.80) and acceptable reliability (r = 0.35 - 0.88, with ¾ of items above 0.65). Number of days spent in a type of activity multiplied by the number of minutes is calculated for each intensity level and the scores for each intensity level are summed. Higher scores indicate higher levels of physical activity (α = 0.69 in the current sample).

Assessment of Health Behavior Importance. Health behavior importance was assessed using one item that required participants to choose the domain of health most personally important to them. Options included alcohol use, nicotine use, diet quality, sleep quality, and physical activity.

Program Evaluation. A series of questions asked participants to rate aspects of the interventions to inform possible development of tools to increase health in college populations. Quantitative items include the helpfulness of the intervention regarding behavior engagement and increased awareness of personal health valuing, both on a five-point scale. Qualitative items included the question “What did you like or not like about the daily text messages?” and an open text box for any other feedback.

3. Results

All cleaning, coding, and analyses of data was conducted using IBM SPSS Statistics (Version 25). For the sake of brevity, a more detailed description of methods used for data preparation, other analytic procedures, baseline analyses, and non-key findings are available for access elsewhere. The original study may be reviewed at no charge at https://digitalcommons.murraystate.edu/etd/257/. Additionally, the dataset and additional materials can be accessed on the Open Science Framework’s data repository at https://osf.io/zhyns/.

3.1. Study Flow and Baseline Descriptives

Of the 252 participants who completed the baseline survey, 118 did not complete the second time point and were removed. The overall attrition rate from baseline to post-intervention surveys was 46.8%, attrition in the valuing condition was 43.6%, and attrition in the informational condition was 49.6%. All cases with more than 10% of missing responses were deleted (n = 10). Those with a score of five or less on the VLQ were removed prior to analysis . This criterion helped to ensure participants valued health, a priori, not due to implied researcher requirements (i.e., pliance). Thirty-three additional participants were removed due to a technical error. See the CONSORT diagram (Figure 1) for details on participant enrollment, assignment to condition, attrition at each stage of the study, and analyses.

Most participants (N = 107) identified as female (n = 71, 66.4%), heterosexual (n = 84, 78.5%), and White (n = 91, 85%). The average age of the sample was 20.3 years of age (SD = 6.00). Participants in the valuing condition (n = 52) and informational condition (n = 55) showed similar demographic characteristics (see Table 1). Descriptive statistics and correlations were calculated for all study variables at baseline (Table 2).

3.2. Change in Valued Health Behavior

The intervention condition was hypothesized to show greater standardized improvements in the selected health outcome of most personal importance relative to pre-post changes in the control condition. This hypothesis was tested using an analysis of covariance (ANCOVA). With the intervention condition as the independent variable. The dependent variable was a standardized (expressed as z-scores) pre-post intervention change score in the participant’s selected health domain with change scores calculated such that positive scores reflected a positive change in health. The standardized baseline score for the relevant health domain was entered for each participant as a covariate. There was no significant effect of intervention condition on health behavior change in participant selected domain after controlling for baseline scores in participant selected domain, F(1, 73) = 0.40, p = .531, ηp² = .006.

3.3. Mediation by Values Awareness and Engagement

Changes in values awareness and engagement were hypothesized to mediate the relation between condition and the change score in selected health outcomes of most personal importance. Intervention condition was entered as the predictor and the standardized, pre- post intervention change score in participants’ selected health domain was entered as the dependent variable for both groups. Pre-post-intervention change score for values progress from the VQ was entered as the mediator. The standardized baseline score for the selected health domain was entered as a covariate.

Results showed that intervention condition was not associated with a change in values progress (b = 0.08, SE= 0.24, t = 0.32, p = .749; Figure 2). Values progress was not associated with a change in health behavior in the domain of most personal importance (b = -0.07, SE= 0.08, t = - 0.93, p = .351). A bootstrap estimation approach was used to compute standardized indirect effects for each of 10,000 bootstrapped samples, with 95% accelerated confidence intervals used to determine significant paths. The indirect effect was not significant, (95% CI [-0.09, 0.04]), indicating a lack of mediation by a change in values progress.

3.4. Overall Health Behavior Improvement

The impact of intervention condition across all health domains was tested using a series of six ANCOVAs to determine if the values intervention led to greater general improvements in health. The independent variable was the intervention, and the dependent variables were each health domain at post-intervention (T2). For all analyses, the covariates were baseline scores for the corresponding health domain. The effect of intervention condition on health behavior change while controlling for baseline score was not significant for all dependent variables. See Table 3 for ANCOVA results. See Table 4 for means and standard deviations by condition.

3.5. Program Evaluation

Participants rated the helpfulness of the interventions and the subjective increase of values awareness on scales of one to five. Helpfulness ratings had a mean score of 3.66 (SD = 0.92). The valuing (M = 3.70, SD = 0.90) and informational (M = 3.60, SD = 1.00) conditions had similar ratings. Overall evaluation of an increase in values awareness had a mean of 3.33 (SD = 0.97). The valuing (M = 3.60, SD = 1.00) and informational (M = 3.20, SD = 0.90) conditions had slightly differing ratings. Two items asked participants “What did you like and not like about the daily texts?” and “Please give any other feedback you chose or type n/a.” Response statements were categorized by theme as per the qualitative content analysis method (Bengtsson, 2016). See Table 5 for frequencies in all categories.

4. Discussion

An ACT-congruent valuing intervention to increase health behaviors in college students was compared to a traditional informational intervention. Previous research points to both values-based and personalized interventions providing positive effects on behavior change (Epton et al., 2015; Gregg et al., 2014; Levin et al., 2021; Lillis et al., 2021). That said, those given the valuing intervention did not show greater positive health behaviors in domains of most personal importance than those given the informational intervention, in contrast with the findings of Jackson et al. (2016). Nor did changes in values progress mediate the relation between intervention conditions and changes in health domains of most personal importance. Additionally, there were no significant differences in overall health behavior change between intervention conditions . However, this study found a positive association between progress toward living in line with one’s values. It was also found that higher levels of unhealthy behavior, such as nicotine and alcohol use are associated with poorer health outcomes, as is consistent with the literature (CDC, 2019).

Of course, non-significant results can be vital to explore in the progression of science as they tell us where to improve and where to move forward. Pilot studies can function as a vital phase in the iterative design process. Rather, they are a necessary step in intervention development (Lerner et al., 2000). Therefore, it would be inappropriate to interpret these null findings as evidence that valuing interventions are never useful in affecting health behavior change. Instead, one conclusion that may be drawn from this study’s findings is that this valuing intervention needs improvement. There is support for this interpretation in the past effectiveness, acceptability, and feasibility of ACT and values-based interventions on physical health delivered by technological intervention (e.g., Levin et al., 2021; Mueller et al., 2023).

4.1. Comparison of Intervention Conditions

Both the quantitative and qualitative program evaluation data indicate that there are favorable elements in the study design, showing moderate social validity and acceptability. Results indicated that varying participants found the interventions motivating, good reminders of their goals and values, personalized, to contain valuable information, and generally positive. However, while multiple participants (n = 6) in the valuing intervention indicated their intervention was motivating, more of those assigned to the informational condition (n = 10) expressed being motivated. This is problematic given that values are conceptualized as motivational augments in ACT (Kissi et al., 2017; Smout et al., 2014; Zettle & Hayes, 1982). Given that many participants created statements containing behavioral goals rather than valuing statements, this stands as evidence that the front-end intervention requires more depth and weight. An example of goal-oriented statement content includes “I would like to get back to going to bed at 10 and getting up at 8 every day to get back on a sleep schedule.” A contrasting example of values-oriented content includes “I want to be able to challenge myself on a regular basis.” The brevity of intervention this study used as a strategy to retain participants’ engagement with the recordings should be balanced with consideration of an ideal dosage. Future interventions might be designed to delineate the difference between values and goals in a more in-depth manner, possibly with multiple exemplars rather than one (e.g., Krafft et al., 2017; Rahal & Gon, 2020). Further, the current exercise could be expanded so that participants have a longer period in which to interact with the content and receive benefits (Li et al., 2022).

4.2. Provider Feedback

Additionally, construction of values usually occurs in clinical settings which allow for significant time and personal, one-on-one feedback. This intervention was based on clinical tools typically used in an in-person setting, wherein the client has access to feedback and ongoing construction of values through discussion. This process is supported in ACT-based interventions on health that utilize one or more session of one to five hours of values work to encourage health outcomes (Lillis et al., 2021) and self-guided digital interventions for health that provide multiple modules (Firestone et al., 2019). Inclusion of telehealth or other one-on-one initial communication may help people requiring assistance in detecting adaptive reinforcing opportunities, discriminating behavior needed to contact reinforcers, tacting appropriate contingencies, or shifting perspective (Hayes & Strosahl, 2013). Artificial Intelligence (AI) is one possible route to real-time, personalized feedback without a clinical provider (e.g., Heinz et al., 2025). For example, younger populations will use AI conversational agents frequently while using mHealth apps, resulting in high engagement (Vertsberger et al., 2022). These have recently been shown to be effective with an ACT intervention (e.g., Naor et al., 2022). However, there are still questions on the veracity and helpfulness of AI in mHealth to ensure quality care that require further exploration in the health (Deniz-Garcia et al., 2023).

4.3. Competing Reinforcer

Another factor to consider, specifically in valuing studies, is competing contingencies. The evocative effect of values statements in this study may not have been as salient as other sources of behavioral control. Existing research points to the need for valuing interventions to target significant behaviors with strong reinforcing functions (Fryling, 2012; Whelan & Barnes-Holmes, 2004). It is possible that the physically healthy functioning targeted in the current study may not have been as salient as other reinforcers, including course credit for research participation. Participants received course credit before completing the second time point, which may have promoted attrition. Further, the reinforcing value of course credit may have competed with the motivating augmental instead of having an additive effect. This is not necessarily detrimental, as academic success is a useful and workable goal for students. However, it suggests that future research in this domain may consider only recruiting college students who express an interest in improving health for its own sake. Such samples of college students could be drawn from previously established campus services, such as student medical services and counseling centers on campus. This recruitment method has the benefit of exploring ways to amplify the benefits of treatment as usual through an adjunct that clients and patients can use anywhere, as well as resulting in greater experimental external validity.

4.3.1. Additional Sampling Methods

The obtained attrition rate of 46.8% is on the higher end of average rates in the reported literature on technologically delivered health trials, which typically fall between 26% and 49%, and up to 60% or more (e.g., Howarth et al., 2018; Meyerowtiz-Katz et al., 2020; Prior et al., 2023). However, health behavior change has lower rates of adherence than other mobile interventions and is not often maintained in naturalistic settings. For example, a recent meta-analysis showed that only nine out of 62 mHealth interventions had completion rates considered successful (no more than one standard deviation below the mean rate; Amagai et al., 2022). Another intervention on New Year’s resolutions showed that 45% of participants were unsuccessful due to a focus on avoidance rather than approach motivation surrounding health (Oscarsson et al., 2020).

4.4. Methodological Lessons Learned

Congruence of study design and analysis with theoretical frameworks used to create interventions should also be considered in future research. Although this study included personalized elements for participants in the intervention (i.e., valuing statement), only one such element was incorporated into the study design and analysis (i.e., the participant selected behavior domain of most personal importance). There are multiple methods which can achieve this aim, such as utilizing single-subject designs (SSD), more longitudinal data collection, network analysis, and the continued inclusion of processes of change variables in studies (Gates & Hellberg, 2022). Use of these methods would additionally allow for more cross-disciplinary comparison of findings.

4.4.1. Intervention & Study Delivery

Synthesis of technologically delivered interventions and traditional treatments in future research could mitigate difficulties constructing values in a non-clinical setting. Daily EMIs through text could be explored as an adjunct to campus therapy and medical centers—a method proven useful in clinical disorders (Clough & Casey, 2011; Heron & Smyth, 2010; Newman et al., 2015). This usage would attenuate front-end difficulties through inclusion of immediate feedback on values construction in therapy centers. Many doctor’s offices and hospitals are now offering Focused Acceptance Commitment Therapy (FACT; Glover et al., 2016; Kanzler et al., 2022), in which medical providers are trained to provide one session ACT interventions focused on changing health behavior through the valuing process, and such services could address these difficulties. These approaches could additionally reduce the burden on these facilities and offer daily support to clients, as well as providing support following the termination of treatment.

Additionally, future studies on health behavior change may consider the limitations of self-report, such as the subjective nature of responses, recall bias, and social desirability (Gmel & Daeppen, 2007; Sallis & Saelens, 2000; Steene-Johannessen et al., 2016; van de Mortel, 2008). However, there are several methods that can help buffer these concerns. For instance, EMA research can address concerns with recall bias and increase the ecological validity of studies (Smith & Juarascio, 2019). If possible, the production of a permanent product of behavior is an ideal strategy to assess behavior change as it provides accurate, objective data from which to draw conclusions. This could be accomplished with technologies such as fitness tracker watches and mobile phone applications, which record a variety of health-related data, such as step count, heart rate, and sleep quality (Chow & Yang, 2020; de Zambotti et al., 2015; Stackpool, 2013).

4.4.2. Intervention Dosing

There is evidence that technological interventions spanning 10-14 days can lead to positive outcomes (Criddle et al., 2021; Jackson et al., 2016; Jeffers et al., 2019; Lazev et al., 2004; Nam et al., 2019). Despite much premature discontinuation of medical and behavioral health services (Miller & Brennan, 2015; NORC, 2018; Reljic et al., 2019; Ti & Ti, 2015), medical treatment can last anywhere from days to a lifetime depending on the health concern. Increasing the duration of EMI delivery may prove effective given the average length of professional sessions that address behavior change and treatment discontinuation rates. A dosage increase would provide additional support for students between professional appointments and following termination. Another way to increase EMI dosage would be a change in content rather than only a change in frequency or duration. Many ACT-based self-help programs and mobile phone applications used without professional services are effective (Bricker et al., 2014; Potts et al., 2020). Another alternative to increasing the duration of EMI delivery or length of front-end interventions is the use of interactive content. For example, micro-engagement with technologically delivered content has been found to increase the effectiveness of interventions (Voorheis et al., 2022).

4.4.3. Mechanisms of Change

Starting valuing intervention development at the single-subject level allows for comparison of response patterns across multiple participants where individual context and characteristics can inform decisions on how, why, and for whom interventions are successful. Furthermore, this approach could lead to further development based in common factors with more precision than traditional mediation and moderation analyses (Hofman et al., 2020). The additional inclusion of processes of change variables at the individual or group level can allow researchers to determine what biopsychosocial processes maintain or change health-related behaviors over the course of an intervention (Hayes et al., 2018). Interventions can then be modified to enhance or omit elements that target specific mechanisms that are or are not driving behavior change.

Results from SSDs and utilization of change process measures applied to the refinement of interventions may also provide a more evidence-based program to be tested at the group level. Group level studies would then provide opportunities for statistical methods such as dynamic network analysis and Group Iterative Multiple Model Estimation which can be used to combine idionomic and nomothetic approaches (Hofman et al., 2020). These analytic strategies allow for the construction of temporal models providing insight into the directionality and multidirectionality of processes of change and into time fluctuating relationships between variables to best design interventions (Hasselman & Bosman, 2020; Jordan et al., 2020; Sanford et al., 2022).

4.4.4. Idiographic Approaches

While general comparative group trials are useful to test the effectiveness of developed products; initial development of interventions would greatly benefit from a combination of nomothetic and idiographic approaches. This could address the issue of individual responding patterns in group research violating the ergodic assumptions underlying common analytic methods (Lowie & Verspoor, 2018; Molenaar, 2007; Sanford et al., 2022). Larger group trials focus on specific behaviors, yet behaviors hold different functions for different people and inferences about populations do not generalize to all individuals (Staines, 2008). Specifically, valuing is an ideographic process in which valued behaviors differ in topography and function across participants.

The control offered by SSDs can ensure that the effects of a treatment are due to the administration of the treatment rather than unaccounted for factors and that its effects can be repeated (Holman & Koerner, 2014). Using SSD to explore valuing interventions can reduce resulting intersubject variability due to contextual and personal factors, as each participant serves as their own control group (Barlow & Hayes, 1979). SSD can also offer vital information on minimum effective treatment dosages, which are high enough to produce results yet low enough to encourage participant engagement—a concern raised by the current study. Additionally, these design methods could reduce the number of participants needed for analysis and allow for reliable results despite constraints such as funding concerns or attrition common in larger trials (Hopewell et al., 2025; Shore et al., 2012).

4.4.5. Analytic Approaches

There are idiographic approaches to statistical analyses as well. While there are existing statistical methods that allow us to examine within-person effects (e.g., SEM; Hoyle, 2012), recent advances in technology are proving useful for person-centered mental and physical health investigations. For example, dynamic network-based analyses allow for the modeling of time-dependent, multivariate relationships at the individual and group levels (see Gates & Hellberg, 2022; Ong et al., 2022). Such analyses can facilitate the understanding of the directionality and interplay of symptoms, process variables, and outcomes such that clinically relevant trajectories and intervention points may be observed (Jordan et al., 2020).

Another statistical consideration in interpreting results is underpowered analyses. While some literature suggests that this study’s sample size is adequate, most guidelines for intervention trials indicate that larger samples are necessary to obtain sufficient power (Bujang et al., 2017; Shieh, 2019). Support for null hypotheses in this study may be the result of an underpowered analyses, and a larger sample could show significant result.

4.5. Program Evaluation

Both quantitative and qualitative program evaluation data can be useful when improving interventions by indicating what elements to modify, to discontinue, or to continue implementing. As mentioned, participants enjoyed the medium of text, indicating that future research should continue to explore this delivery method. Two participants also had positive comments about nightly EMAs delivered tangential to this study, stating they helped them stay focused on health throughout the day and in reflection on the resulting data of their own behaviors. Texting has the benefit of allowing for high frequency, low intensity interventions. Texting is also accessible, cost-effective, and easy to implement as compared to in-person services (Gaziano et al., 2015; Smith et al., 2020). Mobile phone prompts for self-guided interventions or fitness and diet tracking applications could also provide a means to increase engagement with content.

Recent research has indicated other methods to increase the success of mobile health interventions. The integration of behavioral design and design theory allows for interventions to incorporate behavior change strategies with efficient delivery methods and user experience considerations (Voorheis et al., 2022). A bottom-up method of co-creating intervention content and delivery could help ensure engagement and satisfaction of participants (Arevian et al., 2018; Cyril et al., 2015; Walser et al., 2024; Wright et al., 2016). Researcher population collaborations can also ensure that participant and user needs are being met. The target community has valuable input about their needs, values, and preferences that researchers may not be able to anticipate, as well as the inclusion of preferences and needs to ensure more effective and ecologically valid interventions (Lerner et al., 2000). One might say that this is the primary purpose of all research in the social sciences, rather than research for its own sake (Kurtines et al., 1990; Lerner et al., 2000). Methods for co-creation could include student focus groups for varying aspects of intervention content and delivery and expansive program evaluations to inform future iterations of projects.

4.6. Knowledge Synthesis

Finally, if the primary aim of health interventions is to improve lives in a way that is appreciable for the target population, then clinical significance of resulting behavior change should be considered. Though statistical significance can reveal whether an effect is likely, effect sizes can demonstrate the practical significance of results (Harris et al., 2017; Ranganathan et al., 2015). The difference between baseline variables in this study had a range of effect sizes, from none to medium. Effect sizes across health domains could be compared to give information on the effectiveness of interventions for varying health outcomes and inform design. This would also allow future researchers to calculate sample sizes needed to obtain significant results and for inclusion of multiple studies in meta-analyses (Cohen, 1962; Tomczak & Tomczak, 2014). Once there are sufficient studies in the literature, it would be beneficial to follow-up with a meta-analysis of value treatment kernel interventions with an applied focus on values only.

5. Conclusions

Component analyses of valuing as a mechanism of change are vital to predict and influence outcomes in research using process-based therapies such as ACT. Indeed, the recent Report of the Association for Contextual Behavioral Science Task Force (Hayes et al., 2021) on the strategies and tactics of contextual behavioral science research highlights the need for identifying mechanisms of change, technological dissemination, idiographic experimental analogues, and longitudinal studies as paths toward understanding manipulable variables for behavior change.

In summary, there is a dearth of research exploring valuing interventions for health behavior change, especially in college populations. This study compared a valuing intervention to a traditional informational intervention to address this need. The non-significant results highlight the importance of incorporating student feedback and co-development of the design of such interventions. Further, these results point to a need for pulling from design theory with traditional development approaches to better tailor interventions for college populations. Technologically delivered health behavior change interventions can be explored as an adjunct to therapy to enhance the effectiveness of both therapeutic treatments and EMIs. Further, study methods should include research design, measures, data collection strategies, and analytic methods that reflect the ideographic nature of the valuing process in affecting personal change.