Elizabeth Chen
A dissertation submitted to the faculty at the University of North Carolina at Chapel Hill in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department
of Health Behavior in the Gillings School of Global Public Health.
Chapel Hill 2019
Elizabeth Chen: Development and Validation of a New Scale to Measure the Acceptability of Mobile Health Applications Among Adolescents
(Under the direction of Kathryn E. Moracco)
Background. Most U.S. adolescents ages 13 to 17 use smartphones, so there is
tremendous potential to use smartphones to deliver health interventions to adolescents. A health app’s acceptability is an important precursor of usage. However, no validated measures of health app acceptability among adolescents currently exist. Therefore, I aimed to develop and validate a scale to measure the acceptability of health apps among adolescents.
Methods. In Study 1,I conducted interviews with participants (n=19) in grades 7-9 to understand how adolescents assess health app acceptability. In Study 2, I developed an acceptability scale based on the Theoretical Framework of Acceptability, administered it to participants (n=182) testing the Fooducate app, and revised the scale after analyzing the
psychometric properties and conducting exploratory factor analysis. In Study 3, a new participant sample (n=161) tested the Fooducate app and completed surveys that included the revised
acceptability scale items. One week later, I collected data from participants’ phones about their Fooducate usage. I conducted confirmatory factor analysis and used structural equation modeling to assess the relationship between acceptability and Fooducate usage.
from 0.77 to 0.90). EFA results indicated that the 22-item scale had a six-factor structure. With a separate sample of participants, I used CFA and confirmed that all six subscales were
unidimensional and that the 22-item scale had the predicted six-factor structure. However, CFA did not confirm a higher order latent factor model for acceptability. Ethicality (=0.018, p<0.05) and self-efficacy (=5.16, p<0.05) predicted usage after one week.
Conclusion. The 22-item AHAA scale and subscales are reliable and CFA results
I would like to acknowledge the mentors and peers who have supported me in this dissertation journey and throughout my doctoral training. First and foremost, I want to thank Dr. Beth Moracco: Thank you for your patience, guidance, and advocacy over the past five years and for giving me the autonomy to pursue academic and professional training opportunities that aligned with my research interests. Your brave leadership continues to inspire me and I look forward to future collaborations.
To my other committee members, Drs. Susan Ennett, Kirsten Kainz, Kate Muessig, and Deb Tate: Thank you for your thoughtful feedback on my dissertation and for supporting my growth as a researcher throughout my training. Each of you has held me to high expectations and modeled what effective and supportive mentorship looks like.
To my peers: I learned so much from each of you! I am grateful to have completed my training alongside scholars who will be changemakers in their respective fields. In particular, I want to thank Sarah Kowitt, Cristina Leos, and Yesenia Merino for supporting my growth and understanding in a variety of areas, including design thinking, intersectionality, and anti-racist practices and pedagogy. I also want to thank Vichi Jagannathan and Meg Landfried for being outstanding thought partners and for showing me what community-led projects look like.
LIST OF TABLES ... xi
LIST OF FIGURES ... xiii
LIST OF ABBREVIATIONS ... xiv
CHAPTER 1: INTRODUCTION ... 1
Overview ... 1
Dissertation Overview ... 1
Study 1 Overview ... 3
Study 2 Overview ... 3
Study 3 Overview ... 3
Significance to Public Health ... 4
Organization of the Dissertation ... 4
CHAPTER 2: BACKGROUND AND RESEARCH GAPS ... 6
Background ... 6
Conceptual Definition of Acceptability ... 9
Definitions and Operationalization of Related Terms ... 10
Acceptability Research Methods ... 12
Current Acceptability Theories ... 12
Synthesis of Research Gaps ... 15
CHAPTER 3: THEORETICAL FOUNDATION ... 16
Theoretical Framework of Acceptability ... 16
Conceptual Model, Specific Aims, and Research Questions ... 20
CHAPTER 4: ADOLESCENTS’ ASSESSMENTS OF HEALTH AND APP ACCEPTABILITY: A QUALITATIVE STUDY ... 22
Overview ... 22
Background ... 23
Theoretical Framework of Acceptability ... 24
Purpose ... 25
Methods... 25
Overview ... 25
Recruitment ... 26
Data Collection ... 27
Analysis ... 29
Results ... 30
Themes ... 32
Discussion ... 42
Limitations ... 45
Strengths ... 46
Conclusion ... 47
CHAPTER 5: ACCEPTABILITY OF HEALTH APPS AMONG ADOLESCENTS: THE DEVELOPMENT AND VALIDATION OF A NEW SCALE ... 48
Overview ... 48
Background ... 49
Conceptual Definition of Acceptability ... 51
Theoretical Framework of Acceptability ... 52
Purpose ... 55
Study 2 ... 56
Overview ... 56
Methods ... 57
Results... 66
Study 3 ... 82
Overview ... 82
Methods ... 83
Results... 87
Discussion ... 95
CHAPTER 6: SYNTHESIS ... 104
Implication and Future Research ... 106
Limitations ... 107
Strengths ... 108
Conclusion ... 108
APPENDIX A: DEMOGRAPHIC FORM PRE-INTERVIEW 1 ... 110
APPENDIX B: STUDY 1 INTERVIEW GUIDES ... 113
APPENDIX C: CODEBOOK WITH PARENT CODES AND SUB-CODES ... 118
APPENDIX D: INITIAL ITEM POOL ... 123
APPENDIX E: SURVEY FOR COGNITIVE INTERVIEWS AND PILOT TESTING ... 126
APPENDIX F: INTERVIEW GUIDE FOR COGNITIVE INTERVIEWS ... 134
APPENDIX G: STUDY 2 SURVEY ... 136
APPENDIX H: STUDY 3 SURVEY ... 148
LIST OF TABLES
Table 1. Theoretical Framework of Acceptability Domains ... 17
Table 2. Specific Aims and Research Questions ... 21
Table 3. Theoretical Framework of Acceptability Domains ... 24
Table 4. Interview Guide Questions for Theoretical Framework of Acceptability Domains ... 29
Table 5. Study 1 Participant Characteristics, (n=19) ... 31
Table 6. Change within Acceptability Domains Between Interview 1 and Interview 2 ... 33
Table 7. Illustrative Quotes for Reasons for Perceived Effectiveness of Health App ... 37
Table 8. Illustrative Quotes for Reasons for No Perceived Effectiveness of Health App ... 38
Table 9. 2x2 Table for Self-Efficacy Vs. Intention to Use Health App ... 39
Table 10. Theoretical Framework of Acceptability Domains ... 52
Table 11. 7-Step Scale Development Process for Study... 58
Table 12. Sample Characteristics for Exploratory Factor Analysis (EFA) Sample, (n=182) ... 67
Table 13. Means, Standard Deviations, And Range of Responses from EFA Sample... 68
Table 14. Cronbach’s Alpha and Inter-Item Correlations for Each Acceptability Subscale ... 70
Table 15. Factor Loadings and Communalities for 9-Factor Model of 52 Items Included in Original Acceptability Scale ... 72
Table 16. Original and Revised Cronbach’s Alpha with Item Deletions ... 77
Table 17. Factor Loadings and Communalities for Revised 22-Item Scale ... 79
Table 18. Cronbach’s Alphas and Inter-Item Correlations for Subscales of 22-Item Acceptability of Health Apps among Adolescents (AHAA) Scale ... 81
Table 20. Descriptive Statistics for Confirmatory Factory Analysis (CFA) Sample ... 89 Table 21. Fit Statistics for Confirmatory Factor Analysis Single Factor Models ... 91 Table 22. Factor Loadings, R2 Values, and Cronbach’s Alphas for Six Subscales ... 92 Table 23. Factor Loadings, R2 Values, and Cronbach’s Alphas for Higher Order
six-factor Acceptability Model ... 93 Table 24. Latent Variable Predictors and Regression Coefficients from Structural
Equation Modeling Predicting Usage ... 95 Table 25. Acceptability of Health App Scale Items ... 102 Table 26. Simple Linear Regression of Acceptability Domains on Actual Health
LIST OF FIGURES
Figure 1. Theoretical Framework of Acceptability... 13
Figure 2. Technology Acceptance Model ... 14
Figure 3. Theoretical Framework of Acceptability... 18
Figure 4. Technology Acceptance Model ... 18
Figure 5. Proposed Conceptual Model for the Dissertation ... 21
Figure 6. Conceptual Model of Acceptability ... 26
Figure 7. Updated Conceptual Model for Acceptability ... 32
Figure 8. Proposed Conceptual Model for Acceptability ... 54
Figure 9. Measurement Model of Six Factors of Acceptability... 85
Figure 10. Measurement Model of Higher Order Latent Variable Acceptability and First Order Latent Variables... 86
LIST OF ABBREVIATIONS
App Application
CFA Confirmatory factor analysis CFI Comparative fit index DF Degrees of freedom
EFA Exploratory factor analysis eHealth Electronic health
mHealth Mobile health
RMSEA Root mean square error of approximation SEM Structural equation model
SRMR Standardized root mean square residual TAM Technology Acceptance Model
TFA Theoretical Framework of Acceptability TLI Tucker Lewis index
CHAPTER 1: INTRODUCTION
Overview
The acceptability of mobile health (mHealth) applications, hereafter referred to as “health apps”, among adolescents is neither well-defined nor consistently operationalized. Therefore, it can be difficult, time-consuming, and expensive for researchers and app developers to assess the acceptability of health apps among adolescents – whether adolescents would use the health app in their everyday lives – during the app development process. Acceptability is important to assess because health interventions that are more acceptable are more likely to be used, and usage is necessary to bring about the intended outcomes of the health intervention (Sekhon, Cartwright, & Francis, 2017). For my dissertation,I first developed and validated a scale to measure the acceptability of health apps among adolescents, and second, I assessed the
relationship between acceptability of health apps and actual health app usage. This research was intended to result in a measurement tool that researchers and app developers may use to quickly and accurately assess the acceptability of health apps among adolescents during the app
development phase, which will save time, money, and resources, and thereby facilitate the more efficient creation of effective, usable, and scalable health apps for adolescents.
Dissertation Overview
among adolescents, very few adolescents actually download and use health apps. Research shows that interventions that are more acceptable are more likely to be used in the real world, and usage is needed to result in the desired health outcomes (Sekhon et al., 2017). Therefore, it is important to study the acceptability of health apps among adolescents to maximize their adoption and usage in the real world, in order to yield the desired health outcomes.
There are several gaps in the published literature and existing theoretical frameworks and models on acceptability of health apps. First, while the acceptability of health interventions more broadly has been studied, researchers define and operationalize acceptability inconsistently when it comes to the study of acceptability of health apps. Second, existing acceptability frameworks or models were not developed for health apps specifically; many were developed for in-person healthcare interventions on desktop computers. Third, there is a lack of quantitative research to measure the relationship between the acceptability of a health app and actual usage of that same health app. Lastly, the majority of acceptability research on health interventions has been conducted with adults and not adolescents, so additional acceptability research with adolescents is needed, especially in the field of mHealth, given the prevalence of adolescents’ access to phones.
In order to address these gaps, I designed three studies for this dissertation with the following three specific aims:
• Aim 1: Describe how adolescents in grades 7 through 9 assess the acceptability of
health apps.
• Aim 2: Assess the psychometric properties of a new scale developed to measure
• Aim 3: Assess the relationship between acceptability of the Fooducate food
tracker app and actual Fooducate app usage.
Study 1 Overview
The purpose of Study 1 was to describe how adolescents in grades 7 through 9 define and conceptualize the acceptability of health apps and to develop a new scale to measure the latent construct of acceptability of health apps. In the first phase of Study 1 (Aim 1), I conducted pairs of qualitative interviews with adolescents in grades 7 through 9 to understand how
adolescents assess the acceptability of different health apps. Building from the conceptual definition of acceptability put forth by the Theoretical Framework of Acceptability (TFA) (Sekhon et al., 2017) that consisted of seven domains (affective attitude, burden, ethicality, intervention coherence, opportunity costs, perceived effectiveness, and self-efficacy), I employed a Grounded Theory approach to collect and analyze qualitative data to assess the salience of each of these domains of acceptability among adolescents.
Study 2 Overview
In Study 2 (Aim 2), I followed DeVellis’ scale development process (DeVellis, 2016) and administered the draft scale to a sample of adolescents in grades 7 through 9 as they tested the Fooducate food tracker app for the first time. Then, I assessed the psychometric properties of the draft acceptability scale and adhered to Múthen and Múthen’s (2017) guidelines for exploratory factor analysis (EFA) to assess the factor structure of the instrument developed. Lastly, I revised the acceptability scale so that it could be administered to a separate sample in Study 3.
Study 3 Overview
between acceptability of the Fooducate app and actual usage of the nutrition tracker app among a sample of adolescents in grades 7 through 9 in a one-week period.
Significance to Public Health
My dissertation provides the fields of adolescent health and mHealth with a deeper understanding of how to define and conceptualize the acceptability of health apps among adolescents. The scale permits researchers to uniformly assess the acceptability of health apps and collect quantitative data to complement qualitative data on acceptability. These quantitative data will also help researchers as they build and test earlier iterations of their apps so that they can build more acceptable apps during their development phases, thus leading to more effective and more scalable interventions. The development of this scale allowed me to test whether there is an association between acceptability of health apps and actual usage of the health apps among adolescents. The results of this proposed research could efficiently guide the development of future apps with improved acceptability and lead to future research to link acceptability with usage and then intended outcomes (e.g. knowledge, attitudes, behaviors).
Organization of the Dissertation
This dissertation has six chapters. The first chapter includes an introduction to the dissertation, an overview of the three studies proposed, and the significance of research on the acceptability of health apps among adolescents to the broader fields of mHealth and adolescent health. Chapter 2 synthesizes previous empirical research on the acceptability of health
interventions and highlights the research gaps which informed this dissertation study. Chapter 3 reviews the existing theoretical frameworks and models related to acceptability and their
CHAPTER 2: BACKGROUND AND RESEARCH GAPS
Background
Mobile technologies (e.g. smartphones, tablets, personal digital assistants) can deliver mobile health (mHealth) interventions that are affordable, easy to use, and that can be widely adopted across socioeconomic and age groups in the United States (US) and globally (Klasnja & Pratt, 2012). Distinct from the broader field of electronic health (eHealth) which involves using the Internet and related technologies to organize and deliver health services (Boogerd, Arts, Engelen, & van de Belt, 2015; Mechael & Mechael, 2009), mHealth is defined as using mobile communications for health interventions and information and involves the use of portable devices that are capable of creating, storing, retrieving, and transmitting data in real time to improve the safety, quality of care, and health of end users (Akter & Ray, 2014). There are many advantages to mHealth interventions compared to delivering an in-person public health
intervention, including cost-effective dissemination, real-time data collection and feedback, lowered participant burden, and flexibility for tailoring (Tate et al., 2013). As the field of
mHealth continues to grow, there is increasing evidence that mHealth interventions are effective in changing an array of health behaviors and improving health outcomes (Cafazzo, Casselman, Hamming, Katzman, & Palmert, 2012; Free et al., 2013; Klasnja & Pratt, 2012; Tate et al., 2013).
mHealth researchers to assess the acceptability of their interventions early and often during the development process, especially given that building an intervention that involves technology is expensive and time consuming. More specifically, acceptability of an mHealth intervention should not be assessed for the first time during a formal pilot study; if the acceptability of the intervention is determined to be low at that point, valuable time and resources will have been wasted during development that could have been used to build something more acceptable to the priority population.
There is growing interest in leveraging mobile technology to change health behaviors among adolescent populations. Approximately 95% of adolescents ages 13 to 17 have consistent access to smartphones and that percentage continues to increase (Pew Research Center, 2018b). Among adolescents with smartphones, 45% of adolescents say they are on the internet, either on a computer or a cellphone almost constantly, with another 45% that say they are on the internet several times a day (Pew Research Center, 2018b). Many adolescents use social media apps on their smartphones regularly, with 72% of adolescents using Instagram (up from 52% in 2015) and 69% using Snapchat (up from 41% in 2015) (Lenhart, 2015; Pew Research Center, 2018b). Reaching adolescents on their smartphones to target health behaviors may be an effective health promotion strategy, given widespread smartphone ownership among adolescents, and the fact that they spend a lot of time on their smartphones. The literature on mHealth interventions for adolescents is scarcer than that for adults, but there is emerging evidence that mHealth
I will focus on the acceptability of health apps in this dissertation study given that thousands of health apps are readily available for adolescents to access and download on their smartphones.
We know that adolescents currently do not use health apps to the same extent that they use social media apps. In 2015, there were approximately 165,000 health apps available for download (Misra, 2015). These health apps focused on fitness (36%), lifestyle and stress (17%), diet and nutrition (12%), specific diseases (9%), women’s health and pregnancy (7%),
medication reminders and information (6%), healthcare providers and insurance (2%), and other topics (11%) (Misra, 2015). According to a recent study, only 21% of U.S. adolescents have downloaded a health app on their smartphone compared to 36% of U.S. adults (Bhuyan, Lu, Chandak, & Kim, 2016; Center on Media and Human Development, 2015). Among adolescents who have downloaded a health app, 47% hardly ever or never use them, 45% sometimes use them, and only 8% use them often (Center on Media and Human Development, 2015). Many health apps are unused or underused, especially among adolescents, so it is critical that mHealth researchers and app developers measure acceptability of these health apps so that they can be more regularly adopted and used in order to achieve their intended impact (Center on Media and Human Development, 2015).
health), and had a higher income (U.S. $50,000 or more vs. U.S. $0-49,999) (Carroll et al., 2017). If we generalize these findings to adolescents, they would suggest that the majority of the adolescents who use health apps are likely be adolescents who report excellent health or who come from households with parents with higher education levels or higher incomes. These adolescents who are early adopters of health apps are likely not the high-priority sub-groups of adolescents for whom public health researchers are specifically developing new health apps. Therefore, it is important for mHealth researchers to be able to first measure how acceptable health apps are among adolescents (and sub-groups of adolescents), and then understand what makes some health apps more likely to be accepted by some adolescents than others. This will
enable researchers and developers to develop health apps that adolescents will want to use in
their everyday lives to achieve the intended health outcomes.
Conceptual Definition of Acceptability
Researchers define the acceptability of interventions differently, both in their conceptual and operational definitions, and this is problematic because the latent construct of acceptability cannot be measured consistently without a common definition. Conceptual definitions define a construct in abstract or theoretical terms, whereas an operational definition defines a construct by specifying the procedures used to measure that construct (Sekhon et al., 2017). Many studies claim to assess the acceptability of interventions, but the authors do not include conceptual or operational definitions in their published manuscripts, thus limiting others’ ability to compare or interpret results across studies (Ben-Zeev et al., 2014; Holloway et al., 2014; Mullen, Berry, & Zierler, 2004). Recently, Sekhon et al. (2017) recognized the need to conceptualize and
defined acceptability. For the purpose of this study, I have adopted the Samkange-Zeem et al. (2015) definition because it is broad and simple. For this dissertation study, I define the
acceptability of health apps as whether intended users (e.g. adolescents) would use the health app in their everyday lives.
Given the inconsistency in conceptual definitions of acceptability, it is not surprising that there is also inconsistency in the operationalization of the construct in research studies.
Acceptability studies for electronic health (eHealth) and mHealth interventions among
adolescents and young adults often include a range of Likert-scale items (Kothe & Mullan, 2014; Robinson et al., 2015; Sanchez & Bartel, 2015). Among a handful of acceptability studies, one set of researchers measured the acceptability of their online intervention by asking about look and feel, enjoyment, and helpfulness (Robinson et al., 2015). Another team of researchers assessed acceptability of their online intervention by including items on interest, credibility, and personal relevance (Kothe & Mullan, 2014). The final research team measured acceptability by assessing the extent to which adolescents thought the online intervention was fun, how much they liked it, whether it took the right amount of time to play, and whether they liked the overall look of the intervention (Sanchez & Bartel, 2015). Given this variability in items used to assess acceptability in these studies, it is difficult to compare the results of these interventions. There is a need for more rigorous, standardized processes for evaluating mHealth interventions in both domestic and international contexts (Lee et al., 2016; Tomlinson, Rotheram-Borus, Swartz, Tsai, & Niedzinski, 2013; Whittaker, Merry, Dorey, & Maddison, 2012).
Definitions and Operationalization of Related Terms
design feasibility studies, Bowen et al. define feasibility as “the extent to which an intervention can be delivered when resources, time, commitment, or some combination thereof are
constrained in some way” so feasibility focuses on the delivery of the intervention and does not have to do with whether users can or want to use the intervention. Next, usability has been defined as “the ease with which participants could use the respective functions to get the information they desired” (Samkange-Zeeb et al., 2015). Usability focuses more on the
functionality of the mHealth intervention and if users can successfully use the specific functions in the app but still does not address whether users want to use the intervention in their everyday life. The acceptability of mHealth interventions has also been linked with “satisfaction” (Ben-Zeev et al., 2014; Lodhia, Karanja, Lees, & Bastawrous, 2016; Payne, Lister, West, & Bernhardt, 2015) and “appropriateness” in the literature (Gonsalves et al., 2015). However, while feasibility, usability, satisfaction, and appropriateness may represent distinct aspects of acceptability, these terms are still not synonymous. Yet, there is no theoretical model that delineates the relationship among these separate but related constructs. All in all, a clearly-defined and consistently-used conceptual definition of acceptability would facilitate more consistent operationalization of acceptability in research and enable researchers to compare findings across studies and build a more solid evidence-based for health apps.
functions in this system were well integrated” and “I found the system very cumbersome to use” (Lewis & Sauro, 2009).
Acceptability Research Methods
Currently, most researchers rely on qualitative rather than quantitative methods to assess the acceptability of health apps. For example, researchers often describe the use of interviews and think-alouds to collect qualitative data about acceptability (Hickman, Jacobson, & Melnyk, 2015). The development of an acceptability scale would make it easier for researchers to collect quantitative data in formative studies, pilot studies, and larger efficacy and effectiveness studies. The purpose of the proposed approach to developing an acceptability scale is to complement the existing qualitative findings and provide researchers with a tool that they can use to collect quantitative data to inform decision-making in their health app development process.
Additionally, this scale will assist researchers to assess the relationship between acceptability of the health app and actual health app usage.
Current Acceptability Theories
Figure 1. Theoretical Framework of Acceptability
This framework was developed by analyzing and synthesizing published peer-review literature on healthcare interventions, so the TFA may not apply to mHealth interventions like health apps. To date, there are no peer-reviewed publication that explicitly test the TFA for health apps. Additionally, the TFA presents these seven domains of acceptability but did not operationalize these domains so that researchers could uniformly assess these latent constructs. There is also no indication about which of these seven domains is most important for acceptability. Lastly, the systematic reviews that were reviewed for the development of the TFA focused on studies whose participants were adults and not adolescents, so there is a need for further exploration to
determine whether acceptability as conceptualized by the TFA is salient among an adolescent population engaging with health apps instead of healthcare interventions.
Second, the Technology Acceptance Model (TAM) (Figure 2) is another popular theory that was created in the 1980s when desktop computers were gaining popularity (Davis, Bagozzi, & Warshaw, 1989). TAM posits that perceived usefulness (U) and perceived ease of use (E) predict attitude toward using a particular technology (A) (Davis et al., 1989).
Affective(attitude
Burden
Ethicality
Intervention(coherence
Opportunity(costs
Perceived(effectiveness
Figure 2. Technology Acceptance Model
Attitude toward using the technology is predictive of behavioral intention to use the technology (BI) which then predicts actual system use. Although the Technology Acceptance Model (Figure 2) has the word “acceptance” in its name, the term “acceptance” is not found in the model itself. Instead, it is inferred that the acceptance of the technology intervention is captured by two constructs: perceived usefulness (U) and perceived ease of use (E). While this model explicitly references “technology”, the technology available when this model was developed was limited to desktop computers and not portable mobile devices. The TAM and newer iterations of the model have been applied to mHealth interventions, but there is still little consistency for how these constructs are operationalized and no published research on how well this model predicts outcomes (i.e. actual system use), or outcomes among adolescents (Brown, Yen, Rojas, & Schnall, 2013b; Campbell et al., 2017; Mohamed, Tawfik, Al-Jumeily, & Norton, 2011; Mohr, Schueller, Montague, Burns, & Rashidi, 2014).
Theoretical frameworks such as the TAM and TFA were not developed specifically for mHealth interventions like health apps, and health apps likely differ from other types of interventions (e.g. in-person, eHealth). Additionally, current acceptability models and frameworks were developed for adults and not adolescents. Adolescents use technology
differently than adults and the two populations are in different developmental stages (Bhuyan et al., 2016; Center on Media and Human Development, 2015; Pew Research Center, 2018a), so it
and adults. Therefore, it is important to provide further conceptual clarification to elucidate the aspects of acceptability of health apps that are most salient to adolescents and assess the relationship between acceptability of health apps and actual usage of the health apps.
Synthesis of Research Gaps
To summarize, there are several major gaps in the published literature and available theoretical frameworks and models on the acceptability of mHealth app for adolescents that my dissertation research aims to address.
1. The acceptability of health apps among adolescents has not been well studied. 2. There is inconsistency concerning the definition and the operationalization of the
construct “acceptability”, especially in terms of the acceptability of health apps. 3. Existing acceptability frameworks or models were not developed for health apps. 4. There is a lack of quantitative research that measures the relationship between
acceptability of health apps with actual usage of health apps.
CHAPTER 3: THEORETICAL FOUNDATION
The proposed research is based on two complementary conceptual frameworks: the Theoretical Framework of Acceptance (TFA) and the Technology Acceptance Model (TAM). The TFA conceptualizes distinct factors within acceptability, whereas the TAM conceptualizes how the acceptability construct predicts actual (technology) system use. However, neither of these existing models is specific to mHealth interventions nor to adolescents. Through my dissertation research, I aim to build on existing research and conceptual frameworks and develop a new scale to specifically measure the acceptability of health apps among adolescents. As noted in the previous chapter, the majority of adolescents in the U.S. have access to smartphones and use them frequently (Lenhart, 2015; Pew Research Center, 2018b). There is also growing evidence that mHealth interventions lead to improved health outcomes among adolescents (Badawy et al., 2017; Cafazzo et al., 2012; Tate et al., 2013). In order to ensure that the mHealth interventions like health apps for adolescents are in fact used by adolescents, we must improve the acceptability of health apps.
Theoretical Framework of Acceptability
burden, ethicality, intervention coherence, opportunity costs, perceived effectiveness, and self-efficacy. The definitions provided by the authors are described in Table 1 below.
Table 1. Theoretical Framework of Acceptability Domains
Domains Anticipated Definitions
Affective attitude How an individual feels about the intervention
Burden The perceived amount of effort that is required to participate in the intervention
Ethicality The extent to which the intervention has good fit with an individual’s
value system
Intervention coherence
The extent to which the participant understands the intervention and how it works
Opportunity costs The extent to which benefits, profits or values must be given up to engage in the intervention
Perceived effectiveness
The extent to which the intervention is perceived as likely to achieve its purpose
Self-efficacy The participant’s confidence that they can perform the behavior(s)
required to participate in the intervention
Figure 3. Theoretical Framework of Acceptability
In the conceptual model above, each of the seven domains is depicted as a sub-construct of acceptability. Through my research, I will continue to refine this model by adding or deleting constructs as I conduct qualitative research with adolescents and operationalize these sub-constructs for health apps for adolescents.
Technology Acceptance Model
The Technology Acceptance Model (TAM) (Figure 4) also informed my dissertation, as this model links the acceptability of information technology and actual system (e.g., mHealth intervention) use. This model was originally designed as computers gained popularity in the 1980s (Davis et al., 1989).
Figure 4. Technology Acceptance Model
Affective(attitude
Burden
Ethicality
Intervention(coherence
Opportunity(costs
Perceived(effectiveness
Self:efficacy Acceptability
Adapted from the Theory of Reasoned Action (TRA) and the Theory of Planned Behavior (Holden & Karsh, 2010), TAM has been applied to numerous public health issues across a variety of populations (Ahadzadeh, Pahlevan Sharif, Ong, & Khong, 2015; Strudwick, 2015) and adapted several times (Campbell et al., 2017; Shih, 2003; Venkatesh & Bala, 2008; Venkatesh, Davis, Smith, & Walton, 2000). Although the TAM (Figure 4) has the word “acceptance” in the name of the model, the term is not found in the model itself. Rather, it is inferred that the acceptance of the information technology is captured by two constructs: perceived usefulness (U) and perceived ease of use (E). These two constructs predict attitude toward using the information technology (A). Attitude (an individual’s evaluative judgment of the target behavior on some dimension) predicts behavioral intention (BI) (individual’s
motivation or willingness to exert effort to perform the target behavior) which then predicts actual system use (Holden & Karsh, 2010). Attitude and behavioral intention are constructs that are also included in the Theory of Reasoned Action (TRA) and the Theory of Planned Behavior (TPB) (Holden & Karsh, 2010). The TAM differs from TRA and TPB because it focuses specifically on technology use, whereas TRA and TPB can be applied to a variety of health behaviors. With regards to TAM, the definition of use is for one specific behavior of interest performed by individuals with regard to some information technology (IT) system (Holden & Karsh, 2010). TAM also includes a direct pathway from perceived usefulness to behavioral intention.
Moreover, some of these mHealth studies that apply TAM do not explicitly measure TAM constructs. For example, a study by Arteaga, Kudeki & Woodworth (2009) described the development of an mHealth intervention for teenagers to combat obesity. While the authors stated that TAM, TPB, the Theory of Meaning Behavior, and the Big 5 Personality Theory were used to guide the design of the intervention, they did not describe how TAM guided intervention develop. Furthermore, Arteaga, Kudeki & Woodworth (2009) did not explicitly measure the TAM constructs nor test the expected relationships between constructs (Arteaga et al., 2009).
Other studies have not found evidence that supports the linkage between TAM constructs and actual technology system use. For example, Lim et al. (2011) measured TAM constructs while assessing the acceptability of using mobile phones to seek health information among Singaporean women. Lim et al. (2011) found that perceived usefulness predicted behavioral intention to use mobile phones for health information. However, the expected relationship between behavioral intention and actual usage was not significant.
Conceptual Model, Specific Aims, and Research Questions
Figure 5. Proposed Conceptual Model for the Dissertation
My dissertation is divided into three studies. Study 1 focused on understanding how adolescents in grades 7 through 9 defined and conceptualized the acceptability of health apps (Aim 1). In Study 2, I developed a new acceptability scale and administered it to a sample of adolescents in grades 7 through 9 to assess its psychometric properties and validity (Aim 2). Then, in Study 3 I administered the revised acceptability scale to a new sample of adolescents in grades 7 through 9, predicted its factor structure, and measure, and assessed the relationship between health app acceptability and actual health app usage (Aim 3). Table 2 lists the aims of this dissertation as well as the associated research questions.
Table 2. Specific Aims and Research Questions
Specific Aims Research Questions
Aim 1: Describe how adolescents in grades 7 through 9 assess the acceptability of health apps.
RQ1: How do adolescents in grades 7 through 9 assess the acceptability of health apps?
Aim 2: Assess the psychometric properties of a new scale developed to measure the
acceptability of health apps for adolescents.
RQ2: What are the psychometric properties of a newly developed scale to measure the acceptability of health apps among
adolescents?
Aim 3: Assess the relationship between acceptability of the Fooducate app and actual Fooducate app usage.
RQ3: Does the acceptability of the Fooducate health app predict actual Fooducate app usage?
The next two chapters (Chapters 4 and 5) include two manuscripts that describe the three studies designed to answer these research questions. Then, Chapter 6 will synthesize findings across the three studies and propose future directions for research and the studies’ implications for research, practice, and policy.
Acceptability+of+ health+app
Health+ outcomes Actual+health+app+
CHAPTER 4: ADOLESCENTS’ ASSESSMENTS OF HEALTH AND APP ACCEPTABILITY: A QUALITATIVE STUDY
Overview
Adolescents spend a lot of time on their phones engaging with social media applications (i.e. apps) but significantly fewer adolescents have used health apps. While there is a
proliferation of apps – both health and non-health – in the market, there is potential to leverage health apps to improve adolescent health at scale. However, the acceptability of health apps among adolescents is neither well-defined nor consistently operationalized. The purpose of this study was to understand how adolescents in grades 7 through 9 assess the acceptability of health apps and determine whether they want to use them in their everyday life. Using a Grounded Theory approach and building on the Theoretical Framework of Acceptability (TFA), this study collected qualitative data from 19 participants via paired interviews. Results indicated that there were nine domains of acceptability among this adolescent population: affective attitude, burden, ethicality, intervention coherence, opportunity costs, perceived effectiveness, self-efficacy, intervention expectations, and peer norms. In the future, these results will inform the
Background
In the United States, 95% of adolescents ages 13 to 17 have consistent access to smartphones and that percentage continues to grow; additionally, the digital divide between higher income and lower income adolescents has lessened, as well as the divide between White adolescents and non-White adolescents (Pew Research Center, 2018b). Among adolescents with smartphones, 90% say they are on the internet at least several times a day, with half of connected adolescents saying they are on the internet constantly (Pew Research Center, 2018b).
Adolescents use social media apps on their smartphones very often, with 72% of adolescents using Instagram and 69% using Snapchat (Lenhart, 2015; Pew Research Center, 2018b). While there is less literature on health apps for adolescents than literature on health apps for adults, emerging evidence indicates that mHealth interventions like health apps are effective among adolescents in changing behaviors related to a range of health topics, including: obesity prevention, type 2 diabetes management, and medication adherence (Badawy et al., 2017; Cafazzo et al., 2012; Tate et al., 2013). Therefore, reaching adolescents on their smartphones to target health behaviors may be an effective health promotion strategy given widespread
smartphone ownership among adolescents and the fact that adolescents spend a lot of time on their smartphones.
Only 21% of U.S. adolescents have ever downloaded a health app on their smartphone compared to 36% of U.S. adults (Bhuyan et al., 2016; Center on Media and Human
interventions that are more acceptable are more likely to be adopted or used, and usage is necessary to bring about the intended outcomes of the intervention (Sekhon et al., 2017). We define the acceptability of health apps as whether intended users (e.g. adolescents) would use the health app in their everyday lives.
Theoretical Framework of Acceptability
Recent work by Sekhon, Cartwright, & Francis (2017) provided some clarity with regards to the conceptualization of acceptability specific to health interventions. The authors emphasized that acceptability is a necessary but not sufficient condition for effectiveness of an intervention (Sekhon et al., 2017). After employing inductive and deductive methods to synthesize 43 systematic reviews, the authors presented their Theoretical Framework of Acceptability (TFA) consisting of seven domains of acceptability pertaining to health-related interventions: affective attitude, burden, ethicality, intervention coherence, opportunity costs, perceived effectiveness, and self-efficacy. The definitions provided by the authors are described in Table 3.
Table 3. Theoretical Framework of Acceptability Domains
Domain Definitions
Affective attitude How an individual feels about the intervention
Burden The perceived amount of effort that is required to participate in the intervention
Ethicality The extent to which the intervention has good fit with an individual’s value system
Intervention coherence The extent to which the participant understands the intervention and how it works
Opportunity costs The extent to which benefits, profits or values must be given up to engage in the intervention
Perceived effectiveness The extent to which the intervention is perceived as likely to achieve its purpose
Self-efficacy The participant’s confidence that they can perform the behavior(s)
Sekhon et al. (2017) emphasized that acceptability of a health-related interventions should be assessed at multiple timepoints during intervention implementation because there are three types of acceptability: prospective acceptability prior to participating in the intervention, concurrent acceptability while participating in the intervention, and retrospective acceptability after participating in the intervention. It is possible that participants’ perceptions of acceptability change across these three timepoints so it is important to assess acceptability at multiple
timepoints to see if that is the case. Sekhon et al. (2017) also proposed that the TFA should be applied to collect quantitative data via surveys or visual analogue rating scales based on the TFA domains throughout all stages of a health-related intervention’s development and evaluation cycle. Sekhon et al. published their TFA framework in 2017 and few researchers have applied it to mHealth research in the published literature to date.
Purpose
The purpose of this qualitative study was to understand how adolescents assessed the acceptability of a health app (i.e. whether they would want to use a particular health app in their everyday life) because the greater the acceptability of a public health intervention, the greater likelihood of its usage (Sekhon et al., 2017). We planned for this qualitative study to inform the future development of one or more measurement tools designed to measure the acceptability of health apps among adolescents in grades 7 through 9 (see Chapter 5).
Methods
Overview
interviews. The two interviews per participant allowed us to assess whether the acceptability perceptions (across acceptability domains and within acceptability domains) remained stable across time and whether additional factors contributed to adolescent’s assessment of the app’s acceptability after their initial interaction with the app. The second interview also served as an opportunity to ask participants follow-up questions based on findings that emerged in the first round of interviews.
Figure 6. Conceptual Model of Acceptability
Recruitment
We recruited19 participants in grades 7 through 9 from five schools in North Carolina in April and May 2018. These five schools were a convenience sample of area schools whose school leaders responded to email requests from the research team in early 2018. Recruitment letters were sent home with all students in grades 7, 8, and 9 in these schools and interested participants completed a demographic form and submitted it to a designated teacher or school administrator along with completed parental consent and participant assent forms. The
demographic form (Appendix A) asked participants for basic demographic information and how
Affective attitude
Burden
Ethicality
Intervention coherence
Opportunity costs
Perceived effectiveness
their menstrual cycle (as applicable), getting enough sleep, and reducing stress or anxiety. Participants were asked to rate how much they cared about each health topic using a four-point Likert scale varying from “A lot” to “Not at all” with a “N/A” option for the menstrual cycle question.
The study aimed to recruit a diverse sample based on gender (at least 8 of both male and female), race/ethnicity (at least 2 participants who identified as White, at least 2 who identified as African American, and at least 2 who identified as Latino), and urbanicity (at least 2
participants who attended an urban school, at least 2 participants who attended a rural school). The demographic forms and characteristics of the first 20 enrollees were examined and these criteria were met. There were three enrollees who were waitlisted. Participants needed to have their own smartphone to participate in the study, and students who were enrolled in an
Occupational Course of Study (OCS) track were not eligible to participate because our research team did not have the training or skills required to engage with students with their specific learning disabilities. This study was approved for human subjects research by the Institutional Review Board at The University of North Carolina at Chapel Hill (IRB Number 17-295).
Data Collection
Each participant was contacted by phone or emailed to schedule the first of two in-person semi-structured interviews. Participants were asked to download a health app to their phone and evaluate the acceptability of the health app over the course of two interviews at two distinct time points. A second interview was scheduled for two- to four-weeks after the first interview.
topics were deemed “valued” if the participant had indicated that they cared about this topic “a lot” in their demographic form). If participants indicated that they cared a lot for multiple health topics, they were offered health apps for each of those topics to choose from. The health apps we selected for use were recommended for teens by Teen Vogue or were listed as top Health & Fitness apps in the iTunes App Store at the time of the study (May 2018 – June 2018)
(McNamara, 2016; Pennell, 2015). Participants were asked to download one of the following free apps on their own smartphones: BodySpace (exercising), Fooducate (eating healthy),
Headspace (reducing stress or anxiety), Sleep Cycle (getting enough sleep), Spot On (monitoring your menstrual cycle), or WaterLogged (staying hydrated) (Bodybuilding.com, n.d.; Day Logger, n.d.; Fooducate, n.d.; Headspace Inc., n.d.; Planned Parenthood Federation of America, n.d.; Sleep Cycle AB, n.d.). Participants used the app for 3 to 5 minutes and then answered questions in-the-moment about the app’s acceptability.
Table 4. Interview Guide Questions for Theoretical Framework of Acceptability Domains
TFA Domain Sample interview guide question
Affective attitude How do you feel about using the app itself?
Burden How hard do you think it will be for you to use this app every day?
Ethicality How well does this app help you do something that is important to you?
Intervention coherence Describe how you think this app works.
Opportunity costs What might you miss out on by using the app?
Perceived effectiveness Do you think this app will help you [insert app purpose]? Why or why not?
Self-efficacy How confident are you that you can [insert behavior desired by app]? E.g. How confident are you that you can use the app to track your food calories?
In the second interview, participants were asked about their health app experience, including many of the same questions from Interview 1 that aligned with each of the seven acceptability domains proposed by Sekhon et al. (2017). Participants were asked if they still had the app on their phones. If yes, participants were asked if they had used the app at least once after the first interview. If no, participants were asked why they deleted the app. For participants who still had the app on their phone but said that they did not use the health app after the first interview, they were also asked why they chose not to return to the app and about their current acceptability assessment of the app.
The interviews lasted an average of 24 minutes and took place at the student’s school to lessen transportation barriers. Participants received a $30 VISA gift card after the second interview for the completion of both interviews. Each of the interviews was recorded and transcribed verbatim by the principal investigator.
Analysis
tools help analysts distance themselves from the technical literature and personal experience to see new possibilities in the data; avoid standard ways of thinking about phenomena; and stimulate the inductive process (Corbin & Strauss, 2008). Specifically, we used the following analytics tools: questioning, making comparisons, and looking at language (Corbin & Strauss, 2008). We developed a codebook by breaking data into “manageable pieces” and then
constructing concepts out of the data (i.e. open coding) (Corbin & Strauss, 2008). Then, ideas were assigned conceptual names or in vivo codes that closely reflected key words and phrases used by participants that were then grouped under themes (i.e. parent codes) that corresponded to each of the seven TFA constructs (Saldaña, 2015). For example, open coding led to grouping together quotations that described how it would be hard for participants to use the app. These quotations were assigned the in vivo code of “It’ll be hard to use the app” and grouped within the larger theme of “burden.” The codebook of parent codes and sub-codes are contained in
Appendix C.
The codebook was used to systematically code all transcripts using Atlas.ti version 1.0.36, and the principal investigator continued to memo and diagram relationships between the analytic concepts (i.e. axial coding).
Results
Instagram daily, 47% (n=9) reported using YouTube daily, 47% (n=9) reported using a music app daily (e.g. Spotify, Spinrilla), and 37% (n=7) reported using Snapchat daily. However, only 16% (n=3) of participants had a health app installed on their phone beyond the generic health apps that came pre-installed on their phones. One participant had the FitBit app and the other two had the activity app tracker Sweatcoin.
Table 5. Study 1 Participant Characteristics, (n=19)
Characteristic n %
Gender Male 10 52.6
Female 8 44.4
Non-binary 1 5.3
Grade 7th grade 4 21.0
8th grade 9 47.4
9th grade 6 31.6
Age 13 6 33.3
14 11 61.1
15 2 11.1
Race American Indian/Alaska Native 1 5.6
Asian 1 5.6
Black/African American 11 61.1
Native Hawaiian/Other Pacific Islander 0 0.00
White 5 22.2
Other 1 5.6
Ethnicity Hispanic/Latino 3 16.7
Rurality/urbanicity Rural 10 55.6
Non-rural 9 50.00
using the health app at least once after the first interview. Additionally, eight of the 19 participants (42%) reported using the health app at least twice after the first interview.
Themes
In addition to the seven Theoretical Framework of Acceptability domains (affective attitude, burden, ethicality, intervention coherence, opportunity costs, perceived effectiveness, and self-efficacy) there were two additional themes that emerged from the analysis: 1)
intervention expectations and 2) peer norms surrounding the intervention’s health topic. These two themes were added as domains in a revised conceptual model for acceptability. The updated conceptual model is included in Figure 7 below.
Figure 7. Updated Conceptual Model for Acceptability
We were able to collect data over two distinct time points with the two interviews per participant so were also able to assess whether participant views on each of the seven
acceptability domains changed over time (Table 6). We found that some participants’ views on
Affective attitude
Burden
Ethicality
Intervention coherence
Opportunity costs
Perceived effectiveness
Self-efficacy Acceptability
Intervention expectations
intervention coherence and perceived effectiveness changed between the first and second interview but that the views on the remaining domains were consistent. Change status was classified as “some change” when participants used language like “more” or “less” (e.g. “Now I know more about how the app works compared to last time” or “I think that this app would be less effective than I originally thought because it doesn’t send me reminders”) when comparing their current views to the previous ones they held in their first interviews.
Table 6. Change within Acceptability Domains Between Interview 1 and Interview 2
Domain Change status between interviews
Affective attitude No change
Burden Some change
Ethicality No change
Intervention coherence Some change
Opportunity costs No change
Perceived effectiveness Some change
Self-efficacy Some change
The following results are presented by theme.
Affective attitude. For affective attitude, participants did not indicate how the app made them feel; generally, participants ascribed adjectives to the apps like “helpful” and “motivating” and used verbs including “liked”, “disliked”, “enjoyed”, and “hated.” When prompted, many participants articulated why they either liked or disliked the app and many reasons related to other TFA domains listed below (e.g. burden, perceived effectiveness) or specific features. For example, one participant said, “I like that you can post, go live, and the Explore page is filled with funny videos that everybody sees.” (P6, Spot On) Responses to questions about affective attitude did not differ meaningfully from the first interview to the second interview.
With regards to the visual design, one participant said, “I don’t think it’d be that hard [to use the app daily] because the interface is simple.” (P7, Sleep Cycle) The same participant continued on to refer to the simplicity of the app components: “It’s pretty easy because there are only three categories: Alarm, Statistics, and Trends so you gotta do one of those.” (P7, Sleep Cycle) However, while several participants described the burden to use the health app as low, several indicated that they would not want to use apps that were too simple. For example, one participant stated, “It wouldn’t be hard to use [Fooducate], but I wouldn’t want to. I don’t like really simple things – it doesn’t give you much.” (P3, Fooducate)
Between the first and second interview, views on burden remained consistent. Most participants who indicated that the app was easy to use during the first interview maintained that the app got easier to use at the second interview. Language used in the second interview was also more extreme. For example, participants said that the app was “really easy” (P3, Fooducate) or “very confusing” (P12, BodySpace) and no participants had neutral opinions about their app.
Ethicality. With regards to ethicality, participants were able to characterize the health topics’ importance, but did not go further to describe how these health topics related to their value systems. For example, one participant said, “Since staying hydrated is an important thing for me, it will help me stay hydrated because if I fill this whole water bottle up with things that I drink, then I’ll know that I do good and need to keep it up.” (P5, WaterLogged) While
sometimes care about the health topic but that it was contingent on other circumstances. For example, one participant said, “It’s like during the week days, I don’t care about sleep. On weekends I catch up on my sleep” (Participant 7, Sleep Cycle). Views on ethicality did not change between first and second interviews.
Intervention coherence. After interacting with the newly downloaded health app for 3 to 5 minutes, approximately three-quarters of the participants were able to describe how the health app worked in detail (n=14). For example, after their brief interaction, one participant described how the WaterLogged app worked as follows:
You click on water bottle and then it fills it up and if you reach the goal, then you’re doing good. If you don’t reach your goal, then you need to step it up. You fill the water bottle. Say for example you had just drunk two bottles of water, then you click here and then enter two water bottles...I played with the app and made a mistake. I saw that when you click on the water bottle that the water fills up and was like that’s how you do it. (P5) Many participants offered that they figured out how to use the health app through trial and error, and were able to describe at least one feature of the app. On the other hand, approximately one-quarter of the participants (n=5) were not able to describe how the health app worked after interacting with it for a few minutes. When asked if more time spent with the app would help, all but one (n=4) said no. This suggested that participants had formed opinions on intervention coherence within minutes during the initial interaction. The majority of the participants who did not describe how the app worked said that the app was confusing and they did not know what they were supposed to do in the app.
At the second interview, more participants were able to articulate how the app worked. Descriptions were longer and more detailed at this time point than at the initial interview and participants were able to describe multiple features of the app and how they related to one
(P3, Fooducate) described how one could use the Food Finder feature in Fooducate to scan food items and learn about calorie counts for different food options, enter the number of calories consumed in the Health Tracker feature, and then share progress updates in the Community feature.
Opportunity costs. Participants raised very few opportunity costs in either interview and these views were consistent over time. Occasionally, participants offered that they may lose out on spending time on other apps (e.g. Snapchat, Instagram) in order to use the health app or would prefer to do other activities than using the health app. One participant noted, “I prefer going to the gym or getting a Shaun T. video tape…he dances with it…” and indicated that he would rather go to the gym or watch a workout video than use the BodySpace app given these choices and his limited time. (P15, BodySpace) Similarly, others claimed that using the app would be a “waste of time” (P9, BodySpace). One participant who indicated they cared a lot about healthy eating was asked to download the Fooducate app and they already had the Fitbit app installed on their phone. When asked about the new Fooducate app, they said, “…It feels like I’m wasting my time and there’s a better app than this that I’m already using.” (P3, Fooducate)
Perceived effectiveness. Participants quickly assessed perceived effectiveness and their responses were grouped into two broad categories: 1) the app will work for me (n=14) and 2) the app will not work for me (n=5).
Moreover, participants gave a variety of reasons for why the app would be effective in changing their behavior. Many participants liked that the apps delivered real-time feedback in order for participants to improve the behavior, allowed participants to go at their own pace, held them accountable, and motivated them. Table 7 provides examples of illustrative quotes for each of these reasons.
Table 7. Illustrative Quotes for Reasons for Perceived Effectiveness of Health App
Reasons for perceived effectiveness of health app
Illustrative quote
Helps them do something they have already tried to do in the past better
“I work out every day already and if I’m doing the same thing every day then I’m just not gonna want to do it. The app has different things so it’s gonna push me to try different exercises.” (P8, BodySpace)
Delivers real-time feedback
“It helps me better my sleep so…if I’m sleeping poorly, the app would give you everything that happened through the night so that you can do better.” (P7, Sleep Cycle)
Goes at their own pace “[BodySpace] motivates me to complete my goals without
forcing me to do anything and lets me go at my own pace.” (P11, BodySpace)
Holds them accountable “I think that this app would be helpful because I could stay on
myself about drinking water. Am I doing good or bad with my goal?” (P5, WaterLogged)
Motivates them “I’m very competitive and like to beat stuff and complete a goal.
I’m going to set me a high goal so that I can beat that during the day.” (P16, WaterLogged)
Table 8. Illustrative Quotes for Reasons for No Perceived Effectiveness of Health App
Reasons for no
perceived effectiveness of health app
Illustrative quote
The app does not teach them anything new
"[Headspace] is teaching me something I already know. I already know how to meditate." (P1, Headspace)
They already use another app or do something that is better than this app
"It just doesn’t look as effective as my regular workouts." (P15, BodySpace)
"Um, well to me it feels like I’m wasting my time and there’s a better app than [Fooducate] that I’m already using..." (P3, Fooducate)
They do not believe that they are affected by the health problem
“I feel like I don’t see me keeping track of it…because I’m okay with my weight right now…” (P17, Fooducate)
The app is boring "It just doesn’t engage me…” (P15, BodySpace)
A subset of the participants who indicated that the app would not work for them even offered ideal populations who would benefit from the app. For example, one participant testing the BodySpace app said, “It’s a good app for those who have gained too much weight and want to get in shape…” (P15, BodySpace)
In the second interviews, participants were generally consistent with their initial opinions on perceived effectiveness. Among participants who indicated that the app would not be
Self-efficacy. Participants were able to quickly assess their level of confidence in using the health app to perform the desired behavior in the app (e.g. track calories, watch a workout video, listen to guided meditations) within minutes of using the app for the first time, and talked in their first interview about their self-rated confidence along with their intent to use the app beyond the first interview. Three groups emerged from the first interview: one group of participants was confident that they could use the app and intended to use it (74%, n=14); one group was confident that they could use the app but did not intend to use it (16%, n=3); and one group was not confident that they could use the app and did not intend to use it (11%, n=2). See Table 9 below for this 2x2 table.
Table 9. 2x2 Table for Self-Efficacy Vs. Intention to Use Health App
Intention to use No intention to use
High self-efficacy 14 3
Low self-efficacy 0 2
For participants who were confident that they could use the app and intended to use it, they often offered when they planned to use the app next. For example, one participant said, “[I’m] pretty confident [I can use BodySpace] because I’m always by my phone and it’s not that hard to set a goal and make it happen if you’re committed to it...I want to see changes in my own self and build muscle and increase my stamina. If I want to prepare for sports this would be a great way to start...I’ll use it next at home today. Either before or after I take a shower after I’m done with household tasks.” (P11, BodySpace)
nah, I don’t got no interest in it.” (P3, Fooducate) They were able to separate their self-efficacy to use the app from their intentions to use the app.
Lastly, there were those two participants who expressed low confidence in using the app and therefore did not intend to use the app. When asked about his confidence to use the app after the first interview, one interview offered, “[I’m] not that confident cuz I’d probably forget about it because I don’t think it’d help me.” (P18, BodySpace) This self-efficacy assessment was related to the participants’ perceived effectiveness of the app and the potential to forget about the health app. Several participants noted that they might forget about the app because they have so many apps on their phones.
At the second interview, participants’ perceptions of self-efficacy to use the app were consistent with what they expressed in the first interview. Among the 14 participants who indicated high self-confidence to use the app and intentions to use the app, 13 of the 14 participants (93%) shared that they used the app at least once since the first interview and 8 of the 14 participants (42%) said that they used the app at least twice since the first interview. No participant went from reporting low self-efficacy to high self-efficacy or vice versa. However, three participants who expressed high self-confidence to use the app in the first interview expressed even greater self-confidence in the second interview. For example, one participant exclaimed, “At first, I didn’t know I could use an app like this to track my health but now I’m confident I can do it all the time.” (P6, Spot On, second interview)
Intervention expectations. In the initial part of the first interview, participants listed the apps that they used on a daily basis, many of which were social media apps. Often when
app met their expectations for what that kind of health app (e.g. fitness app, period tracker app) would do. For example, one participant who tested BodySpace said, “…When I think of fitness apps I think it tells you a routine or what you should eat or do and this is more do-it-yourself and to just keep track of what you should do.” (P18) In this case, the BodySpace app did not meet his expectation for what a fitness app should do. The same participant went on to complain that “[BodySpace] isn’t Instagram” and that users should not be posting pictures of themselves working out in a fitness app. On the contrary, some participants liked that the intervention reminded them of other social media apps. Another participant who tested out the BodySpace app said, “It’s like another version of Instagram except it’s an exercising app and it seems really cool.” (P9, BodySpace)
Intervention expectations are shaped by experiences with other health apps as well. For participants who already had other health apps installed on their phones, they compared the apps to each other using one as a frame of reference. One participant said:
I’m sure that I can use [Waterlogged]. I’m used to working with technology like newer stuff like apps. It looks like other apps I’ve used before like Sweatcoin. I’m used to Sweatcoin… When you leave the app and you don’t complete your goal or whatever your challenge is for the day, it tells you don’t be lazy and come back to the app and
Waterlogged can do the same thing. (P16, Waterlogged)
