question validated tool for generating a usability “score”. These surveys provide a single score on a scale from 0 to 100, where higher scores indicate better
usability. The average SUS score for Internet-based Web pages and applications obtained from approximately 500 studies in which it was used was 68.47,48
DATA COLLECTION AND STATISTICAL ANALYSIS
Agreement between diet analysis obtained from the MyEnergyBalance website and the ASA24 was examined. The following nutrient items were included in the analysis: energy (kcal), carbohydrate (g), protein (g), total fat (g), sodium (mg), iron (mg), calcium (mg), vitamin C (mg), and beta carotene (ug).
Differences between values for these nutrients/energy obtained from the diet analysis on the MyEnergyBalance website versus the ASA24 methods were statistically analyzed by paired t-tests. We also collected data on gender to determine if there were any differences between females and males for the comparisons.
A correlation coefficient was calculated to determine if there was any statistically significant relationship between the participants’’ System Usability Scale scores and the accuracy of their diet analysis results using the MyEnergyBalance app (relative to the ASA24 analysis). A correlation coefficient was also calculated to determine if there was a statistically significant relationship between the
using only the MyEnergyBalance website, without image reminders (relative to the ASA24 analysis).
RESULTS
There were ten participants (seven females and three males) aged 20 to 22 years that completed this study. The mean age of all participants was 20.5 years (ranging from 20-22 years). Average height was 67 inches and average weight was 149 pounds. The average BMI of all participants was 23.12 kg/m2 (ranging from 18.95 to 32.28 kg/m2).
As can be seen in Table 1 we compared the mean energy and macronutrients intake measured with the MyEnergyBalance app and website to the same day’s diet analyzed using the ASA24 method. When completing the dietary analysis using the MyEnergyBalance website, individuals who used the
MyEnergyBalance app to take images of all foods consumed did not have any statistically significant differences in the estimates of the energy intake compared to the ASA24 results. However, there was a significantly less estimated amount of protein when compared to the estimated amount using the ASA24 recall (50.7g vs 65.8g, respectively; p=0.02). When completing the dietary analysis using the MyEnergyBalance website (without the use of the iPhone app food image reminders), there was no statistically significant differences in the
estimates of the energy intake. However, there was a significantly less estimated amount of cholesterol when compared to the estimated amount using the ASA24
recall (128.3mg vs 193.0mg; p=0.02). In order to determine if there was any gender effects, we separately analyzed the female (n=7) and male (n=3) participants. There was no significant effect of gender on either diet analysis obtained using the iPhone app (with images) or the website alone (without images) compared to the ASA24 results.
Tables 2 and 3 provides data describing the individual System Usability Scale (SUS) scores and the overall raw SUS for the MyEnergyBalance app and website, respectively. The SUS score for the MyEnergyBalance app was 86 (SD, 8), and the SUS score for the MyEnergyBalance website was 69.5 (SD, 18.7). The overall user-friendliness was based on a 7-point scale. In addition, study participants were asked one summary question (“Overall I would rate the user-friendliness of this [app or website] as..”) based on a seven point scale from “worst imaginable” to
“best imaginable.” For this question, the MyEnergyBalance iPhone app scored 5.8 (SD 8) and the MyEnergyBalance website scored 4.6 (SD 0.8).
A correlation analysis was conducted to evaluate if there was a relationship between the System Usability Scale scores and the absolute difference of energy intake between the MyEnergyBalance app (images were used) and the ASA24 recall. Initial analysis found no significant relationship between the System Usability Scale scores and the MyEnergyBalance app (Figure 3). However, when one outlier subject was removed from the analysis, a statistically significant negative correlation was found between the System Usability Scale scores and the absolute difference of energy intake between the MyEnergyBalance app and
the ASA24 recall (Figure 4). Therefore, the higher the System Usability Scale score, the smaller the difference between the absolute difference in energy intake between the MyEnergyBalance app and the ASA24 recall.
A correlation analysis was also conducted to evaluate if there was a relationship between the System Usability Scale scores and the absolute difference of energy intake between the MyEnergyBalance website (no images were used) and the ASA24 recall. There was no statistical significance found between the System Usability Scale scores and the MyEnergyBalance website (Figure 5). However, after removing the same outlier participant from the analysis, a weak negative correlation was noted (r=-0.47, p=0.20) (Figure 6).
DISCUSSION
This was a pilot study to generate preliminary data on the accuracy of the MyEnergyBalance iPhone app and website. The majority of the participants in our study were healthy college students and their BMI indicated that the majority of them had normal weight with an exception of one individual who was obese. Our initial hypothesis was to see if the use of the MyEnergyBalance app (with the use of images) in combination with the MyEnergyBalance website would result in dietary analysis closer to the results obtained using the
Automated Self-Administered 24-hour dietary recalls (ASA24) versus using the MyEnergyBalance website alone (without the use of images). However, we did not see any significant results in the estimates of the energy intake between the
MyEnergyBalance app (with the use of images) in combination with the website compared to ASA24 versus the MyEnergyBalance website (without the use of images) compared to ASA24. Although there was a statistically significant difference with the protein intake in the MyEnergyBalance app and website and the cholesterol intake in the MyEnergyBalance website alone, we believe these results were due to chance and could be better confirmed with a larger sample size.
All participants evaluated the MyEnergyBalance app and website using the System Usability Scale tool. The average SUS score for web pages and
applications obtained from approximately 500 studies in which it was used was 68.47,48 The SUS score for our MyEnergyBalance app was 86, which was higher than the average score of 68. In general, an SUS score of 80 is considered to be in the top tenth percentile, which is closely associated with the likelihood that users would recommend this app to friends.48 Although the overall SUS score for the MyEnergyBalance app was 86, there was a neutral rating to the statement asking if participants would use this app frequently. The SUS score for the
MyEnergyBalance website was 69.5, which is approximately equivalent to the average score of 68 placing the website at the 50th percentile. In addition to this score, there was a low rating to the statement asking if participants would use this app frequently.
As discussed earlier, we did not see any significant differences with the overall energy intake between the MyEnergyBalance app and website compared to the ASA24. However, we were interested to see if there was any correlation between the system usability scale scores and the absolute differences in energy intake with the MyEnergyBalance app and ASA 24 as well as the MyEnergyBalance website and ASA24. After removing one subject outlier from the analysis, a significant negative correlation between the SUS scores and the absolute
differences in energy intake between the app and the ASA24 results was noted.
Participants who rated the MyEnergyBalance app higher on the system usability scale had a smaller absolute difference between the MyEnergyBalance app and ASA 24 energy intake. A similar but not statistically significant negative
correlation was noted between the system usability scale scores and the absolute differences in energy intake between MyEnergyBalance website and ASA24. In a future study we will include focus group analysis to help identify the specific aspects of the app and website that contribute to their usability.
An additional objective of this pilot study was to generate preliminary data from which a sample size power calculation would be determined for a future larger study. Based on the results of this pilot study, we determined that we would need 140 participants to be able to detect a difference of 5% between the energy intake obtained with the MyEnergyBalance iPhone app versus the ASA24 with a power of 0.80 and a two-sided alpha of 0.05. We also determined that we would need 37 participants to be able to detect a difference of 10% between the energy
intake obtained with the MyEnergyBalance iPhone app versus the ASA24 with a power of 0.80 and a two-sided alpha of 0.05.
Due to our small, relatively homogenous sample of young, healthy, college students, our results cannot be applied to the general population. Testing the accuracy and generalizability of a food recall tool will be influenced by the age, sex, and BMI of the study participants.49 Specifically, underreporting of energy intake is more prevalent in obese individuals.50,51 However, due to our small sample size, we did not see this result. Therefore, a future, robust study must be conducted with a larger, diverse study population representing a wide range of BMI’s to evaluate whether the MyEnergyBalance tool can demonstrate a
significant benefit on the efficacy of image-based food recalls.
CONCLUSIONS
This was a pilot study to generate preliminary data on the accuracy and usability of the MyEnergyBalance iPhone app and website. Although we were not able to show that the MyEnergyBalance app (with the use of images) helps with recall of foods consumed, it may be possible to demonstrate this by conducting a study with a larger and more diverse sample size. However, we have demonstrated that our MyEnergyBalance app was rated high for usability.
We are in the process of developing an improved and more user-friendly
“version 2” of the MyEnergyBalance smartphone app (for both iPhone and
of the smartphone app will include an option to receive text message reminders.
In addition, version 2 will have the ability to scan food barcodes and have the food information (name of food, servings, calories, nutrient analysis)
automatically saved to the user’s daily food record and saved in the user’s
“pantry” for easy recall if they eat the same food at another time. Version 2 of the app will also include integration with the iPhone’s build-in “Health” feature that automatically records steps. Users will be able to import their iOS Health app step data directly into the MyEnergyBalance physical activity record, as well as convert these step data to calories expended (which can then be used in the estimation of total daily energy expenditure and energy balance). This pilot study and a future study will both contribute to a better understanding of the role that mobile technologies can play in helping individuals track and improve their diet and exercise health behaviors.
Table 1: Mean energy and macronutrient intake assessed by MyEnergyBalance tool1 compared with ASA244 recall in college students (n=10)
MyEnergyBalance
1MyEnergyBalance tool consists of both an app and website.
2MyEnergyBalance app allows users to take images to enhance food recall;
MyEnergyBalance website was used to analyze their diet.
3MyEnergyBalance app was not used, therefore no images were available to enhance food recall; MyEnergyBalance website was used to analyze their diet.
4Automated Self-Administered 24-hour (ASA24) system is a validated dietary recall tool.
5Mean differences are statistically significant at the p<0.05 level.
Table 2: System Usability Scale Scores for MyEnergyBalance app
4 I think that I would need support of a technical person to be able to use this app.
4.8 ± 0.4
5 I found the various functions of this app were well-integrated.
4.1 ± 0.7
6 I thought there was too much inconsistency in this app. 4.2 ± 1.0 7 I would imagine that most people would learn to use this
app very quickly.
4.9 ± 0.3
8 I found the app very cumbersome to use. 4.2 ± 1.3
9 I felt confident using this app. 4.9 ± 0.3
10 I needed to learn a lot of things before I could get going with this app.
4.9 ± 0.3
SUS Scorea 86 ± 8
11 Overall, I would rate the rate the user-friendliness of this app as:
5.8 ± 0.6
Items 1-10: Based on a 5-point Likert Scale (1=Strongly Agree to 5 = Strongly Disagree).
Item 11: Scale based on 7-point Likert scale (1=Worst Imaginable to 7=Best Imaginable)
aThe SUS Score was calculated by taking the odd numbered items (for Items 1-10) and subtracting one from the user response, and taking the even numbered items and subtracting the user response from five. This scales all values from 0 to 4 (with four being the most positive response). The resulting values are summed and multiplied by 2.5 to convert the SUS Score range from 0 to 100 (see reference 48).
Table 3: System Usability Scale Scores for MyEnergyBalance website
2 I found the website unnecessarily complex. 3.3 ± 1.3 3 I thought the website was easy to use. 3.6 ± 1.2 4 I think that I would need support of a technical
person to be able to use this website.
4.8 ± 0.4
5 I found the various functions of this website were well-integrated.
3.6 ± 1.1
6 I thought there was too much inconsistency in this website.
4.0 ± 1.2
7 I would imagine that most people would learn to use this website very quickly.
4.0 ± 0.8
11 Overall, I would rate the rate the user-friendliness of this website as:
4.6 ± 0.8
Items 1-10: Based on a 5-point Likert Scale (1=Strongly Agree to 5 = Strongly Disagree).
Item 11: Scale based on 7-point Likert scale (1=Worst Imaginable to 7=Best Imaginable)
aThe SUS Score was calculated by taking the odd numbered items (for Items 1-10) and subtracting one from the user response, and taking the even numbered items and subtracting the user response from five. This scales all values from 0 to 4 (with four being the most positive response). The resulting values are summed and multiplied by 2.5 to convert the SUS Score range from 0 to 100 (see reference 48).
Figure 1: Homepage Menu of the MyEnergyBalance app
Figure 2: Experimental Design Flow Diagram
Figure 3: Scatter plot of individual system usability scale scores of the MyEnergyBalance app and the absolute difference in energy intake (kcal) between the MyEnergyBalance app and ASA24 recall (with all subjects)
Figure 4: Scatter plot of individual system usability scale scores of the MyEnergyBalance app and the absolute difference in energy intake (kcal) between the MyEnergyBalance app and ASA24 recall (without outlier)
Figure 5: Scatter plot of individual system usability scale scores of the
MyEnergyBalance website and the absolute difference in energy intake (kcal) between the MyEnergyBalance website and ASA24 recall (with all subjects)
Figure 6: Scatter plot of individual system usability scale scores of the
MyEnergyBalance website and the absolute difference in energy intake (kcal) between the MyEnergyBalance website and ASA24 recall (without outlier)
REFERENCES
1. Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States, 2011-2012. JAMA. 2014;311(8):806-814.
2. Alley DE, Chang VW. The changing relationship of obesity and disability, 1988-2004. JAMA. 2007;298(17):2020-2027.
3. Reuser M, Bonneux LG, Willekens FJ. Smoking kills, obesity disables: a multistate approach of the US Health and Retirement Survey. Obesity (Silver Spring, Md.). 2009;17(4):783-789.
4. Chen H, Guo X. Obesity and functional disability in elderly Americans.
Journal of the American Geriatrics Society. 2008;56(4):689-694.
5. Flegal KM, Graubard BI, Williamson DF, Gail MH. Excess deaths associated with underweight, overweight, and obesity. JAMA.
2005;293(15):1861-1867.
6. Calle EE, Teras LR, Thun MJ. Obesity and mortality. The New England journal of medicine. 2005;353(20):2197-2199.
7. McGee DL. Body mass index and mortality: a meta-analysis based on person-level data from twenty-six observational studies. Annals of epidemiology. 2005;15(2):87-97.
8. Adams KF, Schatzkin A, Harris TB, et al. Overweight, obesity, and mortality in a large prospective cohort of persons 50 to 71 years old. The New England journal of medicine. 2006;355(8):763-778.
9. Lawlor DA, Hart CL, Hole DJ, Davey Smith G. Reverse causality and confounding and the associations of overweight and obesity with mortality. Obesity (Silver Spring, Md.). 2006;14(12):2294-2304.
10. Finkelstein EA, Brown DS, Wrage LA, Allaire BT, Hoerger TJ. Individual and aggregate years-of-life-lost associated with overweight and obesity.
Obesity (Silver Spring, Md.). 2010;18(2):333-339.
11. Fontaine KR, Redden DT, Wang C, Westfall AO, Allison DB. Years of life lost due to obesity. JAMA. 2003;289(2):187-193.
12. Flegal KM, Kit BK, Orpana H, Graubard BI. Association of all-cause mortality with overweight and obesity using standard body mass index categories: a systematic review and meta-analysis. JAMA. 2013;309(1):71-82.
13. Hill JO, Wyatt HR, Peters JC. Energy balance and obesity. Circulation.
2012;126(1):126-132.
14. L. Kathleen Mahan JLR, Sylvia Escott-Stump. Krause's Food & the Nutrition Care Process. Saunders; 13 edition (September 7, 2011); 2011.
15. Bethesda. Clinical Guidelines on the Identification, Evaluation, and Treatment of Overweight and Obesity in Adults: The Evidence Report. National Heart, Lung, and Blood Institute1998 Sep.
16. Spear BA, Barlow SE, Ervin C, et al. Recommendations for treatment of child and adolescent overweight and obesity. Pediatrics. 2007;120 Suppl 4:S254-288.
17. Barlow SE. Expert committee recommendations regarding the prevention, assessment, and treatment of child and adolescent overweight and
obesity: summary report. Pediatrics. 2007;120 Suppl 4:S164-192.
18. Sodlerlund A, Fischer A, Johansson T. Physical activity, diet and
behaviour modification in the treatment of overweight and obese adults: a systematic review. Perspectives in public health. 2009;129(3):132-142.
19. McConnon A, Kirk SF, Ransley JK. Process evaluation of an internet-based resource for weight control: use and views of an obese sample. Journal of nutrition education and behavior. 2009;41(4):261-267.
20. Klem ML, Wing RR, McGuire MT, Seagle HM, Hill JO. A descriptive study of individuals successful at long-term maintenance of substantial weight loss. The American journal of clinical nutrition. 1997;66(2):239-246.
21. Baker RC, Kirschenbaum DS. Self-monitoring may be necessary for successful weight control. Behavior Therapy. 1993;24(3):377-394.
22. Brownell KD. The LEARN Program for Weight Management. Dallas, TX:
American Health Publishing Company; 2000.
23. Wadden TA, Crerand CE, Brock J. Behavioral treatment of obesity. The Psychiatric clinics of North America. 2005;28(1):151-170, ix.
24. Hartmann-Boyce J, Johns DJ, Jebb SA, Aveyard P. Effect of behavioural techniques and delivery mode on effectiveness of weight management:
systematic review, meta-analysis and meta-regression. Obesity reviews : an official journal of the International Association for the Study of Obesity.
2014;15(7):598-609.
25. Burke LE, Wang J, Sevick MA. Self-Monitoring in Weight Loss: A Systematic Review of the Literature. Journal of the American Dietetic Association. 2011;111(1):92-102.
26. Tsai C, Lee G, Raab F, et al. Usability and Feasibility of PmEB: A Mobile Phone Application for Monitoring Real Time Caloric Balance. Mobile Netw Appl. 2007;12(2-3):173-184.
27. Cambliss H. Persuasive Technologies for Health. In: Thompson TL, ed.
Encyclopedia of Health Communication2014 1054-1055.
28. Burke LE, Conroy MB, Sereika SM, et al. The effect of electronic self-monitoring on weight loss and dietary intake: a randomized behavioral weight loss trial. Obesity (Silver Spring, Md.). 2011;19(2):338-344.
29. Burke LE, Warziski M, Starrett T, et al. Self-monitoring dietary intake:
current and future practices. Journal of renal nutrition : the official journal of the Council on Renal Nutrition of the National Kidney Foundation.
2005;15(3):281-290.
30. McCabe-Sellers B. Advancing the art and science of dietary assessment through technology. J Am Diet Assoc. 2010;110(1):52-54.
31. Long JD, Stevens KR. Using technology to promote self-efficacy for healthy eating in adolescents. Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing / Sigma Theta Tau. 2004;36(2):134-139.
32. Breton E, Fuemmeler B, Abroms L. Weight loss—there is an app for that!
But does it adhere to evidence-informed practices? Behav. Med. Pract.
Policy Res. 2011;1(4):523-529.
33. Van Horn L. Assessing dietary intake: new ideas and better approaches. J Am Diet Assoc. 2006;106(10):1533.
34. Penn L, Boeing H, Boushey CJ, et al. Assessment of dietary intake: NuGO symposium report. Genes & nutrition. 2010;5(3):205-213.
35. Schap TE, Zhu F, Delp EJ, Boushey CJ. Merging dietary assessment with the adolescent lifestyle. Journal of human nutrition and dietetics : the official journal of the British Dietetic Association. 2014;27 Suppl 1:82-88.
36. Hongu N, Pope BT, Bilgic P, et al. Usability of a smartphone food picture app for assisting 24-hour dietary recall: a pilot study. Nutrition research and practice. 2015;9(2):207-212.
37. Six BL, Schap TE, Zhu FM, et al. Evidence-Based Development of a Mobile Telephone Food Record. Journal of the American Dietetic Association.
2010;110(1):74-79.
38. Martin CK, Han H, Coulon SM, Allen HR, Champagne CM, Anton SD. A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method. The British journal of nutrition. 2009;101(3):446-456.
39. Martin CK, Correa JB, Han H, et al. Validity of the Remote Food
Photography Method (RFPM) for estimating energy and nutrient intake in near real-time. Obesity (Silver Spring, Md.). 2012;20(4):891-899.
40. Rollo ME, Ash S, Lyons-Wall P, Russell A. Trial of a mobile phone method
40. Rollo ME, Ash S, Lyons-Wall P, Russell A. Trial of a mobile phone method