Our hypothesis that LIWC 2007 would be more sensitive to emotional expression than LIWC 2001 was not supported. LIWC 2007 was able to increase the previously established strength of LIWC 2001 in the identification of overall emotional expression and positive emotions. However, LIWC 2007 exacerbates the existing weakness of LIWC 2001 in that many of the words it identifies as emotion are not labeled as
emotional by human raters. In regards to identification of nonemotional words, there was no improvement by LIWC 2007. Both LIWC 2001 and LIWC 2007 were excellent with respect to identification of nonemotional words. In other words, this research indicates that while LIWC 2007 had higher levels of emotional identification, more words were also inaccurately classified as emotion. Therefore, while both LIWC 2001 and LIWC 2007 measure a number of domains other than emotional expression, our findings suggest that both LIWC 2001 and LIWC 2007 have excellent sensitivity for detecting emotional expression, but LIWC 2001 is superior with respect to positive predictive value- the words it identifies as representing emotion are more likely than LIWC 2007 to be in agreement with human raters.
Pennebaker et al. (2007) made a number of alterations to LIWC 2007 based on the previously established weaknesses of LIWC 2001. For instance, they increased the dictionary size and removed the subcategories of optimism and positive feelings. Despite these changes, LIWC 2001 remains superior with respect to positive predictive value. The alterations to LIWC 2007 resulted in improvements in sensitivity. However, these
the improvement in overall emotional identification as well as the increase in false positive error that occurs in LIWC 2007 is due to the alterations that have occurred to the LIWC 2007 dictionary. There were a number of words defined as emotional in the LIWC 2007 dictionary that were previously categorized as non-emotional (e.g., confident, champ, resolve). In addition to a reclassification of preexisting words, LIWC 2007 added additional emotional words that were not originally included in the LIWC 2001
dictionary (e.g., grace, jaded, joke, openness, rancid) and removed emotional words that were in the LIWC 2001 dictionary (e.g., sensitive). Finally, the LIWC 2007 dictionary classified the roots of words as emotional (e.g., stammer) that may be perceived as nonemotional by human coders in an extended form (e.g., stammered, stammering). The alterations to the LIWC 2007 dictionary may have resulted in the increased emotional identification but decrease in the precision of the identification.
The sensitivity levels for both LIWC 2001 and LIWC 2007 indicate strength in regards to identification of emotional content, such that both were highly sensitive to the identification of emotional expression. However, the positive predictive value was fairly poor for both LIWC 2001 and LIWC 2007. LIWC 2001 produced a significantly superior performance in regards to positive predictive value than LIWC 2007. Evaluation of the F- score, which evaluates both the positive predictive value and sensitivity, revealed that LIWC 2001 was superior to LIWC 2007 in emotional identification of overall affect, positive emotions, and anxiety. The remaining categories were not significantly different, indicating that LIWC 2001 and LIWC 2007 performed similarly in their identification of those emotion categories (e.g., negative emotion, anger, sadness). These results indicate that LIWC 2001 is more inclined to accurately identify emotion in accordance to human
rater when compared to LIWC 2007. Considering human coders are the gold standard in emotional identification, and LIWC 2001 provides results most similar to that of human coders, LIWC 2001 is superior to LIWC 2007.
LIWC 2001 may present as superior in its emotional identification over LIWC 2007, yet the accuracy in its performance is highly dependent upon the population being evaluated. Positive predictive value is dependent upon the prevalence in the population, meaning it can vary based on the sample utilized while sensitivity may stay the same despite what population is being evaluated (Altman & Bland, 1994). More specifically, cancer patients have been found to express more emotion than the health population (Linden, Vodermaier, Mackenzie, & Greig, 2012), meaning the prevalence of expressed emotion is higher for the sample utilized in this study than that of the general population. Considering prevalence rates or emotional expression in the cancer population, LIWC 2001 and LIWC 2007 are likely to produce poorer positive predictive values if being utilized with the emotional expression of a nonclinical population. LIWC 2001 and LIWC 2007 currently have a high rate of false positives, which may increase when evaluating a less emotional population or decrease when evaluating a more emotionally expressive population. Ultimately, the LIWC programs would benefit from further
validation utilizing alternative populations with varying levels of emotional expression. Pennebaker and his associates may have produced improvements in emotional identification in textual data had they utilized a more definitive validation process than simple correlation analyses. A correlation analysis describes the strengths of a
relationship between variables but does not provide information regarding what
allows users to see the weaknesses and strengths of that the relationship and what factors contribute to the strengths of that relationship. It may have been more beneficial for them to review text data obtained from their sample population along the LIWC results to ensure that the classification identified what emotions they intended to express with their written emotional expression. Emotions are multifaceted, making them much more difficult to accurately identify when simplified down to one modality. Based on the limitations involved in evaluating a single modality of emotion, obtaining a peer review on their validation procedures would have bolstered their utilization of their validation process.
It must be noted that there are some limitations to this study. The narratives utilized in this study were obtained from women diagnosed with breast cancer. Research has indicated that women cancer patients express more emotion than male counterparts (Linden, Vodermaier, Mackenzie, & Greig, 2012). Additionally, cancer patients are more inclined to endorse affective disorders, such as anxiety, which may impact their
emotional expression (Mitchell, Ferguson, Gill, Paul, & Symonds, 2013). Additionally, based on the specific circumstances these women faced (e.g., cancer diagnosis, treatment, and outcomes) this may have limited the range of emotions that may have been discussed compared to a healthy population. Based on the population utilized, results may be limited to cancer survivors rather than the general population. Finally, there were very few emotions evaluated (e.g., overall affect, positive emotions, negative emotions, anger, anxiety, and sadness), which does not reflect the full range of emotions experienced. This limited range of emotions measured may not accurately reflect emotions expressed (e.g.,
frustration, excitement, fear). Taking those things into consideration, this may limit the generalizability of this research to a healthy population.
While human coders are the gold standard for emotional identification in text data, due to the time and cost associated with evaluating such large volumes of data human coders are not always reasonable. Based on the importance of positive predictive value in addition to sensitivity, LIWC 2001 is superior to LIWC 2007 and is the
suggested modality for analysis of text data. Positive predictive value is highly dependent upon the prevalence of emotion in the specific population, such that the more emotion presented in a population the more accurate the analysis will likely be. Considering the high prevalence of emotion in a cancer population, and that LIWC 2001 performed significantly better than LIWC 2007, this indicates that for a population with much less emotional expression LIWC 2007 will still perform significantly poorer than LIWC 2001. LIWC 2001 seems to have a good validity in emotional identification and presents as a viable tool for identification of emotion in text-data, which is important in the
REFERENCES
Adolphs, R. (2002). Recognizing emotion from facial expressions: Psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews 1(1), 26-62.
Alpers, G. W., Winzelberg, A. J., Classen, C., Roberts, H., Dev, P., Koopman, C., Taylor, C. B. (2005). Evaluation of computerized text analysis in an internet breast cancer support group. Computers in Human Behavior, 21, 361-376.
Altman, D. G., & Bland, J. M. (1994). Statistics Notes: Diagnostic tests 2: predictive values. The BMJ, 309(6947), 102.
Austenfeld, J. L., & Stanton, A. L. (2004). Coping through emotional approach: A new look at emotion, coping, and health-related outcomes. Journal of Personality,
72(6), 1335-1364.
Baddeley, J. L., Daniel, G. R., & Pennebaker, J. W. (2011). How Henry Hellyer’s use of language foretold his suicide. Crisis, 32(5), 288-292.
Bantum, E. O., & Owen, J. E. (2009). Evaluating the validity of computerized content analysis programs for identification of emotional expression in cancer narratives.
Psychological Assessment, 21, 79-88.
Bhatia, M. S., Verma, S. K., Murty, O. P. (2006). Suicide notes: Psychological and clinical profiles. International Journal of Psychiatry in Medicine, 36(2), 163-170. Blood, A. J., & Zatorre, R. J. (2001). Intensely pleasurable responses to music correlate
with activity in brain regions implicated in reward and emotion. Proceeding of the
National Academy of Sciences, 98(20), 11818-11823.
Bonanno, G. A., Papa. A., Lalande, K., Westphal, M., & Coifman, K. (2004). The importance of being flexible: The ability to both enhance and suppress emotional expression predicts long-term adjustment. Psychological Science, 15, 482-487. Brissette, I., Scheirer, M. F., & Carver, C. S. (2002). The role of optimism in social
network development, coping, and psychological adjustment during a life transition. Journal of Personality and Social Psychology, 82(1), 102-111. Broderick, J. E., Junghaenel, D. U., & Schwartz, J. E. (2005). Written emotional
expression produces health benefits in fibromyalgia patients. Psychosomatic
Medicine, 67, 326-334.
Brotheridge, C. M., & Grandey, A. A. (2002). Emotional labor and burnout: Comparing two perspectives of “people work”. Journal of Vocational Behavior, 60, 17-39.
Chambless, D. L., & Hollond S. D. (1998). Defining empirically supported therapies.
Journal of Counseling and Clinical Psychology, 66(1), 7-18.
Chochinov, H. M., Wilson, K. G., Enns, M., & Lander, S. (1998). Depression,
hopelessness, and suicidal ideation in the terminally ill. Psychosomatics, 39, 366- 370.
Cordova, M. J., Giese-Davis, J., Golant, M., Kronnenwetter, C., Chang, V., McFarlin, S., & Spiegel, D. (2003) Mood disturbance in community cancer support groups: The role of emotional suppression and fighting spirit. Journal of Psychosomatic
Research, 55, 461-467.
Cukrowicz, K. C., Ekblad, A. G., Cheavens, J. S., Rosenthal, M. Z, & Lynch, T. R., (2008). Coping and thought suppression as predictors of suicidal ideation in depressed older adults with personality disorders. Aging & Mental Health, 12(1), 149-157.
De Moor, C., Sterner, J., Warneke, C., Gilani, Z, Hall, M., Amato, R., & Cohen, L. (2002). A pilot study of the effects of expressive writing on psychological and behavioral adjustment in patients enrolled in a phase II trial of vaccine therapy for metastatic renal cell carcinoma. Health Psychology, 21(6), 615-619.
De Ridder, D., Geenen, R., Kuijer, R., & van Middendorp, H. (2008). Psychological adjustment to chronic disease. Lancet, 372, 246-255.
Ekman, P. (1970). Universal facial expressions of emotion. Research Digest, 8(4), 151- 158.
Ekman, P. (1991). Who can catch a liar? American Psychologist, 46(9), 913-920.
Ekman, P. (1993). Facial expression and emotion. American Psychologist 48(4), 376-379. Ekman, P. (1999). Basic Emotions. Handbook of Cognition and Emotion, 98, 45-60. Esterling, B. A., Antoni, M. H., Fletcher, M. A., Margulies, S., & Schneiderman, N.
(1994). Emotional disclosure through writing or speaking modulates latent
Epstein-Barr virus antibody titers. Journal of Consulting and Clinical Psychology,
62(1), 130-140.
Farber, E. W., Mirsalimi, H., Williams, K. A., & McDaniel, S. (2003). Meaning of illness and psychological adjustment to HIV/AIDS. Psychosomatics, 44, 485-491. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible
statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.
Folkman, S. & Greer, S. (2000). Promoting psychological well-being in the face of serious illness: When theory, research and practice inform each other. Psycho-
Oncology, 9, 11-19.
Giese-Davis, J., Conrad, A., Nouriani, B., & Spiegel, D. (2008). Exploring emotion- regulation and autonomic physiology in metastic breast cancer patients: Repression, suppression, and restraint of hostility. Personality and Individual
Differences, 44, 226-237.
Grandey, A. A. (2003). When “the show must go on”: Surface acting and deep acting as determinants of emotional exhaustion and peer-rated service delivery. Academy of
Management Journal, 46(1), 86-96.
Gross, J. (1989). Emotional expression in cancer onset and progression. Social Science &
Medicine, 28(12), 1239-1248.
Gross, J.J., & John, O. P. (2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of
Personality and Social Psychology, 85(2), 348-362.
Gross, J. J., Richards, J. M., & John, O. P. (2006). Emotion regulation in everyday life.
Emotion Regulation in Couples and Families: Pathways to Dysfunction and Health, 2006, 13-35.
Hoyt, M. A., Stanton, A. L., Irwin., M. R., & Thomas, K. S. (2013). Cancer-related masculine threat, emotional approach coping, and physical functioning following treatment for prostate cancer. Health Psychology, 32, 66-74.
Hoyt, T., & Yeater, E. A., (2011). The effects of negative emotion and expressive writing on posttraumatic stress symptoms. Journal of Social and Clinical Psychology,
30(6), 549-569.
Iwamitsu, Y., Shimoda, K., Abe, H., Tani, T., Okawa, M., & Buck, R. (2005). The relation between negative emotional suppression and emotional distress in breast cancer diagnosis and treatment. Health Communication, 18(3), 201-215.
John, O. P & Gross, J. J. (2004). Healthy and unhealthy emotion regulation: Personality processes, individual differences, and life span development. Journal of
Personality 72(6), 1301-1334.
Joormann, J., Hertel, P. T., Brozovich, F., & Gotlib, I. H. (2005). Remembering the good, forgetting the bad: intentional forgetting of emotional material in depression. Journal of abnormal psychology, 114(4), 640-648.
Kennedy-Moore, E., & Watson, J. C. (2001). How and when does emotional expression help? Review of General Psychology, 5(3), 187-212.
Kidd, S. A. (2002). The role of qualitative research in psychological journals.
Psychological Methods, 7(1), 126-138.
Leff, J., Wig, N. N., Ghosh, A., Bedi, H., Menon, D. K., Kuipers, L., ... & Sartorius, N. (1987). Expressed emotion and schizophrenia in north India. III. Influence of relatives' expressed emotion on the course of schizophrenia in Chandigarh. The British Journal of Psychiatry, 151(2), 166-173.
Lieberman, M. A., & Goldstein, B. A., (2006). Not all negative emotions are equal: The role of emotional expression in online support groups for women with breast cancer. Psycho-Oncology, 15, 160-168
Linden, W., Vodermaier, A., MacKenzie, R., & Greig, D. (2012). Anxiety and depression after cancer diagnosis: Prevalence rates by cancer type, gender, and age. Journal
of Affective Disorders, 141, 343-351.
Low, C. A., Stanton, A. L., Bower, E. J., & Gyllenhammer, L. (2010). A randomized controlled trial of emotionally expressive writing for women with metastatic breast cancer. Health Psychology, 29(4), 460-466.
Ludwig, S., Ruyter, K., Friedman, M., Bruggen, E. C., Wetzels, M., & Pfann, G. (2013). More than words: The influence of affective content and linguistic style matches in only review on conversion rates. Journal of Marketing, 77, 87-103.
Lyubomirsky, S., Sousa, L., & Dickehoof, R. (2006). The costs and benefits of writing, talking, and thinking about life’s triumphs and defeats. Journal of Personality and
Social Psychology, 90(4), 692-708.
Mitchell, A. J., Ferguson, D. W., Gill, J., Paul, J., & Symonds, P. (2013). Depression and anxiety in long-term cancer survivors compared with spouses and healthy
controls: a systematic review and meta-analysis. The Lancet Oncology, 14(8), 721-732.
Morris, J. A., & Felhman, D. C. (1996). The dimensions, antecedents, and consequences of emotional labor. The Academy of Management Review, 21(4), 986-1010. Oatley, K., & Jenkins, J. M. (1992). Human emotions: Function and dysfunction. Annual
Review of Psychology, 43, 55-85.
Owen, J. E., Giese-Davis, J., Cordova, M., Kronenweter, C., Golant, M., & Spiegel, D. (2006). Self-report and linguistic indicators of emotional expression in narratives as predictors of adjustment to cancer. Journal of Behavioral Medicine, 29(4), 335- 345.
Parker, J. S., Stewart, G. S., & Gantt, C. (2006). Research and intervention with adolescents exposed to domestic violence. Family Therapy, 33(1), 45-52.
Peden, B. F., & Carroll, D. W. (2008). Ways of writing: Linguistic analysis of self- assessment and traditional assignments. Teaching of Psychology, 35, 313-318. Pennebaker, J. W. (1993). Putting stress into words: Health, linguistic, and therapeutic
implications. Behaviour Research and Therapy, 31(6), 539-548.
Pennebaker, J. W., Boothe, R. E., & Francis, M. E. (2001). Linguistic inquiry and word
count: LIWC-2001 operations manual. Austin, TX: LIWC.net
Pennebaker, J. W., Chung, C. K., Ireland, M., Gonzalez, A., & Booth, R. (2007). The
development and psychometric properties of LIWC2007. Austin, TX: LIWC.net
Pennebaker, J. W., & Francis, M. E. (1999). Linguistic Inquiry and Word Count. Mahwah, NJ: Erlbaum.
Quartana, P. J., Bounds, S., Yoon, K. L., Goodin, B. R., & Burns, J. W. (2010). Anger suppression predicts pain, emotional, and cardiovascular responses to the cold pressor. Annals of Behavioral Medicine, 39, 211-221.
Robinson, M. D., & Clore, G. L. (2002). Brief and feeling: Evidence for an accessibility model of emotional self-report. Psychology Bulletin, 128(6), 934-960.
Robinson, M. D., & Clore, G. L. (2002). Episodic and semantic knowledge in emotional self-report: Evidence for two judgment processes. Journal of Personality and
Social Psychology, 83(1), 198-215.
Rosenberg, H. J., Rosenberg, S. D., Ernstoff, M. S., Woldord, G. L., Amdur, R. J., Elshamy, M. R., Bauer-Wu, S. M., … & Pennebaker, J. W. (2002). Expressive disclosure and health outcomes in prostate cancer population. International
Journal of Psychiatry in Medicine 32(1), 37-53.
Scherer, K. R. (2005). What are emotions? And how can they be measured? Social
Science Information, 44, 695-792.
Schlatter, M. C., & Cameron, L. D. (2010). Emotional suppression tendencies as predictors of symptoms, mood, and coping appraisals during AC chemotherapy for breast cancer treatment. Annals of Behavioral Medicine, 40, 15-29.
Soriano LeBovidge, J., Lavigne, J. V., Donenberg, G. R., & Miller, M. L. (2003). Psychological adjustment of children and adolescents with chronic arthritis: A meta-analytic review. Journal of Pediatric Psychology, 28(1), 29-39.
Sloan, D. M., Feinstein, B. A., & Marx, B. P. (2009). The durability of beneficial health effects associated with expressive writing. Anxiety, Stress, & Coping, 22(5), 509- 523.
Sloan, D. M., & Marx, B. P. (2004). A closer examination of the structured written disclosure procedure. Journal of Consulting and Clinical Psychology, 72 (2), 165- 175.
Smith, S., Anderson-Hanley, C., Langrock, A., & Compas, B. (2005). The effects of journaling for women with newly diagnosed breast cancer. Psycho-Oncology, 14, 1075-1082.
Smyth, J. M. (1998). Written emotional expression: Effect sizes, outcome types, and moderating variables. Journal of Consulting and Clinical Psychology, 66(1), 174- 184.
Stanton, A. L., Danoff-Burg, S., Cameron, C. L., Bishop, M., Collins, C. A., Kirk, S. B., Swororski, L. A., & Twillman, R. (2000). Emotionally expressive coping predicts psychological and physical adjustment to breast cancer. Journal of Consulting
and Clinical Psychology, 68(5), 875-882.
Stanton, A. L., Danoff-Burg, S., Sworowski, L. A., Collins, C. A., Branstetter, A. D., Rodriguez-Hanley, A., … Austenfeld, J. L. (2002). Randomized, controlled trial of written emotional expression and benefit finding in breast cancer patients.
Journal of Clinical Oncology, 20(20), 4160-4168.
Stirman, S. W., & Pennebaker, J. W. (2001). Word use in the poetry of suicidal and nonsuicidal poets. Psychosomatic Medicine, 63, 517-522.
Valentiner, D. P., Hood, J., & Hawkins, A. (2006). Differentiating thought suppression and the suppression of emotional expression. Cognition and Emotion, 20(5), 729- 735.
Vuilleumier, P. (2005). How brains beware: neural mechanisms of emotional attention.
Trends in Cognitive Sciences, 9(12), 585-594.
Walker, B. L., Nail, L. M., & Croyle, R. T. (1999). Does emotional expression make a difference in reactions to breast cancer. Oncology Nursing Forum 26(6), 1025- 1032.
Weisz, J. R., & Hawley, K. M. (2010). Finding, evaluating, refining, and applying
empirically supported treatments for children and adolescents. Journal of Clinical
Child Psychology, 27(2), 206-216.
Zabowski, S. G., Herzer, M., Barrett, S. D., Milligam, J. G., & Beckman, N. (2011). Who benefits from emotional expression? An examination of personality difference among gynaecological cancer patients participating in a randomized controlled emotional disclosure intervention trial. British Journal of Psychology, 102, 355- 372.
Zellars, K. L., & Perrewe, P. L. (2001). Affective personality and the content of
emotional social support: Coping in organization. Journal of Applied Psychology,