THE INFLUENCE OF CONTENT TYPE
OF FACEBOOK MESSAGES ON THE
EFFECTIVENESS OF THE MESSAGE
"And the moderating role of consumer brand relationship"
Master Thesis Communication Studies
11 April 2014
2
Abstract
The communication between business and consumers is shifting to social networks online, one of these online social networks is Facebook. Facebook provides companies to create their own brand pages, from where they can send messages to consumers. There are many different ways to communicate on these brand pages. From a business perspective the question arises what content type of Facebook messages is the most effective, and does this differ between the consumers with a high consumer brand relationship, and consumers with a low consumer brand relationship. This study focused on the influence of content type of Facebook messages on the effectiveness of the messages, and how consumer brand relationship moderates this.
A pre-test was conducted to see if the content of the four different messages (informational, entertainment, promotional, and social) significantly differ from each other, the conclusion from the pre-test was that the post with social content was not perceived as significantly different from the other post types. , so the social message type is left out of this study. For the main study, a questionnaire (N=191) was developed in order to measure the message effectiveness and the consumer brand relationship.
3
Acknowledgement
After finishing my HBO in Commercial Economics, I was pretty determined to also get a master degree. When starting with communication studies, I needed some time to get used to the new way of learning and analysing. Nevertheless I am pretty glad I have chosen to try and get my master degree, because the way of thinking and reflecting problems is extremely different from the HBO way of working. I think that this study gave me a whole new way of thinking, and I have broadened my knowledge not only with theoretical knowledge, but also the practical side of analysing a problem or statement.
I would like to make use of this opportunity to give my word of thanks to a number of people. First of all I would like to thank my supervisor Dr. Sabrina Hegner, for her patience and for the quick answering on the ongoing questions from my side. I would also like to thank Dr. Svetlana Bialkova. I also would like to thank all the respondents for filling in my pre-test, and questionnaire. Furthermore, I would like to thank Peter, for reading, and helping me with my thesis. Last but not least I would like to thank all my family and friends for helping my throughout the years.
4
Table of Content
Abstract 2
Acknowledgement 3
1. Introduction 6
1.1 Background 6
1.2 Problem statement 6
1.4 Research question and research model 10
1.5 Academic relevance 10
1.6 Practical relevance 11
1.7 Structure of the research 11
2. Theoretical framework 12
2.1 Message types 12
2.1.1 Message types on Facebook brand pages 14
2.2 Effectiveness of Facebook messages 17
2.2.1 Attitude toward the message 18 2.2.2 Attitude toward the brand 18 2.2.3 Purchase intention 19 2.2.4 Engagement rate 19
2.3 Consumer brand relationship 21
3. Research method 24
3.1 Pre-test 24
3.2 Main Study 27
3.2.1 Respondents 27 3.2.2 Questionnaire 28 3.2.3 Measurements 29
4. Results 30
4.1 Manipulations check 30
4.2 Analysis of main effects 30
4.3. Analysis of interaction effects 32
4.3.1 Attitude toward the message 32 4.3.2 Attitude toward the brand 33 4.3.3 Purchase intention 34 4.3.4 Engagement rate 35
5. Discussion 37
5.1 Theoretical implications 37
5 5.3 limitations and directions for further research 41
References 43
Appendix 49
A. Facebook messages Nike 49
B. Literature framework 51
C. Facebook definitions and interaction types on Facebook 52
D. Stimuli - message types 53
E. Items effectiveness of the message 54
F. Items consumer brand relationships 55
6
1. Introduction
1.1 Background
Social networking sites are rapidly growing in the number of people using it, and is changing the purpose and functionality of the internet (Kelly, Kerr & Drennan, 2010). Facebook is the biggest social network site, and has more than one billion users around the world (Wasserman, 2012). Social network sites are the top online destination, and reach around 60 percent of the active internet user. Furthermore, is the majority of time online spent on social network sites (Nielsen, 2011).
According to Park (2011) online social networks have changed the way people communicate in today’s world, and how they work, play, and how people consume products and brands. The social media has broadened the purchase experience (Marsden, 2010a), consumption experience as product discovery (ATG Research, 2010), product usage behavior (Algesheimer, Dholakia & Herrmann, 2005 and Trusov, Bodapati & Bucklin, 2010), product referral (Kozinets, Valck, Wojnicki & Wilner, 2010, Stephen & Toubia, 2010), and finally the product co-creation (Hennig-Thurau, Malthouse, Friege, Gensler, Lobschat & Rangaswamy, 2010). These developments dramatically impacted and changed marketing behavior for companies globally. Social media (like Facebook) has become a critical marketing tool to establish a strong consumer brand relationship. By ways of building a direct personalized relationship between the brand and the consumer, the companies can change the way they are related to its customers (Marsden, 2010a).
Nowadays, an increasing amount of organizations (profit and non-profit) do implement their campaigns with social networks sites (like Facebook and twitter). The social networking sites provide brands to capture costumers and engage with these brands. Anderson, Sims, Price and Brusa (2010), Madden (2009) found that social network sites are radically changing the marketing environment, and are ought to become a critical driver of successful marketing communication.
1.2 Problem statement
7 Research studies were not able to keep up with the explosively growing popularity of social media, and the way businesses are using these social media channels like Facebook as a new marketing channel or tool.
This study focuses on the influence of content type of Facebook messages on the effectiveness of the messages, and how consumer brand relationship moderates this. The content type of the messages that are displayed on Facebook brand pages are very divergent. Some of the messages are informative, for example by explaining new product introductions. Figure 11 shows an informative message of Heinz from their Facebook brand page. This messages could be within the informative message types because the message explains a new brand (product introduction), and provides information about this specific product. On the other hand other messages are intent to entertain, by showing movies or messages that contain funny pictures. Figure 22 shows an entertainment message, this messages belongs to the entertainment message types, because it is a video that is aimed at entertaining the consumers on Facebook with a message containing a video about the one million fans Heinz reached on Facebook. Other messages types could be focusing on the social aspects, by asking questions or aiming on interaction. Figure 33 shows a Facebook message with social content, this message is aiming on interaction by asking a question. There are also message that are concentrating on the promotion of the brand or product, through contest or promoting a specific product. Figure 44 shows a message which focuses on a contest. In the appendix (A) are also four examples of the four message types from the Facebook page from Nike, to show that these post types apply for different types of companies, in this situation for a sport product company (Nike) and a company (Heinz) with products in fast moving consumer good product market.
1 https://www.facebook.com/photo.php?fbid=10152261326469292&set=a.287305849291.143129.263814699291& type=1&theater 2 https://www.facebook.com/photo.php?v=10100531920354196&set=vb.263814699291&type=3&theater
8 Figure 1. Informative message on the Heinz Facebook brand page.
9 Figure 3. Social message on the Heinz Facebook brand page.
Figure 4. Promotional message on the Heinz Facebook brand page.
[image:9.595.76.458.361.661.2]10 From a business perspective the question arises: what and how to communicate for companies (like Nike, and Heinz) on these Facebook brand pages. There are many different ways to communicate and to present these messages on Facebook. The question is which of the four message types creates the most effectiveness measured in: attitude toward the message, attitude toward the brand, purchase intention, and engagement rate.
Research of Bagozzi and Dholakia (2006) showed that Facebook brand pages play a role in the consumer brand relationship with this brand, people who are fan of the brand pages incline to be more loyal and committed to the company. Research from Aggarwal and Law (2005) show in their study that consumer brand relationship influences what information becomes relevant for the consumer that is interacting with a brand. Therefore, we will also investigate the effect of consumer brand relationship on the influence of message type on the effectiveness of the message.
1.4 Research question and research model
"Does the content of the message (informational, entertainment, promotional, or social) influences the
effectiveness of the message (attitude toward the message, attitude toward the brand, purchase
intention and engagement rate), and to what extent does the brand relationship moderates this?"
From this research question we derive the below research model:
Message types Effectiveness of Facebook messages
Informational Attitude toward the message
Entertainment Attitude toward the brand
Promotional Purchase intention
Social Engagement rate
Consumer brand relationship
Figure 5. Research model.
1.5 Academic relevance
11 of research on the way consumer brand relationship could also be a moderator variable instead of an outcome in relation to Facebook. There is a possibility that a higher consumer brand relationship may result in different consumer interests compared to consumers with a low consumer brand relationship. In the literature no research has been conducted on how consumer brand relationship moderates the effectiveness (measured in attitude toward the message, attitude toward the brand, purchase intention, and engagement rate) of a message on Facebook. Also the research about which content type is the most effective on social media is limited. This study will contribute to both the literature on consumer brand relationship as well as the literature about Facebook (messages), with results from this study about the most effective content type of messages on Facebook brand pages.
1.6 Practical relevance
Because of the continuously growing magnitude of social networks, not only Facebook but also Twitter and for example LinkedIn, it is of extreme importance to use these social network sites in the best possible way to yield the highest possible outcome. People that like a brand page on Facebook could have very different rationale for “liking” that brand page, and may be interested in different content published by the brand. For example people that are already have a high consumer brand relationship are perhaps interested in different messages than people that have a low brand relationship. Furthermore, the consumer brand relationship could also play a role in the interest for specific content on the brand page. This research will investigate which content of the message on brand pages are most effective, and how it is (potentially) affected by the consumer brand relationship. The results of the research could be used to help brands decide which kind of content they should use on their Facebook brand page.
1.7 Structure of the research
12
2.
Theoretical framework
This chapter provides an overview of existing research that has been published on different content
types of social media messages. In addition it shall outline the effectiveness of Facebook message by
using the literature of advertisement effectiveness. Finally the theory of consumer brand relationship
is explained and applied to the subject of this thesis.
2.1 Message types
Facebook marketing is still at a relatively early stage and has to be studied from different perspectives.
Although the studies on this subject are increasing significantly, the focus on the content type of
messages on Facebook is still limited. For that reason, this research is focused towards the different
content types of the messages, as well as research about the motivation of users to use Facebook brand
pages.
One of the studies that identified different content types for social media messages is the study from Cvijikj and Michahelles (2011). Their research was about the effect of post characteristics posted by moderators (posts that are posted by the company itself on the Facebook brand page). They defined seven different categories: (1) Product(s) announcements: announcements of new product launch. (2) Information: information regarding a sales location, number of page fans etc. (3) Designed question: posts in form of questions with a goal to engage users in a dialog. (4) Questioner: using the Facebook Poll to obtain answers on a specific question5. (5) Competition: posts related to competition i.e. announcements, rules, winners etc. (6) Advertisement: advertisement of existing products (mostly used in a form of photo post). (7) Statement: Posts in form of statement, stating opinion on certain topic. The Product(s) announcements posts belong to informative posts in this thesis, because the announcement gives “new” information to the consumers on Facebook (e.g. “Check out the new Ketchup flavor, in store now!”) as mentioned above, also the post information that Cvijikj and Michahelles (2011) stated gives information about the company and/or product. The designed questions, and statement posts that Cvijikj and Michahelles (2011) stated, create interaction between the consumers, and the company (e.g. designed question: “What is your favorite ketchup?”, and e.g. a statement post: “Ketchup is better than Mayonnaise”). Since the designed questions invites the consumer to answer the question, whereas the statement invites the consumers to respond on this and explain why the consumers agree or disagree. Consequently these two post type require interaction, and thus belong to the social post type, because the characteristic of a social post in this thesis is that the post enables/encourages interaction. The competition post that Cvijikj and
5 The poll function is deleted from the Facebook brand page, so this options it not available anymore. So the
13 Michahelles (2011) stated belongs to promotional posts because posts about competitions have a promotional characteristic (e.g. “Join this contest, and win a Ketchup bottle”) in this example they promote a specific contest to win a ketchup bottle. The post type advertisement that Cvijikj and Michahelles (2011) stated is not taken into account in this thesis because the focus is on non-paid messages on the own Facebook brand page.
In the research from Ryan, Peruta, and Chouman (2013), about how companies communicate on social media, five different post types were used: (1) Event: these posts are promoting a time-base activity. (2) Contest: this post encourages to participate by competing with each other (with or without rewards). (3) Special promotion: the post promotes a special offer. (4) Product promotion: the product is being presented. (5) Brand related: the post makes a reference to the brand itself. The post type event, product promotion, and brand related posts belong in this thesis to the informative post type, this because the event post gives information about the event or the activity from or related to the company (e.g. event: “This weekend the Ketchup taste event from Heinz at your local shopping mall!”). While the posts with product promotion gives general information about a specific product from the company (e.g. “New: Ketchup with jalapenos!”) , and the brand related posts that they defined give information that is related to the brand/company, and therefore belong in the category informative posts in this thesis (e.g. “Heinz was established in Canada in 1909.”). The contest post type that is defined by Ryan, Peruta, and Chouman (2013) belongs as mentioned earlier to the promotional posts, because these posts promote specific contests or competitions, and therefore, they have promotional characteristics (e.g. “Join this contest, and win a Ketchup bottle”). Also the post type special promotion belongs to the promotional posts, because these posts promote special offers (e.g. “Print this coupon and get 10% off for all the Heinz Ketchup this weekend”)
14 Hong (2011) researched the users' motivations for communicating and using business Facebook pages through the users and gratifications theory. Hong (2011) classified in this research four different post categories. (1) Entertainment postings: are meant to amuse consumers on Facebook. (2) Information postings: to provide information to the consumers (3) Promotional postings: These posts highlight a contest, promotion, coupon or any other offer aimed to attract attention. (4) Social postings: to encourage user participation, for example by asking a question. The entertainment posts (e.g. “Watch this video to see the first painting made with Heinz Ketchup”) belongs to the
entertainment post category in this thesis. The information postings (e.g. “New: Ketchup with jalapenos!”) belong in this thesis to the informational posts. The promotional postings (e.g. Join this contest, and win a Ketchup bottle”) belong in this thesis to the promotional posts, and the social postings (e.g. What is your favorite ketchup?”) in this thesis belong to the social posts.
Based on the above literature four message types are distinguished: Informational messages, entertainment messages, promotional messages, and social messages. These four message types will be further described in the following paragraph. Table 1 shows the above research plotted in the new categories, with corresponding examples. In the appendix (B) a summary can be found of the research used to define the four message types.
2.1.1 Message types on Facebook brand pages
Informational posts
The first post type contains information about the brand or the product. The general definition of information is: facts provided or learned about something or someone (Oxford online dictionaries, 2014). An informational message is therefore a message with information about a company/brand/product that is based on facts about them. Vries, Gensler and Leeflang (2012) described informative brand posts as posts that contained information about the company, brand, and/or its products. In the study of Hong (2011) information posts were specified as posts that provide information to the consumers. All the above mentioned content could be seen as informative content, and therefore, an informative post.
Entertainment posts
15 (2012) entertainment brand posts were defined as content that is not specific related to the company/brand, such as general funny movies or anecdotes. Hong (2011) mentioned that entertainment posts are intended to amuse Facebook users. Hong (2011) has given examples of the content these posts could consist of: Links to multimedia, Facebook games, and fun trivia about the brand or its products. All of the above mentioned content could be seen as entertainment content, and therefore, an entertainment post.
Promotional posts
The third type relates to post with promotional content. The general definition of a promotional message is relating to the publicizing of a product, organization, or venture so as to increase sales or public awareness (Oxford online dictionaries, 2014). Promotion is also used to refer to a specific activity that is intended to promote the company and/or its products, by promoting contests or advertise about specific product promotions. In the study of Cvijikj and Michahelles (2011) competition posts are specified including all posts that are related to a competition. In addition, Ryan, Peruta and Chouman (2013) specified promotional posts as posts that consist of content about a contest: the post encourages to participate by competing with each other (with or without rewards). Ryan, Peruta and Chouman (2013) also specified posts with a special promotion: meaning that the post promotes a special offer (like 2 for 1 or 30% discount), in this study this type of content belongs to promotional content. Furthermore, Hong (2011) defined promotional posting as posts that highlight a contest, promotion, coupon or any other offer aimed to attract attention. In this thesis posts that contain: competitions, contests, special offers, promotion, and coupons, belong to promotional posts.
Social posts
16 Table 1. The above research plotted in the four new categories, with examples of the four different post types.
Post type Covered content Post examples
Informational Product announcements (e.g. announcement of a new product launch), Information (e.g. posts about sales locations, number of fans etc.) (Cvijikj & Michahelles,2011). and hong) Event (e.g. posts that are promoting a time-base activity) Product promotion (e.g. Posts were the product is being presented), brand related (e.g. posts that make a reference to the brand itself) (Ryan, Peruta, & Chouman, 2013). Informative brand posts (e.g. posts that contain information about the company/brand and/or its products) (Vries, Gensler, and Leeflang, 2012) Information postings (e.g. to provide information to the fans) (Hong, 2011)
“Check out the new Ketchup flavor, in store now!”
“This weekend the Ketchup taste event from Heinz at your local shopping mall!”
“Heinz was established in Canada in 1909.”
“New: Ketchup with jalapenos!”
Entertainment Entertainment brand posts (e.g. content that is unrelated to the company/brand, funny movies or anecdotes (e.g. (Vries, Gensler, and Leeflang, 2012). Entertainment postings (e.g. are meant to amuse Facebook fans) (Hong, 2011)
“Watch this picture to see which type of Heinz person you are!”
“Watch this video to see the first painting made with Heinz Ketchup”
Promotional Competition (e.g. posts related to competition i.e. announcements, rules, winners etc.) (Cvijikj and Michahelles, 2011). Contest (e.g. post that encourage to participate by competing with each other with or without rewards), Special promotion (e.g. posts that promote a special offer) (Ryan, Peruta, & Chouman, 2013). Promotional postings (e.g. these posts highlicht a contest, promotion, coupon, or any other offer aimed to attract attention) (Hong, 2011)
“Join this contest, and win a Ketchup bottle”
“Print this coupon and get 10% off for all the Heinz Ketchup this weekend”
Social Designed questions (e.g. posts in form of questions with a goal to engage users in a dialog), Statements (posts in form of statement, stating opinion on a certain topic) (Cvijikj and Michahelles, 2011). Social postings (e.g. to encourage user participation, for example by asking a question) (Hong, 2011)
“What is your favorite ketchup?”
17
2.2 Effectiveness of Facebook messages
The dependent variable in this study is the effectiveness of the messages. However, the effectiveness
of the message (on Facebook) is not a specific, and known variable; hence in this study the effectiveness
of the message is composed on the basis of research on advertising effectiveness. The variable
effectiveness of the message is divided in four different constructs explained in this paragraph.
Since there is only a modest amount of research about the content of the Facebook messages, specifically on brand pages. As well as the way consumers perceive these messages, and whether the Facebook messages have any effect. For those reasons this study yielded on the literature and research of advertising effectiveness is used and related this to the Facebook message effectiveness. Advertisement effectiveness could be measured in attitudes as proven in many studies (Fishbein, 1967 and Mehta, 2000). Fishbein (1967) stated that views and opinions in advertising are mostly measured through attitudes. According to researchers, attitudes are one of the best predictors of a consumer’s behavior, and can predict if a person will try a product or make a purchase (Jee & Lee, 2002). Kotler and Keller (2003) describe attitudes as an individual’s enduring favorable or unfavorable evaluation, emotional feeling, and action tendencies toward some object or idea, people can have attitudes toward almost everything. Lutz, Mackenzie and Belch (1983) suggested that there are multiple dimensions in advertising effectiveness. They stated four different types of attitudes in relation to advertisement effectiveness: ad cognitions, brand cognitions, attitude toward the ad, and attitude toward the brand. In this research they also mentioned that purchase intention is in addition important according to advertising scholars researched by Lutz et al. (1983). Brand attitudes of a brand have an impact on purchase intention (Chen et al., 2008; Lee and Lee, 2009). This would indicate that more favorable brand attitudes lead to a higher intention to purchase the brand. Gresham and Shimp (1985) point out that exposure to an advertisement influences consumers attitudes toward the brand. According to Gresham and Shimp, (1985) attitude toward the advertisement influences attitude toward the brand. Taking this research in account it is quite reasonable to think that when people are positive toward a message of a specific brand page on Facebook it is likely that their attitude toward the brand is also positive. The connections between attitude toward the advertisement and attitude toward the brand is close, hence why these two attitudes should be researched in combination.
18 below section the definitions of the variables will be provided and the variables will be discussed. In addition the relationship between the independent and dependent variables is discussed.
2.2.1 Attitude toward the message
Attitudes toward the message are the receivers' affective reactions (e.g., like-dislike) to the message itself (Lutz et al., 1983). Mehta (2000) indicated that consumer’s beliefs and their attitude toward advertising are essential indicators of advertisement effectiveness. Respondents with a more positive attitude toward advertising advertisements recalled a higher number of advertisements the day after seeing these advertisements and were more persuaded by those advertisements. Chen and Wells (1999) suggested there could be a value in exploring the effect of attitude toward the website in evaluating the effectiveness of web advertising. In research about mobile advertising the results show that entertainment is the main attribute affecting consumer attitude toward mobile ads (Tsang, Ho & Liang, 2004). In a study of Bauer et al. (2005) also about mobile marketing advertising, is shown that consumers will develop a positive attitude toward the advertisement when the advertisement embodies messages that are entertaining or provide a high information value. For promotional messages, Hahn et al. (1995) stated that the consumers that reviewed an advertisement with a coupon generated favorable attitudes toward the advertisement. A study from Taylor, Lewin, and Strutton (2011) showed that people have the most positive attitude toward entertainment ads, and after the entertainment ads the informative ads were ranked most positive on attitude. So research found that the ad type seems to play a role, as different researchers found differences in the effect of ad types. Taken the above research into account, and translating it to message types on Facebook the following hypothesis is proposed:
H1 There is a difference in attitude towards the message depending on the message type (informational message, promotional message, entertainment message, and social message).
2.2.2 Attitude toward the brand
19 H2 There is a difference in attitude towards the brand depending on the message type
(informational message, promotional message, entertainment message, and social message).
2.2.3 Purchase intention
Purchase intention is defined by Lutz et al. (1983) as receivers' assessments of the likelihood that they will purchase the brand in the future. We also want to see whether the different content types of the messages have an effect on the decision to purchase products of this brand. Purchase intention is used in previous literature as a variable to measure advertisement effectiveness (Lutz et al. , and 1983; Chen et al., 2008;Lee and Lee, 2009) for these reasons purchase intention is used in this research. In this research not every message is promoting a physical product so the purchase intention of the brand is examined and not the purchase intention of a specific product. Hahn et al. (1995) stated that the consumers that saw an advertisement with a coupon (promotional content) generated favorable attitudes toward the advertisement and the purchase than the consumers that did not saw an advertisement with a coupon. In the research before is shown that the message type plays a role, as researchers found differences in the effect of message types. Therefore the following hypothesis is proposed:
H3 There is a difference in purchase intention depending on the message type (informational message, promotional message, entertainment message, and social message).
2.2.4 Engagement rate
20 were related to competitions (in this thesis this is an element of promotional messages). Cvijikj et al. (2011) had a variable, designed questions, in their study (in this research designed questions belong to the social messages). The results of the research of Cvijikj et al. (2011) shows that designed questions have a significant larger comments ratio. A study from Cvijikj and Michahelles (2013) on the influencing factors on customer engagement of posts on Facebook brand pages showed that remuneration (promotional) posts have a positive effect on the comments ratio, while the shares ratio showed no effect, and the likes ratio showed a negative effect. Because the engagement rate is a combination of these three variables it is expected that the promotional post shows no influence on the engagement rate. In the research is shown that the message type plays a role, as different researchers found differences in the effect of message types. Therefore the following hypothesis is proposed:
21
2.3 Consumer brand relationship
The moderating variable in this study is consumer brand relationship. In this paragraph the definition
of this construct is explained, and how it is related to Facebook brand pages. Furthermore, the
relationship between the message type and the effectiveness of the message is discussed.
Consumer brand relationship can be described as the overall relationship that a consumer haves with a brand. There is an interpersonal relationship between the consumer and the brand. (Fournier & Alvarez, 2012). Consumer brand relationship has to be seen as a result of an interactive process between the consumer and a brand instead of a simple pattern of repurchase based on consumer satisfaction with a product (Fournier, 1998). Fournier (1998) created a brand relationship quality construct. This construct measures the whole brand relationship (quality, depth and strength). In this thesis consumer brand relationship is chosen because research has shown that this construct is far more intense than simple liking a brand or being involved with a brand. Consumers can experience a love-like feeling toward a brand (Caroll & Ahuvia, 2006). Research has shown that brand pages on Facebook have a role in the consumer brand relationship with these brands. Consumers who are fans of brand fan pages tend to be loyal and committed to the company, and are more open to receiving information about the brand (Bagozzi & Dholakia 2006). Harter et al. (2010) also argued that social media is important for the humanization of the company or brand, because through social media companies create a personal customer experience.
In this thesis consumer brand relationship is not investigated as an outcome variable but as a moderating variable. Although various research showed that consumer brand relationship is an outcome variable, we believe that consumer brand relationship could also be seen as a moderating variable. Research from Aggarwal and Law (2005) support the thought that consumer brand relationship is not only an outcome variable. This study is about relationships norms as a moderator of information processing. Results from this study suggests that when a consumers is interacting with a brand, the type of consumer brand relationship influences what information becomes relevant (Aggarwal & Law, 2005).
22 shifts to a cognitive evaluation. This defines Fournier (1998) as emotional or psychological connection to a brand (consumer brand relationship). According to the study of Greenwald and Leavitt (1984) viewers of advertisements that are highly involved (thus have a high consumer brand relationship) show the message elaboration of the viewers is increased. Also the cognitive evaluation of the viewers of advertisements dominates over their affective processing. The research of Ahn and Edwards (2002) cited by Yang (2004) showed that consumers with a high involvement (thus have a high consumer brand relationship), are more likely to click on banner advertisements than consumer in a low involvement situation. The study of Zaichkowsky (1985) shows that involvement influences the motivation to process information centrally. Low involved consumers require less product information because they perceive the risk of a decision as less relevant, and therefore dedicate less attention to advertising stimuli. Petty and Cacioppo (1986) introduced the Elaboration likelihood model. This is dual process theory about how attitudes are formed and changed. Attitudes can be a result from a couple of things, but persuasion is a prime source. The elaboration likelihood model features two routes of persuasive influences: the peripheral and central route. The most important variable in this process is involvement. When people are motivated to think about the content of a specific message the elaboration is high, and as a consequence the consumer brand relationship is higher. When elaboration is high the central route will probably occur. The peripheral route is likely to occur for low elaboration. The peripheral route process do not involve elaboration of the message by cognitive processing of an argument. Petty and Cacioppo (1986) demonstrated by their Elaboration likelihood model that peripheral cues are more significant for advertising effectiveness for low involved consumers. Consumers with a higher involvement, and a higher brand relationship are geared toward processing the message more centrally. For the central route, people will have a systematical and cognitive processing of the information. In the central route will the arguments be weighed against each other. For the peripheral route, the content of the message hardly plays a role, in that case the shallow characterizes are more important cues, like music, humor, or attractive pictures.
Considering the above definitions and explanations it is likely that informative messages will be processed through the central route. Whereas the entertainment, social and promotional messages will be processed more peripheral. A message that displays informative content will be more effective for consumers who are highly involved (with a high consumer brand relationship). Messages with entertainment, promotional or social content will be more effective for consumers that are low involved, thus have a low consumer brand relationship.
23 H 6.1: The effectiveness of the messages (attitude toward the message, attitude
toward the brand, purchase intention and engagement rate) with informational content is higher for consumers with a high consumer brand relationship.
24
3.
Research method
In this thesis the brand Heinz (Beanz) is used. Heinz was chosen because it is a well know brand, it is worldwide known for its ketchup and tomato products. Heinz distributes in more than 50 countries. They communicate via Twitter, Facebook and their website. For this study the messages were used from the Facebook page from Heinz Beanz United Kingdom. In this study an experimental design was applied: an in between subject design is used with one factor.
3.1 Pre-test
25 entertainment (M = 4.90, SD = .29, p < 0.001) and informational messages (M = 4.98, SD = .29, p < 0.001) on the promotional scale (F(2,29) = 186.96, p <0.001).
Table 3. Questionnaire items pre-test. Entertainment scale items
(Strongly disagree - Strongly agree, nine-point Likert scale) (Ducoffe, 1995) 1. This message is entertaining
2. This message is enjoyable 3. This message is pleasing 4. This message is fun to use 5. This message is cool 6. This message is exciting Informational scale items
(Does not apply at all - Completely applies, five-point Likert scale) (Jourdan, 1999) 1. Thanks to this message, I have learned something new about the product
2. After having seen this message, I know what is important to look for when buying this product 3. With the information supplied by this message, I am more capable of comparing the product of
this brand to its competitors
4. This message speaks of choice criteria for the product of this brand, which I find important 5. I feel more capable and more competent to choose and evaluate this type of product after
having seen this message Promotional scale items
(Strongly disagree - Strongly agree, seven-point Likert scale) 1. This message is a promotional message
2. this message gives me the information about a contest 3. this message displays an offer
4. This message show me a coupon Social scale items
(Strongly disagree - Strongly agree, seven-point Likert scale) (Gao et al., 2010) 1. The message makes me feel the company wants to listen to its customers 2. The message provides me an opportunity to give my feedback
3. The message can create a conversation between the company and the customer
4. The message gives me the opportunity to respond in more than one way (e.g. like, share, react) 5. The message information seems not interpersonal (r)
6. I feel like having an interpersonal communication when receiving the message
26 Table 4. Results From The One Way ANOVA, Post Hoc For The Social Factor
Message Type Mean Difference (I-J) SD P
Social
Informational
Entertainment 2.33 3.31 .486
Promotional .89 3.31 .791
Social -4.93 3.31 .145
Entertainment
Informational -2.33 3.31 .486
Promotional -1.44 3.31 .666
Social -7.25 3.31 .035
Promotional
Informational -.89 3.31 .791
Entertainment 1.44 3.31 .666
Social -5.81 3.31 .088
Social
Informational 4.93 3.31 .145
Entertainment 7.26 3.31 .035
Promotional 5.81 3.31 .088
Table 5. Results from the One way ANOVA, post hoc for the informational, entertainment, and promotional message.
Message Type Mean Difference (I-J) SD P
Entertainment
Informational Entertainment -6.48 .88 .000
Promotional -.34 .88 .703
Entertainment Informational 6.48 .88 .000
Promotional 6.14 .88 .000
Promotional Informational .34 .88 .703
Entertainment -6.14 .88 .000
Informational
Informational Entertainment 2.86 .37 .000
Promotional 2.38 .37 .000
Entertainment Informational -2.86 .37 .000
Promotional -.48 .37 .200
Promotional Informational -2.38 .37 .000
Entertainment .48 .37 .200
Promotional
Informational Entertainment -.08 .29 .801
Promotional -4.97 .29 .000
Entertainment Informational .08 .29 .801
Promotional -4.90 .29 .000
Promotional Informational 4.98 .29 .000
[image:26.595.70.525.334.643.2]27
3.2 Main Study
3.2.1 Respondents
Participants were invited to fill in the online questionnaire using the link that was displayed and distributed on personal social networks (Facebook and Twitter). The questionnaire was online available for two months. There was one criteria before people could start with the questionnaire, and that was they needed a Facebook account to fill in the questionnaire. This criteria was needed to be sure that the respondents knew the mentioned (Facebook) terms and were familiar with the look and feel of the (Facebook) messages that were displayed.
[image:27.595.69.556.498.757.2]For this research 398 started the survey, and in total only 191 ended the survey. The response rate is therefore: 47%. After deleting the invalid data the total number of respondents was N=191. Gender was divided in 58% men (N=111) and 41% women, and one person did not indicate his or her gender (N=79). The average age of the respondents was 29 years. The education distribution was as follows: people with high school as the highest completed education was 5% (N=9), people with vocational education was 12% (N=23), a bachelor 47% (N=90), and a master 35% (N=67). Facebook experience was measured with a 5 points likert scale, from strongly disagree (1) to strongly agree (5). The average Facebook experience for the respondents was 3,9, this means that they were relatively experienced. The respondents were evenly distributed over the 3 stimulus conditions. Table 6 displays the social demographic data and Facebook experience of the respondents.
Table 6. Social demographic data and Facebook Experience
% N M (SD.)
Gender Male 58.1 111
Female 41.4 79
Age 28.9 (9.5)
Education
High School 4.7 9
Vocational education 12 23
Bachelor 47.1 90
Master 35.1 67
Facebook experience
I know how to create a Facebook profile
I know when a Facebook profile or pages is fake I understand the way Facebook works
I understand how to use and apply apps on Facebook I understand almost all Facebook terms
I can usually fix any problems I encounter when using Facebook
I help others who are learning to use Facebook
I download and install software updates from Facebook when necessary 4.6 (0.6) 3.5 (0.9) 4.2 (0.8) 3.8 (1) 4.2 (0.8) 3.9 (0.9) 3.5 (1.1) 3.6 (1.2)
28
3.2.2 Questionnaire
29 experience. The rephrased scale was initially the Internet Knowledge measurement scale, created by Potosky (2007). The Facebook experience will be presented in a six-point Likert scale. The following options were presented: “I don’t understand this statement and cannot respond” (0), strongly disagree (1), disagree (2), neither agree nor disagree (3), agree (4), and strongly agree (5). The “I don’t understand…” option was included to increase the likelihood of measuring actual knowledge instead of each respondent’s ability to guess or to select a response that they did not understand” (Potosky, 2007). In the appendix (E,F,G) all the questionnaire items are shown. After the questions about the Facebook experience there were some general questions about age, gender, and education.
In this study a MAN(C)OVA was used to analyze the results from the questionnaire. The main effects of the independent variables (message type) on the dependent variables (message effectiveness; attitude toward the message, attitude toward the brand, purchase intention and engagement rate) were analyzed via a MANOVA. A MANCOVA was used to analyze the moderating effect of consumer brand relationship.
3.2.3 Measurements
30
4.
Results
4.1 Manipulations check
To ensure the developed messages manipulate the right way a manipulation check was measured for the three manipulated conditions: the informational message, the entertainment message, and the promotional message. For the manipulations check the same items were used as in the pre-test (table 3) only the social items were left out. The participants were randomly assigned. The message manipulation was successful, below the results of the manipulation check are explained.
Informational message vs. promotional and entertainment messages
Results of the one way ANOVA (post hoc, Bonferroni) showed that the informational message (M = 2.41, SD = 1.45) differed significant from the entertainment message (M = 4.38, SD = 2.03, p < 0.001), and the promotional message on the informational scale (M = 3.99, SD = 1.00, p < 0.001) (F(2,190) = 26.55, p < 0.001).
Promotional message vs. informational and entertainment messages
Results of the one way ANOVA (post hoc, Bonferroni) showed that the promotional message (M = 3.99, SD = 1.00) differed significant from the informational message (M = 2.41, SD = 1.45, p < 0.001), and the entertainment message on the promotional scale (M = 4.38, SD = 2.03, p < 0.001) (F(2,190) = 46.84, p < 0.001).
Entertainment message vs. informational and promotional message
Results of the one way ANOVA (post hoc, Bonferroni) showed that the entertainment message (M = 4.38, SD = 2.03) differed significant from the promotional message (M = 3.99, SD = 1.00, p < 0.001), and the informational message on the entertainment scale (M = 2.41, SD = 1.45, p < 0.001) (F(2,190) = 13.23, p < 0.001).
4.2 Analysis of main effects
31 Table 7. An overview of the means of the dependent variables and the message type.
Dependent variable Message type Mean SD N
Engagement rate
Promotional 2.44 1.60 62
Informational 2.19 1.32 66
Entertainment 2.52 1.40 63
Total 2.38 1.44 191
Attitude toward the message
Promotional 2.68 .96 62
Informational 2.80 .74 66
Entertain 2.71 .87 63
Total 2.73 .86 191
Attitude toward the brand
Promotional 2.21 .68 62
Informational 2.35 .72 66
Entertainment 2.33 .73 63
Total 2.30 .71 191
Purchase intention
Promotional 2.50 .89 62
Informational 2.69 .90 66
Entertainment 2.64 .94 63
Total 2.61 .91 191
Table 8. Main effects of message type on attitude toward the message, attitude toward the
brand, purchase intention, and engagement rate, and the validity of the hypotheses.
Independent variable Hypotheses Dependent Variables df Mean Square F P
Message type
H1 ✗ Attitude toward the message 2 1.59 2.46 .08 H2 ✗ Attitude toward the brand 2 .67 1.49 .23
H3 ✗ Purchase intention 2 .02 .02 .98
H4 ✗ Engagement rate (ER) 2 .15 .09 .92
[image:31.595.71.527.463.574.2]32
4.3.
Analysis of interaction effects
A MANCOVA was conducted that examined the effect of consumer brand relationship on the influence
of message type on the effectiveness of the message. In this paragraph the results of this analysis is
presented. They are presented as four dependent variables as explained earlier. These dependent
variables are split into: Attitude toward the message, attitude toward the brand, purchase intention,
and engagement rate. An overview of all the interaction effects can be found in table 8.
4
.3.1 Attitude toward the message
A MANCOVA was conducted that examined the effect of consumer brand relationship on the influence of message type on the attitude toward the message. There was a significant interaction between the effects of consumer brand relationship, and messages type on the attitude toward the message, F(3, 167) = 6.637, P = < 0.001. The figure (6) shows that consumers with low brand relationship, promotional and entertaining messages lead to a higher attitude toward the message. For consumer with a high brand relationship the informational messages lead to a higher attitude
[image:32.595.71.527.256.362.2]toward the message. Furthermore, the difference in attitude toward the message between the three message types is greater for consumers with a high consumer brand relationship.
Table 8. Moderating effects of consumer brand relationship on the influence of message type on the
attitude toward the message, attitude toward the brand, purchase intention and engagement rate.
Independent variable Moderating variable Dependent variable DF F P
Message type Consumer brand relationship
Attitude toward the message 3 6.64 .000 Attitude toward the brand 3 6.61 .000 Purchase intention 3 3.86 .011
Engagement rate 3 8.01 .000
Figure 6. The interaction effect of consumer brand
relationship on the relationship between message type
[image:32.595.262.519.300.625.2]33 The results of this research support the hypothesis (6.1): That the attitude toward the message with informational content is higher for consumers with a high consumer brand relationship. The results of this study also show that hypothesis 6.2 is accepted: attitude toward the message with entertainment, promotional, or social content is higher for consumers with a low consumer brand relationship
4
.3.2 Attitude toward the brand
A MANCOVA was conducted that examined the effect of consumer brand relationship on the influence of message type on the attitude toward the brand. There was a significant interaction between the effects of consumer brand relationship and messages type on the attitude toward the brand, F(3, 167) = 6.613, P = < 0.001. Figure 7 shows that consumers with a low brand relationship, the promotional and entertainment messages have a greater influence on their attitude towards the brand. For consumers with a high brand relationship the informational messages lead to a higher attitude toward the
brand. Additionally, the difference in attitude toward the brand between the three message types is greater for consumers with a high brand relationship.
The hypothesis (6.1) : Attitude toward the brand with an message with informational content is higher for consumers with a high consumer brand relationship, is accepted. Furthermore, the results of this study showed that the attitude toward the brand with messages with entertainment, promotional, or social content is higher for consumers with a low consumer brand relationship, for that reason hypothesis 6.2 is also accepted.
Figure 7. The interaction effect of consumer brand
relationship on the relationship between message type
[image:33.595.266.522.196.429.2]34
4
.3.3 Purchase intention
A MANCOVA was conducted that examined the effect of consumer brand relationship and message type on the purchase intention. There was a significant interaction between the effects of consumer brand relationship and messages type on the purchase intention, F(3, 167) = 3.860, P = 0.011. Figure 8 shows that the purchase intention of consumers with a low brand relationship is the same for all message types, so the purchase intention is not influenced by the message type. On the other hand, looking at consumers with a high brand relationship, the informational messages lead to a higher purchase intention compared to the entertaining or the promotional message.
The hypothesis (6.1): The purchase intention with messages with informational content is higher for consumers with a high consumer brand relationship, is accepted in this study. On the other hand, the hypothesis (6.2) is rejected: The purchase intention with messages with entertainment, promotional, or social content is higher for consumers with a low consumer brand relationship.
Figure 8.. The interaction effect of consumer brand
relationship on the relationship between message type
[image:34.595.270.523.84.310.2]35
4
.3.4 Engagement rate
A MANCOVA was conducted that examined the effect of consumer brand relationship and message type on the engagement rate. There
was a significant interaction between the effects of consumer brand relationship and messages type on the engagement rate, F(3, 167) = 8.008, P = < 0.001. Figure 9 shows that consumers with a low brand relationship are less influenced by the message type, they just show to some extent a higher engagement rate when they see a promotional message compared to the informational and entertainment message types. Whereas, consumers with a high brand relationship show a higher
engagement rate when they see an entertaining or promotional message compared to an informational message.
The hypothesis (6.1): The engagement rate for messages with informational content is higher for consumers with a high consumer brand relationship, is not supported. Similarly, the hypothesis (6.2): The engagement rate for messages with entertainment, promotional, or social content is higher for consumers with a low consumers brand relationship, is rejected. Below are the hypotheses displayed, in table 9 is an overview of the validity of these hypotheses.
H 6: Consumer brand relationship has a moderating effect on the relationship between message type and the effectiveness of the message.
H 6.1: The effectiveness of the messages (attitude toward the message, attitude toward the brand, purchase intention and engagement rate) with informational content is higher for consumers with a high consumer brand relationship.
H 6.2 : The effectiveness of the message (attitude toward the message, attitude toward the brand, purchase intention and engagement rate) with entertainment, promotional, or social content is higher for consumers with a low consumer brand relationship.
Figure 9. The interaction effect of consumer brand
relationship on the relationship between message type
[image:35.595.73.521.102.395.2]36 ✔ Hypothesis accepted, ✗ Hypothesis not accepted.
Table 9. Validity of hypotheses.
Hypotheses Effectiveness of the message Result
H6 Consumer brand relationship has a moderating effect on the relationship between message type and the effectiveness of the message
✔
H6.1
The “effectiveness of the messages”… with informational content is higher for consumers with a high consumer brand relationship.
..”Attitude toward the message” ✔ …”Attitude toward the brand” ✔
…”Purchase intention” ✔
…”Engagement rate” ✗
H6.2
The “effectiveness of the message” … with entertainment, promotional, or social content is higher for consumers with a low consumer brand relationship.
…”Attitude toward the message” ✔ …”Attitude toward the brand” ✔
…”Purchase intention” ✗
37
5.
Discussion
This thesis focused on Facebook messages on brand pages and their effectiveness, besides this the
influence of consumer brand relationship on the effect of this messages was checked. The content of
Facebook messages are getting more and more important for companies. Not only because Facebook
on itself is expanding, but also more and more business are creating their own brand page on Facebook.
This study consisted of research about three different Facebook brand page messages. The message
types were: informational messages, entertainment messages, and promotional messages (dependent
variables). The dependent variables in this research were, the attitude toward the message, the
attitude toward the brand, purchase intention, and engagement rate. The moderating variable was
consumer brand relationship. In this chapter the managerial implications, limitations and directions for
further research are discussed.
5.1 Theoretical implications
The primary goal of this research was to understand better if there is a difference between the different message types and the effect the message type has on the effectiveness of the message. The effectiveness of the message was measured via four variables: attitude toward the message, attitude toward the brand, purchase intention, and engagement rate. The other goal was to see how consumer brand relationship would moderate this relation. The main research question was: "Is there a difference in the effect of the different message types, and how does consumer brand relationship
moderates this?".
Main effects
38 in attitude toward the brand between the three message types (hypothesis 2).For purchase intention Hahn et al. (1995) stated that there was a difference in purchase intention for people who saw a message with a coupon, than for consumers that saw the message without a coupon. Therefore, was expected that there was a difference in purchase intention between the three message types (hypothesis 3). Surprisingly no differences were found in this research. For engagement rate (which is measured in likes, comments, and shares) was expected that there is a difference in engagement rate between the three different message types (Hypothesis 4). Taken the research from Cvijikj and Michahelles (2013), in account which showed that Informative posts got more likes than for example promotional posts. Whereas, social, and promotional posts got more comments. The results of this study show that there is no significant difference in engagement rate between the three message types. So in contrast with previous findings, however, no evidence for differences in engagement rate between the three messages types was detected. A reason for this could be that all the messages have the possibility to respond, maybe people like an informative message faster, but would share an entertainment message. The engagement rate consist of three different constructs with different meanings. This could be a reason that the engagement rate on the three messages show no significant difference.
Moderating effects
Furthermore, this study tested whether the moderator, consumer brand relationship, influences the relation between message type, and message effectiveness. Results indicated that there is a significant influence. Research from Aggarwal and Law (2005) showed that the level of consumer brand relationship influences what information becomes relevant. The research of Petty and Cacioppo (1986) showed that people that are highly involved (thus have a higher consumer brand relationship) are geared toward processing the message more centrally (informative messages are processed more centrally). While consumer that have a lower involvement (thus have a low consumer brand relationship) are geared toward processing the message more peripherally (promotional, and entertainment messages are processed more peripherally). Therefore, the hypotheses that were used in this thesis were:
H 6: Consumer brand relationship has a moderating effect on the relationship between message type and the effectiveness of the message.
39 H 6.2 : The effectiveness of the message (attitude toward the message, attitude
toward the brand, purchase intention and engagement rate) with entertainment, promotional, or social content is higher for consumers with a low consumer brand relationship.
The results of this study indicate that consumers with a low consumer brand relationship, have a higher attitude toward the message when the messages contains promotional or entertaining content. While consumers with a high consumer brand relationship the informational messages lead to a higher attitude toward the message. For attitude toward the message Hypothesis H6.1, and H6.2 are confirmed. The results for attitude toward the brand show the same relationship as attitude toward the message. Additionally, the difference in attitude toward the brand between the three messages types is greater for consumers with a high consumer brand relationship. For attitude toward the brand Hypothesis H6.1, and H6.2 are confirmed. A reason for this result could be that people that are already really interested in the brand and have a high consumer brand relationship will probably not be interested in entertainment messages or promotional messages, because they will by the brand anyway.
The results of this study show that the purchase intention of consumers with a low consumer brand relationship is the same for all message types. On the other hand, consumers with a high consumer brand relationship, that were shown messages that contain informative content, have a higher purchase intention compared to entertaining or promotional messages. Therefore, hypothesis H6.1 is confirmed. However, hypothesis H6.2 is not confirmed, because the results for the low consumer brand relationship show no differences.
40 The hypothesis H6: Consumer brand relationship has a moderating effect on the relationship between message type and the effectiveness of the message (H6) is accepted. The results show a significant
moderating effect of consumer brand relationship. In other words a high or low consumer brand relationship show a different effect on the relationship between message type and the effectiveness of the message.
From the results of this study it can be concluded that there is no significant difference in message effectiveness for the three message types. Nevertheless the moderating effect of consumer brand relationship on the relationship between the message types and the effectiveness of the message is shown in this study. See an overview of the accepted and/or rejected hypotheses in table 7 (for the main effects) and table 9 (for the moderating effects).
5.2 Managerial implications
This chapter discusses how companies can make use of the results of this stud. The results can help companies to decide which content to use in the Facebook messages.
The results of this study show no differences between the different Facebook message types and the effectiveness of them. Except, when looking at consumer brand relationship, and the way this variable influences the relationship between message types, and effectiveness there is some significant differences between message types. This study shows that consumer brand relationship plays a role in the effect of the different messages. So is the attitude toward the messages and attitude toward the brand for consumers with a high consumer brand relationship the most positive when it comes to informational messages, while messages that have promotional or entertainment aspects are more appreciated by consumers with a lower consumer brand relationship. On Facebook brand pages you can select which people see your message, when you decide to promote the message it is also possible to reach "non-fans" of you brand page. People that have liked a brand page on Facebook are called fans. People that did not liked the brand page are called non fans. With the results of this study it is recommended to use an informational message for people that are already a fan of your brand page. For the non-fans it is wise to use message with promotional or entertainment content.
41 "comment" on their Facebook messages then companies should use an entertainment message. Consumers with a high brand relationship are more willing to engage than people with a low engagement rate. Consumers with a low engagement rate are more likely to engage with promotional messages. For this reason it is important marketers should focus on what type of consumers are liking their Facebook brand page and adapt their messages to their audience.
So it is advisable to reach consumers that are already fans in a different way than consumers that are not yet a fan of your Facebook brand page. For companies it is possible on Facebook to select a specific target group for messages and advertisements. In this thesis is assumed that people that are fan of brand page (in this case of Heinz) have a higher consumer brand relation toward Heinz than people that are not a fan of the Heinz brand page. On Facebook it is possible to target fans or non-fans with advertisement or Facebook messages. Depending on the goal of the message (reaching fans or non-fans) it is possible to adapt the message to the target group and only display them to this specific group. So it is also possible to target the fans of your brand differently than the non-fans. As mentioned above people with a low consumer brand relationship have interest in different messages than people that have a high consumer brand relationship. Taking the results of this study in account it is wise to adjust the messages on Facebook to the right target group. This because these results show that the effectiveness of the informational, entertainment, or promotional messages differ for high or low consumer brand relationship.
5.3 limitations and directions for further research
Limitations of this research were that only the brand Heinz was used, and within this brand there was chosen for the product Heinz Beanz. This is a quite specific product and brand, therefore these results cannot be generalized. Another limitation is the product use. Heinz Beanz is a product that is not that familiar in the Netherlands, and because it was a food product it is plausible to think that respondents filled in the questionnaire with in mind that they liked or disliked the taste of the product and let that play a role in their evaluations. Additionally the group of respondents was quite young, with an average age of 29. Younger people are more present on Facebook than older people, and this could have influenced the results, and therefore the result are not generalizable.
