NHSvax corpus contents
4.5 Corpus Analysis: Methodological considerations
4.5.4 Analysing contextual information
4.5.4.1 Forum posters: frequency and ideological stance
As Partington (2008) observes, one way in which CADS methods deviate from the standard Sinclairean approach to corpus analysis is that the CADS analyst acquaints him- or herself as much as possible with the data. The elements of the JABS corpus data which demanded extra attention were the discussion forum data. A particularly salient feature concerned variations in the frequency with which different individuals posted on the site. Many only post once or twice but some are highly frequent posters. It was important to discover if there was a difference in the motivations between low-frequency and moderate- or high-frequency
posters. There was also evidence that certain high-frequency posters were not vaccine-critical.
In 2007 and 2008, the JABS forum attracted a small but vociferous number of members who expressed an overtly pro-vaccination and pro-science stance.46 The pro-science posters were banned from the site at the end of 2008. While they were active on the site, their contributions attracted a lot of response, so the threads in which they participated tended to become very long. The arguments between vaccine-critical and pro-science posters on the forum were
46 A number of these individuals were readers of Ben Goldacre’s Bad Science blog http://www.badscience.net/
and one, who calls herself Becky Fisseux, had her own blog: ‘JABS Loonies - Justice, Awareness, Basic Support and Mind Blowing Stupidity’, http://jabsloonies.blogspot.co.uk/.
136
interesting from the point of view of discovering how people on the site constructed their arguments. It was therefore necessary to read sections of the data online in order, firstly, to find out how posts were distributed among posters, and, secondly, to discover whether low- and moderate- to high-frequency posters had different motivations for posting, and, thirdly, to distinguish the pro-science posters from the vaccine-critical ones.
There are 517 posters represented in the JABS forum sub-corpora. Most of them, 278, to be precise, are based in the UK, eighteen are located in the USA, five in Ireland, four in Canada, and two in New Zealand.47 As tends to be the case with online forums, the majority of posts are made by a minority of posters, with a large number of posters making only one post. The 1,989 threads represented in the JABS forum sub-corpora comprise 20,250 posts. As Table 4.4 below shows, just over half the posters (279) make only one or two posts and the vast majority (406) make ten or fewer.
No. of posts
1 2 3 4 5 6 7 8 9 10 Total
No. of posters
200 79 52 18 19 12 4 11 3 8 406
Table 4.4 Number of posters posting one to ten posts
Individuals who post only once typically ask for advice or information. These posters tend not to be overtly vaccine-critical, although the fact that they are posting on the forum indicates that they are certainly sceptical about the safety of the vaccines and resistant to accepting the
47 In 210 cases, location information is not given.
137
advice or directives of health professionals uncritically. The example shown in Figure 4.13 is typical of this kind of request. The poster expresses uncertainty: I am unsure what to do for the best; I really don’t know what to do; I feel so confused. She also expresses a lack of trust in her GP, since she questions what the GP has told her: Does anyone know if this is true?
Instead she directs her request at anyone on the JABS forum who might read her post: Can anyone help? been putting it off for so long(she is now 5 months), that my Doctor has told me if i dont hurry up and make my mind up then it will be too late to give it to her, as you can only give it up to 11 months. Does anyone know if this is true? Also my son who is now 3 yrs, had the old baby jab minus the whooping cough, but ive told that he can't have it that way now when he has his booster.
I really dont know what to do. I was so sure when it came to the MMR(he did not have it). I feel so confused.
Can anyone help?
Victoria.
<endpost/>
Figure 4.13 Example of a request for advice from a low-frequency poster
By contrast, the ten most frequent posters collectively make over 10,000 posts (see Table 4.5 below). Most posts made by moderate- to high-frequency posters on the forum (those who make more than three posts) are not requests for advice but observations and expressions of opinion or responses to the observations and arguments of others. Most moderate- to high-frequency posters are overtly vaccine-critical.
138
Table 4.5 Number of posts by the ten most frequent posters on the JABS forum.
As is typical of reformist vaccine-critical groups, many of the moderate- to high-frequency posters claim that they are not opposed to vaccination per se, but believe vaccines can cause neurological problems. Such a stance is illustrated in the example shown in Figure 4.14 below.
We believe our daughter was made severely deaf by MMR. However with regards to autism I have come across several parents whose children have been diagnosed/being diagnosed with a disorder on the autistic spectrum and the parents are adamant that it was not vaccine related as they saw certain signs prior to the MMR vaccine being administered at about 13 months. This I can understand but it seems to me that many people seem to think MMR is the only vaccine to cause problems. I'm not sure that they have even considered the earlier/new born jabs to be a concern and that if these caused a problem then the side effects will be seen before 13 months. I am not anti-vaccine and I believe that anti-vaccines have a place in society if used with great caution and with fully informed choice.
We can only do our best
<endpost/>
Figure 4.14 An example of a JABS forum post from a moderate frequency poster
139
As we can see in Figure 4.14, the author states that she is ‘not anti-vaccine’, but believes MMR is responsible for her daughter’s deafness and suggests that other vaccines administered in early life may cause autism. A large number of threads, especially those posted on the News and Comment sub-forum, are initiated by a poster drawing attention to a story in the news and, very often, uploading the story into the post. An example of the beginning and end of such an initial post and a response to the initial post is given in Figure 4.15 below.
<thread>
<topicid>3105</topicid>
<title>Baby vaccine could increase the risk of asthma</title>
<page>page 1 of 1</page>
A vaccine given to babies could increase the risk of childhood asthma By Beezy Marsh
Last updated at 12:12 AM on 21st October 2008
A vaccination given to babies has been linked to asthma.
Experts believe the diphtheria, tetanus and whooping cough jabs might provoke an immune system response which predisposes the body to the lung condition. […] A Department of Health
spokesman said 'Several large studies have looked at whether childhood vaccines can cause asthma or allergies - they have found no evidence for this.'
<endpost/>
This is why Department of health Spokesmen are desperate to cover up any report of vaccine damage at all costs by blanket denials and even refuse to look at any evidence like the above.
Any study you like as long as we are the ones doing it, what a joker.
Vaccinations cause chronic diseases is as fully an accurate things to say as "smoking kills"
warnings on cigarette packs.
<endpost/>
<endpage/>
</thread>
Figure 4.15 Part of a thread from the News and Comment sub-forum
140
Most high-frequency posters are vaccine-critical, but, interestingly, the pro-science posters are among the most frequent posters. The names, number of posts, and ideological stances of all individuals with 40 or more posts are shown in Table 4.5 below.
Name No of
Table 4.5 Posters with 40 or more posts, number of posts, and stance towards vaccination
As mentioned above, contributions from pro-science posters provoke a lot of response from vaccine-critical members. Threads to which pro-science posters contribute tend to run to
141
several pages. Figure 4.16 below shows a post from a pro-science poster. The degree of animosity she attracts from one of the high-frequency vaccine-critical posters is evident in the quoted section of text in her post.
<author>Becky </author>
<country>United Kingdom</country>
<posts>287 PostsPosted</posts>
<datetime>07/09/2008 16:37:44</datetime>
<beginpost/>
<q>Originally posted by GUS THE FUSS
Dear Becks one observation of you you defend the Lucifer himself Sir(to you) Roy Meadows hardly balanced are you ,wheres your people evidence Becky anon the fishy fish...to refute Dr Martin??(wont hold my breath) or in your case your breath has to be worse than your bite...</q>
MMR RIP Gus,
I've never defended Cybertiger's bête noire Prof Sir Roy Meadows - and frankly I see no reason why it would be relevant here, even if I had.
I'm not just suggesting that there aren't "stealth viruses" in vaccines - I'm suggesting that it's rather suspicious that such an important development in our understanding of viruses has only ever been discovered by one researcher, and that no-one has managed to replicate his work. Neither has he put such important work up for peer-review. You have to ask yourself why. This isn't just a vaccine matter, it's potentially a massive step forward in the study of viruses. You can't simply use the old
"oh, he knows it wouldn't be worth putting it forward for peer review because every other scientist in the world has been bought off by big pharma" argument, as there would be someone prepared to do the work, and, if Dr Martin's right, to back him up. "Oh, I'm so scared of the conspiracy..." just doesn't wash.
Follow the money.
Kind regards, Becky
<endpost/>
Figure 4.16 Example of a post from a pro-science poster
4.5.4.2 Using expanded context
As discussed already, it is sometimes necessary to read extended context around a particular concordance line. This is especially the case where a section of text is quoted and one needs to see how it is framed or when a news text is included and one needs to scroll to the top of the article to discover how it is framed or how the person posting it has framed it. In
142
WordSmith Tools, by clicking on a concordance line, the researcher can access the entire text file in which the line occurs. This was done frequently throughout analysis.
4.6 Procedure
Once the JABS and NHSvax corpora had been compiled and sections of the data had been read in order to gain a general idea of the nature of the contents, the following steps were taken.
• Frequency lists of the JABS and NHSvax corpora were drawn up. WordSmith Tools settings were adjusted to exclude header information from the word count.
• In order to complete the cleaning of corpus data, instances of html codes in the word list were noted. These were converted to plain text using WordSmith Tools Utilities Text Converter.48
• Following conversion of html codes, frequency lists of the JABS and NHSvax corpora were drawn up afresh.The number of tokens in the corpora and their various sub-corpora were recorded from the WordSmith Tools statistics.
• Keyword lists of the JABS and NHSvax corpora were compiled using the BNC World version wordlist as a reference corpus. Log-likelihood was used as the statistical measure of probability. The parameters were set as follows: Log-likelihood threshold 24; maximum p value 0.000001; maximum number of keywords, 6,000; negative keywords were excluded.49 In order to limit the number of keywords to a manageable quantity but still capture the most central ones, the minimum number of occurrences
48 The symbol #, which is used by WordSmith to represent numerals, proved impossible to remove but, fortunately, it does not corrupt any lexical items in the corpus.
49 WordSmith allows the option to draw up a list of negative keywords, that is, words which occur markedly infrequently when compared with a reference corpus. Since the JABS corpus keyword list was to be compared with the NHSvax corpus keyword list, it was not considered necessary to draw up a list of negative keywords.
143
for keywords was set at 15 for the NHSvax corpus, and, since the JABS corpus is 6.8 times the size of NHSvax, the minimum frequency for JABS was set at 100
occurrences.
• Analysis began with an examination of the most frequent words and statistically most significant keywords in the NHSvax corpus, in order to gain an idea of the key features of mainstream medical-scientific discourse about health, risk, immunity, and the MMR-autism causal hypothesis.
• The 200 most frequent words in the NHSvax corpus were scanned in order to identify key themes and to provide a means for gauging the reliability of the keyword list.
• The keyword list was scanned and cross-compared with the NHSvax word list. In order to limit the number of items selected for closer investigation to a manageable quantity, all keywords which, according to the NHSvax wordlist statistics supplied by WordSmith Tools, occurred in fewer than 10% of NHSvax text files were excluded.
• The remaining keywords were grouped into semantic and functional categories and these categories were further arranged into sub-categories. Some of the semantic categories contain both lexical and grammatical items on the grounds that there is, to a certain extent, overlap between semantics and grammar (for example, word following can be used as a verb, an adjective and a preposition; conjunctions are used to indicate meaning relations; prepositions such as following, after, and so on, suggest temporal relations). A number of semantic categories were identified and named on an ad-hoc basis.50 Where it was not immediately apparent how a word should be categorized, concordance lines were examined in order to see the typical context of use. Words
50 For example, several nouns were found which denoted diseases, types of vaccines, medical conditions, and so on, and a number of nouns, verbs and adjectives were found which related vaccination and healthcare in general or to risk and causation.
144
which were found to be used in more than one way were categorized according to the more frequent use.51 The following reference works were consulted to aid the
categorization of some items: Halliday’s (1994) Introduction to Functional Grammar;
the Collins COBUILD English Dictionary (Sinclair et al., 1995); the Collins
COBUILD Grammar Patterns 1: Verbs (Francis, Hunston and Manning, 1996); the Collins COBUILD Grammar Patterns 2: Nouns and Adjectives (Francis, Hunston and Manning, 1998).52
• In order to render the task of examining the corpus even more manageable, a short-list of keywords was drawn up for close examination. The suitability of the members of each sub-category of words as candidate terms for closer examination was assessed on the basis of the likelihood that analysis of a given term might usefully help fulfil the research objectives of the thesis. Concordances were consulted to aid assessment and a list of candidate terms was drawn up.
• Concordances for each of the candidate keywords were examined in detail. Where there was a lot of data, the WordSmith Tools collocation and cluster programs were used to identify the most frequent patterns. MI3 was used as the statistical measure of significance with which to calculate significant collocates.Concordances were
examined relating to the collocational patterns and strings identified.
• Analysis continued with an examination of the most frequent words and statistically significant keywords in the JABS corpus.
51 For example, the word may was used as an epistemic modal in 86% of occurrences and as a reference to the calendar month in 14% of citations. It was therefore categorized as an epistemic modal.
52 The semantic categories of those nouns which occur in the N-that pattern were identified according to the categorization scheme described by Francis, Hunston and Manning (1998: 108-113) and discussed in Section 3.4.4 of this thesis. Some of the labelling was adapted to reflect more accurately the uses of particular items in the contexts of the JABS and NHSvax corpora. For example, the group of words labelled by Francis, Hunston and Manning (ibid.) the ‘happiness’ group (which contains words such as concerns) was labelled ‘affect’ in this thesis.
145
• The 200 most frequent words in the JABS corpus were scanned in order to identify key themes and to provide a way of gauging the reliability of the keyword list.
• To limit the number of candidate terms to a manageable size, keywords with less than 10% distribution across the JABS corpus texts were excluded, using the same
procedure as was followed during analysis of the NHSvax corpus.
• The remaining keywords were then grouped into semantic and functional categories and sub-categories. Categories and sub-categories were identified using the procedure with which the NHSvax keywords were categorized.
• The similarities and differences between the JABS and NHSvax corpus keywords were noted.
• A short-list of terms for close examination was drawn up. As was the case when drawing up the NHSvax keyword short-list, the suitability of the members of each sub-category of words as candidate terms for closer examination was assessed on the basis of the likelihood that analysis of a given term might usefully help fulfil the research objectives of the thesis. Concordances were consulted to aid assessment and a list of candidate terms was drawn up.
• Concordances of each of the candidate keywords were examined in detail. Where there was a lot of data, the WordSmith Tools collocation and cluster programs were used to identify the most frequent patterns. MI3 was used as the statistical measure of significance with which to calculate significant collocates. Concordances were examined relating to the collocational patterns and strings identified.
• Analysis of concordances and collocational patterns started with the keywords which were identified which related to the causal hypothesis, risk, health and immunity. MI3 was used to calculate the strength of collocational relationships. Typical lexical and
146
grammatical patterns were analysed, and comparisons were drawn with the patterns observed in the NHSvax corpus. Where necessary, expanded context was used to shed light on broader contextual features, such as framing.
• Analysis of the JABS data continued with an examination of words identified in the keyword analysis, which were considered useful to uncovering the ways in which intertextual elements are reframed and warrants for expertise are expressed. Lexico-grammatical patterns related to expressions of factivity, to the framing of reported speech, and to the expression of identity were examined. Z-score was used as the measure of statistical significance with which to calculate the significant collocates of
• Analysis of the JABS data continued with an examination of words identified in the keyword analysis, which were considered useful to uncovering the ways in which intertextual elements are reframed and warrants for expertise are expressed. Lexico-grammatical patterns related to expressions of factivity, to the framing of reported speech, and to the expression of identity were examined. Z-score was used as the measure of statistical significance with which to calculate the significant collocates of