Sampling and Screening

The present study sampled from the threads initiated between the first week of August 2010 and the third week of October 2010 (1,390 discussion threads overall). At the time when the data were collected, all of these threads were inactive for over at least one month.

The population was narrowed by removing from consideration the threads falling outside a specific range for the following parameters: the number of messages and the number of contributors. This was done to focus on threads meeting the following conditions: (a) the threads should be large enough to allow knowledge construction and (b) the threads should be small enough to allow coders to deal with each thread as a single unit. In very small threads, there is little, if any interaction, and, therefore, the concept of knowledge construction is not applicable. On the other hand, in very large threads, the number of messages is too large for a coder to remember them at a time; moreover, in a very large thread the nature of discussion is likely to shift over time (for example, an on-topic thread may eventually turn into an off-topic thread, or even into a flame war). Thus, it may be difficult to code very large threads as single units, and the validity of the resulting codes may be in doubt.

Removing the threads that are too small or too large is similar to removing outliers. Outliers are observations that are so different from the rest that they are likely to be aberrations not representing the population (Hair, Black, Babin, & Anderson, 2009). By retaining only typical threads, this study went beyond the common practice of removing outliers, as usually only very small numbers of cases are removed to ensure that the remaining cases still represented the population (Hair et al., 2009). Because a relatively large number of threads were removed in the present study, the results did not

necessarily generalize to all threads, but rather to the “typical” threads as defined in this study. I do not believe that this restriction undermines the ability of this study to

address the stated research questions. Under any circumstances, it is not possible to sample from the population of all discussion threads in existence. It is sufficient to test the hypotheses for a class of threads that are representative (in an analytical rather than a statistical sense) of a typical thread in a typical discussion. I believe that in terms of achieving this, not including all threads in the discussion, but instead only threads that are not too large or too small, does not undermine the internal or the external validity of this study (validity is discussed in sections 4.11.5 and 4.11.6).

Figure 10. Definition of “typical” threads (given by the shaded area).

The parameters used to establish cut-offs and the exact cut-off values were based on my (subjective) observations and correspond to what I believed to be a "typical" thread size. Threads with the number of messages from four to seven and the number of contributors from three to five were retained. This roughly corresponded to threads falling within the second and the third quartile of the distribution of threads by the

number of messages and by the number of contributors (the shaded area in Figure 10). As the result, 501 threads were judged as typical and retained for further consideration. The following three paragraphs consider the rationale and the consequences of using these cut-off points in more detail.

Number of messages. Of the threads that were removed, 361 threads contained three or fewer messages (arguably, too short for an argument to develop—indeed, a thread consisting only of one or two messages can hardly be called a thread), and 418 threads contained eight or more messages. As seen in the histogram in Figure 11, there was a sharp drop between the numbers of threads of length seven and of length eight, suggesting seven as a cut-off value.

Figure 11. Distribution of threads by the number of messages in a thread. Bins

corresponding to typical threads are shown in grey.

Number of contributors. Of the threads that were removed, 345 threads involved just one or two contributors (arguably, not enough to present multiple points of view), and 298 threads contained six or more contributors. As seen in the histogram in Figure 12, there was a sharp drop between the numbers of threads that involved five contributors and that involved six contributors, suggesting five as a cut-off value.

0 50 100 150 200 250 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627 Numb er o f thre ads Number of messages

An alternative to setting cut-off values based on my subjective observations would be to conduct a separate content analysis study (involving coders) to distinguish threads that are too small for the argument to develop and long threads with the nature of the discussion shifting over time. This would necessitate the development of a coding scheme designed to distinguish such threads, and the coding scheme would have to be developed from scratch—to the best of my knowledge, a coding scheme of this type has never been developed. Moreover, applying such a coding scheme would considerably increase the costs of conducting the present study, because threads of all sizes would need to be coded. In terms of time and funding available for the present study, it was not feasible to use an extra round of content analysis to distinguish typical threads.

Figure 12. Distribution of threads by the number of contributors. Bins corresponding to

typical threads are shown in grey.

After distinguishing typical threads, out of the pool of the 501 typical threads, 120 threads were drawn randomly for analysis, following the standard sampling procedure used in content analysis. Because the threads were chosen at random, they represented the population of typical threads—drawing threads at random did not introduce bias. The number of threads selected at random for analysis (120 threads out of 501 typical threads available in the sample) was based on two considerations. First, the budget of the study was limited, and hiring coders to code 120 threads was within budget. Second, the statistical techniques used to test the structural model in Figure 8 required approximately 100 cases (see section 4.13 for a more detailed discussion of this aspect).

0 50 100 150 200 250 300 350 1 2 3 4 5 6 7 8 9 10 11 12 13 Numb er o f thre ads Number of contributors

The reduction in the number of threads reduced the statistical power of the study; however, because statistically significant relationships were detected, and various reliability and validity criteria were met (see section 5.6), this number was sufficient.

The screening of threads is summarized in Table 4-2 in terms of the number of threads and the number of messages retained after each phase.

Table 4-2 Screening of Threads

Screening phase Number of threads Number of messages Threads initiated between

August and October 2010 1,390

8,816 After extracting typical

threads 501 2,732

After random selection 120 708

After data extraction 108 633

The 120 discussion threads obtained in random selection were inspected by two independent coders for the presence of specific recommendations addressing the issue of weight control. The details of the inspection procedure are given in section 4.8. Recommendations for weight control were discovered in 108 threads. These threads were treated as task related and used for further data analysis.

Figure 13. Distribution of task related threads by the number of messages in a thread.

Figure 14. Distribution of task related threads by the number of contributors.

Histograms for the distributions of threads by the number of messages and by the number of contributors for the 108 task related threads that were coded are given in Figure 13 and Figure 14.

8 29 32 39 0 5 10 15 20 25 30 35 40 45 4 5 6 7 Numb er o f thre ads Number of messages 15 41 52 0 10 20 30 40 50 60 3 4 5 Numb er o f thre ads Number of contributors