Support of High-Level Hypotheses

The purpose of this study was formulated in section 1.6; the specific research questions were formulated in the same section as two high-level hypotheses. This section

discusses the implications for these hypotheses of the results of the model testing conducted in Chapter 5.

5.9.2.1 H1:KnowledgeConstructionResultsinHigherPerceivedInformation Quality

The hypothesis H1 that knowledge construction results in higher perceived information quality was tested in a number of models, with constructs at different levels.

2x2 model. In the model in section 5.7, knowledge construction was conceptualized as a two-dimensional construct with the dimensions of explicitation and evaluation, with perceived information quality also conceptualized as a two dimensional construct, with the dimensions of relevance and usefulness. Explicitation affected both relevance and usefulness with medium effect sizes, but evaluation had no effect on them. The effect of explicitation on usefulness was slightly higher than the effect on relevance. The model had a substantial discriminant validity problem with relevance and usefulness highly correlated (as discussed in sections 5.6 and 5.9.1).

1x1 model. In the model in section 5.8.1, both knowledge construction and perceived information quality were conceptualized as one-dimensional constructs. Knowledge construction affected perceived information quality with a medium effect size. However, the explanatory power of the model (in terms of the amount of variance explained) was somewhat lower than the explanatory power of the model in section 5.7 (labelled as “2x2 model” in this section).

2x1 model. In the model in section 5.8.2, knowledge construction was conceptualized as a two-dimensional construct with the dimensions of explicitation and evaluation, with perceived information quality treated as a one-dimensional construct. Explicitation affected perceived information quality with a medium effect size, but evaluation had no effect on perceived information quality. The explanatory power of this model was similar to the model in section 5.7 (labelled as 2x2 model in this section) and higher than for the 1x1 model. On the other hand, the 2x1 model did not have the discriminant validity problems of the 2x2 model. Thus, of the models considered, the 2x1 model fitted the data the best.

Overall, all of the models analysed in Chapter 5 (including the three models discussed in this section) supported the H1 hypothesis. The models treating knowledge

the H1 hypothesis. The knowledge construction activities that contributed to perceived information quality were at a low level of knowledge construction (the activities represented by the explicitation construct). The results were consistent with a view that higher level knowledge construction activities (the activities represented by the

evaluation construct) had no effect.

Thus, refinement (improving or perfecting by pruning or polishing), elaboration (developing in intricate and painstaking detail), clarification (interpretations that remove obstacles to understanding), confirmation (additional proofs that some facts or hypotheses are correct), and negation (statements that are a refusal or denial of some other statement) contributed to higher perceived information quality of the transcripts of health support group online discussions. Conversely, critical discussion (discussion characterized by careful evaluation and judgment), argumentation (a discussion in which reasons are advanced for and against some proposition or proposal), reasoning (presentation of reasons and arguments), and justification (defending or explaining or making excuses for by reasoning) did not. In other words, clarifying the facts

contributed to perceived information quality, but reasoning about the facts did not. This is consistent with the findings by Kanuka and Anderson (2007) (discussed in section 2.6.2.1), who found little evidence of high levels of knowledge construction in online discussions by professionals and interpreted the findings by suggesting that high-level knowledge construction activities are not as relevant in online discussions in non- educational contexts as they are in educational contexts.

Clarification of facts contributes to perceived information quality in terms of making the facts more accessible, but reasoning and argumentation from a variety of

perspectives do not necessarily clarify the facts and might even obscure them. Lasker et al. (2005) found that the main rationale for people to participate in health support group online discussions was gaining access to information (discussed in section 2.3). High levels of knowledge construction do not necessarily make it easier to access

information, but clarification and refinement (lower level knowledge construction activities) do make it easier to access information.

5.9.2.2 H2:KnowledgeConstructionActivitiesinHealthSupportGroupOnline DiscussionsResultinGreaterPrevalenceofEvidence‐Based

Knowledge

Prevalence of evidence-based knowledge was conceptualized as information integrity. The hypothesis H2 that information integrity is affected by knowledge construction was tested in a number of models (see sections 5.7 and 5.8). The strongest test was reported in section 5.8.5, where the hypothesis that evaluation and explicitation (the dimensions of knowledge construction) affect information integrity was tested at the level of a recommendation. In all tests, the hypothesized effects on information quality were not found to be statistically significant. Thus, the results were consistent with the view that knowledge construction does not affect information integrity (and thus, knowledge construction does not result in greater prevalence of evidence-based knowledge).

A possible reason for this finding was that the operationalization of information

integrity was not valid enough in view of low inter-rater reliability (see section 5.3 for a detailed discussion of this aspect).

Taking a view that the effects on information integrity were not discovered because there were no effects (rather than because of problems with the measure, or because the statistical power of the study was not sufficient), this suggests that knowledge

construction activities in health support group online discussions are not robust enough to distinguish high-quality recommendations from recommendations that are doubtful or even risky.

5.10Summary

This chapter presented the results of data analysis.

Descriptive statistics (minimum, maximum, mean, and standard deviation for each item, separately by coder) and inter-rater reliability (in terms of Pearson’s correlation and Spearman’s rho) were presented. Inter-rater reliability was the highest for the dimensions of knowledge construction, followed by the relevance and usefulness dimensions of perceived information quality. Inter-rater reliability for the

correlation between ratings by different coders was not statistically significant), that the construct was excluded from further analysis. Inter-rater reliability for information integrity was also low, but was judged to be high enough to retain it in further analysis. For the overwhelming majority of the indicators, the inter-rater reliability was below the threshold of 0.7; nonetheless, this was consistent with similar prior studies.

Normality tests suggested that the data were close enough to normal (according to kurtosis and skewness values); however, the Shapiro-Wilk’s multivariate normality test suggested that the data were not multivariate normal.

Preliminary testing of convergent and discriminant validity using exploratory factor analysis suggested possible discriminant validity issues between the dimensions of knowledge construction (explicitation and evaluation) and between the dimensions of perceived information quality (relevance and usefulness).

The results of measurement model analysis were overall consistent with EFA results; nonetheless, the discriminant validity issues were minor enough to allow the analysis of the structural model.

Structural model analysis suggested that explicitation affected both relevance and usefulness; the rest of the hypotheses were not confirmed (no effects were discovered for evaluation, and no antecedents—for information integrity).

Post hoc analyses involving variations of the research model suggested that the best model fit (in terms of maintaining the explanatory power while avoiding discriminant validity issues) is achieved in a model with explicitation and evaluation treated as separate constructs, but with relevance and usefulness combined in a one-dimensional perceived information quality construct.

In a separate post hoc analysis, coders’ conceptions of the target constructs were modelled explicitly, resulting in a hierarchical model. Coders’ conceptions were found to be close enough to the underlying constructs to result in a good fit of the model.

The results of a post hoc analysis involving testing variations of the model at the level of a recommendation were consistent with the initial analysis in not discovering any

relationships. Moreover, a one-way ANOVA analysis suggested that there was no significant difference between the levels of information integrity in different threads.

Data analysis results across all of the analyses conducted were summarized and interpreted in section 5.9. The results suggested that knowledge construction in health support group online discussions improves perceived information quality, but does not affect information integrity. Thus of the two high-level hypotheses (H1 and H2) in the high-level research model in Figure 2, only the hypothesis H1 was confirmed.