Trend and Inconsistency Effects in stage 5

Now we will turn to trend and inconsistency until stage 5 of the experiment or study and their related mediation analyses. When running the mediation analyses in Process for trend and inconsistency to overcome the unstable confidence interval problem in some of the analysis, 10,000 bootstrapping samples were used in the two trials or analyses, so once with 95% CI and once with 90%. First, looking at the tests of paths a (from trend and inconsistency to mediators measured in stage 5; see Baron & Kenny, 1986; Edwards & Lambert, 2007) as presented in tables G-9 and G-17, in the case of positive and negative trend as predictors (with neutral as the reference category), the only path that is significant (in terms of both p-values and CIs) is that of positive trend to PA Self with a coefficient of 0.428 at a p-value<0.05 and a R square of 5.2% (the coefficient here needs to be interpreted relative to the neutral category and throughout the

presentation of results in this section since in these mediation analyses, the Xs or independent variables are tested as a multi-categorical variables).

However, indirect effects may sometimes be present regardless of insignificant paths a and these indirect effects will be discussed here but not emphasized since they point to possible indirect effects when other control variables or moderators are included. Also, sometimes paths b would be insignificant but indirect effects are present and this may be due to including all

mediators together in the same model. The omnibus test results are not discussed in this study but a significant effect means there is a possibility that the mediated effect changes when the predictor category used as reference (in this case neutral trend and in the case of initial valence, neutral valence while in the case of inconsistency it is consistency) changes.

In the case of inconsistency, even though only attribution and negative affect towards the feedback system are hypothesized to be mediators in the study, positive affect towards the system was also included in the mediation analyses. However, in terms of a significant paths “a” from feedback inconsistency to mediators, only the path from feedback inconsistency to internal attribution is significant with a coefficient of 0.32 (p-value<0.05; R square only 1.4%), i.e. internal attribution average in the data increases by 0.32 over average of neutral category or those respondents given consistent feedback across the five stages (the average in this case is the constant of 2.87).

Now we turn to test of paths b and the total, direct and indirect effects from mediators to outcomes when the predictor is included in the model (Baron & Kenny, 1986), discussing first the results for trend and then for inconsistency for each outcome. When the outcome is task satisfaction and the predictor is trend, two of the mediators show significant effects PA self (coefficient=0.5158, p-value<0.01) and Self-efficacy (coefficient=0.2347, p-value<0.01) with the model including both trend and mediators as predictors explaining 43.95% of the variance in task satisfaction around its mean (F (5, 279) = 36.03). The total effect of trend on task satisfaction is represented by a significant relative effect of positive trend (coefficient=0.47, p<0.01) with a R square of 4.9% (F (2,282) =7.02) while there is no significant direct effect when the mediators are included (so the relative path c’ is not different from zero) and the only relative indirect effect present in the data is that of positive trend through PA self (relative effect=0.2207, SE=0.101, 95%=0.0413 to 0.4388). This effect is also the only indirect effect significant when 90% CIs are used but the effect of negative trend through NA Self also becomes significant only

when control variables (that were allowed to co-vary with both the mediator and outcome) are entered, mainly here the big five personality variables (see table G-9).

In the case of inconsistency as predictor and task satisfaction as outcome (see table G- 24), testing paths b shows that three of the mediators have significant effects, all p-values<0.05: PA Feedback System coefficient=0.6043, NA Feedback System coefficient=-0.1676, and Internal Attribution coefficient=0.2247 with a R square for the model including inconsistency equal to 50.48%, F(5, 279)=78.72. As for the total effect and direct effect, both are not

significant but the relative indirect effect of feedback inconsistency through Internal Attribution is the only one significant (effect =0.0769, SE=0.0429, 95%CI=0.0098 to 0.1814); same result is repeated with a 90%CI.

As for the outcome of learning intentions, when trend is the predictor and included in the model with the mediators, two of the mediators are significant in their effects: PA self with a coefficient of 0.44 (p-value<0.10) and self-efficacy with a coefficient of 0.14 (p-value<0.01) with a R square of the model of 16.96 (F (5, 279) =9.54). Trend has a total effect (path c) on learning intentions, specifically, positive trend which has a significant relative effect of 0.37 (p<0.10) and a significant R square of 1.8% (F (2, 282) =2.81) but no relative direct effect (so relative path c’ is zero) which points to mediation; it has an indirect (mediated) relative effect (paths a*b) through PA Self of 0.0588(SE=0.0442, 95%CI=0.0021 to 0.1857), however. The relative indirect effect of negative trend through PA Self (effect=-0.04, SE=0.03) becomes significant with a 90%CI (see table G-10).

In the case of inconsistency as the predictor (see table G-23), in testing paths b from mediator to outcome, two of the mediators only are significant and inconsistency is insignificant: PA Feedback system with a coefficient of 0.21, p-value<0.05) and Internal Attribution with a coefficient or effect of 0.44 (p-value<0.01); R square for the model is 26.55% (F (5, 279) =19.66). There are no significant total or relative direct effects (so paths c and c’ are most probably not different from zero in the population) but feedback inconsistency has an indirect effect (relative to feedback consistency) through Internal Attribution of 0.15 (SE=0.07, 95% CI= 0.02 to 0.32).

When motivation is the focal outcome and trend is the predictor, the three mediators are significant when trend is also included in the model but not trend itself: PA Self

coefficient= 0.4162, p-value<0.05) and a model R-square of 19.79 (F (5, 279) =13.81). There are no significant relative total or direct effects however but positive trend exerts a relative indirect effect (positive) through PA self of 0.0907(SE=0.0543, 95%CI=0.0144 to 0.2315). The relative indirect effect of negative trend (negative) versus PA self (effect=-0.0565; SE=0.0413) and the positive relative effect of negative trend versus NA Self (effect=0.0306, SE=0.0268) become significant when 90%CIs are used but when control variables enter the picture the most stable and consistent effect is that of positive trend through PA Self (see table G-11).

In the case of inconsistency as predictor, two mediators have significant paths b: PA Feedback System coefficient=0.2251 and Internal Attribution coefficient=0.4785, both p-

value<0.01; the model R square is 33.59% (F (5, 279) = 31.52). There are no significant relative total or direct effects of inconsistency on motivation (see table G-22) but feedback inconsistency has a positive relative indirect effect through Internal Attribution of 0.1636 (SE=0.0793, 95% CI=0.0207 to 0.336).

As for the outcome of improvement intentions, in the case of trend as a predictor, when testing the significance of b paths only self-efficacy has a significant effect of 0.4693 (p- value<0.01) and so does positive trend (coefficient= 0.3431 or it increases average of

improvement intentions by 0.34 over when trend is neutral with a p-value<0.05) with a R square of 17.84% (F (5,279) =10.403). As for relative total effects, positive trend has an effect of 0.4521 (p-value<0.05) with a significant R square of 2.2% (F (2,282) =3.28) and a relative direct effect of 0.3431 (p-value<0.05). However, there are no relative indirect effects and so it seems that the relative direct effect of positive trend is an effect of controlling for the mediators especially self- efficacy and not due to their mediation. However, when controlling for some of the control variables like the big five and using 90%CIs, the positive relative indirect effect of positive trend via PA Self (effect=0.0403, SE=0.0386) becomes significant (see table G-12).

When inconsistency is the predictor (see table G-21), only internal attribution has a significant paths b with a coefficient of 0.5601(p-value<0.01) and the model has a significant R square of 0.3405 (F (5, 279) =26.06). There no significant total or direct effects but feedback inconsistency has a positive relative indirect effect through Internal Attribution of

0.1915(SE=0.092, 95%=0.0239 to 0.3872; same result with 90%CI).

With respect to task performance satisfaction as the outcome of focus, when both trend dummy variables and the mediators of PA Self, NA Self and self-efficacy are included together

in the model, all three mediators and both dummy variables are significant: PA Self

coefficient=0.5352(p-value<0.01), NA Self coefficient=-0.3232(p-value<0.05), Self-efficacy coefficient=0.2642(p-value<0.01). negative trend coefficient=-0.5537 (p-value<0.05) and

positive trend coefficient=0.6399 (p-value<0.01). This model explains 34.17% or the variance of task performance satisfaction around its mean (i.e. its error variance around the mean; F (5, 278) =25.47).

The total effects that positive and negative trend have (relative to neutral trend) are both significant and they are as follows: for positive trend the effect= 0.91 (SE=0.2485, p-value<0.01) and negative trend =-0.797 (p-value<0.01); R square explained when only trend is included in the model equals14.55% (F (2,281) =23.42). A for the relative direct effects they are as follows: negative trend=-0.5537 (p-value<0.05) and positive trend (effect=0.6399, p-value<0.01). The only significant relative indirect effect at the 0.05 significance level is 0.229 (SE=0.1047, 95% CI=0.0452 to 0.4598) but at the 0.10 level using 90% CIs, the indirect through PA self of negative trend (effect=-0.1496; SE=0.0948) also becomes significant; however, the effect of negative trend through PA Self is less stable or consistent when controlling for other variables (see table G-13).

When inconsistency is the predictor and task performance satisfaction is the outcome, running a model with all four mediators and the dummy variable that represents feedback

inconsistency leads to the following significant results: PA Feedback System coefficient=0.6578 (p-value<0.01), NA Feedback System coefficient=-0.4798 (p-value<0.01) and Internal

Attribution coefficient=0.1901 (p-value< 0.05) with R square of 24.13% (F (5, 278) = 15.81). However, no total or indirect effects of feedback inconsistency compared to feedback

consistency are significant (see table G-20). However, feedback inconsistency has a relative indirect effect through Internal Attribution of 0.0646 (SE=0.0469, 95% CI=0.0014 to 0.1961) but this effect did not withstand controlling for initial valence and trend so it is weak and can be explained away by the effect of other feedback variables instead.

In the case of desired grade or performance goal, the only significant path b is that for self-efficacy (coefficient=5.3381, p<0.01) when trend is the predictor while there are no significant relative total effects or direct effects for either positive or negative trend but only a relative indirect effect of positive trend through PA self with 90%CI (effect=0.7971, SE=0.66).

However, this effect is not very stable because it is no longer present when controlling for general affect and implicit theory (see table G-14).

As for expected grade and standard grade for both, only the mediators are significant when testing paths b and all three mediators are significant. For example, in the case of expected grade PA Self coefficient= 5.8298 (p<0.01), NA Self coefficient = -3.2706 (p<0.05) and Self Efficacy coefficient= 6.7383 (p<0.01) with a R square of 24.62 (F (5, 277) =19.73. For expected grade there are also no total or direct relative effects but there is one significant indirect effect of positive trend through PA Self of 2.4165(SE=1.1948; 95% CI=0.4903 to 5.3029) and the indirect effect of negative trend via PA Self (effect =-1.49, SE=1.04) become significant at the 0.10 significance level or with 90% CI but only the positive trend effect is resilient when controlling for other variables (see table G-15). Similar results to expected grade occur in the case of standard grade when trend is the predictor (see table G-16)

When inconsistency is the predictor, in the case of all three outcomes, desired grade, expected grade and standard grade, the three mediators: PA feedback system and NA Feedback System have significant b paths. For instance, for standard grade, the b path for PA feedback system is 6.90 (p<0.05) while it is -3.80 (p<0.01) for NA feedback system with R square of 14.15% (F (5,277) = 11.26) while for expected grade the results are 7.08 for PA feedback system and -4.96 for NA feedback system, both p-values<0.01 and for desired grade the b paths are 4.02 and -4.05 respectively, again both p-values<0.01.

There are no significant relative total effects or direct effects for both desired grade and expected grade while for standard grade feedback inconsistency has a total relative effect of - 5.8973 (p<0.05) with R square of 1.3% (F(1,281)=3.93) and a relative direct effect of.-6.06 (p- value<0.05) However, only NA Feedback system seems to channel any kind of indirect effect for all three outcomes but because there is no significant paths a for this mediator and it is only a mediator when control variables are included or the effect disappears when control variables are included and because moderated mediation analyses showed mostly non-significant results, it will be concluded here that there are no mediated effects or significant mediators for those three outcomes when inconsistency is the predictor (see tables G-17-19 for details).

In terms of whether mediation was relatively partial or full, when trend is the predictor, evidence in terms of differences between total, relative direct and relative indirect effects exists (and showed in the tables) that mediation in the case of task performance satisfaction and desired

grade is partial while there is full mediation (with significant total and relative indirect but not direct effects) in the case of task satisfaction and learning intentions. In the case of expected grade, standard grade, motivation and improvement intentions, there are no significant relative total effects or relative direct effects but there are significant mediation or indirect effects thus pointing to one form of full mediation. This latter form of full mediation is also present when feedback inconsistency is the predictor in the case of all outcomes with the exception of expected grade and standard grade where there is no evidence for any form of mediation (when no control variables are used).

Thus, in terms of hypotheses 5 for initial valence (that perceived competence or self- efficacy and affect towards task are mediators), 11 for trend (that performance control or self- efficacy here and affect towards self are mediators) and 16 for inconsistency (that negative affect towards system and external attribution are mediators), there is partial support for them because none of the mediators had universal mediation/ indirect effect on all predictor-outcome

relationships. However, in terms of the direction of the effects predicted, there is evidence that supports it. For instance, for most of the outcomes and regardless of significance, negative trend has a negative indirect effect through positive mediators (like PA self, PA Task, PA Feedback system and self-efficacy) but a positive indirect effect through negative valence mediators like NA Self, NA Task and NA Feedback system. There are exceptions of course like learning intentions in stage 5 where both positive and negative trend have positive effects (but not significant) through NA self.

As an additional analysis here, injustice perceptions (in stage 5) were tested as mediators between inconsistency and outcomes but found insignificant for all of the outcomes. Also, the effect of inconsistency (compared to inconsistency) on outcomes through attribution is through internal as opposed to external attribution and the relative indirect effect is positive rather than negative in valence while the effect through NA towards feedback system is always negative as predicted.

Moreover, it is important to note that when control variables and/or trend and initial valence are controlled for, some of the mediation effects did not prove to be resilient (i.e. no longer existed with the control variables). For instance, NA Self for motivation and task

satisfaction in stage 5, PA Self for learning intentions, improvement intentions and desired goals in stage 5, NA Feedback system for desired goals in stage 5 and Internal Attribution for task

performance satisfaction in stage 5. These results can point to potential moderators (that need to be explored in future research) in the relationships between predictor, mediator and outcome. Now we turn to the moderation effects actually hypothesized and tested here.