Reading hypothesis test results

and perceptions. Interpreting interactions of multiple categorical independent variables from a regression table is difficult, especially with nonlinear dependent variables (e.g., counts of queries issued). As such, the interpretation of model results focuses on the predicted values of the dependent variable (e.g., the time pressure predicted by the model for the no time limit condition and time limit condition). As the fully specified models include multiple interactions of time limit condition and other categorical independent variables, the interpretation of the effects of time limit condition focuses on themarginal effectsas recommended by Mize (2019). Marginal effects indicate the change in the predicted value of the dependent variable as a result of a change in an independent variable holding other independent variables at specified values. Predicted values and marginal effects are presented graphically and in the text. For a nominal independent variable like time limit condition, the marginal effect for an observation represents the difference in the predicted time pressure if the observation was in the time limit condition vs. if it was not in the time limit condition.

To assess the impact of time limit condition, two types of marginal effects of time limit condition are of primary interest: the average marginal effect (AME) of time limit condition, and the marginal effect of time limit condition at representative values (MER) of specific independent variables. First, to understand

whether there is an overall effect of time limit condition on the dependent variable, theaverage marginal effect(AME) of time limit condition is analyzed. An average marginal effect of the time limit condition is the average of the marginal effects of time limit condition for all observations. The average marginal effect of time limit condition was calculated for dependent variables derived from search and decision behavior measures as well as post-task questionnaire measures.

Second, to understand whether there is an effect of the time limit condition at important levels of select categorical or continuous independent variables, themarginal effect at representative values(MER) of the categorical or continuous variables were analyzed. Marginal effects of time limit condition were calculated for dependent variables derived from post-task questionnaire measures at representative values of key independent variables derived from search and decision behavior (e.g., task times, query counts, count of nonSERPs viewed from SERPs) and recommendation quality (e.g., recommendation specificity, recommendation accuracy). In some cases, the marginal effects for other search and decision behaviors were examined if relevant (e.g., clock views for time monitoring). Representative values for categorical independent variables such as recommendation specificity and recommendation accuracy are the categories, and the representative values for the search and decision variables were selected to represent meaningful values (i.e., if didn’t search, the mean or median, and a value on the upper end of the range typically the 95th percentile).

The findings are interpreted in the results section with graphical representations of predicted values and marginal effects; tables with the models and predicted values with marginal effects are included in Appendix S. For all hypothesis tests, two tables are included in Appendix S for each dependent variable. The first table is the regression model table which includes the fixed and random effects and model statistics. Dependent variables derived from the post-task questionnaire have the full model interpreted in the results section as well as a reduced model which does not include an interaction of time limit and search behaviors.19The fixed effects included in the models are at the top of the tables: experimental variables (time limit, topic, time limit and topic interaction, and topic order), individual characteristics (student status, age, search self-efficacy), and pre-task perceptions (interest, prior knowledge, ability to make recommendation without searching, expected difficulty, expected difficulty stopping, and task self-efficacy). For dependent variables derived from post-task questionnaires, the models also include covariates for search and decision behaviors: clock view, decision

19_{By not including the interaction terms, a reduced model provides estimates of the average effect of a 1-unit change in the search}

time, queries issued, max view rank, hover count, SERP views, nonSERPs from the SERP, and nonSERPs viewed from other SERPs. For Hypothesis 7 examining the effects of individual differences on perceived time pressure, additional variables (tendency to procrastinate, sensitivity to pressure, and interactions) are added to the model. Table S.1 in Appendix S summarizes the independent variables including transformations (e.g., mean-centering age, centering interest). The second table in the Appendix for each hypothesis contains the predicted values for the dependent variable by time limit condition and (average) marginal effects of time limit condition overall, at each level of recommendation specificity, and at each level of recommendation accuracy. The bottom half of the second table contains the predicted values for the dependent variable for each topic and task order as well as the results of post-hoc pairwise comparisons.

Exact, uncorrected p-values are reported in the text, in tables, and in graphs.20 Model coefficients for individual characteristics and pre-task perceptions were considered statistically significant ifp<.05. For dependent variables derived from post-task questionnaire measures, p≤.003 is used as the threshold for statistical significance (α=.003) for the average marginal effect of time limit condition and marginal effects at representative values of time limit condition given multiple marginal effects calculations run after model estimation.21 For dependent variables derived from search and decision measures,p≤.003 is also used as the threshold for statistical significance (α=.003) for the average marginal effect of time limit condition for consistency across dependent variables even though only the average marginal effect is calculated and no additional marginal effects at representative values are calculated. For pairwise comparisons for topic and order effects, to be consistent with Study 1, topic or order effects were considered significant ifp<.0083.

6.4.2 H1: More perceived time pressure, time inadequacy, and time monitoring. The first hypoth-