Task adaptation

required more work in adaptation. In addition, the analyze tasks in general also needed adaptations to reach the same level of task requirements as the evaluate tasks. The following explains the steps taken to adapt analyze tasks to evaluate tasks and details additional strategies taken to control for the range of self-generated queries in the dissertation research.

The first step to adapt analyze tasks into evaluate tasks involved comparing the

similarities and differences in task descriptions between the two task types. In both task types, the original task descriptions required participants to identify and describe options, yet it was only in evaluate tasks participants were asked to compare the options and make decisions. In order to increase the cognitive complexity level of analyze tasks and keep the task descriptions uniform, one question was added at the end of the original task description for analyze tasks to urge decision making, such as “which one would you recommend?” or “which method do you think is the best?”

After the above mentioned adjustment, all six study tasks conformed to the same task description format: first, a scenario was presented to motivate information need; second, all tasks asked participants to find information in order to identify options and describe the differences

65

among the options; lastly, participants were asked to make a judgment or to indicate a preference based on the differences among these options.

From the “Task Selection” section it was demonstrated that during the search for the analyze task in the Entertainment domain significantly more queries were submitted than other tasks to address the information need, and many of them involved specific extreme sport names. Such differences from other tasks can potentially be resolved by narrowing down the task requirement. Instead of asking participants to evaluate “extreme sports”, the requirement was changed to evaluate “extreme sports on the water”. There were fewer extreme water sport names to submit, and even if participants chose to submit specific water extreme sport names to the system and the search results were about another water extreme sport, the results would not appear to be as non-relevant as compared to when the results were about miscellaneous types of extreme sports.

Lastly, since participants tended to use terms directly from task descriptions as search queries, in most task description the keywords were rephrased in at least two different ways so that participants would have more word choices. For example, in the extreme sports task, “extreme sports on the water” and “action water sports” were used interchangeably in the task description. The goal of providing more word choices was to decrease the likelihood of a participant submitting idiosyncratic query terms, which could result in a mismatch between queries and preselected search results and suspicion on the part of the participant.

The resulting six study tasks included two health tasks, two science and technology tasks, and two entertainment tasks (Table 10). Tasks #1, #2, and #3 were originally analyze tasks and tasks #4, #5, and #6 were evaluate tasks to begin with. Tasks #1, #2, and #3 were used to

66

the effect of information scent pattern; this decision was made so that participants would experience a wider range of SERP characteristics during the experiment as opposed to only ISL or ISP treatments. The order of the tasks was rotated using a Latin Square design to balance order effects, as shown in Table 11.

Domain _{Information Scent Level Tasks Information Scent Pattern Tasks}

Task

ID Task Description Task ID Task Description Health 1 Having heard some of the recent reports

on risks of natural tanning, it seems like a better idea to sport an artificial tan this summer. What are some of the different types of artificial tanning methods? How risky are they? Which one would you recommend?

4 One of your siblings got a spur of the moment tattoo, and now regrets it. What are the current available methods for tattoo removal, and how effective are they? Which method do you think is best?

Science &

Technology 2 You recently became involved with a conservation group that picks-up trash from local waterways. One of the group members told you that your work was important because it helps keep pollution out of the ocean. What are some of the different types of marine pollutants? What environmental risks are associated with each pollutant? Which one seems to be the most harmful to the environment?

5 Many people believe that social media has many benefits to our life, but your sibling argues that people are losing their ability to communicate face-to-face and that social media makes people lonelier and more isolated. What are the positive and negative consequences of using social media? Do you believe that social media is detrimental to the development of personal relationships?

Entertainment 3 Your sister is turning 25 next month and wants to do something exciting for her birthday. She is considering some type of extreme sport on the water. What are some different types of action water sports in which amateurs can participate? What are the risks involved with each one? Which one would you recommend?

6 For his 16th birthday, your nephew has asked you for a video game that is rated "M" for mature audiences because it contains intense violence. You are unsure about whether to purchase this game because you recently overheard two people discussing the effects of violent video games on teenagers. What are some positive and negative effects of playing violent video games on teenagers? Would you buy a video game rated "M" for him?

Table 10 Study Tasks

67 Table 11

Latin Square Design of Task Order