This study was done in combination with Study-1. The participants were first inter- viewed and then given a demonstrative walkthrough. Thus, the set of participants for this study included 11 subjects from age 25 to 58 and with diverse backgrounds. These subjects were graduate students and faculty members in the fields of information sci- ence, library science, social science, and journalism. In general, these participants were mature researchers and/or academicians in their respective fields. A majority of these participants also taught various courses relating to information and library science, and media and journalism. They, therefore, could talk not only from their personal experiences, but also from those of their students.
I walked the participants through Coagmento’s workflow in the following sequence.
1. First, the participants were given a brief introduction of how Coagmento is de- signed for serving two or more people working together in an information seeking task.
2. They were then shown Coagmento’s main screen (Figure 3.22). Various compo- nents of the display were explained.
by any member of the team, as well as any other attributes about a document in the result set, such as that it is saved or under discussion.
4. A document in the result set was clicked by me and, when the actual document was displayed in the browser, the participant was shown how Coagmento’s toolbar (Figure 3.23) can be used for saving that document, putting it in the discussion box, or collecting a snippet from it.
5. Possible uses and scenarios of the boxes on the right side of Coagmento screen were explained.
After giving this demonstrative walkthrough, I asked the participants questions regarding the usability and functionality of the system. The questions that guided the interviews follow.
Q1. Which features of this interface have you already used in some existing system? Q2. Are there features that you haven’t seen before? If so, which do you expect would be useful for doing collaborative searches? Describe one use of each of the features in collaborative searching. How do these features map to what you have previously mentioned about your group project experience and/or wish list.
Q3. If “no” to new features, ask participants to map the familiar features to what they previously mentioned about group project experience and/or wish list.
Q4. Now that you have seen the functionalities of this interface, what else would you like to have for carrying out effective collaborative searching?
Q5. Would you suggest changing something on this interface to
1. make it more usable
If answered “yes” to any of the questions, ask how.
One of the first questions asked was about identifying the components of the in- terface that they had seen or not seen before. Not very surprisingly, no one had seen all of the components in the same place. Most people had not seen color coding of the documents (based on views) or the query history. Most of the participants were also not aware of a system where they could collect the snippets, save the documents that are useful or keep them for later discussion. These are also the features that these participants found most appealing. Almost everyone appreciated having all the components and saw the value of them even in the situations where one was not doing collaboration. As one participant said, “This is so cool; I could use such a tool even if I’m not working with others.” Such remarks provided the support for appropriation for Coagmento, i.e., the unintended situations in which it can be used. It was noted earlier in Section 3.2.4 that it is important to design a CIS system keeping in mind that often people working in collaborative projects may desire to work individually. Demonstrative walkthroughs of Coagmento confirmed this aspect of the system.
On the flip side, some participants felt the need to extend certain features. For instance, about half the participants reported that they would like to see more meta- data about the queries and the saved/discussion documents, especially time-stamps. Similarly, one participant asked for a way to track the progress of the collaborative group by means of a timeline of the events (query issued, document viewed or saved, etc.). Three of the participants suggested having a workspace such as Google Docs to consolidate viewed, discussed, and collected information.
Most users felt comfortable having all the actions and results stored on the system server, but some wanted a way to export all that information so that they could use it in other applications. A couple of participants desired to have more configurable screen
layouts and customizable components.
The findings from this study can be summarized as follows.
• Ability to effectively collect and share information is highly desired from a CIS system. People typically do this using traditional tried-and-tested tools such as email and IM, without realizing that there could be more effective ways of doing it in collaborative projects. Coagmento allows the users to easily collect and share snippets of information, which most participants reported to be a unique feature. However, one may not want to use a CIS system such as Coagmento just because it has a couple of unique features. As noted before, we need to understand the larger context and environment in which people collaborate for information seeking projects.
• It was a pleasant discovery that the participants saw Coagmento’s use even in non- collaborative projects. Such appropriation is important to have for a successful groupware system, as Grudin (1994) noted.
• The participants asked to see more information on the interface, including time stamps. This was important feedback, yet at the same time created a design challenge. How much information is enough and how much is too much? Should we show day, date, as well as time in hour, minutes, and seconds? We need to develop systems that allow the users to choose this. Having a configurable interface can let the users work through its features without getting overwhelmed.
Based on the lessons learned from the demonstrative walkthroughs, Coagmento was enhanced primarily to support a real life testing of the system. The TREC ciQA (Complex Interactive QA) 2007 data-set was used as the collection for this version of Coagmento. This data-set had nearly one million documents from various news
sources. The collection was indexed using the Lemur Toolkit,7 and a modified Indri8 search service served the requests in the background.