Feedback on preliminary interface

The participatory design sessions for Fall 2009 were different in that, by this time Coagmento was developed and ready to be tested. This led to letting the students of Dr. Marchionini’s Fall 2009 HCI seminar class try out an earlier version of Coagmento that supported multi-session projects, but not collaborations. The intention was to get feedback on various features of the system without involving collaboration.

This version of Coagmento was based on a new design architecture that is described in the next section. The participants were walked through its installation and given a brief tutorial on its usage. About a month later, they were asked to fill in an online

questionnaire rating various features of the system. These questions are given below and the participants’ responses (on the scale 1 to 7, with each scale ordered from less to more) to them are summarized in Table 3.2. Only eight students responded, as providing feedback was optional.

Q1. It was easy to save relevant information (documents and snippets) using the toolbar functions.

Q2. Making annotations on webpages was useful.

Q3. Display of the project name in the toolbar was useful.

Q4. Display of various statistics about a displayed webpage (view count, snippets, and annotations) in the toolbar was useful.

Q5. Display of my personal history (queries used, and documents and snippets saved) in the sidebar was useful.

Q6. Ability to write notes using the sidebar was useful. Q7. Writing a note using the sidebar was easy.

Q8. Log information about my activities (visited and saved pages, snippets, and annotations records) was useful.

Q9. Creating a new project was easy.

Q10. It was easy to learn to use this system.

Q11. I believe I became productive quickly using this system.

Q12. It is easy to find the information (logs and other objects, histories, etc.) I need.

Q13. The organization of information on the system screens (toolbar, sidebar) is clear.

Q14. This system has all the functions and capabilities I expect it to have. Q15. I am able to efficiently complete my work using this system.

Q16. Overall, I am satisfied with how easy it is to use this system. Q17. Overall, I am satisfied with this system.

Table 3.2: Feedback on Coagmento (version without collaboration support), averaged over eight responses.

Question # Mean s.d. 1 5.50 0.75 2 4.38 1.50 3 5.25 1.38 4 4.50 0.75 5 5.25 0.71 6 4.75 2.18 7 4.50 2.13 8 5.00 0.75 9 6.00 0.53 10 5.00 1.60 11 4.13 0.99 12 4.50 1.30 13 3.88 0.99 14 3.63 0.91 15 4.25 0.46 16 4.50 0.92 17 4.75 1.16

As we can see, the participants found it relatively easy to learn the system, use it for its intended purpose, and appreciate the usefulness of its various components, especially those related to providing awareness. In the free-form field asking them about two aspects that they liked about Coagmento, a majority of the participants identified the ability to save (bookmark) any webpage and collect snippets from anywhere as the biggest positive points. One of the participants suggested incorporating a citation builder like Zotero10 _{with snippets collection. In general, the participants appreciated}

having ready access to their history (saved pages, snippets, and queries).

The participants, however, expected several other features not offered by the version of Coagmento they tried. This came up in Q14 (Table 3.2) as well as in the responses to an open-ended question asking them to list a couple of aspects of Coagmento that they did not like. One of the common feature requests that Coagmento did not have was the ability to save snippets from PDF files. This was identified as a highly valuable function in academia, since many scholarly articles appear in PDF format. Another requested feature was the way to organize collected snippets and notes.

In the discussion session followed by online feedback, the following additional issues came up.

• The participants reported experiencing an additional overhead for using Coag- mento. They found the installation and learning to use Coagmento to be easy, but incorporating it in existing practices a bit hard. Due to the overhead involved with different actions, starting with login to selecting a project, and finally re- visiting the collected information, the participants found themselves not utilizing Coagmento much.

• As a feature, Coagmento allows one to explicitly turn on and off its ability to record one’s browsing. While this feature was provided as a way to facilitate privacy, it ended up being an obstacle sometimes as the participants forgot to turn it on. The participants were divided on the issue of making the recording on or off by default.

• Another issue the participants reported facing was of remembering to label a new project. Some of the participants also indicated that it was difficult for them to remember the actual name they gave to a project several days back. In other

Overall, the participatory design sessions provided a platform to engage in interac- tive discussions with several experienced participants, identify some of the key issues in designing a CIS system, and obtain feedback on early versions of Coagmento. These explorations helped drive the development of a new version of Coagmento, which is described in the next section.