Technology Probe

To complement the participant observation research I designed a technology probe study. I refer to this as a “technology probe” approach because the

technology is being used to probe students’ practices and understandings of PIM (Chi, 2003). During the same semester, I recruited students to use MyLifeBits system on their personal laptops to manage their digital information (such as emails, files, saved Web pages, etc.). The goal of this study was to explore students' use of MyLifeBits in typical educational tasks and situations.

Sample

Students enrolled in the BIO 101H class were recruited for this part of the study. Prior to recruitment, they were shown a demonstration of MyLifeBits in class. An e-mail message requesting volunteers to use the system was then sent to all members of the class. Participants were offered a $50 gift certificate and upgraded RAM as compensation for their time. Eight students responded to my request for volunteers. Of these, one declined to participate after learning further details about the study.

Technical problems prevented MyLifeBits from operating properly on three of the participants' computers. One student was running Windows Vista which proved to be incompatible with MyLifeBits. One student had a non-CCI computer or which was simply not powerful enough to run the software adequately. One student's computer could not run the required web server software due to a configuration problem. Two of the students who had technical problems remained in the study as a "control group." I was able to discuss with

them their natural use of PIM software such as e-mail clients and Microsoft OneNote.

Procedure

I met with each of the seven volunteer students individually to install

MyLifeBits on their personal laptop computers. Before installation, participants signed an informed consent agreement. After installing the software, I gave participants’ a brief walk through of MyLifeBits' functions, and made sure they felt comfortable using all of its features. For students who had 1 GB or less of memory installed, I obtained an additional gigabyte of RAM and had it installed in their laptop. Participants then continued to use MyLifeBits for the remainder of the semester and exam period. Overall, the technology probe lasted from six to eight weeks depending on when I met with the participant to install the software.

After the semester, all of the participants (including the control group) met with me for a two-hour participatory design session in a small classroom with whiteboards and drawing materials. The motivation for this “participatory design” was to involve students in envisioning what future student-centered PIM systems might look like. The participatory design session took the form of

interactive focus group, in which students participated in exercises and group discussions to create ideas for PIM systems. Students were asked to identify key tasks and scenarios, and then develop prototype designs. Students participated in the following specific exercises to stimulate discussion:

• Exercise 1:

Interview another student about his or her PIM practices, using these questions:

o How do you keep track of important Web pages?

o How about important emails?

o How do you organize files on your PC (like documents,

spreadsheets, presentations, etc.)?

o How do you keep track of tasks and due dates for class?

o How do assess how much you’ve learned for a class (how prepared

you are, whether you need to study more, etc.)?

• Exercise 2:

As a group, brainstorm some specific scenarios for how a student could manage information, related to each of the areas you covered in the interview.

• Exercise 3:

Based on the ideas we have discussed, work with a partner to sketch out ideas for technologies that could help solve one of the specific problems we identified. Then, switch partners and discuss your ideas.

I then reviewed and summarized the ideas generated as a result of these exercises, which helped inform my own design concepts. The specific ideas proposed by students are presented in the Results section.

At the conclusion of the semester’s exam period, I met with each participant to interview him or her, and collect log data and screenshots from MyLifeBits. I then removed MyLifeBits from the participant’s computer.

Data collection

I investigated students' use of MyLifeBits in four ways. First, I used informal opportunities (e.g., the time before class began) to discuss with students their use and impressions of MyLifeBits. I summarized these discussions in my daily field notes. Second, I sent periodic e-mails to all the participants with questions about MyLifeBits and example ways to use it. Third, I set up individual interviews with all participants to discuss their PIM practices and use of MyLifeBits in greater detail. The interviews were guided by reviews of the captured materials in their MyLifeBits archives. Fourth, I collected usage logs from each MyLifeBits

database at the end of the semester. Taken together, these activities allowed for multifaceted evaluation of the system, including both usability (ease of learning and using the system and interface) and usefulness (ability of the system to support students’ learning and PIM practices).