Development of protocol analysis codes for the interactive, electronically- electronically-assisted interview method

Another methodological contribution worthy of mention is the candidate’s development of a protocol analysis coding scheme for the results of the interactive, electronically-assisted interviews. In their original proposal for the ESRC-funded project, the researchers had indicated a desire to use verbal protocols, or the analysis of think-aloud techniques, to provide a record of citizens’ online search processes when using parliamentary websites. At the time, previous studies of information seeking that had utilised protocol analysis had tended to focus on measuring the frequencies of particular types of verbal statements or behaviours, where each statement or behaviour, regardless of its duration, was treated as an individual ‘segment’ and coded accordingly (e.g., Blackshaw and Fischhoff, 1988; Nahl and Tenopir, 1996). Other authors, however, had suggested that recording the time spent by participants on particular protocols might be a more meaningful measure (e.g., Gilhooly and Green, 1996). In considering the first option, the candidate believed that transcribing and coding every individual statement would be exceptionally time-consuming and, in many

96

cases, would be of little practical value (e.g., when participants had simply read aloud the contents of a particular website page). It was felt, therefore, that measuring the time spent by participants on particular types of verbal statements or behaviours, complemented by the transcription of meaningful, illustrative examples of these statements and behaviours, would be a more manageable yet equally illuminating method. With this in mind, as the transcription of the 2001 roadshow interviews progressed, the candidate devised a unique series of 19 codes which reflected the nature of the comments made and the behaviour exhibited during the online sessions, by both the interviewee and the interviewer. These codes appeared in Output 3 (Appendix A, pp.388-389) and in Output 5 (pp.157-158), and are reproduced below, beginning with 12 interviewee codes with an IE prefix:

IE Search. When the interviewee had structured a search, with little or no assistance or guidance from the interviewer.

IE Browse. The interviewee had no specific search outcome in mind, but simply browsed the website.

IE Navigate. The interviewee asked navigational questions or questions about the website’s design.

IE Read aloud. The interviewee read aloud the contents of a particular page of the website.

IE Read internal. The interviewee read the contents of a particular page of the website 'internally'.

IE Positive. The interviewee made a positive qualitative comment about the content or particular design features of the website.

IE Negative. The interviewee made a negative qualitative comment about the content or particular design features of the website.

IE Parliament. The interviewee made comments on, or asked questions about, parliamentary procedure or terminology.

IE IT. The interviewee made comments on, or asked questions about, the software and hardware being used, or computers and the Internet in general.

IE Project. The interviewee asked questions about, or commented on, the research project.

IE Political. The interviewee voiced his/her opinions on particular political and current issues.

IE Personal. The interviewee gave personal information, relating to work or study experiences, previous use of computers, or to a specific life incident.

97

Six interviewer codes with an IR prefix were also assigned:

IR Search. The interviewer provided the interviewee with specific instructions or advice on formulating a search.

IR Navigate. The interviewer provided specific navigational instructions or advice, or answered questions on website design.

IR Question. The interviewer asked questions when the interviewee failed to maintain a constant verbal report of their thoughts and actions, or when a specific matter of interest arose during the search.

IR Parliament. The interviewer provided an explanation of parliamentary procedure or terminology.

IR IT. The interviewer provided an explanation of, or answered questions on, the technology used in the project, or on computers and the Internet in general.

IR Project. The interviewer provided further details of the research project.

The last of the 19 codes was Interruptions, indicating that the interview was interrupted.

These may have been verbal, by the interviewee's family or friends or other individuals; or they may have been for technical reasons.

While the coding scheme relates specifically to citizens’ online behaviour on parliamentary websites, the candidate believes that most of the 19 codes might be applied equally to information seeking on any type of online resource; and that, with minimal effort, they might be adapted to meet the needs of other LIS researchers.

The coding resulted in a significant amount of quantitative data, which was presented in full in Appendices XII to XVII of the final report to the ESRC (Marcella, Baxter and Moore, 2002), and which was discussed briefly in Outputs 3, 4 and 6. In particular, Figure 1 of Output 3 provided a summary of the protocol analysis of the search sessions, and clearly

demonstrated the participants’ need for support in the navigation of parliamentary websites, with 23% of the search time consisting of the interviewer providing navigational instructions and advice, compared with just 12% of the time being occupied by unassisted search formulation on the part of the interviewees.

While this protocol coding approach was analytically valuable, it did remain relatively time-intensive. With this in mind, when the candidate decided to reprise the interactive,

electronically-assisted interview method, in the 2011 voter information behaviour study, a

98

lack of time and other resources unfortunately precluded the inclusion of this additional analytical layer.⁶³