QUALITY OF COMMUNICATION EVALUATION 119

". . . human conversation does not follow the kind of message-passing or exchange model that formal, mathematical theories of communication posit. Rather humans coconstruct the mutual intelligibility of a conversation through an extraordinarily rich array of embodied interactional competencies, strongly situated in the circumstances at hand (the bounds and relevance of which are, in turn, being constituted through the same interaction)."

Suchman (2007)

Quality of Communication, or QoC, is a socially aware evaluation abstraction, as described in Sections 6.5.2 and 7.5.2, that follows on the Softbridge stack (see Figure 8-4). QoC is to the Softbridge stack as QoS is to the OSI stack. QoS, as described in Section 2.2, addresses user perception and network performance with respect to temporality variation and latency across various layers in the OSI stack. QoC reaches beyond the user to include the user's social environment in the appraisal of communication quality, as Suchman's (2007) quote above evokes. QoC is more concerned with an assessment of how ICT enables "mutual intelligibility", whereas QoS is more concerned with system performance (Bouch et al., 2000). Thus, QoC takes the view that while QoS is a 'nice to have'; it is not necessary, as

argued by Purbo (2004a, 2004b). QoC is a holistic appraisal of intermeshed Softbridge layers. QoC employs RA/RI criteria to examine how computer mediated communication functions with respect to social mechanisms and challenges. QoC is therefore more challenging to measure than QoS. We used qualitative approaches for QoC measurement precisely because it is so people-oriented. Thus, QoC and QoS can also be viewed as complementary in the same way that Softbridge and OSI stacks are complementary.

Figure 8-4 Quality of Communication

Just as QoS examines characteristics of each layer in the OSI stack, QoC examines characteristics at Softbridge layers. However, in the case of Softbridge, there is intermeshed interaction between technical and social issues. QoC characterises communication quality within that interaction. While QoC may make use of QoS performance metrics, most QoC is mostly expressed qualitatively in terms of RA/RI criteria.

During the opening phases of both field studies, we thought QoC could be expressed like QoS in terms of MOS (ITU, 2003a), or correlating macro latencies to user acceptance and perception of communication prototypes. We instrumented prototypes to collect latency data and usage statistics, and also conducted semi-structured interviews to learn what participants thought of the system. Over time, however, two matters caused us to abandon that approach. Firstly, the participants in both field studies did not use the prototypes enough to enable statistical significance. Neither were there a large number of participants. Secondly, the macro latencies yielded larger implications for communication software design and evaluation. Where QoS is relevant to real-time voice and video communication, QoC is not limited only to 'perfectly' operational systems with minimal micro-scale latency. QoC applies to both asynchronous and synchronous systems.

The related work covered in Chapter 3, and both field studies, attest that asynchronous communication is quite applicable to 'imperfect' and challenging ICT4D situations. It is

Real Access/Real Impact People Temporality Media Device Network Power Interface People Temporality Media Device Network Power Interface Quality of Communication

arguable that asynchronous communication, e.g. email and Instant Messaging (IM), has become just as necessary and ubiquitous as synchronous communication in developed regions. Email and IM approximates synchronous communication when exchanges happen quickly enough (Tyler & Tang, 2003) and IM could demonstrate 'blended synchrony' (Erickson, 1999). For difficult scenarios, fault tolerance and large delays are part and parcel of ICT4D deployment (Brewer et al., 2005; Pentland et al., 2004). Pentland et al. (2004) considered asynchronous connectivity to be a stepping-stone toward full synchronous connectivity. In our view, temporality in human communication is actually a continuum between asynchronous and synchronous exchanges, and therefore primarily semi- synchronous. The SIMBA and MUTI prototypes demonstrated various forms of semi- synchronous communication with multiple media modalities.

Accordingly, QoC includes several very non-traditional notions of service quality. Firstly, QoC is more concerned with the use of an ICT4D system within a user's social environment than with an individual user's perception. Secondly, Softbridge, hence QoC, layers are not insulated from one another. Rather, they are intermeshed and often influence one another. As with Softbridge, QoC intermeshing between and across layers is made complicated by the social entanglements in the environments in which ICT4D prototypes are deployed. Informed by RA/RI criteria, QoC offers a way to bridge 'evaluation-reality gaps' just as the Softbridge offers a way to help avoid Heeks' (2002) 'design-reality gaps'. The design and evaluation processes are complementary. Together, the Softbridge stack and QoC can address the alignment of technical and social factors during design and evaluation of ICTD, respectively. Table 8-1 compiles crosscutting people issues in terms of RA/RI criteria abstracted from the two field studies.

Physical access: Provide mobile and shared devices with wireless connectivity. Appropriateness: Technological appropriate solutions may not be socially appropriate.

Affordability: Design for top end IP-based services. What is expensive today will be cheaper later. Human capacity and training: Train up NGO members so they can pass on skills to the community. Locally relevant content: Application-independent infrastructure lets users choose their own content. Integration to daily life: Focus on lower Softbridge layers first and get authority figures to use ICT. Socio-cultural factors: Start from an existing communication task/norm before extending circles. Trust in technology: The trust of the NGO in the researcher and the research extends to the community. Local economics: Sustainability is ultimately the responsibility of the community, not the researcher. Macroeconomics: South African macroeconomics does not benefit the disadvantaged as intended. Legal and regulatory: Using restricted technologies can help lead to their legalisation.

Public support and political will: Projects like these improve awareness of ICT4D realities.

Table 8-1 Crosscutting Real Access/Real Impact criteria

This table lists RA/RI criteria that were clearly evident in both field studies. The similarity of issues underscores the use of the RA/RI criteria to inform consideration of the people issues in the Softbridge stack.

The alignment of social and technical imperatives is a tricky balance. For example, at one point we thought that Deaf participants were not so interested in text or communicating with hearing people (Cycle A.7). Then they briefly showed some enthusiasm for the DeafChat prototype (Cycle A.8). However, Deaf participants actually wanted to communicate in SASL with both Deaf and hearing people. We now feel that a sign language interface to video relay would appeal to DCCT participants. Despite the relative failure of the text relay prototypes, their introduction enabled ICT infrastructure to take root in the Deaf community. We had not expected so many Deaf people to use the PCs so often, and certainly had not expected the DCCT NGO to assume payment for the broadband connectivity.

Similarly, our understanding of the appropriateness of the MUTI system evolved over the course of the rural telehealth field study. We came to understand that telehealth needed to include more people than just the nurses and doctors. Improving QoC meant enabling nurses to communicate with other nurses and their superiors, and actually for doctors and nurses to communicate with whomever they wanted. We had not provided Internet access at Tsilitwa because we had concentrated solely on telehealth. By 2007 both Nessie Knight and Canzibe hospitals had their own VSAT for Internet access.

It appears that QoC is easier to attain at lower levels in the Softbridge stack, closer to where QoS operates. However, one could also argue that QoC needs to be built at lower levels first, including some modicum of QoS, so that QoC can be attained at higher levels later. This process takes time, and requires a number of cyclical interventions.