The explanation of how information is communicated via computer-mediated communication in complex socio-technical systems and the potential for that information to lead to team cognition creates the core of this dissertation. Team members use computer- mediated communications during operation to communicate timely, useful information. During these operations team members also seek information from multiple sources in an attempt to make optimal decisions within given time constraints (Palen, Vieweg, Liu, & Hughes, 2009; Vieweg et al., 2010). The first objective is to identify what messages contribute to team cognition, specifically to SA and TMS, and what specific cognitive behavior indicators they contain. This leads to the second objective, which is to identify how to communicate cognitive information at a linguistic level. This process generates the
foundation for tools that can automatically extract pertinent, valuable information. Such automated tools need to be trained to correctly “understand” human language which involves the identification of the words team members use to communicate in computer-mediated communication.
In natural language understanding, data coding is an integral component of identifying words. Methods for data coding and language understanding incorporate named entity
recognition (Bikel, Schwartz, & Weischedel, 1999), semantic role labeling (Hovy, Marcus, Palmer, Ramshaw, & Weischedel, 2006; Hwang, Bahtia, Bonial, Mansouri, Vaidya, Xue, & Palmer, 2010) or syntactic parsing (Gabbard, Markus, & Kulick, 2006), which employ a supervised machine learning approach that relies on annotated corpora. For a machine to successfully locate utterances in computer-mediated communication that contain information about team cognition, it needs to be trained to identify computer-mediated communication text that is most like to present such information (e.g., Palmer, Glidea, & Xue, 2010). This process will require identifying behaviors of information team members communicate (cf. Verma et al., 2011); for example, of information about the social, built, and physical mission environment or more specifically, information about team members’ positions related to other
92
team members and their coordinated effort. Computer-mediated communication are
annotated with various categories with further analyses revealing how information leading to team cognition and specifically how SA and TMS cognitive behavior are constructed
linguistically. In the research presented here, this further analysis involves an examination of the speech act, as well as verb and verb tense used to convey team cognitive information. Particular given that in the English language, the verb is generally the central element in a sentence which organizes all other elements (Manning & Schütze, 1999), and which conveys the meaning of the events taking place (Palmer et al., 2010).
While existing measures of SA and TMS provide a theoretical ground for developing measures of cognitive behaviors, in which cognitive behavior indicators may group and form into classifications to enable an automated driven analysis; a model can determine how the cognitive behavior indicators from participant’s messages are derived. The various reviewed measurements of team cognition present different approaches taken (i.e., phenomenological, causal or actionable) to measure cognitive processes (e.g., Cook et al., 2005; Dwyer et al., 1997; Johnston et al.,1997; McNeese & Reddy, 2000; McMillan et al., 2004; Orasanu, 1990; Serfaty, Entin, & Johnston, 1998; Smith-Jentsch et al., 1998; Walker, 2005). Fundamentally, cognitive behavior indicators are construct factors used for measurement. In a first step, Figure 7 depicts the formalization of the description of these cognitive behavior indicators and factors.
All relevant aspects of the models dimensions such as cognitive processes, behavioral indicators, and data coding are displayed. Data coding of message utterances, specifically of verbs, the cognitive behavior indicators are classified and are the lowest level of analysis. Thereon, an attribution of the cognitive behavior indicators means that one of the cognitive behavior indicators results because of data coding. These behavior indicators then identify the corresponding cognitive process. To understand the pattern of cognitive behaviors in team communication, speech acts (e.g., request, announcement, question, reply, confirmation, read-back) are identified by means of data coding of communication patterns and utterances (Parush et al., 2011). The behavior indicators map out according to the occurring speech act as seen in the following example:
93 Team Member 1: “Is there any fire?”
Speech-act: Questions; Cognitive behavior indicator: Perception.
The objective of such a sound data coding process is to identify communications that reflect cognitive behavior indicators of developing and maintaining cognitive behavior in communication.
Figure 7. Data Coding Process Model.
This model is capable of examining team communication to assess the degree of SA cognitive behavior to which team members are sharing perceived data (level 1), interpreting data to understand the current situation (level 2), and project what will happen in the near future (level 3; Endsley, 1998). In addition, TMS cognitive behavior indicators will also examine the areas of specialization expertise, credibility of others’ knowledge, and
coordination of procedures using this model. An ongoing assessment will provide a running indicator of SA/ TMS cognitive behaviors that could, if not addressed, lead to reduction of team effectiveness or failure. Each occurring event in complex socio-technical systems is unique; however, regularities exist as to how events transpire. Previous experience with these types of situations (e.g., when a wildfire ignites) provides team with background knowledge necessary to predict future states as the event unfolds. Correspondingly, by uncovering, understanding, and describing regularities of cognitive behavior indicators in team communication, automatic methods can locate vital information that we can expect team members to communicate, in complex event driven situations. By creating background
94
measurement techniques that can collect data from team members’ communications in the background, cognitive process measurement can be more easily included in a wide variety of system development and exercises. It can significantly extend the ability to detect shortfalls in cognitive processes during early team development phases, or prior to significant problems occurring in the field. For analysts and trainers to gain knowledge about cognitive behavior, transmitted via computer- mediated communication, automated text analysis must be able to identify these pertinent cognitive behavior indicators. A fundamental step in this process is the creation of classifiers to locate cognitive behavior indicators, often described by particular verbs (Verma et al., 2011; Vieweg et al., 2010).