Variables and Measures

In line with the hypotheses, the dependent variables of interest were requirements size,

requirements quantity, and requirements completeness (depth and breadth). Requirements

size was measured as the number of words used to construct the requirements message.

Requirements quantity was the total number of requirements provided by a participant

(e.g., Browne and Rogich, 2001). In line with Pitts and Browne (2007), requirements

completeness was comprised of the measures of depth and breadth of requirements.

Breadth was measured as the number of distinct requirements categories (from the

Browne and Rogich requirements taxonomy shown below) provided by subjects. Depth

was measured as the number of requirements within each category (from the taxonomy)

provided by subjects. The taxonomy has been used in many studies to code requirements

data (e.g., Browne and Rogich (2001), Pitts and Browne (2007)). For the purpose of

analysis involving requirements quantity and completeness, a coding procedure (similar

to Browne and Rogich (2001)) for the textual requirements data was used (described

later).

The taxonomy (Pitts and Browne, 2007) is shown in Table 24.

Table 24. Requirements taxonomy (Pitts and Browne, 2007) GENERIC REQUIREMENTS CATEGORIES

(Pitts and Browne, 2007, p.101)

DESCRIPTION (Pitts and Browne, 2007, p.101) GOAL

Goal state specification Identifying the particular goal state to be achieved

Goal specification Comparing existing and desired states

Difficulties and constraints Identifying factors inhibiting goal achievement Ultimate values and preferences Stating the final ends served by a solution Difficulties and constraints Identifying factors inhibiting goal achievement Ultimate values and preferences Stating the final ends served by a solution Means and strategies Specifying how a solution might be achieved Causal diagnosis Identifying the causes of the problematic state Knowledge specification Stating facts and beliefs pertinent to the problem

128

Existing support environment Existing technological environment to support new system

Stakeholders Organization units, customers, suppliers,

competitors

PROCESS

Process description Steps or tasks designed to produce a product Process knowledge specification Facts, rules, beliefs, decisions required to perform

process

Difficulties and constraints Facts that may prohibit process completion Roles and responsibilities Individuals/departments charged with performing

process

TASK

Task description Sequence of actions required to complete a task

Task knowledge specification Facts, rules, beliefs, decisions required to perform task

Performance criteria Statement that link outcome with

condition/constraints

Roles and responsibilities Individuals/departments charged with performing task

Justification Explanation of specific action to be/not to be taken

INFORMATION

Displayed information Data to be presented to users; paper/electronic format

Interface design Language and format used for displayed

information

Inputs Data that must be entered in to system

Stored information Data saved by the system

Objects and events Physical entities and occurrences relevant to the system

Relationship among object/event How one object/event is associated with another object/event

Data attributes Characteristics of objects and events

Validation criteria Rules that govern the validity of data

Computations Information created by the system

Some control variables that were expected to be present in the context of

requirements generation in OSS development were also measured. The experience and

knowledge of software project team members and users can influence the outcomes of the

requirements capture stage of a software project (Chatzoglou and Macaulay, 1997).

Technical knowledge (e.g., knowledge about commonly occurring functional

requirements in the context of information systems development, general knowledge

129

important knowledge sources during requirements determination for systems

development (Vitalari, 1985). Thus, one’s technical background (technical career and

technical education) can potentially influence requirements generation in OSS

development and were included as covariates in the analysis. Questionnaire items for

measuring for the extent to which the participants’ career and education were technical

were obtained from Enns et al. (2003).

Web-based requirements generation is different from the traditional face-to-face

setting of requirements elicitation, as web-based requirements generation is mediated by

the internet. Hence experience with the relevant web-based applications could be a

potential confounding variable. One such variable is the crowdsourcing experience of

participants, which is the number of crowdsourcing tasks (hosted on some crowdsourcing

platform) that participants had completed in the past. For the crowdflower platform, this

would be the total number of crowdflower tasks (tasks hosted on that platform) that a

participant has completed. This measure was obtained for each participant from their

Crowdflower profile. Crowdsourcing platforms, such as Crowdflower, are web-based

platforms/applications that support crowdsourcing tasks in a variety of domains (Doan et

al., 2011). Example categories of such crowdsourcing tasks are reviewing (e.g., books,

movies), tagging (e.g., webpages) and constructing/sharing textual knowledge (e.g.,

answering questions) (Doan et al., 2011). Participants who have successfully completed

many such crowdsourcing tasks could potentially gain knowledge of many different

application domains. They may become adept at common web-based activities such as

130

important role during requirements generation in OSS development. Hence,

crowdsourcing experience of participants was included as a covariate.

The prior experience of participants with medical software (e.g., personal health

record software) could also be a potential source of knowledge for them during

requirements generation; hence, it was also included as a covariate. Participants were

asked to indicate their usage experience with electronic medical record software systems,

personal health record software systems, and other health related software systems on a

Likert scale and the scores were aggregated and included in the analysis as the control

variable, experience with medical software.

4.5 Data Analysis and Results