Validity Assessments

validity. After I generated the initial item pool, the items were examined for face and content validity to ensure that the measures created for the construct of interest could be operationalized.

Hardesty and Bearden (2004) use the dartboard as an analogy to make distinctions between content and face and content validity. In order to establish content validity, the darts should land randomly on the dartboard. Should the darts be grouped on one side or the other of the dartboard, content validity cannot be established as the items may only partially represent the construct of interest. Similarly, in assessing face validity the items must at a minimum hit the dartboard (p. 99).

In this study, face validity (hitting the dartboard) was first assessed by my partner in research (Dr. Lloyd Duman) and by the members of my dissertation committee. Each item was evaluated based on the strength of the content to uniquely express what it was intended to measure (i.e., knowledge sharing, network cohesion, and creativity). Next, I solicited feedback from my methodologist, Dr. Donna Chrobot-Mason, Director for the Center for Organizational Leadership at the University of Cincinnati.

Based on Dr. Chrobot-Mason’s feedback, four major revisions were made. First, in response to feedback that items were too broad, some items were reconstructed to better reflect the original definition. Second, I modified the survey to clarify the focus on the study. Third, the items were initially found to shift between individual-level and organizational-level responses.

Based on this feedback, all survey items were revised to reflect a holistic view of the

organization. Finally, redundant and double-barrel questions were identified and revised for clarity and conciseness.

Content validity. Content validity assessments were initially established through the

solicitation of various experts that included both scholars and practitioners in various fields related to organizational ambidexterity and network resilience. Hardesty and Bearden (2004) offer this cautionary advice to researchers: “This validity assessment is necessary since

inferences are made based on the final scale items and, therefore, they must be deemed face valid if we are to have confidence in any inferences made using the final scale” (p. 99).

This process facilitated the pruning of the initial item pool from 52 to 40 items. For the 40 items, a six-point Likert partitioned scale was used. Each item had six possible options for a response: 1 = strongly disagree, 2 = disagree, 3 = somewhat disagree, 4 = somewhat agree, 5 = agree, and 6 = strongly agree. I chose six partitions as appropriate to the sensitivity of the scale based on the recommendation from Abell et al. (2009) that “imposition of the numbers helps defend interpretations claiming that if respondents perceive the distance between two and three as equal to the distance between three and four, they will make the same associations regarding the text labels” (p. 49). Table 3.2 illustrates the types of validity assessed in this study:

Table 3.2

Types of Validity Assessed

Type of validity Plan to assess validity Goals

Face validity: The degree to which the assessment appears to measure what it claims to measure (DeVellis, 2015).

The scale will be presented to my dissertation committee and PhD colleagues

To ensure the connection between the scale and the construct of interest remain aligned.

Content validity: Refers to the adequacy with which a measure assesses the domain of interest.

(Hinkin, 1995).

Solicit content experts and practitioner insights, through a brief survey on how well the construct of interest is

represented in the scale items

To ensure the scale content conforms to what it intends to measure.

Construct Validity: Concerned with the relationship of the measure to the underlying

attributes it is attempting to assess (Hinkin, 1995).

The extent to which the scale measures the construct of

Construct validity. Considering the relevance of clear conceptualizations of the construct

of interest in scale development, Clark and Watson (1995) argue that “the most precise and efficient measures are those with established construct validity; they are manifestations of construct in an articulated theory that is well supported by empirical data” (p. 310). Due to the basic exploratory nature of this study, a simplified statistical approach that would allow for the investigation of various paradigms potentially influencing the dimensions of a theoretical model was necessary. Worthington and Whittaker (2006) describe exploratory factor analysis as a

“dynamic process of examination and revision, followed by more examination and revision, ultimately leading to a tentative rather than a definitive outcome” (p. 808). Williams, Onsman and Brown’s (2010) objectives for exploratory factor analysis were followed in this study to,

reduce the number of variables, examine the relationship between variables, assess the unidimensionality of a theoretical construct, evaluate the construct validity of a scale, development of parsimonious (simple) analysis and interpretation, address

multicollinearity (two or more variables that are correlated), and to develop theoretical constructs, and used to prove or disprove theories. (p. 2)

In considering an appropriate sample size, this study followed Clark and Watson’s (1995) recommendation of assessing a minimum of 300 respondents. Abell et al. (2009) also

recommended following a range of five to 10 respondents per scale item when conducting exploratory factor analysis (p. 64). All indicators were measured using a conventional multi-item Likert scale and higher scores were interpreted as indicators of greater amounts of the construct of interest. Statistical data was then analyzed in Statistical Program for Social Sciences (SPSS). Items were grouped based on the tentative indicators identified in the literature review.

With the understanding that all forms of validity are interrelated, convergent and discriminant validity assessments were conducted. Convergent validity ensures that variables that are

expected to correlate do so, while discriminant validity assesses the degree variables that should not correlate, do not (Abell et al., 2009). This final step verified the expected relationships between the new scale results and the construct of interest being examined. Simms (2008) contends that discrepancies between results and theoretical assumptions suggest that, “(i) the measure does not adequately measure the target construct, (ii) the theory requires modification, or (iii) some combination of both” (p. 428).

Step 3. Refining the content of the RNA scale. Creswell (2014) reasoned that, “the