Chapter 5 : Methods and Analysis – Phase Two (Student Rating Exercise)
5.4 Revisions to the Scale and Final Selection of Items
making sense of the results of the analysis. In particular, while the results of the
ANOVAs appeared relatively clear, subsequent reviewing and reflecting on the content of the questionnaire items themselves was an important step. For example, I examined some of the items where the mean rating scores across dimensions were not significantly different but were similar to try and understand what might be confusing the raters in the wording. Additionally, I also examined where mean rating scores were not significant but clearly higher on an unintended dimension. This examination led to some general concerns about the overall wording and phrasing of the questionnaire items. While the
items had been adapted from past published culture studies (e.g., Taras, 2011), it became apparent that there may have been issues throughout with ambiguous or confusing wording. While the items did not suffer from exceedingly problematic and obvious issues such as "double-barreled" phrasing or confusing double negative word structures, it became clear that the items needed further refinement. For example, some of the items referred to both individual and organizational targets (e.g., "I feel..." and "people feel..."). This is problematic because while culture indeed reflects individual (as part of a
collective) and group aspects, they are not the same. Using both types of targets in the same series of questions runs the risk of actually tapping subtle but distinct ideas. Self- reflective individual targets might capture a different sense of the organization compared to reporting on the perceived group sentiment or behaviour.
The phrasing of statements was also inconsistent from question to question which may have made the items needlessly more difficult to parse. As noted in Chapter 4.5, the dimension definitions were revised to be more consistent with one another. Upon reflection, this consistency check should have occurred with the questionnaire items as well. This is partly due to mixing and matching past culture survey items without more careful consideration as to how the items would operate together. Questions that are worded inconsistently are not only problematic for respondents to interpret but may also confuse the conceptual meaning of the items. For example, while all the questions may have been ultimately about the organization, the phrasing with respect to referencing the organization varied between questions. Questions were phrased about "the organization", "my organization", "our organization", or referenced "people here" with the organization being implied. While these different phrasing structures may reflect the same underlying setting (i.e., the organization in which the respondent works), it adds unnecessary
complication.
After consulting with faculty advisors and several students who provided feedback on the exercise, I decided to seek additional resources on survey design as well as culture
questionnaires. This consultation was motivated by the results of Phase Two, namely the observed confusion between items and their resultant ratings, compared to their expected ratings. I was directed towards classic questionnaire item development resources (e.g.,
Converse & Presser, 1986; Spector, 1992) for tried-and-true guidance on structuring clear, unambiguous, items that conceptually accessible for a varied respondent audience. The purpose of revisiting and revising the items was to retain the same meaning and intent as the original iterations but with enhanced clarity and conciseness. This process was also guided by patterning the structure of the items after a different and far more coherent set of published culture items. The first iteration had used a more scattered approach, generating items based off of a catalogue of previously published culture questions. Instead, in this iteration, in order to ensure that the items focused on a direct, unambiguous, and consistent organizational culture framing, the questions were guided by ethical organizational culture items as developed by Trevino and colleagues
(Schaubroeck et al., 2012; Trevino, Butterfield, & McCabe, 1998; Trevino &
Youngblood, 1990). This included clarifying the statement target as the organization and clearly and consistently orienting the questions around organizational behaviours and values. For example, the majority of the items took the form "In my organization..." which resolved both the target and consistent phrasing issues. Other items varied the wording slightly for variety but consistently referenced the respondent's own organization as the focal point. This kind of phrasing removed the ambiguity in the target of the question instead emphasizing the descriptive culture elements (e.g., values, assumptions, practices). The items developed and validated by Trevino and colleagues had a much clearer and well-established history and demonstration in psychology and organizational behaviour practice, which made them much more suitable templates. Ideally both the survey development and culture questionnaire resources noted here would have been identified earlier in Phase One. However, survey item development is a process that often requires multiple iterations before being completely validated (cf. MacKenzie et al., 2011). In this case, the results generated in Phase One and Two and certainly hindsight were instrumental in this refining process. The result of this subsequent development and refining work is the final 25 item questionnaire as shown in Table 19.
Table 19 25 Item Entrepreneurial Culture Scale
Organizational Enthusiasm
OE1 In my organization, people are passionate about the organization’s mission. OE2 In my organization, people are enthusiastic about what our organization does. OE3 In my organization, people care about the organization’s vision for the future. OE4 In my organization, people care about company values or guiding principles. OE5 In my organization, people are enthusiastic about achieving organizational goals. Stakeholder Alignment
SA1 In my organization, people are actively concerned with building relationships with the organization’s stakeholders (e.g., customers, suppliers, partners, investors).
SA2 In my organization, it is expected that you will do what is right for the organization’s stakeholders (e.g., customers, supplier, partners, investors).
SA3 My organization disapproves of people who treat the organization’s stakeholders (e.g., customers, suppliers, partners, investors) poorly.
SA4 In my organization, people have a strong sense of responsibility to the organization’s stakeholders (e.g., customers, suppliers, partners, investors).
SA5 How our decisions effect organizational stakeholders (e.g., customers, suppliers, partners, investors) are a primary concern in this organization.
Cohesiveness
C1 In my organization, people support one another.
C2 Trust and mutual respect for one another are a major consideration for everyone in this organization. C3 In my organization, people care about group interests above other considerations.
C4 In my organization, people pull together during times of crisis.
Table 19. 25 Item Entrepreneurial Culture Scale con’t.
Learning and Development Support
LDS1 This organization actively encourages employees to learn and improve themselves. LDS2 In my organization, each person is expected to continuously learn and develop.
LDS3 In my organization, people are actively concerned with improving their skills and abilities at work. LDS4 My organization rewards people who learn and develop themselves.
LDS5 People in this organization are very concerned about learning new things. Opportunity Driven Change
ODC1 My organization rewards employees who introduce improvements and innovations. ODC2 Our organization is actively concerned with exploring new opportunities.
ODC3 In my organization, people are open to changes in their work routine.
ODC4 People in the organization are actively concerned about changing work processes when they can be improved. ODC5 It is expected that individuals will change work routines in order to pursue an organizational opportunity.
5.5 Conclusion
This chapter outlined the objective, procedure, sample and selection, analysis and results of Phase Two of the dissertation data collection. This phase was based on a content validity rating exercise with doctoral students. The purpose of this phase was to evaluate the prospective entrepreneurial culture questionnaire items against their intended
definitions and others. The exercise was important for establishing that participant's ratings of items were significantly different across the various definitions. The results of the exercise and the subsequent analysis indicated not only the items which were
correctly associated with their intended definitions, but also areas of item deficiency. These deficiencies, such as incorrect association or low associations in general, may have been attributed to poor or confusing wording of the items. As a result of these issues, the questionnaire items were revisited in light of recommendations for structure and wording (i.e., suggested survey design best practice) as well as to more closely match a template for organizational culture surveying (i.e., consistent targets and focal cultural elements such as behaviours and values). These revised questionnaire items formed the basis for the entrepreneurial culture survey which was subsequently deployed in Phase Three.