Validation Process

4.4.3.1 Pre-test

The questionnaire was pretested with a group of information systems PhD students to collect feedback on the overall structure of the questionnaire. The pre-test respondents were asked to complete the instrument first and then provide feedback on matters such as format, content, understandability, terminology, and ease and speed of completion, as recommended by Lewis et al. (2005). Several minor mistakes were detected, spelling errors and duplication of items. The questionnaire was refined accordingly. Using the refined questionnaire, a pilot test was undertaken to further purify the instrument.

4.4.3.2 Pilot study

The pilot study was conducted with respondents similar to the target population. Three people with experience in health were asked to complete the questionnaire and to offer suggestions for improvement, such as adding items they felt were missing. The addition of one item relating to the role of leadership in providing the funding was suggested. This item was then added to the measure of the leadership construct. Overall responses suggested that the items adequately covered the content of their constructs.

4.4.3.3 Content Validation

Content validity of the measurement instruments was further tested in a formal content validity test, to ensure that measurement items reflect all of the important aspects of their constructs (Boudreau, Gefen, & Straub, 2001). This study used a quantitative approach

involving a content evaluation panel consisting of subject matter experts—individuals knowledgeable about healthcare and knowledge management. By using a quantitative approach (rather than a qualitative approach), I was able to include a large number of experts and could take into account their judgements objectively.

A quantitative approach developed by Lawshe (1975) was employed. Lawshe‘s approach uses the content validity ratio (CVR) to quantify the degree of consensus among experts to assess the content validity of items. Lawshe’s quantitative approach has been successfully applied to validate content validity of information resource management instruments (Lewis, Snyder, & Rainer, 1995).

The content validity study involved preparing a content validity questionnaire, identifying subject matter experts, applying for ethical approval from Massey University Human Ethics Committee, and sending the questionnaire packet to the subject matter experts.

The content validity questionnaire included descriptions of constructs, with items listed under their constructs. The subject matter experts were asked to rate each item’s relevance to the content domain of its construct using a scale from 1 to 3: (1) not relevant, (2) important (but not essential), and (3) essential. As additional information, the subject matter experts were asked to provide free-text feedback for the construct and for the items, or to suggest any new items in the space provided. The culture of sharing and the subjective norm constructs were not covered by the content validity study; the experts involved in the content validity study were not seen as experts in distinguishing between these two constructs (for a discussion of the content of these constructs refer to section 2.3.2).

The subject matter experts were chosen from both industry and academia. It is recommended to have at least three experts on the panel. However, a larger number is better as it allows access to a broader range of expertise and addresses the risk that some of the experts do not respond (Rubio, Berg-Weger, Tebb, Lee, & Rauch, 2003). The total number of subject matter experts in this study was thirty. Five were chosen based on their research in the area of KM, twenty were randomly chosen from the list of presenters at Health Informatics New Zealand Conference 2009 (HINZ2009), and five were researchers who have published papers in the areas of KMS in healthcare and KMS success in top ranking journals such as Information and Management Journal and MIS Quarterly, or at

peer-reviewed conferences. The subject matter experts were from New Zealand, USA, and Malaysia.

The questionnaire was sent by email to the potential respondents with known email addresses and to the rest by post. The questionnaire packet consisted of a cover letter with a web link to the online version of the questionnaire, information sheet, and the content validity questionnaire (as a hard copy or as a Word document). The respondents had a choice to use the on-line version of the questionnaire or the alternative.

Massey University ethics approval procedures were followed. Low risk notification was submitted to Massey University Human Ethics Committee.

Out of the thirty potential participants contacted, 17 responded. One respondent did not rate the items, but did provide comments on how to revise the measures. The response rate was 53 percent. Sixteen panellists responded and completed the form. A panel of sixteen experts is within the range recommended for content validity studies in methodological literature (Rubio et al., 2003). To determine whether an item has content validity, a content validity ratio (CVR) was computed for each item. The content validity ratio indicates the degree of consensus among the experts.

Although Lawshe (1975) utilized only the “essential” response category in computing the CVR, following the suggestion by Lewis et al. (1995), a less stringent criterion was employed to compute CVR in this study. This was because the categories of both “important (but not essential)” and “essential” mean that the items are considered to be relevant to the content of their constructs. Responses that did not provide a rating on a given item were not used in the calculation of the CVR for that item.

A content validity ratio (CVR) was computed for each item from the following formula:

CVR =ቀଶே೐

ே ቁ െ 1, (1)

where CVR is the content validity ratio, ܰ_௘ is the number of panellists rating the item as either “3 = essential” or “2 = important (but not essential)”, and ܰ is the total number of subject matter experts who rated the item.

The total number of subject matter experts in this content validity study was 16. According to a table in Lawshe (1975, p. 568), the minimum CVR value required for an item to be acceptable when 16 subject matter experts are involved is 0.50. That means that if the number of subject matter experts in the panel is 16, for an item to be judged as having content validity at least 12 of the subject matter experts need to rate the item as acceptable. A summary of the CVR values is provided in Table 4-5. Overall, 66 items out of the 71 included in the content validity study met the criterion for content validity.

Table 4-5Content Validity Ratios (CVRs)

CVR Number of items 0.90-1.00 17 0.80-0.89 20 0.70-0.79 16 0.60-0.69 12 0.50-0.59 1 0.40-0.49a 2 0.30-0.39a 1 0.20-0.29a 1 0.10-0.19a 0 0.00-0.09a 1 a

Do not meet the criterion for content validity.

Five items, four from the incentive construct and one from the perceived security construct, did not meet the criterion for inclusion according to the CVR value. Table 4-6 presents the CVR value for each item in the constructs incentive and perceived security. After considering item and construct content, I decided to drop the item that did not meet the formal criterion for inclusion from the perceived security construct. However, the items for incentive that did not meet the formal criteria for inclusion were retained because, based on the understanding of the incentive construct gained from the literature, it was concluded that they are important.

Table 4-6Original Items for Incentive and Perceived Security

Construct Item CVR

Incentive I will receive financial incentives (e.g. higher bonus, higher salary) in return for my knowledge sharing.

0.07a

I will receive increased promotion opportunities in return for my knowledge sharing

0.38a

I will received increased job security in return for my knowledge sharing

0.25a

Knowledge sharing is built into and monitored within the appraisal system.

0.43a

Generally, individuals are rewarded for teamwork. 0.62

Perceived Security

I believe that the knowledge I share will not be modified by inappropriate parties.

0.87

I believe that the knowledge I share will only be accessed by authorized users.

0.88

I believe that the knowledge I share will be available to the right people.

0.88

I believe that people in my organisation do not use unauthorized knowledge.

0.73

I believe that people in my organisation use other’s knowledge appropriately.

1.00

I believe that KMS have mechanisms to avoid the loss of critical knowledge.

0.88

I believe that KMS have mechanisms to protect knowledge from being stolen.

0.47a

In my opinion, top management in my organisation is entirely committed to security.

0.87

Overall, I have confidence in knowledge sharing via KMS. 1.00

a

Items that did not meet the formal criterion for inclusion.