Phase two: Item refinement

To assess the all initial items generated in phase one, step two, content validity and face validity procedures were employed. Content validity refers to the definition of constructs and is defined as the degree to which elements of an assessment instrument are relevant to and representative of the targeted construct for a particular assessment

167

(Haynes et al. 1995). Face validity is a component of content validity and refers to the degree that the items within an assessment instrument are appropriate to the targeted construct and assessment objectives (Haynes et al. 1995; Anastasi, 1988; Nevo, 1985). Face validity explores the readability of the items (Cavana et al. 2001). To assess the items in the survey the researcher undertook four stages as expert-judge of face validity, decision rules for removing and/or keeping preventative items, pretesting, and final questionnaire.

5.4.2.1.2.1. Step 1: Expert-judge of face validity

Face validity involves having the generated items assessed by expert judges for their content and face validity. In this study, the recommendations of Morgan et al. (2012), Ngo and O’Cass (2013), and Zaichkowsky (1984) were followed. Items were submitted to expert judges for content validation, feedback and item reduction if necessary. This process is based on the suggestion of Morgan et al. (2012) and Zaichkowsky (1984) and involved an initial subjective assessment by judges for deletion of unrepresentative items in line with the appropriate meaning of the terms, definitions and then a finer judging. The content and face validity of the items was examined by the experts in two ways through the item list. First for initial deletion of recognisably poor items and then again for more rigorous refinement of the items which remained as recommended by O’Cass and Siahtiri (2013), Heirati et al. (2013), Morgan et al. (2012), Ngo and O’Cass (2011), and Zaichkowsky (1984).

To examinee face validity, the researcher invited nine senior academics in the marketing and management disciplines who were provided with a set of instructions for judging and asked to evaluate the conceptual definition of the constructs with the corresponding items (Morgan et al. 2012). The senior academics were asked to rate each item as either: not representative, somewhat representative, or very representative of the construct’s definition (e.g., O’Cass and Siahtiri 2013; Heirati et al. 2013; Ngo and O’Cass 2013; Zaichkowsky 1985). Within the face validity stage, instead of giving the nine expert judges a lengthy draft survey of all items, three sub draft surveys were created from the initial draft survey and each sub draft survey was given to a group of three expert judges in the field. Totally, there were three groups each containing three expert judges corresponding to three sub-surveys.

168

5.4.2.1.2.2. Step 2: Decision rules for removing and/or keeping preventative items After expert judge’s evaluations on content validity, a specific decision rules should be considered to determine which items should be retained (or modified) in the survey (e.g., O’Cass and Siahtiri 2013; Heirati et al. 2013; Cillo et al. 2010). There are three decision rules to retain or remove the item(s) from the item pool (draft measures for surveys): (1) the sum score approach (the total score for an item across all), (2) the complete approach (the number of judges that rated an item as completely representative of the construct), and (3) the not representative decision rule (the number of judges indicating that the item was not representative of the construct of interest) (Ngo and O’Cass 2013; Hardesty and Bearden 2004).

Based on the recommendation of Ngo and O’Cass (2013) and Hardesty and Bearden (2004), after receiving feedback from expert judges, decisions about which items to delete or retain were made. The first decision was based on the sum score, the second decision was based on the complete approach, and the last decision made was based on not representative rule. Of the 89 initial items, 17 items were dropped based on the suggestions and comments from these expert judges. Consequently, 72 items were retained in the refined item pool, which are shown in Appendix II, III, and IV.

Furthermore, two more questions were designed to capture the knowledge level and confidence of respondents to answer the questions. This decision was made in line with the suggestion of Boso et al. (2013), Morgan et al. (2012), De Luca and Atuahene- Gima (2007) and Atuahene-Gima (2005) to ensure the integrity and reliability of the responses obtained. The respondents were first asked to evaluate the extent that they are knowledgeable about their firms’ business operations, characteristics, business processes, performance and business environment (at the beginning of the questionnaire). Second, they were asked to identify their confidence in possessing the necessary knowledge to complete the statements asked throughout the questionnaire (at the end of the questionnaire) using a seven-point Likert scale with scale pole Strongly Disagree/Strongly Agree. Adopting this procedure requires a judgment about keeping and removing respondents and it was decided that any respondents who answered below 4 on the two questions were to be dropped from the study (Bose et al. 2013; Morgan et al. 2012).

Further, nine firmographics and demographic items and two marker variables to control for common method variance were added to the draft pool of item. The demographic items (1-9) and marker variables (10-11) are:

1. Company age; 2. Company size;

169 4. Service sectors

5. Age of respondents; 6. Gender of respondents;

7. Level of graduation of respondent;

8. The position of respondent and the year of experience in the same position; 9. Previous position and years of experience in the position.

10. I like the company Microsoft. 11. My life is enjoyable.

Having chosen the scale poles and measurement items for all constructs, the physical layout of the survey becomes a critical component in the design stage (Ekerljung et al. 2013). The layout is argued to directly affect the appeal and ease of administration of the survey (Ekerljung et al. 2013; Toepoel and Dillman 2011; Fuchs 2009; Aaker et al. 2004). As such, issues involving opening instructions and question sequence were addressed at this stage (Podsakof et al. 2003). To minimise possible errors and biases, every attempt was made to ensure that the instructions were clear and simply stated when developing the draft survey for pilot testing. Demographic questions were placed at the end of the survey (Burns and Bush, 1995). The final three surveys are presented in Appendix II, III, and IV.

Table 5.5

Refined item pool and demographic items

Constructs Number of Items

Service solution 9

CCSP 6

Employee brand building behaviour 6 Brand specific transformational

leadership 15

Deep and broad customer knowledge 7 Deep and broad technical knowledge 4 Knowledge assimilation 6 Customer based brand equity 5

Market effectiveness 4

Profitability 4

Environment turbulent 6

Demographic 9

Marker variable 2

Respondent’s knowledge & confidence 2

170

5.4.2.1.2.3. Step 3: Pretesting

After undertaking the face validity assessment, a pilot test was conducted. The main reason to conduct the pilot test was to identify any problems in readability and understandability of items in the survey (Aaker et al. 2005). Pilot testing can be implemented using two different approaches: qualitative pretesting (Churchill 1979; Spector 1992) or quantitative pretesting (Presser et al. 2004). As the quantitative pilot testing is more difficult in the context of B2B due to the finite number of respondents (Cavana et al. 2001), this research adopted a qualitative pilot test approach. Further, the employment of the qualitative pilot test has been widely used by researchers in the marketing and management literature (see, O’Cass and Ngo 2011 and Kohli et al. 1993). To pilot test the survey, a set of interviews were conducted with senior managers within B2B PSFs, which are the target respondent for this study. Senior managers of 10 PSFs in Taiwan were invited to take part in the pilot-test. Their contact details were obtained from the list provided by Ministry Economic Affair, R.O.C. in Taiwan.

Senior managers were initially contacted via phone. A detailed explanation of the purpose of the study was given to them. Upon their consent to participate in the pilot- testing, in-depth interviews were conducted with the ten senior managers. In the pilot testing phase step, the interviewees were asked to explain why they responded the way they did on each item or there is any other way to interpret the question. Further, interviewees were asked to provide feedback on question sequence, items duplications, and any other points of concern with the survey instrument that respondents had. The pilot test revealed no particular problems with the survey’s terminology, clarity of instructions, or response formats, showing acceptable face validity (Cavana et al. 2001).

However, the feedback obtained from the pilot-testing demonstrates a good readability of the questionnaire and did not result in any reduction in questions and items in the survey.

5.4.2.1.2.4. Step 4: Final surveys

As shown in Figure 5.3, the last phase of measurement development is finalising surveys. The information presented in Table 5.5 shows 85 items were included in the draft item pool. As discussed in Section 5.2 and shown in Table 5.1 this study adopts a multiple informant design. This approach was adopted in accordance with the work of Zhou and Li (2012), Arnold et al. (2011), Slotegraaf and Atuahene-Gima (2011), Vorhies et al. (2011), and De Luca and Atuahene-Gima (2007) aiming to decrease the effect of common method bias. Further, it is acknowledged that multiple-informant design (e.g., data from three hierarchical levels) provides high quality data, with less bias problems

171

than single-informant designs (Damanpour et al. 2009). Following the process outlined by a number of researchers, three surveys were developed for three different management positions in PSFs (labelled as Survey A, B, and C for three different positions in each PSF).

The respondents to Surveys A, B and C were one senior (CEO or equivalent) and two mid-level mangers’ in different organisational positions within each PSFs. The use of managerial perception has been extensively adopted within marketing and management research (e.g., O’Cass and Sok 2013; Morgan et al. 2012; Morgan et al. 2009; Vorhies et al. 2009; Newbert 2008), because they are in a good position to respond to measures pertaining to firm routines and firm performance. It is also argued that managerial perception is appropriate and yields reliable information (Morgan et al. 2009; Ngo and O’Cass 2009; Newbert 2008; Vorhies and Morgan 2005). This view is also held as there are significant constraints in obtaining objective data because of confidentiality (Ward et al. 1996; Blindenbach-Driessen et al. 2010). Further, research has shown that there is a high correlation between objective performance indicators and subjective performance items used in performance measurement (Morgan et al. 2012), which validate the application of subjective data.

Survey A was completed by CEOs, who answered questions related to the knowledge assimilation, employee brand building behaviour, and environmental turbulence. The reason for the allocation of knowledge assimilation to CEOs is based on the suggestion of Zhou and Li (2012) who believe CEOs are in a better position to answer questions about knowledge assimilation. The reason for allocating employee brand building behaviour to CEOs is based on the work of Liao and Chuang (2007) who believe leaders are in the better position to answer questions about employee behaviour. The CEOs should also know the level of changes in the environment, therefore; they are considered as suitable respondent to answer the items related to environmental turbulence.

Survey B was completed by marketing managers or sales managers, who answered questions related to the deep and broad knowledge (customer and technical), CCSP, service solution, market effectiveness and profitability. Marketing managers or sales managers were deemed suitable position in the PSF to answer these questions. This assumption has been built on the work of Vorhies et al. (2011), because marketing managers or sales managers should have appropriate knowledge of customer and technical knowledge to offer service solutions. Further, as they design marketing activities they are in a suitable position to judge performance in the market and the results obtained from their marketing activities (Vorhies et al. 2011).

172

Survey C was completed by customer service managers or customer relationship managers who answered questions related to the customer commitment and brand specific transformational leadership. The underlying reason for asking questions about customer commitment is that customer service managers or customer relationship managers are more likely to be in contact with customers and more aware of relationships with customers (Ernst et al. 2011). Further, they are considered more suitable to evaluate their supervisors’ transformational behaviour since subordinates are the target of the leader’s influence and are thus most likely to observe their behaviour (Cho et al. 2011). If there were no such positions in a PSF, the surveys were directed to managers in equivalent positions based on the instruction provided to the CEO, as the CEO was responsible for introducing the other two managers. The three surveys (A, B and C) are provided in Appendix II, III, and IV.

Survey translation. As the data were to be collected in Taiwan, it was necessary to translate all surveys. Following Slotegraaf and Atuahene-Gima (2011) the double- translation method was used to translate the survey from English to Mandarin. Following this approach, the survey was first prepared in English and then translated into Chinese by a certified translator and then back into English by another certified translator to evaluate the translation accuracy (see also O’Cass et al. 2014; Tang et al. 2008; De Luca and Atuahene-Gima 2007). Any conflicts were discussed by the researcher and translators until agreement was reached (O’Cass and Sok 2013 and 2011; Lee and Zhou 2012; Zhou et al. 2008).