Quantitative data collection

Guided by the CIPP evaluation model and the research questions, a quantitative questionnaire (see Appendix B for details) was developed to measure the extent of transformational leadership development, the development of leadership skills and capabilities, satisfaction with the program design and the training, demographics and other program design measures, such as the derived outcomes and impacts. The questionnaire also contained open-ended questions to map the progress of program’s impacts measured during Phase I, which included individual, workplace, tourism destination and community outcomes and impacts.

The CIPP model has been developed by David Stufflebeam (Stufflebeam 2005; 2014) and is a widely used model in program evaluations due to its established credibility and validity, developed over the years in the education field and the health sector. According to Stufflebeam and Coryn (2014), program effectiveness could be assessed through four non- linear evaluations. These evaluations include Context evaluation, Input evaluation, Process evaluation and Product evaluation. Each evaluation focuses on a different aspect of a program, and each evaluation is able to inform all other evaluations. Thus, in this study, each evaluation stage provided a focus on the type of data that needed to be collected at each stage. Through the ‘Context evaluation’ stage, the researcher collected data which were needed to assess the MTLP program’s ability to meet the needs of the program participants, and the

tourism leadership needs established through the literature, Phase I and also the views derived from the program’s stakeholders.

The ‘Input evaluation’ data collection was aligned to program design, and specifically the program content. Measures were established to assess whether the program content was relevant and had addressed the program participant’s developmental needs. Data relating to leadership skills and capabilities, knowledge, and leadership behaviours were the focus of this evaluation and data collection stage.

During the ‘Process evaluation’ stage, the researcher focused on the collection of data relating to program training approach to understand how the program enabled the program participants to learn and to transfer their learning into their contexts, such as their workplace, the tourism destination, and their community.

The final ‘Product evaluation’ stage focused on the collection of data pertaining to the program outcomes. Indicatively, it assessed the level and extent of outcomes derived from the MTLP participant’s learning. The outcome category levels included individual, workplace, tourism destination and community level outcomes, and were measured through seven levels of program outcomes proposed by Armstrong (1996).

In regard to the leadership development measures within the questionnaire, several leadership theories have guided the questionnaire design. The literature review (Chapter 2) uncovered that within the theory of leadership styles and the tourism context, transformational leadership theory is the most suitable style to consider. Specifically, the transformational leadership model and assessment tool of Kouzes and Posner (2007) that is used to measure the extent of transformational leadership was found to be suitable. The LPI is one of the most widely used leadership assessment instruments in the leadership practice and the research field today (Kouzes et al. 2010).

The transformational leadership model developed by Kouzes and Posner (2002) was used to collect data to understand how the program participants are currently able to apply the transformational leadership in their leadership roles. This leadership model consists of five behavioural practices: Model the Way, Inspire a Shared Vision, Challenge the Process, Enable Others to Act, and Encourage the Heart. Through the Model the Way practice,

leader’s credibility is assessed. It is seen as the most important personal quality that other people look for in a leader. The practice also assesses the leader’s awareness of their own values and whether they act consistently with these values. Through Inspire the Way, the assessment is focused on leader’s ability to enlist others in a shared vision by encouraging them to work together towards a common goal. To do so, leaders need to have a clear picture of needed organisational developments that are possible within their organisations. In Challenging the Process, the leader assessment is focused on the ability of the leader to look for opportunities and innovative ways to change, grow and improve. To do so, they need to be able to experiment, take risk and continuously learn from mistakes. Enable Others to Act assesses leader’s ability to foster collaboration by supporting cooperative goals and building trust. To do so, leaders need to be able to strengthen others by sharing power and decision making. In Encourage the Heart, it is assessed whether a leader is able to recognise individual and exceptional contributions, celebrate results and successes and thus build a strong sense of collective identity and team spirit (Kouzes et al. 2010).

Theories relating to leadership skills and competencies were also reviewed (Chapter 2) and in line with the tourism context and the findings in Phase I, a list of skills and competencies was created to form the assessment base in this study. A total of 22 leadership skills and competencies were identified and assessed in this study. To assess whether and to what extent these leadership skills and competencies were perceived as important by the program participants, and to also assess to what extent the participants had been able to develop in these areas since the completion of MTLP, the Importance-Performance tool was used to perform the analysis.

The Importance-Performance Analysis was developed by Martilla and James (1977) in the field of marketing, and has been since used in other fields, such as education, program design and evaluation studies, and student evaluation of teaching (Anderson et al. 2016; Huybers 2014). These studies were used to guide the current IPA analysis.

In this study, the 22 leadership skills and capabilities were assessed to explore what skills and capabilities are perceived as important from the perspective of the program participants, who are the current leaders within Victoria’s visitor economy. These skills and capabilities were rated on a five-point Likert scale, ranging from ‘not very important’ (1) to ‘very important’ (5). The same skills and capabilities were then measured for participants’ perceived

performance. Based on this analysis, it was possible to identify skills and capabilities that were performed well, and which needed further attention. Thus, leadership effectiveness was established in terms of the fit between perceived importance and performance. The smaller the gap, the better the effectiveness.