Data-Collection Methods

Three methods of data collection were used to conduct this research project: surveys, focus groups, and interviews (see Figure 2). Surveys are often used in education research to describe trends and identify attitudes (Bryman, 2012; Creswell, 2008). The surveys for this project were designed to gauge student engagement with utilising the on-demand resources, and to further the development of the delivery methods and content. They were created by myself as the researcher and evaluated by my academic supervisors to increase reliability. They focused on how the students utilised the resources and on what improvements could be made to the structure of on-demand resources to maximise student engagement. These surveys used Likert scales to measure attitudinal responses, and included some open questions regarding the resource project since open-ended questions allow researchers to let the participant guide the conversation (Bryman, 2012; Creswell, 2008). In total, three different surveys were conducted with a range of students from across the cohort during the project.

Two surveys were conducted during the pilot study, gathering background information and student impressions on how they saw on-demand education. These surveys were used to

On-Demand Learning Resources IBOOK Downloads Blackboard You Tube Facebook Page Studio Computers

investigate the idea that they would utilise this kind of initiative. The first survey was conducted at the commencement of the pilot study. It was created by myself as the

researcher, and asked ten questions to develop some background knowledge of student year level, previous experience with music technology, technical issues they had faced and other relevant background information. The second survey was conducted at the end of the pilot study and asked seven questions. The aim of this survey was to identify student awareness of the on-demand resources, types of information accessed and delivery formats. The first survey was conducted on-line through the learning management system Blackboard. It was open to students from all levels of the program, who were free to participate or decline participation. The second survey was conducted in paper forms. It was also created by myself as the researcher and was open to students within the program and distributed during the assessment period at the end of the semester. Students were given the option to

participate if they were interested. Both of these surveys used yes/no responses and Likert scales.

The third survey was conducted at the conclusion of the project with students from across the program. It was done as a paper based survey and students were all offered the opportunity to participate if they were interested in offering an opinion on the on-demand resources. This survey used five questions in total. Two questions were open questions, two were likert scales and two yes/no questions.

Data collected from the surveys were analysed using both bivariate and univariate data analysis techniques where appropriate. Univariate analysis presented an overview of the central tendencies, dispersion, and frequency distribution related to student usage of the resources, and student attitudes to how the on-demand resources were structured.

Information related to these research questions was also analysed using bivariate analysis, with the results analysed to identify relevant trends. This analysis explained relationships between the student’s year level and their attitude regarding on-demand resource delivery.

Focus groups were also conducted during the research project. The first focus group, conducted during the pilot study, was convened via social media in a semi-structured way

through the creation of a Facebook group. This focus group was asynchronous and conducted over a period of two weeks, with students commenting when they had the opportunity to engage with the conversation. Social media is being used increasingly to connect with students and create a sense of community by higher education institutions (Davis III, Deil-Amen, Rios-Aguilar, & Gonzalez Canche, 2012). This Facebook group was used to facilitate an informal discussion about the project, with questions posted and 10 participants then asked to contribute their opinions on the subject. The use of a focus group allowed the participants to drive the discussion using both structured and semi-structured questions, with the semi-structured questions allowing the participants more flexibility to guide the topics.

There is a high level of interaction between alumni, current students, and staff within the program through social media, making it a fertile space to engage the participants in discussion surrounding the project. Considerations regarding the sample of students invited to participate in the focus group interviews were relevant to ensuring the credibility of the research. This meant that having representatives from the various year levels of the program was a fundamental consideration, because design considerations could differ based on the specific needs of each group. One method that is commonly used in qualitative research for selecting research samples is purposive sampling. This sampling technique allows the researcher to choose participants with relevance to the research questions to be answered, while ensuring that key characteristics are varied enough to gain different perspectives (Bryman, 2012; Creswell, 2008). This research project required an even spread of participants, with specific characteristics related to the cohort. Because of the time and budget restraints of the research project, a purposive sample of 10 students were asked to take part in the focus group. Representative characteristics of all of the recognised groups were considered while choosing the potential pool of participants. This method of sampling aligns strongly with the aims of the research, because it delivered a sample from a cross section of perspectives. For the purpose of this study an even representation of students from each year level of the BPM program were invited to participate in the research. The pilot

study focus group was conducted at the conclusion of that stage of the project to assist with the development of initial design principles for delivering these resources. Ethics approval was sought and granted to conduct the focus group through the Facebook site. Participants were free to withdraw from the focus group at any time and no participants were directly identified during the research, as it was conducted as a closed group, with only participants and researchers involved. The data collected from this focus group was collated as text and thematically analysed to identify recurring themes.

Other focus groups were conducted throughout the research project. A focus group was conducted to evaluate the initial design of the BPM Studios mobile phone application. This focus group provided feedback on the design elements of the application before it was released to the student cohort. This focus group was open to all interested students within the program, and was conducted face-to-face with the four participants who volunteered to participate. The audio recordings of the focus group were transcribed and thematically analysed. A final focus group was conducted at the conclusion of the project. This focus group discussed usage of the mobile application after it had been in use by the student cohort for one teaching period. This final focus group was conducted with four students in face-to- face mode and the audio recording transcribed and thematically analysed.

This research used thematic analysis as a methodological basis for the analysis of the qualitative data collected through the online focus groups. Thematic analysis is a framework for the analysis of qualitative data through the identification of themes and sub themes emerging from research (Bryman, 2012; Denzin & Lincoln, 2005; Gay & Airasian, 2003). In this case the thematic analysis was conducted of the group conversations gathered through Facebook. The analysis then developed a matrix through the identification of core recurring themes occurring during the focus groups. These data were then used to represent the sub themes emerging from the research. The thematic analysis represented a detailed student perspective allowing for the development of design principles for delivering on-demand resources.

Interviews were conducted with eight Queensland Conservatorium staff members to gain their perspective of on-demand education, and how they saw this area emerging. Interviews are a rich source of data for qualitative research (Bryman, 2012; Creswell, 2008). These interviews specifically sought staff members’ perspectives on the effectiveness and role of on-demand education, the design principles they felt should be incorporated in this area, and the characteristics and technology education requirements of popular music students. The interviews were conducted towards the end of the pilot study to assist with the further development of the resources and to provide a background perspective on the student cohort. These interviews were conducted in person, with the interview recorded for the purpose of later transcription. The researcher was a colleague of the academic staff

members. Ethics approval was sought for the interviews, with the anonymity or participants provided and informed consent granted by the participants. Coding and thematic analysis are common methods for analysing interview data (Bryman, 2012; Creswell, 2008; Denzin & Lincoln, 2005) and all of the interviews conducted with staff were coded using thematic analysis to identify recurring themes. This was used to lead to the development of the design principles for delivering and creating on-demand resources.

Figure 2. The research methods employed across three iterations of the study. RESEARCH ITERATION 1: PILOT STUDY

• Student Focus Group • Pre-Delivery Student Survey • Post Delivery Student Survey • Academic Staff Interviews

RESEARCH ITERATION 2: MOBILE APPLICATION DEVELOPMENT

• Student Focus Group

RESEARCH ITERATION 3: MOBILE APPLICATION EVALUATION

• Student Survey • Student Focus Group