Reflections on the research methodology

5.3.1.1 Question 1: Appraisal of the SBV framework

Notwithstanding to the requirements of the experts, they had different backgrounds and different opinions. It was of interest that all these opinions were proportionally considered, since all experts developed their opinions from different perspectives that were relevant for this research. In future research, the discrepancies between experts would be used again. Perhaps these discrepancies would be tried to become greater, to expose more perspectives. By doing this, the findings of this study might become more well-founded. Furthermore, experts gave advices for modifications of the SBV framework. This study focused on experts’ perceptions, since these experts talked about the expected usefulness and this would enhance the practicality of the framework. If this study focused more on theory and less on experts’ perceptions, other modifications could be recommended.

Doing interviews meant that the experts needed to make some free time to be interviewed. This had influence on the selection of respondents, and therefore the findings, because some experts refused to co-operate, even if they wanted to or even if they were best selected, mostly because of lacking time. Besides, lacking time also influenced to which extent experts explored the SBV

framework before starting the interview. When a study like this would be executed again, more time should be reserved during interviews. This time should be used to let experts explore the SBV framework during interview. Extra time was required, since it was not feasible for all experts to prepare for the interview by exploring the SBV framework before the start of the interview.

5.3.1.2 Question 2: Current practices and perceptions

Also here, doing interviews meant that teachers should make time to be interviewed. Not all selected teachers could co-operate, which influenced participation of respondents, and therefore the findings of this study. In future research, more respondents should participate. Furthermore, more

information about the respondents should be asked, for example other work experiences, previous levels of students they taught, and which educations they have completed.

In future research the sequence of measurements should remain the same. By observing teachers first, the observation is a good starting point for the interview. During interview, fragments of the lesson of the observation were used to ask more and deeply about a specific subject. By using different methods to measure teachers’ use of 21st _{Century Skills, the findings of this study could be} seen as quite well-founded.

Respondents participated voluntary. This probably meant that only teachers who were interested in such a study, participated. Only a few respondents were labelled by their colleagues as less skilled with the use of ICT in their daily practice, but still these teachers needed to participate

voluntary. This could cause decreased variation in respondents, which could have influence on the findings.

5.3.1.3 Question 3: Characteristics of an application based on the SBV framework

The WegWijzer was planned to be available from January 2014. This study should use the WegWijzer to investigate if the use of the WegWijzer had influence on the practice and perceptions of teachers. Since the WegWijzer was launched in April 2014, since then named Doe IT, this study used a

prototype of the WegWijzer, so the study would not retard too much. The prototype contained about the same content, but the layout was different, which could influenced the improvements of the WegWijzer given by teachers.

However, the prototype of the WegWijzer was available from January 2014. In January and February, the second teacher phase was executed to investigate if and how teachers used the

WegWijzer, which meant that teachers could only use the prototype for one month before they were interviewed. This could affect the frequency as well as the practice of the prototype, and therefore the findings, since teachers needed time to habituate to use the prototype. Besides, it was of interest to consider a prototype was used while writing recommendations to improve an application, since the prototype could have other improvements than the final application. Since Doe IT was not yet available in January 2014, using a prototype was the best option to execute without wasting time. For future research, it will be better if the application should be available instead of a prototype. This could be reached by making a strict planning.

Furthermore, the WegWijzer was an example of an application of the SBV framework.

Improvements for the WegWijzer could be used as characteristics when making another application of the SBV framework, but the improvements were not focused on this eventual new application. Recommendations to the WegWijzer should be handled with care when applied to another application based on the SBV framework.

5.3.1.4 Question 4: Changed practices and perceptions

Questionnaire1 was filled in by 16 teachers. Questionnaire2 was filled in by only 13 teachers. Both numbers were quite low, which might have influenced the findings of this study. Besides, since questionnaire2 was filled in by less teachers, this also meant not all teachers filled in both

questionnaires. This could have caused a difference in results, especially while comparing

questionnaire1 with questionaire2. Furthermore, teachers filled in the questionnaire two times, once before and once after the use of the WegWijzer. While filling in questionnaire1, most teachers did not know about 21st _{Century Skills. During questionnaire}

2, teachers were assumed to know more about 21st _{Century Skills, since they have been interviewed on this subject and they could have seen 21}st Century Skills in the WegWijzer. Assuming teachers knew more about 21st _{Century Skills after} interview1 and the use of the WegWijzer, it was possible teachers scored themselves lower in questionnaire2 than in questionnaire1, since they knew better what 21st Century Skills were and how they could be implemented. Nevertheless, two questionnaires and two interviews should be

executed again when executing this study a second time. By executing both questionnaire and interview, an image of teachers was exposed from different perspectives. By executing these measurements two times, discrepancies between first and second time became clear.

However, when executing this study again, more contact information of the respondents should be gathered. Initially, the investigator had only contact with teachers by using e-mail. Using e- mail was sometimes hard, because respondents cannot be contacted in person and the investigator could not influence respondents to read their e-mail. When respondents did not read their e-mail, the investigator had no other options than waiting or sending an e-mail again. After round 1, one teacher did not respond at e-mail at all. Therefore, he did not participate round 2, which had influenced the findings.