Phase 2: Towards recommendations

The second phase of this research project was the phase which leaded towards

recommendations. In this phase an answer to research question 2 was given. Different activities were undertaken to answer research question 2: conclusions and recommendations

were written and a formative evaluation was undertaken by interviewing the lecturers of the two courses and the students.

3.2.1 Research question 2

In this phase research question 2 was answered:

2. How can the quality of the RMME and OTE courses of the LMEF Masters program be improved, by looking at the degree in which the courses can serve adults with several learning styles?

a. Which of the characteristics of adult learning styles can be improved in the RMME and OTE courses?

3.2.2 Activities

To answer the question above different activities were undertaken. Each activity will be explained, the selected respondents will be mentioned and the way of processing and analyzing the retrieved information will be explained.

Writing conclusions and recommendations

To answer research question 2 and the belonging sub-question, conclusion and

recommendations had to be written, based upon the results of the evaluation phase. The following reasons were used to decide when a recommendation was adopted:

- The suggested improvements of the AITEC questionnaire 2003 were not taken into consideration in the rest of the research, when the suggested improvements in the AITEC questionnaire were only mentioned in 2003 and did not come back in the questionnaire of 2004, the developed questionnaire or in the interviews with the students and the lecturer.

They were not relevant anymore and had most likely been improved in the last two years.

- A recommendation was adopted when according to the different activities undertaken, the education would be improved.

- A recommendation was also adopted when it was very important according to the

literature (see chapter 2), even if according to the interviews/developed questionnaire, the lecturer or most students did not like the recommendation. There was analysed how significant a comment of the student or lecturers was.

- Some recommendations were adjusted, based on the results of the formative evaluation of the recommendations. Recommendations were adjusted, when some words had to be added to the recommendation to make it more specific, so textual changes.

Recommendations also were adjusted when the idea behind the recommendation was good, but the recommendation was not usable the way it was, because it took too much time for the students or lecturer the way it was suggested.

- The actual evaluation criteria can be read in chapter 2.

In this phase conclusions and recommendations were written to ensure best teaching practice of the RMME and OTE course of the LMEF Masters. Recommendations were given

concerning the improvement of the degree in which the courses can serve adults with several learning styles. The written recommendations are combined with the results of the evaluation phase from Klink (2006).

The recommendations covered different topics:

- Interaction

- Interaction (teaching) methods - Feedback

- Assignments

- General recommendations Formative evaluation

After drawing conclusions and writing recommendations based upon the results of phase 1, a formative evaluation was undertaken with the lecturers and students to hear their opinion about the recommendations.

A formative evaluation is applied to materials that are being developed (Tessmer, 2001). In this research project, the developed materials are the recommendations about the degree in which the courses serve adults with several learning styles, combined with the

recommendations about the use of interaction methods in the blended learning environment that Klink (2006) had written.

Usually a formative evaluation is conducted with a small sample size (Worthen & Sanders, 1987). In this research project the formative evaluation was undertaken with 2 lecturers and 11 students (6 RMME: 5 face-to-face , 1 online student; and 5 OTE students: 2 face-to-face, 3 online students).

Interviews lecturers

In the second round of interviews a formative evaluation was undertaken to evaluate what the lecturers thought of the found conclusions and suggested improvements. The emphasis was – from their courses point of view – on the underlying idea if the suggested improvements for their course were practical. Did they have more suggestions for improvement concerning the way the courses served adults with several learning styles? After that round of interviews, the suggested improvements were adjusted one more time, to make sure it were practical

improvements that could be used in real time.

The RMME and OTE course had their own lecturer who was responsible for the education of that course. The two interviewed lecturers were the only lecturers teaching this course.

Interviews students

In the second round of interviews a formative evaluation was undertaken to evaluate what the students thought of the found conclusions and recommendations for improvement. The emphasis was – from the students point of view – on the underlying idea if the suggested improvements were practical. Did they have more suggestions for improvement concerning the way the courses served adults with several learning styles? After this round of interviews, the suggested improvements were adjusted one more time, to make sure it were practical improvements that could be used in real time.

The choice was made to also use the same students that were interviewed in the evaluation phase (n=8), because these students were more involved in the research project and that way they would give more detailed information. They knew what the research project was about and what the purpose of the project was.

To get more additional information about the students opinion of the recommendations, all students of the RMME and OTE course were sent an e-mail with the question to react on the recommendations. Even after sending a reminder, only two face-to-face students of the RMME course and one online students of the OTE course responded.

After the formative evaluation the conclusions and recommendations were adjusted.

Finally, an implementation plan was written based on the conclusions and recommendations to improve the degree to which the courses serve adults with several learning styles. This implementation plan gave an answer to the main question:

‘How can the ‘Research Methods in a Multidisciplinary Environment’ (RMME) and

‘Operational Test and Evaluation’ (OTE) courses of the LMEF Masters program ensure best teaching practice, so that several adult learning styles are addressed? ’. The implementation plan itself can be found in appendix 9.

Validity

For the formative evaluation of the recommendations, the same students were interviewed as during the interviews in the evaluation phase to answer research question one. To improve the internal validity, the recommendations were sent to all the students, with the request to respond to it within 3 weeks. This was done because the more responses received, the higher the internal validity. A high internal validity is important when one wants to be sure that the right meaning to the results was given and that the right conclusions were drawn (Van Berkel

& Starren, 1993). Even after sending a reminder, only two students of the RMME course and one student of the OTE course responded. One cannot speculate what the students who did not respond would say about the suggested recommendations.