Of the three schools the EduVentures staff presented to in the pre-visits, Okondjatu Combined School (CS) was the best candidate to pilot the modules. This is because Okondjatu CS had been contacting EduVentures for over a year, expressing their interest in the program. After developing the modules and selecting the school, the next step was to implement the program in the Ombombo mobile classroom. The team departed with EduVentures to test the modules during the seventh week in Namibia. The EduVentures educators taught all four modules to a pilot group of learners from Okondjatu Combined School. The field test followed the schedule shown in Appendix KK. Day One, the EduVentures staff taught the Introduction to
Renewable Energy and Wind Energy Modules to the 20 learners in the program. On day Two they taught the Solar Energy and Bush-to-Energy Modules. Day Three, consisted of the learners going out into the village and
learning about bush encroachment and beginning work on projects. The assigned project was a debate between the classroom to argue for or against allowing bush encroachment to continue in Namibia. The team for bush encroachment used arguments such as bushes act as carbon sinks, they provide food and shelter for animals, and they prevent soil erosion. The team against bush encroachment argued that bushes reduce biodiversity, they are a good source for renewable energy, and you can use bushes as animal feed. On the fourth day, the project team and the EduVentures staff assisted the learners with their projects. Finally, the learners debated in front of the administration of Okondjatu Combined School on Day Five. Although this project’s work focused on Days One and Two, the project team assisted the EduVentures staff in completing the rest of the five-day program.
48
3.5.1 Pre-Testing and Post-Testing
To collect data on learner retention rates, the educators distributed a pre-test at the beginning of the first module and a post-test at the end of the fourth module (see Appendices LL and SS). These
anonymous tests depicted the learner’s growth throughout the program. The educators asked each learner to put their name on the top. The team assigned each learner a number based on the name at the top of the pre-test, then wrote their name and assigned number on the post-test. The purpose of this was to ensure that only the learners who took the pre-test would take post-test. To save anonymity, the researchers did not include any names in this report, simply the assigned numbers. The questions on the pre-test and post-test were the same, however the questions were in a different order. The investigators collected all of the tests and corrected each test marking the answers correct or incorrect. The use of this test allowed the study to observe trends in learner retention of the material presented and the ability of the learners to apply their new knowledge to real world situations.
The questions created were an assortment of rhetorical, guiding and essential questions to test the range of thinking form the learner. Section 2.3.6 provides the definitions of these questions. For simplicity, the team will refer to rhetorical questions as concrete questions throughout the rest of this report. This is because the definition provided by McTighe and Wiggins differs from the more commonly known definition of a rhetorical question. One of the project team members created the Staircase to Inquiry, shown in Figure 3.9, from information in Essential Questions: Opening Doors to Student Understanding ((McTighe & Wiggins, 2013).
Figure 3.9: The staircase to inquiry
The bottom stair is concrete questions. Learners were more likely to answer concrete questions because that information is easily accessible. These questions demonstrate a learner’s ability to recollect specific information but shows no further inquiry into the topic (McTighe & Wiggins, 2013). Although these questions show no further inquiry, they do show what the learners already know about renewable energy. Concrete questions are the most basic questions, but provide good insight into the educational foundation or background of the learner. The middle stair is guiding questions. Guiding questions challenge the learners to come to an answer by recalling some information and also making inferences (McTighe & Wiggins, 2013). The middle stair is important for the teacher to get the learners thinking more about the bigger picture. The top stair is essential questions. Essential questions stimulate inquiry and ongoing thinking about a certain topic (McTighe & Wiggins, 2013). The hope of the modules is to get learners thinking more about renewable energy and climate change in Namibia. The team used the Staircase to Inquiry to modify the four modules
Concrete
questions
Guiding
questions
Essential
questions
49
accordingly, based on conclusions drawn from the the pre-visit data. The wind module, for example, included several different levels of questions to stimulate learner inquiry:
1. Concrete Question: What is wind energy? 2. Guiding Question: How does wind energy work?
3. Essential Question: Why should Namibia use wind energy?
In the pre-test and post-test, there are four concrete questions, three guiding questions, and two essential questions. Table 3.4 shows the questions in the pre-tests and post-tests. The type of question is denoted in the colors listed above in Figure 3.9, light green is concrete, green is guiding, and teal is essential. Table 3.4: List of questions of pre-test and post-test
Pre-Test Question #
Question Post-Test
Question #
1 What is renewable energy? 9
2 How do encroacher bushes affect the environment? 2
3 What is wind energy? 3
4 Why is solar energy good to use in villages? 8
5 What is solar energy? 4
6 What is encroachment bush? 6
7 Why are wind turbines best for coastal regions? 5 8 Why is renewable energy important to Namibia? 7 9 How can you help battle the effects of climate change? 1
The researchers assessed these tests in the same way as the pre-visit surveys but included side-by- side comparisons of the pre-tests and post-tests. The team then analyzed the tests based on the learners’ performances on the three types of questions, the classroom’s performance on each question, and overall improvement.
3.5.2 Module Observation Form
The team decided to use an adjusted observation form during field test evaluation, as shown in Appendix NN. The new form targeted learner engagement attitudes and behaviors only, while the previous classroom observations recorded both teaching styles and learner engagement. For the field test of the modules at Okondjatu Combined School, the module observers looked for similar behaviors in the form: asking questions, talking in side conversations, etc. However, the designer of the observation form also included learner interaction with the SMART Board. It is important that a variety of learners are utilizing the technology and actively participating in the lessons. Two team members were each responsible for counting the behaviors for half of the learners in the mobile classroom, both of which were not the designer of the module. These two members sat at the front of the mobile classroom on either side of the SMART board. While the modules observations occurred, the other two team members were analyzing data. The investigators used these module observations to gain insight on the level of engagement throughout the entirety of the lesson.
50
Design
The module observation form was a modification of the version of the learner observation form mentioned above in Section 3.2., seen in Appendix O. This form tallied the number of learners performing various positive (shown in blue) and negative behaviors (shown in red) throughout the entirety of the module. The observer, at each five-minute mark throughout the lesson, counted the number of learners performing the following behaviors:
Sitting up and listening
Maintaining eye contact with the teacher Smiling
Expressing confusion or anger Talking in side conversations
During the five-minute time periods, the team members observing the classroom counted the number of learners exhibiting the following behaviors:
Asking questions Answering questions
Coming up to the SMART board to participate Looking out the window
Not fully participating in group work
The module observations provided information regarding learner engagement throughout the lesson. The observation form allowed for the identification of successful activities in each module.
Assessment
Throughout the lesson, the team members counted the number of learners performing each action. However, because each team member was responsible for half of the class; after the observation concluded the investigators entered the data into excel sheets individually as well as coalesced their observations together to have data representing the entire classroom. The next step was to compile the number of learners performing positive behaviors and then negative behaviors. Then the team divided each compilation by the number of learners present to obtain percentages of learners performing positive and negative learner behaviors. To display the percentage of learners performing each action over time, the research group graphed learner behaviors over the module time period. This study then looked at learner engagement and matched it with specific sections of the modules. For example, if 96% of the class was sitting up and listening, the team referred to the lesson and check to see what part of the module was causing such a high
engagement rate.
3.5.3 Learner Evaluation
In order to gather information on the learner experience throughout EduVentures’ program, the team developed a learner evaluation form located in Appendix UU). The learners ranked a variety of statements (Questions 1-8), on a scale of one to five, and answered questions based on their experiences during the modules (Questions 9-10). Table 3.5 shows the questions from the learner evaluation.
51
Table 3.5: Learner evaluation questions
Question #
Question
1 The information throughout the entire program was easy to understand.
2 I think the games and activities in the lessons were fun.
3 The lessons were more fun than my classes at school.
4 I would want to participate in another EduVentures program
5 I feel like I know much more about renewable energy now than before I started this program.
6 If my school started an environmental club, I would join it.
7 I plan to learn more about renewable energy after the end of this program.
8 What I learned in the program will change the way I think about energy generation.
9 What was the most interesting topic to learn about?
10 What was the least interesting topic to learn about?
For each statement, the learners ranked their responses from strongly disagree (1) to strongly agree (5). The learners were not able to answer with the number three, which would be the neutral or no opinion category. As the team wished to know what the learners liked and disliked, the neutral option was
intentionally left out so the learners had to state whether they liked or disliked something, and the extent to which they felt this way. Figure 3.10 shows an example of the learner evaluation. From this survey, the researchers hoped to better understand which aspects of the program the learners liked and which aspects they think should change. The researcher group used this information to modify the lesson modules after the conclusion of the program.
Figure 3.10: An example of a partially completed learner evaluation
3.5.4 Module Modification
The team used the information obtained from module observations, pre-tests and post-tests, and learner evaluations to determine modifications to the modules. From each form of data collection, the team members evaluated the important positive and negative aspects and used these to change the modules accordingly.
From module observations, the investigators identified when the learners were most engaged in the class and the times at which they were most distracted. When anything other than lecturing occurred, the observer noted what activity was occurring, whether it was group work, a SMART activity or some other form of teaching. Any additional comments made about both the successful and unsuccessful aspects of each lesson were taken into consideration. The researchers then correlated this information with learner
52
engagement to determine which activities worked and which did not. The team also used this information to gauge how long the lessons should be to maintain learner engagement.
From the pre-tests and post-tests, the team evaluated the knowledge the learners gained from the four modules. As discussed in Section 3.5.1, the questions remained the same in both tests, but the order in which the questions appear on the form changed. This was so that the learners did not memorize the order of the form. If there was one subject which the learners were still not performing well on, the module designers took this into consideration and modified the module related to those questions accordingly.
From the learner evaluation, the team was able to gauge learners likes and dislikes throughout the program. From their rankings, the analysts calculated an average for each question and used this information to modify their modules accordingly.
At the conclusion of the second day of the program, the team had the information needed to make the necessary module modifications and, due to time constraints, began the modification process while the EduVentures’ Program was still ongoing. This was so that the designers could deliver four final, modified versions of the Renewable Energy modules to EduVentures prior to their departure from Namibia.
3.5.5 Curriculum Development for the Ministry of Education
Upon the completion of the modifications to the modules, the team developed an in-depth summary of each lesson in the form of a complete curriculum for the EduVentures’ program, shown in Appendix BBB. The designers based their curriculum off the 2015 WPI EduVentures team, by placing relevant information about each module into it. This curriculum will provide the Ministry of Education with a summary of the information discussed in each of the four modules. This was so that the Ministry of Education could ensure the information EduVentures’ educators presented to the learners throughout the five-day program was relevant to the topics of the renewable energy program. By filling out this curriculum for the EduVentures staff, the team believes that this is the only information that is required for approval by the Ministry of Education.
The team also provided EduVentures with their final presentation (see Appendix CCC), so that they could present the results of the field test to the Ministry of Education. The team anticipated that the EduVentures staff would present to the Ministry of Education after their departure from Namibia, and that EduVentures would deliver the detailed, written curriculum to Ministry members in attendance. By gaining approval from the Ministry of Education for this new curriculum, EduVentures will be able to bring these modules throughout Namibia. This will allow EduVentures continue their work of bridging the gap between rural and urban education and spreading Renewable Energy awareness throughout Namibia.
53
4 Results and Analysis
This study had two main objectives: to develop four interactive SMART modules and assess the effectiveness of the modules during a field test at Okondjatu Combined School. The data collected prior to the implementation of the modules included classroom observations, stakeholder interviews, and the school pre-visit surveys. During the pilot testing of the modules, the team collected information from module observations, pre-tests and post-tests, and a learner evaluation. This collected information enabled the team to analyze the effectiveness of the lessons and then modify the modules in particular areas, if necessary. This chapter will discuss how these preliminary data collection methods influenced the development of the modules and how the pilot testing data collection influenced the final modifications of the modules.