Feature design

Course wide lists

The lists varied among rooms in the same course. This resulted in some rooms who did help students that had questions of week three, while in other rooms students were only helped if they had a question about week four or later weeks. During the meetings of the TAs and the teachers of the course they determined which tutorial weeks would be covered that week, and what weeks would not be helped anymore, because there was not enough time anymore, or the deadline for those weeks was already passed. Despite agreeing with each other which weeks would be covered, some rooms would still create lists that were in contrast with the agreements. This was considered unfair, because the students would not be given the same chances in the same course, depending on the room in which they were scheduled. This resulted in the design choice to create course wide lists. While setting up the course the course wide lists could be specified. The two generic subjects; Questions and Sign off, were given as standards, however the teacher could also choose to make course wide lists with different titles. This feature was not designed for the purpose of improving the efficiency or quality of help seeking and giving. However, this feature does satisfy an important user need, namely the system being fair.

Categories

The first feature that was designed to improve the quality of help seeking is the category feature. The function of the categories is to help the students think about the type of question they have. When a students want to be added to a list they first need to choose the category that matches their help request. This engages the prior knowledge. As was mentioned in section 2.2.1engaging the prior knowledge prepares the brain to receive help better. For instance when a student wants to be added to the question list, they click the ”add me to list” button, then they need to select the exercise they were working on and subsequently choose the concept that matches their search request. By thinking about what concept matches their help request the students engage their prior knowledge. When a student wants to be entered in the sign off list, they click ”add me to list” and choose what week they want to sign off. The course manager is responsible for making the categories when they set up the course in TA-HelpMe.

Hypothesis 1: By selecting the category of their help request, students will take more time to think about what kind of help they need, resulting in an improvement in the quality of help seeking.

The categories made the course wide lists possible, students could now specify their list entry with a category instead of the teaching staff making a list for each category. The TA now no longer needs the list that specifies ”Sign off before week 4” they can see in which week the entry belongs.

An additional benefit is that the teaching staff can prepare the category before they go to the student to help with the help request. The preparing can vary from reading the exercise the student student has mentioned in their entry, to choosing to let a different TA handle the topic. This way more experienced TAs can handle the harder topics, and new TAs can help the basic topics.

Hypothesis 2: By reading the category before going to the help request the teaching staff can prepare for the help request, this leads to a higher level of help.

Hypothesis 3: By distributing the experience over the categories, the teach- ing staff can increase the self reported efficiency of help.

Inspired by Eurekos and the Paper cup technology the chosen categories of students will be visible in every view of the room. Eurekos and the Paper cups shared the progress and the struggles the students had and that created a sense of community. By sharing the category of the entry students can see that other students struggle with the same question. If the weeks are a category the students can see if they are on schedule compared to other students, this sense of community might motivate students.

Typing the question

Frequently asked questions The feature to type out questions, was designed due to the results of the interviews with the teachers. The question ”What happens when several students have the same question?” was leading for the design of this feature. One of the teachers mentioned that he gave the explanation over and over again on purpose. This was a really experienced teacher who had a good reason for doing so, namely the students do not work at the same pace. Some students are more experienced in programming and will be at the more advanced questions, the teacher mentioned that he did not want to distract and bore these students with a general explanation of a frequently asked questions. Other students that he did not want to disturb, were the ”lagging” students who where not yet at the frequently asked question. He did not want to discourage the ”lagging” students by giving an explanation to a question that they had not yet read. He also mentioned the worst case scenario; the students who were working on the frequently asked question, but

3.7. REALIZATION PHASE 77

had a different approach than the teacher. According to the teacher these students would stop and listen to the general explanation, then doubt their own approach, and eventually stop thinking and take on the approach of the teacher. This was the worst case in the eyes of the teacher, because programming can be done in many ways, and the solutions can be found via various approaches. When students stop their approach, they will not trust their own insights the next time.

Learn from each others question In order to make the helping of frequently asked questions more efficient, the following feature was designed: group-help func- tion. This function will be explained in the next section. In order for this function to work the students needed to enter their question into the online application. In the course setup the teacher could choose if a textual entry was mandatory for a list. The students would start by clicking on the button to add themselves to the list, then they needed to specify the category of their entry, and eventually they had to type out their question or choose from the questions of other students. Several teach- ers mentioned that inexperienced students lacked the skill to formulate what their question was. TAs also mentioned in the focus group interviews that student would raise their hand and not know what was wrong. To stimulate the formulation of ques- tions, and to be able to pick the same question as another student, students should be able to see the questions of other students. That students can learn from each others questions and strategies is supported by several studies [1], [11], [12].

Hypothesis 4: Letting students read the questions other students asked at the same category will help students formulate better questions.

Improve the quality of help seeking By typing out the help request, the student has to think about what they want to ask before they can get into the queue. Letting the student formulate or choose a previously written question matches the ”establish a specific need for help” step of the Mercier model [27], see figure2.4. The goal of this step is to get more specific questions from students, and to reduce the amount of help requests that are a general statement of confusion.

Hypothesis 5: Letting students type out their question will improve the qual- ity of help seeking.

Editing questions In this design the teaching staff is responsible for editing the questions students ask. The ability to edit questions was intentionally restricted for students, because students could otherwise type something random to get into the

list and change the random statement to their question if they had a question or remove themselves when they had no question once it was their turn. This would result in an unfair queue. However, to make sure that only the good questions entered the options from which students could choose, the TA had the option to edit the questions of students that they had just helped. The system would show in the top of the screen the last three entries with their question and a button to edit the question.

Group-help function

This feature can only be activated when the students have to type out their question to get on the list. When two or more students have the same question and are wait- ing on a TA to help them, and the TA clicks on the highest entry with this question, the TA gets the following option; help this one student, or give all the students with the same question the notification to come to the front of the room to be helped si- multaneously. This way group help can be offered to students who work at the same pace. The feature reduces the amount of questions a TA has to answer, because they can help several people at the same time.

Hypothesis 6: The group-help feature makes the helping of the TA more effi- cient, without boring or discouraging students who are working at a different pace.

3.7.3

Design choices

Dark theme

In the focus group interviews the TAs mentioned that they preferred to be mobile, they did not want to walk to their laptop every time they wanted to give someone help. TA-HelpMe was already responsive, therefore mobile friendly. On top of that the design choice was made to give the web application a dark theme. Google re- searched energy saving methods for mobile devices and found that using a dark theme saved battery [76]. Muharum supports this claim, in his research about bat- tery saving application setting, he found that dark pixels use less power [77]. When the background of the application is set to dark pixels the majority of the pixels is dark and the application will cost less energy.

3.7. REALIZATION PHASE 79

Lists

In the figures of 3.13, the various designs for the digital room can be seen. The first figure 3.13a shows the design for the student view of the lists. In the figure several students are added to both lists, students can only see each others name and what categories the other students have chosen. Students cannot see each others question. This was purposely done, to keep the threat to self-worth low. As was mentioned in section 2.2.1, public attention can threaten the self-worth of the student and result in less help seeking [7], [8].

In the second figure3.13b the teaching staff design of the room page is shown. Figure 3.13b shows that the teaching staff can read who has what question and which categories were chosen. Next to every name is the option to help (green button) or reject (red button) the students help request. When a student has a non- informative question TAs can choose to reject the help request, this might stimulate students to ask better questions. In the projector view the same view as the student view will be shown, minus the ”add me to list” buttons. Further, the room code will be displayed in a large font just as the previous version of TA-HelpMe did.

(a) Student view (b) TA view

Figure 3.13: Designs for TA-HelpMe version 2.0, digital room pages

Categories and question entry design

The following two figures show the design for the categories and question display. After clicking ”add me to list” the students will be redirected to the first category page, see figure3.14a. In this page the first category that was made by the teaching staff will be displayed. In this example the tutorials work with a weekly sign off, the first category for a question entry is therefore the week in which the students are working. After choosing what week the student is working at the student can choose the second category, in this example the type of question. After clicking the second category in the drop-down button the students will be redirected to the question page, see 3.14b. Students then see an text field where they can enter their own question and below the text field the previously asked questions in this category are

displayed. The students can choose to either write their own question or pick one. After giving their question the students are entered to the list and their screen is redirected to the room view.

(a) Questions category page (b) Question entry

Figure 3.14: Design for the categories and question entry

Course setup

The panel of the course setup that can be seen in figure 3.15a, deviates from the course setup of figure 3.11a. The new course setup has tabs to navigate through the various options. After entering and saving the lists for the course, the course manager can click the blue ”subcategories” button. This will redirect them to a new tab where the first categories of that list can be defined, see figure3.15b.

(a) Course setup page (b) Course setup page

Figure 3.15: Design for the course setup pages

3.8

Chapter conclusions

RQ1: What are the pitfalls of programming tutorials for each user group? RQ4: What technologies are available to improve tutorials?

3.8. CHAPTER CONCLUSIONS 81

the user needs and the MRQ? In this design chapter the human centered design cy- cles were run through. The main pitfalls for the user groups were that the teaching staff wanted students to ask better questions and the students wanted a fair system that managed the help requests. In the state of the art section several technologies that could be used were described. The technologies were tested if they met the requirements of section3.3. TA-HelpMe met the most requirements and was there- fore chosen to expand. Showing the progress of other students of Eurekos and the Paper cups was combined in the design of the categories. The sharing of questions and strategies that was mentioned in section2.2.1, was applied in the question entry design. The disadvantages that teachers mentioned of classical instruction during a lecture were bypassed with the group-help design, that feature allows teaching staff to gather students that have the same question and help them simultaneously. For this design solution three features were designed.

Categories; students need to specify what type of help request they have.

Question entry; students need to write their own question or choose a previously asked question of that category.

Group-help; the teaching staff can choose to let students that have the same question to come to the front so they can offer group help.

The hypotheses of the effects of the features are summarizes below. Feature hypotheses summarized

1. By selecting the category of their help request, students will take more time to think about what kind of help they need, resulting in an improvement in the quality of help seeking.

2. By reading the category before going to the help request the teaching staff can prepare for the help request, this leads to a higher level of help.

3. By distributing the experience over the categories, the teaching staff can in- crease the self reported efficiency of help.

4. Letting students read the questions other students asked at the same category will help students formulate better questions.

5. Letting students type out their question will improve the quality of help seeking. 6. The group-help feature makes the helping of the TA more efficient, without

Chapter 4

Method

In the sections below the procedures and data manipulations will be discussed. The context of the procedures is handled in sections4.0.1-4.0.5. In section data collec- tion the method of data collection is discussed. To get the opinion of the participants on version 2.0 questionnaires were distributed, the method of distribution and data manipulations of the questionnaires are discussed in section4.0.7. Lastly, the TAs were asked to participate in a focus group interview for an in depth evaluation of version 2.0, the method of that focus group is discussed in section4.0.7.

4.0.1

Subject selection

For the subject selection several course were considered. The criteria on which the courses were analyzed were:

• Programming in the tutorials • Group size

• Period 2 or 3

• Are they familiar with TA-help.me

The study that had a course that met the criteria was computer science. That study had a course in the second period that contained programming tutorials for more that 300 students.

4.0.2

Participation

The participation in this research was based on the opt-in principal. Students of the course had worked with the web-application in previous modules, were they were not allowed to ask questions outside of the application. However, for the opt-in to

work the TAs were instructed to allow students to ask questions by raising their hands, in order for the use of the site and the participation to the research to be opt-in. The students were briefed on the opt-in procedure by an announcement on the web-application, that informed the students of the procedure and the possibility to participate in the research.

4.0.3

Anonymity

The data of the students was processed anonymously. The students could even ensure their own anonymity by logging in with a nickname or an alias.

4.0.4

Research time

The students would have exams after the Christmas break therefor the week before the holidays was considered as the best week to test without interfering with the students schedule. In this week three tutorial sessions were scheduled.

4.0.5

Study location

The students of the course were divided over eight classrooms. Most of the class- rooms were in the same building, three of them were located two other buildings on campus. Each classroom had a projector or screen available for projection. In all of the classrooms the TAs could sit behind a table designated for a teacher that faced the students. The students could all see the projector screen, in one classroom that was very narrow and long the back of the classroom had extra tv screens in order