Ph.D. Research Proposal: An Android Game for Improving Code Reading Skills

Ph.D. Research Proposal:

An Android Game for Improving Code Reading Skills

Advisor: Professor Stephen Edwards

Project Summary

In recent years, much attention has been directed to the use of educational games for learning computer science concepts. The motivation of game-based learning with positive experience has been deeply studied in the literature, but game design for learning programming languages have much room for improvement. One possible solution to enhance the knowledge of programming languages is to improve code reading skills. Being good at reading a code is important as a professional developer. To address this issue, we defined a new educationally-oriented mobile game application, aimed at promoting the development of code reading skills in a new and fun way. The strategy of this game is to find errors in pieces of codes. At each level, students should find syntactic and semantic errors in the code in a certain time in order to advance to next levels.

Of the numerous programming languages we chose Java because it is one of the most popular programming languages and in many colleges, it plays an important role in introductory courses. Our vision is to allow instructors to employ the game in their introduction to programming in Java course. In addition, we hope it could be adapted to use in other introductory courses using different programming languages.

The collected data from the project will help evaluate the impact of game-based learning on code reading in programming languages. We will ask undergraduate students at the department of computer science at Virginia Tech who enrolled in the “Introduction to Programming in Java” to play with the game. The collected data will be analyzed and the use of game-based learning will be evaluated.

Intellectual Merit: The proposed work aims to improve the teaching of introductory courses in Java in various aspects. First, playing the game will improve code reading skills which are not sufficiently highlighted by lectures and projects. Second, while the existing games are mostly concentrated on improving writing codes, the proposed game is designed to improve the knowledge of code reading skills. In addition to these, the game provides an effective environment for practicing programming. We hope that if students’ improve their ability to read codes, they will understand codes more efficiently. In conclusion, the game will enhance their code writing skills and they will be better programmers in the future.

Broader Impact: The significance of this project extends beyond improving code reading skills through a mobile game. Playing the game to learn applies to teaching and learning in general. This project will demonstrate the impact of providing an effective and motivational learning environment not just in computer science education but also in any educational settings. Additionally, after releasing the game on Google Play, everyone in the world using their own Android device can take advantage of this game, rather than just Virginia Tech students.

1

Introduction

Computer science education has invented several new and innovative ways of teaching programming languages in recent years. Game-based learning is considered one of the novel ways of learning pro-gramming languages. There exist significant works in the design and development of mobile games aimed at promoting the knowledge of programming languages. Having educational games on mobile phones provides a convenient means for teachers to encourage students for learning programming concepts. Considering the effectiveness of using mobile games in learning new concepts, we antici-pate mobile games will be useful for our project. However, providing useful content as well as being fun is essential in order to find a balance between entertainment and educational value.

There have been developments in educational mobile games with many of them teaching the basics of programming in a fun and accessible way. However, the majority of mobile games assign-ments are dedicated to the development of games rather than playing them [8, 2, 3]. In these works a game development project defined for computer science courses in order to engage students and make the course more fun. While writing codes is essential for learning programming languages, understand the meaning of code and reading proficiency also plays a critical role in programming. Focus on improving code reading abilities is still lacking in educational games.

Developing an educationally-oriented game to enhance code reading skills provide an environ-ment for students to understand the meaning of their codes. In addition, the game reduces the reliance of the programmer to compiler and other debugging tools which results in finding errors in code faster and avoid common mistakes.

1.2 Solution Overview

Considering the existing lack of mobile-based application for improving code reading skills at the college level, we think that it’s time for developing an educationally-oriented Android game for enhancing such skills. The essential part of this educational mobile game is to provide full breadth of content that would be considered in introductory courses. As a result, we develop an educationally-oriented game which includes essential programming concepts. The main idea of the game is that players should find errors in the code and each level has different difficulty of errors; lower level consist some simple syntax error and higher levels include more semantic and run-time errors. The vision of the proposed game is to enhance code reading skills in Java. Our game can be effective at countering the impact of both ordinary mistakes and misunderstood Java language features. By finding errors in the code, we hope that students can find errors easier in their own code or in team projects’ code.

1.3 Evaluation Plan Overview

In order to evaluate the game, we chose “Introductory to Programming in Java” as a main focus, and we will ask instructors of the course to introduce the game in their courses. “Introductory to Programming in Java” provides an introduction to object oriented programming using Java. In this course breadth concepts of object oriented programming like classes, objects, methods, instantiation of classes, and sending messages to objects are covered. We developed the Android game in a way that covers important materials of the course so that students can benefit from playing it. The introductory courses at Virginia Tech includes many programming assignments. However, these

assignments are mostly concentrated on writing codes rather than examine the code reading skills. We will release the game for undergraduate students of Virginia Tech who enrolled in the course and collect data from the game students played during one semester. This data includes time that each student spent on playing the game, the amount of levels that each of them completed, and errors that they found in pieces of codes. We then evaluate and analyze the collected data to find how this game improves their knowledge of programming language. We will use surveys after the semester, as well as analyzing the collected data in order to directly ask students about their experiences using the game.

2

Motivation

Literature review in the field of computer science education as well as an educationally-oriented iOS game for English, have played strong roles in the idea of code reading game for Java. “The Grading Game” (http://www.gradinggame.com/) is an iPhone and iPad game for finding typos and grammar errors in English language. The main idea of the game is to improve students’ editorial eye. The player of “The Grading Game” are giving the role as a grader for a professor who is paying for grading students’ writings. The ability to read and find errors in the code is similar to finding typos and grammatical errors in an English article. The idea of “The Grading Game” was employed for programming languages and it has led to the development of a new educationally-oriented game for reading codes.

The theory of using educational software in undergraduate courses at the department of com-puter science at Virginia tech has been tested before. This resulted in motivation to implement a game for the course “Introduction to Programming in Java” for undergraduate students to encour-age them to pay more attention to enhancing reading code skills as well as writing codes.

Another motivation factor is students’ common mistakes in programming assignments which can only decrease by practice. For instance, students in their first years of college do not have enough programming experiences to know about all common mistakes. A game with the dedicated goal of finding errors can be an effortless way to overcome common programming mistakes. We will include a breadth of knowledge requirement for introductory programming courses in our game.

3

Prior Work

There exist so many efforts in computer science education aimed at using game-based pedagogy in teaching computer science courses. Some of efforts are related to specific topics of computer science [3], and some dedicated to certain courses like [1]. While, most of the research done in this topic stems from good ideas but often lack concrete evidence that their approaches are actually working and it will be extendable to broader area. It would be useful to look at some examples of research that has been done. Below are some examples of research in the area.

Baker et al. [1] developed an entertaining card game that helps students learn about the software engineering process. “Problems and Programmers” is a competitive card game in which students play the role of project manager in a waterfall project. All players lead the same project. Players must balance several competing concerns including budget and the client’s demands regarding the reliability of the final software. Who finishes first, wins in the game. Baker et al. evaluated their card game by having students who had taken a software engineering course play the game, and then answer questions on a survey that asked for opinions on the game’s effectiveness as a teaching tool.

Although this card game has many benefits, it has some limitations in the context of professional software engineering. For instance, this card game is primarily a competitive game and have limited or no collaborative aspects which is not suitable for real life projects.

Hainey et al. [4] implemented similar game for software engineering course. They developed their game in which players take on specific roles (project manager, systems analyst, systems designer, team leader). The systems analyst moves their avatar through the game world to collect requirements by asking questions of game characters. When the analyst thinks they have all requirements, they prepare a requirements document and send it to the project manager, who must decide whether to proceed with the project.

Hainey et al. [4] describe the use and evaluation of game based learning application to teach requirements collection and analysis. The students played a game where they had to manage and deliver a number of software development projects. They used a pre-test and a post-test and a control experimental group to evaluate the effect of game based learning compared to the traditional methods. They found out that game based learning can be a suitable approach in teaching requirements collection and analysis. They had students from two education levels and they concluded that initial knowledge should be taken into account when applying game based methods in teaching. In their experiment, the students with better prior knowledge benefited more from the games. Although their results are useful, there isn’t enough evidence for possibility of extending this game into other computer science areas.

Lee et al. [7] examined how adding assessments to a game affected people’s performance and engagement with the game. They created a storyline based game assessment to improve novice programmers engagement in learning programming. They results show that those who were given assessment levels not only completed significantly more levels than those who were not given as-sessment levels, but also completed the same set of levels faster than their counterparts. These studies indicate that people are engaged playing the game, and demonstrate that they are learning programming concepts by debugging the faulty code. Their results only show the impact of adding assessment to a game, and they do not evaluate the impact of game-based assignments on learning programming concepts which is more related to our concern.

Lee and Ko in another work [6] developed an educational games aiming to teach a specific unit of learning which is named Gidget. In this particular game there is a robot called Gidget, who has had a part of his software destroyed and thus it cannot fulfill its goals. Students are called to help Gidget, either by completing missing lines of code, or by fixing existing lines of code that have been written wrong. During the game, users receive constant feedback regarding their progress. The game aims to teach students how to design and analyze basic algorithms. The game’s users can write programs using a simplified language that was created especially for the game. This is a web-based game web-based on the HTML5 and Javascript programming language and the jQuery library. They found that participants completed more levels of the game, and thus acquired additional skill in programming, if the goal was framed in terms of helping a personable robot rather than an inanimate terminal. The uniqueness idea of this project is they focus on both reading and writing code skills.

Recursion is notoriously difficult to teach, and is a fundamental concept that is necessary to understand in order to do well in Computer Science. One issue with teaching recursion is that concepts are easier to understand if they can be likened back to past knowledge, and there are few instances of recursion in everyday life. Tessler et al. [11] used the Cargo-Bot video game for introducing recursion to novice programmers. In this game players control virtual robots by creating

programs using a simple visual language. The goal is to control a robotic arm so that it moves a set of crates to a specialed goal configuration. One of the interesting features of Cargo-Bot is that it supports recursion but not looping. The authors aimed to measure if students who played Cargo-Bot were able to transfer their experiences in playing Cargo-Cargo-Bot to solve recursive problems in Java. Results from a controlled experiment showed significant improvements in students understanding of recursion using this technique.

Lee et al. [5] also studied the use of a commercial iPad video game, Cargo-Bot, to help students learn recursion. In Cargo-Bot, recursive procedure calls are the only construct for repetition. This makes the game a natural context in which to use recursion. For their study, they used an experimental group which approximately two hundred college students in CS 2 used Cargo-Bot in a more structured way. In addition, a control group who is received lectures, was not introduced to Cargo-Bot. The control group lectures included examples in Java and were identical to the content used for several years. Assessment was done with pre-test and post-test quizzes. Students who played the game were statistically better at writing recursive methods than the control group. The investigators concluded the improved performance was due to the additional practice of writing recursive methods compared to the control group. The result illustrates that students practice recursive procedure by playing Cargo-Bot. While playing this game will improve knowledge about recursive in programming in Java, the game does not cover other programming languages concepts. Computer games have been successfully applied to improve knowledge of computer science concepts. Students need a classroom experience that is creative and challenging, and makes a difference in preparing them for careers. [8] address this challenge by introducing a “game first” as motivators and examples in introductory programming course. The approach of this game is a 2D game development as a unifying theme in an introductory course in computer science. Using games allows students to better learn in a familiar and playful environment. Moreover, it promotes collaborative learning and encourages students to learn. The results of the paper show that “game first” have positive influence on students and motivates them without any loss in the technical skills. Based on these results, it is well recognized that integrating computer gaming into introductory courses is a promising strategy for recruiting and retaining potential students.

Although Leutenegger and Edgington show that game development is both a serious occu-pation and a compelling way to motivate students to learn challenging technical concepts such as programming, their game is not completely proper for an introductory course. Most students in the introductory course are not familiar with graphics and they have not gaming background. Hence, designing a graphical game might be difficult for students and faculty members without graphic background. In addition, students have to build a game not to play, so the approaches cannot be considered as games.

There exist some more works in design and development of computer games serves as an excellent framework for teaching software engineering. Claypool and Claypool describe a game development project for a software engineering course whose goal is to engage the students and make the course more fun [2]. In this course, the students worked with one game project where they had to go through all the phases in a software development process. The preliminary results of comparing the game-based software engineering course with a traditional software engineering course showed that the game version had higher enrollment, resulted in higher average grades, a higher distribution of A grades, and had a lower number of dropouts. The feedback from the students was also very positive. Though Claypool and Claypool in their paper indicate that students can be motivated by game programming, their work is students’ centric development and the purpose of the game is

not learning programming.

Game-assisted learning has developed as an important trend in education. Moreno-Ger et al. introduce an educational video games named e-Adcenture [9]. The goal of this project is to facilitate the introduction of the games in the learning process. In the paper, the authors show that learning progression needs to be reflected by the instructional strategy and the difficulty progression in the game. If there is no challenge or if the challenge is insufficiently great, a game can be perceived as boring and is usually abandoned. Consequently, the main problem in constructing educational tools based on games is the question of finding a balance between entertainment and educational value. This article more focus on reasons why games have the potential to improve level of learning rather than providing an empirical evidence.

Eagle and Barnes introduce a game to help students learn basic programming techniques: basic loops, arrays, and nested for loops [3]. The name of this game is Wu’s Castle. Wu’s Castle is a 2D RPG where players must build armies of snowmen to escape a mirror universe. The authors have conducted extensive studies with Wu’s Castle and have shown playing the game before doing a programming assignment to learn loops and arrays can increase overall learning. They found that interactive instruction with small steps can help lead the players through the learning process while still keeping the players engaged and feeling as if they are “playing a game”.

Eagle and Barnes compare Wu’s Castle to traditional programming assignments and find signif-icantly higher learning gains for students assigned to do Wu’s Castle for homework. In the study, students had an additional week to complete a programming assignment designed to simulate the learning exercises in the game. The author used a strong switching replications experimental design comparing students who played the game one week and wrote a very similar program the next week, with the Control group who wrote the program first and then played the game. They conclude from this study that playing the game first might help students build a better conceptual understanding of for loops, nested loops, and arrays, facilitating increased learning over time. Although this game have been used by students for educationally purposes, the concepts which students learn through this game are limited to basic loops, arrays, and nested for loops.

Papastergiou [10] did similar work for high school students. In her work, 88 Greek high school students (46 boys and 42 girls, aged 16-17 years); randomly assigned to two groups, one used the gaming application and the other one the non-gaming one. A Computer Memory Knowledge Test (CMKT) was used as the pre-test and post-test in the work. Students were also observed during the interventions and, following the intervention, students’ views on the application were elicited through a questionnaire. The game was more effective in promoting students’ knowledge of computer memory concepts and more motivating than the non-gaming approach. The learning gains that boys and girls achieved through the use of the game did not differ significantly, despite boys’ greater involvement with, liking of and experience in computer gaming outside school and significantly greater initial knowledge of the subject matter. No significant gender differences in students’ views on the overall appeal, quality of user interface, and educational value of the game used.

The aim of a study by Papastergiou was to determine the learning effect and motivational appeal of an educational game about learning computer memory concepts. The gaming approach appeared to be effective in improving students’ knowledge about computer memory, as well as more motivating than the non-gaming approach. Even though boys were more involved in computer games than girls as far as enjoyment, experience and domain knowledge were concerned, there was no significant difference in the extent to which boys and girls learned by using the game. Boys and

girls also experienced the game as equally motivating. Based on the results of this study the author concludes that digital game-based learning can be used as an educational environment within high school education, because educational games can increase knowledge of subject-matter, as well as improve students’ enjoyment, involvement and interest in the learning process.

There are many discussions and thoughts on how best to teach introductory programming in computer science courses. These discussions vary from introducing computer programming using a game first approach [8], to using visual games [5, 6, 3]. While there are inevitably benefits and drawbacks to any teaching approach, we feel that the use of mobile devices in teaching introductory programming could provide far more benefits to the student than other techniques as we discuss in this paper. Students have easier access to these gaming applications, since they carry their mobile devices all times. The other motivations of game-based learning with positive experience has been deeply studied in the literature[6], but game design for learning programming languages have much room for improvement. The aim of most gaming assignments taught in introductory courses in computer science is to develop a game for learning programming languages and the goal rarely is to play the game along with development.

Some games focus on limited skills in programming, like loops, arrays, and nested for loops [3], whereas in our paper we are not limited by specific skills. Although some works have been done for middle school and high school students [3], there is much more evidence showing that using mobile games in introductory courses at the college level of computer science have positive impact. In our game we develop a new educationally-oriented mobile game, and provide it for students to play with in the “Introduction to Programming in Java” course. However, our game will not be limited to the mentioned course and it will be expandable for other programming languages and courses. In addition to that, unlike most prior works, target of our work is not limited to the specific people, after releasing the Android game on Google Play everyone can benefit from the game.

4

Proposed Work

Developing a mobile game is challenging due to a numerous amount of details in implementation. For our proposed educational game, we anticipated those challenges and found possible solutions for each challenge. Before start implementation of the game we thought about different aspects of game development and establish a prototype version of our proposed work. Figure 1. shows a screen shot of the first prototype of our game. The Figure shows an example the game which a player found one mistype in a piece of a program. After finding the mistake, the final score increased (shown as a star in the prototype) and the mistyped word became green. The application will show some tips and directions which explains strategies to win the game.

After we completed prototyping the idea of our game, we started developing the game and provided a list of all tasks which needs to be done during implementation. Some of these works are including developing the front-end and back-end of the application, collecting a breadth content of Java codes, collecting a list of common mistakes in Java, and finding an algorithm to inject errors in the code. In addition, we will evaluate and analyze the collected data from students played games. We hope to find evidences from the evaluation result which shows that students who spent more time on playing the game and with more amounts of levels completed, got better scores in their programming assignments.

We started development process with collecting students common mistakes in Java. Compilers and other debugging tools help students in writing their codes. However, sometimes students rely

Figure 1: Screen shot of Prototype, showing the user’s selected mistake in green.

on compilers to find the correct syntax. Our game helps students to read and understand the code without compilers and will improve their knowledge of code reading skills by finding errors in pieces of codes. These compile errors include missing semicolon after variable definition, missing parentheses “(” and “)” in method call without arguments, supplying the variable type for an argument in the parameter list of method call, missing return statement in non-void methods, etc. In addition, most students in their introductory courses face some common runtime errors which will can only be captured when the code is actually running. We include those kinds of errors in our game in order to familiarize students to runtime errors. Some of these mistakes includes referencing to null types, indexing to out of an array range, forgetting to initialize a variable, etc. We then categorize the suggested list of common mistakes into different groups by their difficulty levels.

The goal of our game is to cover all collected Java mistakes in different levels. Our game has multiple levels. Students should find errors at each level in order to be able to access to further levels. The strategy of this game is to click on a word which players think it is not correct. This error can be a compile or runtime error. If the selected word was chosen correctly, the color of that particular code will become green which means that the player successfully finds one of the errors in the code. Otherwise, the color of the selected word will become red. Players of the game should find certain amount of errors in the code and if they click on correct parts of the code, their score will be decreased. If their scores become negative, they will not allow to continue the game, however, they can replay the same level.

A number of technologies are available for implementing the game. We chose writing Single-Page Applications (SPAs) which allow users more native experience and redraw any part of the UI without requiring a server roundtrip to retrieve HTML. Single Page Application was chosen to launch the game on time and it dynamically update pages as the user interacts with the application. Moreover, since our game is research based, we may change different parts of the game during the development which is often more costly, time-consuming and requires extra effort in traditional

native applications. In order to make the development process easier and more structured, we decided to develop the game based on the Model View Controller (MVC) pattern and using Ember.js framework. Ember.js is an open JavaScript library that makes it easier to write client-side web applications that are competitive to native applications (http://emberjs.com/). Ember.js helps us to develop client-side of the game. We also need back-end for our game in order to be able to run our game on the server. For satisfying this need we use Ruby on Rails framework for the back-end of the game (http://rubyonrails.org/) and will storing our database on the default database of Ruby on Rails which is SQlite.

Since our goal is to develop an Android game, after identifying possible solutions, we decided to use free and open source framework Adobes PhoneGap. PhoneGap uses standardized web APIs which enables developers to create a mobile application that runs on a variety of mobile devices and platforms (http://phonegap.com/). After using Ember.js which is mostly HTML5, JavaScript and CSS, we will use PhoneGap turn the web application into a mobile application, most likely Android.

The next step is to inject common mistakes in the original codes to the database. We use “Java Software Solutions: Foundations of Program Design” written by Lewis and Loftus and “Introduc-tion to Java Programming, Comprehensive Version” written by Liang books which are among of common books used in introductory courses as our main reference for collecting pieces of codes. The length of each code piece should be small in order to fit into mobile screen. We picked codes which cover full breadth of content to be able to inject all of our collected common mistakes. For implementing this part of the game, we wrote regular expressions for several pieces of codes which identify codes with common mistakes. We first compiled each regular expression, specified as a string, into an instance of Pattern class in Java and then tried to match these regular expressions to a stored original database of codes. After finding the appropriate match, we changed the original code and display the erroneous code on the mobile screen.

After developing the game, we will ask instructors of the “Introduction to Programming in Java” course to introduce our game as a part of their curriculum. During the semester we collect the data from the games students play and capture students’ grade for their programming assignments. At the end of the semester, we will evaluate and analyze that data. There are many possible research that we can do with the collected data. First, we analyze data to find a relationship between the amounts of levels that each student completed in the game and the scores in programming assignments. We hope that game-based learning enhances the students’ ability of code reading to improve their grades in programming assignments. In order to find a relationship between playing with the game and knowledge of programming language, we store the amount of time spent on game and completed levels for each student. We will also store students’ grades for programming assignments for our future evaluation.

Second, we use collected data in order to find common mistakes which are easier and harder for students to find. In general, errors that more students can find are considered easier than errors that less students find. We will categorize mistakes by difficulty levels using our collected data. This result may help instructors and researchers to pay more attention to those difficult mistakes and try to reduce students’ mistakes in programming.

Another conclusion that we hope to get from our results is that students who have better reading proficiency can find errors in codes easier and debug them faster. The benefit of these results is not limited to just programming assignments, but also it is beneficial in larger team projects. People who can understand codes better, will be able to work in a group and read other people’s code

more efficiently. In addition to these, we anticipate more benefits will emerge as the data from the game is collected.

Nowadays so many applications are available on Google Play and people spend more time working with their portable devices. Considering prior successful educational applications like ”The Grading Game”, we proposed that developing and publishing similar game for computer science concepts will be beneficial. Our broad goal is to release the Android game on Google Play so that not just Virginia Tech students and faculties can use the educational game, but also everyone can use this game. The target of this game is computer science majors students, non-major computer science students, and even high school students.

5

Work Plan

Our goal is to develop an educationally-oriented Android game for the purpose of enhancing stu-dents’ expertise of reading code. Additionally, we aimed to evaluate the impact of the game in learning programming languages. The following estimated schedule is proposed for our work plan:

Year 1 (2015):

• Develop the prototype version of the code in order to make the project more clear.

• Provide a breadth in content database of pieces of codes.

• Provide a collection of students common syntactic and semantic mistakes in writing program-ming in Java.

• Create a database of codes using SQlite to create, delete, and execute SQL commands.

• Establish an algorithm using regular expression to inject errors in original code.

Year 2 (2016):

• Develop the front-end of the game using Ember.js framework.

• Develop the back-end of the game using Ruby of Rails.

Year 3 (2017):

• Convert the Single Page web Application to an Android game that is supported in different platforms.

• Release the game for undergraduate students who are enrolled in the “Introduction to Pro-gramming in Java” at the department of computer science at Virginia Tech.

Year 4 (2018):

• Analyze and evaluate the collected data from the game students play and other programming assignments in addition to the game.

References

[1] Alex Baker, Emily Oh Navarro, and Andr´e van der Hoek. An experimental card game for teaching software engineering processes. J. Syst. Softw., 75(1-2):3–16, February 2005.

[2] Kajal Claypool and Mark Claypool. Teaching software engineering through game design.

SIGCSE Bull., 37(3):123–127, June 2005.

[3] Michael Eagle and Tiffany Barnes. Experimental evaluation of an educational game for im-proved learning in introductory computing. In Proceedings of the 40th ACM Technical Sym-posium on Computer Science Education, SIGCSE ’09, pages 321–325, New York, NY, USA, 2009. ACM.

[4] Thomas Hainey, Thomas M. Connolly, Mark Stansfield, and Elizabeth A. Boyle. Evaluation of a game to teach requirements collection and analysis in software engineering at tertiary education level. Comput. Educ., 56(1):21–35, January 2011.

[5] Elynn Lee, Victoria Shan, Bradley Beth, and Calvin Lin. A structured approach to teaching recursion using cargo-bot. In Proceedings of the Tenth Annual Conference on International Computing Education Research, ICER ’14, pages 59–66, New York, NY, USA, 2014. ACM.

[6] Michael J. Lee and Andrew J. Ko. Personifying programming tool feedback improves novice programmers’ learning. InProceedings of the Seventh International Workshop on Computing Education Research, ICER ’11, pages 109–116, New York, NY, USA, 2011. ACM.

[7] Michael J. Lee, Andrew J. Ko, and Irwin Kwan. In-game assessments increase novice program-mers’ engagement and level completion speed. InProceedings of the Ninth Annual International ACM Conference on International Computing Education Research, ICER ’13, pages 153–160, New York, NY, USA, 2013. ACM.

[8] Scott Leutenegger and Jeffrey Edgington. A games first approach to teaching introductory programming. SIGCSE Bull., 39(1):115–118, March 2007.

[9] Pablo Moreno-Ger, Daniel Burgos, Iv´an Mart´ınez-Ortiz, Jos´e Luis Sierra, and Baltasar Fern´andez-Manj´on. Educational game design for online education. Comput. Hum. Behav., 24(6):2530–2540, September 2008.

[10] Marina Papastergiou. Digital game-based learning in high school computer science educa-tion: Impact on educational effectiveness and student motivation. Comput. Educ., 52(1):1–12, January 2009.

[11] Joe Tessler, Bradley Beth, and Calvin Lin. Using cargo-bot to provide contextualized learning of recursion. InProceedings of the Ninth Annual International ACM Conference on Interna-tional Computing Education Research, ICER ’13, pages 161–168, New York, NY, USA, 2013. ACM.