Citation:
Prigmore, MS and Taylor, R and De Luca, D (2016) A case study of autonomy and motivation in a
student-led game development project. Computer Science Education, 26 (2-3). pp. 129-147. ISSN
1744-5175 DOI: https://doi.org/10.1080/08993408.2016.1210854
Link to Leeds Beckett Repository record:
http://eprints.leedsbeckett.ac.uk/2822/
Document Version:
Article
The aim of the Leeds Beckett Repository is to provide open access to our research, as required by
funder policies and permitted by publishers and copyright law.
The Leeds Beckett repository holds a wide range of publications, each of which has been
checked for copyright and the relevant embargo period has been applied by the Research Services
team.
We operate on a standard take-down policy.
If you are the author or publisher of an output
and you would like it removed from the repository, please
contact us
and we will investigate on a
case-by-case basis.
A case study of autonomy and motivation in a
student-led game development project
M. Prigmore
a∗∗, R. Taylor
band D. De Luca
ba
Leeds Beckett University, Headingley Campus, Leeds, UK
;
bUniversity of Huddersfield, Queensgate, Huddersfield, UK
June 20, 2016
Abstract
This paper presents the findings of an exploratory case study into the relationship between student autonomy and motivation in project based learning, using Self-Determination Theory to frame the investigation. The case study explores how different forms of motivation affect the students’ response to challenges and their intention to complete the project. Earlier studies have made little explicit use of theoretical perspectives on student autonomy and motivation, a weakness this study attempts to address. As an exploratory case study seeking to evaluate the suitability of a particular theoretical framework we chose a small case: three students on a one-term computer games development project. Given the small scale the approach is necessarily qualitative, drawing on project documentation and one-to-one interviews with the students. Our conclusion is that the concepts of Self-Determination Theory provide a useful framework for analysing students’ motivations to undertake project work, and its predictions can offer useful guidance on how to initiate and supervise such projects.
Keywords: autonomy; motivation; Self-Determination Theory; project based learning
1
Introduction
This exploratory case study has two goals: to study the relationship between student autonomy and motivation in project based learning, and to examine the effectiveness of Self-Determination Theory (SDT) (Deci & Ryan, 1985) as a conceptual framework for case studies of this sort. For this exploratory case study, seeking to evaluate the suitability of a particular theoretical framework, we chose a small case: three students on a one-term computer games develop-ment project. This allows us to evaluate the suitability of SDT for case studies
of this nature, ready for further case studies seeking to validate our initial, ten-tative findings. The research context is also explored, and reasons for the choice of SDT over other possible theoretical frameworks are discussed. Note that it is the students on the project team that form the unit of analysis for this case study, not the project or the game itself.
1.1
Problem statement, research objectives and context
Experience of practical, real-world projects is widely recognised as being ben-eficial in undergraduate learning. These may be practical projects undertaken as part of the student’s course (Bosni´c, ˇCavrak, Orli´c, ˇZagar & Crnkovi´c, 2011; Larsen, Nielsen & Zhou, 2013; Marra & Wheeler, 2000) or work placements (Stead, 1997; Rawlings, White & Stephens, 2005; Towl & Senior, 2010). Such projects often have clear real-world requirements (Larsen et al., 2013; Marra & Wheeler, 2000; Towl & Senior, 2010) or can be specified by academic staff without compromising their real-world quality (Bosni´c et al., 2011).
The term “project work” is used in higher education to cover a range of activities, from a literature review to a satellite launch, and ranges from being closely directed by academic tutors or workplace mentors, to being fully student led. For this study we are interested in project work that is: student-led; centres around a meaningful, real-world problem investigated first-hand; creates a tan-gible product or output; and involves the opportunity to draw on expert input (Belagra, Benachaiba, Draoui & Guemid, 2012). These characteristics capture the practice at Canalside Studios at the University of Huddersfield (Canalside Studios, 2014).
Placements offer one way for undergraduate students to experience project work. Finding suitable placements for games students can be difficult. This problem was addressed at the University of Huddersfield by setting up an in-house computer games studio, Canalside Studios, an initiative that fed into improved teaching outcomes and helped graduates secure jobs in the games industry (Taylor, 2013). Canalside Studios recruits four games design and four games programming students for a one-year placement. The placement students undertake student-led projects, grounded in real-world games industry practices, which produce working computer games. These provide scope for autonomy and for developing professional competence through real-world experience. Finally, the involvement of experts from academia and the games industry provides contextual support to our early career game developers.
Studios have experience of this model (Taylor & De Luca, 2014).
Simulating such projects in the classroom can be difficult. In the first and second years of an undergraduate course some students overestimate their own technical competence and underestimate the difficulty of particular game devel-opment tasks (Bosni´c et al., 2011). Others are too timid and propose projects that are too easy to offer interesting opportunities for learning. Choosing a project which provides an optimal challenge, one which is difficult enough to engage the students but not so hard that they become disheartened, is tricky when the students are just starting out. In these early stages academic super-visors must carefully vet the scope of a project proposal and if necessary veto it altogether: student autonomy must often be limited in order to achieve an opti-mal challenge. However, once students have gained some technical competence the balance shifts and greater autonomy is possible. This raises the interesting question: how much autonomy?
The relationship between autonomy and motivation in the context of a student-led game development project is the focus of this research. A key ob-jective was to see how a student team dealt with a high level of autonomy. The intention was to push the degree of autonomy towards that experienced in many commercial contexts, while ensuring the project team had the necessary tools and contacts to complete the project.
On an assessed team project module ethical considerations preclude tak-ing this hands-off approach. Academic staff have both ethical and contractual obligations to support students, hence direct intervention in a team to provide guidance on day-to-day management, technical problems and team building is part of their role. This contrasts with the more strategic perspective of the game publisher.
The Canalside Studios placement is an ideal setting for the research. It takes eight third year undergraduate students on a one year work placement, providing a variety of real-world game development experiences. Students can work on several quite different projects during their placement year, each providing a different mix of practical experiences and requiring a different level of managerial oversight.
2
Related work
The origins of this study lay in a desire to make use of approaches to motivating students working autonomously on team projects that had good empirical evi-dence grounded in established theory. Searching available databases for studies of project work in University computer science courses that explicitly addressed the topics of autonomy and motivation produced limited results. Accordingly the search was expanded to include project work in higher education from other engineering disciplines and in secondary school. This section begins by consider-ing the use of theory in the design of project based learnconsider-ing and outlinconsider-ing SDT. It then discusses a selection of articles that discuss autonomy and motivation in project based learning and computer science education, interpreting these studies using the conceptual framework of SDT.
2.1
Theory
Most of the identified studies of learning through real-world projects while in higher education did not make explicit use of any particular theoretical per-spective on student autonomy and motivation. Only Marra & Wheeler (2000) used an established theory, situated learning, and a validated measure of stu-dent motivation, the Motivated Strategies for Learning Questionnaire (MSLQ). Other authors developed their own questionnaires or used qualitative interviews to gather information on student perceptions of the specific teaching strategies being used. Even so, the concepts of student autonomy and motivation recurred. Only Stead (1997) is explicit in rejecting existing theory, arguing that, at the time, the conceptual base for mentoring was “clouded” (Stead, 1997, p.219) and offering suggestions of how to develop the theoretical framework.
Some recent workplace (Lam & Gurland, 2008) and school based studies (Liu, Wang, Tan, Koh & Ee, 2009a; Lam, Cheng & Ma, 2009) have made use of SDT and concluded that the theory provides a good match for their empirical findings. SDT seeks to understand our motivations for doing what we do. It differs from earlier theories by switching away from control as a central concept: “Control refers to there being a contingency between one’s behaviour and the outcomes one receives, whereas self-determination refers to the experience of freedom in initiating one’s behaviour” (Deci & Ryan, 1985, p.31).
It is clear that there are different forms of motivated behaviour: building and launching a small satellite because the task interests you (see Section 2.2) is different from doing a literature review to get an easy high grade. SDT distin-guishes five kinds of motivational behaviour, forming a continuum from wholly self-determined intrinsically motivated behaviours, through four forms of in-creasingly externally regulated extrinsically motivated behaviours. Intrinsically motivated behaviours are performed for the interest and pleasure they provide. Extrinsically motivated behaviours would not be performed without some form of external regulation, or prompt. Deci, Ryan & Williams (1996) note that there is a natural tendency to internalise the external regulatory forces present in our social world: the spectrum of extrinsically motivated behaviours develops naturally from our desire for relatedness.
The nature of the external force regulating our behaviour determines the degree of self-determination. Deci & Ryan (1994) posit four forms of regulation for extrinsically motivated behaviours. External regulationof a motivated behaviour is present when the intention behind the behaviour is to avoid a punishment or gain a reward: the behaviour itself is in some sense inciden-tal, merely a means to an end. Such behaviours are not self-determined. In-trojected regulationof a motivated behaviour is present when a previously external regulation has been partially internalised, leading to some degree of self-determination: there is no actual punishment or reward, merely a nagging feeling that you ought to do this. For example, external regulation is in action when a student completes an allocated task because they want a good peer assessment grade; introjected regulation is in action when they do so because, having agreed to do it, they don’t want to let their teammates down. Next comes identified regulation of a motivated behaviour, when the behaviour has become personally valued, so moderately self-determined: the student who completes an allocated task because the success of the team, or the approval of their peers, is important to them. Identified regulation involves an instru-mental choice: doing one thing to attain a different, personally valued goal.
Integrated regulationof a motivated behaviour is present when, in addition to being personally valued, the behaviour is integrated with other aspects of their personal identity. Testing a computer game provides a nice example, since few computer games students see themselves primarily as testers. For a student who identifies as a concept artist or 3D modeller testing will typically be under identified regulation: the main personal objective is to produce a great game, and the student recognises that effective testing will assist in achieving that desired goal. A student who identifies as a games programmer may experience integrated regulation: testing is a key tool for any programmer, and great pro-grammers are good testers, so testing is integrated into the mix of behaviours that make up the personal identity of a games programming student. Integrated regulation is largely self-determined, though not wholly because our identities are to a significant extent socially determined.
learn in more controlled ways or more determined ways, and it is the self-determined forms of motivation that positively predict high quality learning and personal adjustment in school.” (Deci & Ryan, 1994, p.11). For example “[w]hen students display more autonomous self-regulation, they evidence greater conceptual understanding” (Deci et al., 1996, pp.171-2), however introjected regulation is also correlated with greater anxiety about failure, and an inability to cope when things do go wrong.
So, SDT predicts that project-based learning will be most effective when students are intrinsically motivated, and will be gradually less effective as their motivation to undertake the project work moves along the spectrum towards externally regulated behaviour. It is worth noting that autonomy is a nec-essary, rather than a sufficient, condition for developing intrinsic motivation: “the results of a range of studies support the view that contextual supports for competence and autonomy are the key ingredients for maintaining intrinsic motivation” (Deci & Ryan, 1994, p.9). We will return to the issue of contextual support within a student-led project when we discuss future work in Section 5.
2.2
Practice
Larsen et al. (2013) discuss problem oriented and project based learning at Aalborg University in Denmark, where engineering students have undertaken a series of student-led projects to build and launch a small satellite, known as a “cubesat”. These projects were team based and addressed a commercially realistic problem, similar to the situation in Canalside Studios. To design, build and launch a cubesat is too large a task for any single group of students. The work was organised into a series of linked sub-projects focused on designing and building the various subsystems. Each sub-project was suitable for a group of three to six students and directly related to their course’s parallel taught content. This is a model that could be adapted for student-led computer games development projects, which involve multi-disciplinary teams working on clearly defined sub-tasks (e.g. developing a game level requires input from environment artists, level designers, 3D modellers and programmers).
The experience of Larsen et al. (2013) was that, while curricular and financ-ing issues must be managed by the academic staff, the students must manage the actual sub-projects themselves; thus there is a high degree of student auton-omy. The students assigned tasks collectively based on the skills and preferences of team members. Larsen et al. (2013) argue that allowing self-selected tasks helped motivate the students, a claim consistent with SDT.
motiva-tions but are oriented towards personal goals, suggesting identified or integrated regulation.
Marra & Wheeler (2000) analyse student motivations for undertaking a sim-ilar two year project, SPIRIT, at Penn State University to launch a rocket payload. Like the Aalborg projects these were student-led with a student team leader, although initial team formation was instructor-led. The instructor also provided necessary underpinning lectures, ensured different teams shared their experiences, and mentored each team’s leader; traditional contextual supports for project work. SPIRIT was intended to provide a motivating environment. To assess whether this goal was achieved all students were asked to fill out the MSLQ for SPIRIT and for a more traditional comparison course; some students also participated in focus groups. A key finding from the MSLQ was “students reported that the SPIRIT project was significantly less driven by extrinsic mo-tivation factors such as grades and credit hours than their comparison courses.” (Marra & Wheeler, 2000, p.12). The focus groups provided some insight into this, with positive comments about the real-world, student-led nature of the project. There were more negative comments about team working, though the students recognised that team working would be a fact of their working lives making this experience valuable, if not enjoyable. They conclude “That student-centred, authentic, hands-on activities can positively affect student motivation.” (Marra & Wheeler, 2000, p.12).
Bosni´c et al. (2011) report on distributed software engineering projects un-dertaken by teams consisting of students from both M¨alardalen University in Sweden and the University of Zagreb in Croatia. The goal of the research was to elicit from the students the factors that students consider relevant to their motivation to undertake the distributed software engineering project, and those which have the greatest impact. There is no explicit use of theory, and they developed their own student questionnaires, supplemented by staff observations of the course.
In contrast to Marra & Wheeler (2000) Bosni´c et al. (2011) report that 47% of their students were motivated by Grade, an external regulation of their motivations to undertake the project. However, Grade also had the lowest perceived impact on motivation, suggesting students were conscious of the final grade but did not make a direct link between their grade and the quality of their work on the project. 33% of students rated Challenging Project as motivating, and it had the second highest impact overall. However just over 40% of students on successful projects rated Challenging Project as motivating, while on projects that had significant problems just under 25% of students did so. Students on challenging projects are likely to encounter technical problems that they struggle to overcome. SDT predicts that in such situations the more self-determined students will cope better; this is something we examine more closely in our case study analysis, below.
were from Croatia and 2 from Bosnia Herzegovina). Accordingly Bosni´c et al. (2011) investigate the effects of cultural background on the self-reported motivating and de-motivating factors, finding interesting differences. For exam-ple only 10% of students at M¨alardalen University were motivated by Project Success while 35% of students at the University of Zagreb were, suggesting a difference in the likelihood of students being intrinsically motivated. Given the importance SDT attaches to intrinsic motivation this suggests that cultural dif-ferences in perceptions of project work could be a significant moderating factor in some cases.
Demotivating factors are analysed too, showing clear differences between those projects that encountered significant problems and those that ran fairly smoothly. Students on more successful projects identify Technical Issues and Lack of Time as demotivating; those on troubled projects Personal Attitude and Communication Issues. Here, Personal Attitude “[represents] personal-level conflicts which are hardly ever resolved during project duration. Such conflicts result in broken communication channels and determine project’s collabora-tion patterns, significantly influencing final results (both process and product).” (Bosni´c et al., 2011, p.33). From the perspective of SDT this finding is consistent with the large number of students who were motivated by Grade, an external regulatory factor.
Projects are also used in secondary schools, and researchers in this area have made more explicit use of theory. Liu et al. (2009a) use SDT to investigate the effect of project work on the learning experience of high school students in Singapore. Although studying a quite different cohort from undergraduates this study is notable for its use of SDT, which has not been used much to study project work (Liu et al., 2009a). In a second study they use Achievement Goal Theory (Liu, Wang, Tan, Koh & Ee, 2009b) but do not find this theory as effective in explaining their observations. This influenced the choice of SDT for our study.
Liu et al. (2009a) note that although the teachers are familiar with project work the students are not. Project themes are chosen by the School and “can be as varied as ’innovation and you’ to ’saving the earth’.” (Liu et al., 2009a, p.144). A key finding is that “students who embark on [project work] with more autonomous forms of regulation tend to have more positive experience and perceive greater learning in [project work]” (Liu et al., 2009a, p.142). This suggests students’ motivations for undertaking project work have an impact on satisfaction as well as on learning.
with students working individually often on a single clearly specified task. This indicates a low level of student autonomy which SDT predicts would lead to less self-determined motivations for completing the work.
Herman (2012) used SDT to design learning experiences on a higher edu-cation computer science module that supported student autonomy. The study examines the design of a Computer Architecture module based on Contributing Student Pedagogies (CSP), which promotes student autonomy by giving them some control over the curriculum. For example, academic tutors provided con-textual support for autonomy by identifying what topics must be taught (the strategic core) and which could be chosen by the students (negotiable elements). Students were then grouped into learning sets based on what they hoped to gain from taking the module (their intention, or purpose) and jointly agreed which negotiable elements to study. The students also designed the grading scheme that would be used to assess their performance on the negotiable elements. This design for the module incorporated SDT by ensuring that individual intentions were explicitly recognised and that both autonomy and a sense of relatedness, to tutors and other students, were fostered by the classroom set-up.
Analysis of the module revealed “positive learning gains . . . positive moti-vational outcomes, and no evidence of self-selection bias” (Herman, Somerville, Goldberg & Green, 2012, p.13). Three interesting vignettes from the field notes illustrate the potential for SDT to provide predictions. Two learning sets, one of electrical engineering students uninterested in computer architecture and an-other of computer engineering students aiming for careers in computer architec-ture, used the negotiable elements to slant their learning towards their personal interests. Since both groups showed evidence of intrinsic motivation SDT would predict good performance from them: both teams gelled, and scored well. A third learning set “was composed of students who were primarily concerned about getting good grades and good jobs” (Herman Et Al., 2012, p.18). This suggests that regulation of their behaviour was partially introjected and par-tially external. SDT would predict less good results from this learning set. In particular the introjected regulation would predict anxiety about failure and an inability to cope when things do go wrong. The description of their attempt to identify the easiest way to get a good grade, failure to complete work, persistent attempts to change the grading scheme and lack of support for one another are all consistent with this prediction. Although this was a taught module, rather than project work, the findings highlight some of the benefits and pitfalls of designing learning experiences with a high degree of student autonomy, and the need to pay close attention to the nature of students’ individual motivations.
3
Case study design and methodology
to examine the effect of a high degree of self-determination and autonomy on project-based learning at undergraduate level.
Two research questions help to frame the study:
RQ1: How do autonomy and self-determination moderate the students’ motivation to undertake and complete the project?
RQ2: Under the conditions of high autonomy in running the project, what is the impact of technical challenges on motivation to complete the project?
The following hypotheses draw on SDT and the earlier studies above to suggest expected answers to these research questions.
RQ1-H1: Allowing students some autonomy over their involvement in the project will lead to them having more self-determined motivations for participating in the project.
RQ1-H2: The high degree of autonomy in running the project will lead the students to have more self-determined motivations for completing the project.
RQ2-H1: The more self-determined the student’s motivations the more effectively and autonomously they will be able to deal with challenges.
Note that there is a third, meta-research question that forms a precursor to RQ1 and RQ2:
RQ0: Does SDT provide a suitable conceptual framework for case studies of autonomy and motivation on undergraduate student projects?
This exploratory case study of a small team project can only offer tentative answers to RQ1 and RQ2. However it does provide a good opportunity to examine RQ0. If the evidence gathered through the case study can naturally be analysed in terms of the concepts and predictions of SDT then there is a strong case for using this approach to analyse future student-led projects. Also, since SDT is well supported by previous research, a good alignment between our case study and SDT would increase the validity of the tentative conclusions drawn as they would be adding to the evidence base for SDT.
3.1
Case and subject selection
The chosen case was a speculative game development project run during a sin-gle semester based on an idea for a Reading Game suggested by the Project Sponsor1, an academic with experience of games programming. It is the three
placement students that form the units of analysis for this paper, not the game or the project. The students brought an appropriate range of technical skills: J was a games programming student with strong programming skills; R a games
design student interested in 3D modelling and the design of the game mechanic; and F was a games design student interested in 2D art and game narrative de-sign. Note that the Project Sponsor was not involved in recruiting the students; this was handled by a second academic, responsible for managing Canalside Studios, the Studio Manager2. At the start of the project the Project Sponsor explained to the placement students that the project would be used as a case study. All three were happy with this and agreed to be interviewed after project completion.
To ensure the placement students undertaking the project were both self-determined in their involvement with the Reading Game project, and had a high degree of autonomy during it, the following measures were adopted. First, a high level overview of all the available projects was discussed when the stu-dents were interviewed for the placement job, and initial project assignments settled in consultation between the students and Studio Manager at the start of their placement: the placement students chose to be involved in the Read-ing Game project. Second, unobtrusive management3 practices were used to provide appropriate contextual support. Necessary resources (desk space, com-puters, software licenses, etc.) were organised by the Studio Manager, with some additional equipment provided by the Project Sponsor.
The Project Sponsor provided more visible contextual support through email and fortnightly meetings; the frequency of these meetings was decided by the students in one of their first project management decisions. The Project Spon-sor also put the students in touch with colleagues from the School of Education and Professional Development, who could offer contextual support in the form of specialist knowledge and who helped arrange for the software to be tested in a local school. Finally the placement students in the project team could, if they chose, draw on additional expert contextual support through Canalside Studios, including: the two academics running the studio4, who have between
them significant experience of game art and 3D modelling, games programming, and project management (Taylor & De Luca, 2014); industry experts visiting the studio; and the other placement students. These people are all available because the Canalside Studios placement involves working a traditional 09:00 to 17:00 day in a dedicated office separated from the teaching rooms and run as a commercial games studio. However the Project Sponsor did not require or attend meetings with external parties, nor did they allocate tasks to team mem-bers: the operational management of the project was left solely to the placement students to ensure autonomy. Again, this is similar to the management practices described by Larsen et al. (2013).
During the project the placement students provided weekly progress reports and additional written reports on meetings with external parties; email commu-nications were also kept. These documents formed the basis for examining how
2The second author.
3The term “unobtrusive management” comes from Bosni´c et al. (2011). Larsen et al. (2013)
use the term “invisible management”. The Canalside Studios academic team had developed similar practices themselves (Taylor & De Luca, 2014).
the project ran. After the project the placement students gave semi-structured interviews about their experiences to the Project Sponsor. The semi-structured interviews were fairly standard (see e.g. Hove & Anda, 2005). To minimise social dissonance (Myers & Newman, 2007) each subject was interviewed individually in the Project Sponsor’s office. This office has comfy chairs and a coffee table and was the room used for the fortnightly project meetings, so was a familiar space associated with the project for all participants. A simple script (see Ap-pendix A) was used to achieve a degree of commonality in the topics covered with each interviewee but the conversation was allowed to follow its natural course. Mirroring and flexibility were employed during the interview (Myers & Newman, 2007). All three interviewees agreed to the interviews being digitally recorded. Recordings were manually transcribed by the lead author with each transcription sent to the interviewee for checking; one transcription error was identified and corrected. Anonymity was guaranteed.
Note that the other five placement students in Canalside Studios worked exclusively on externally funded projects, with the second and third authors as project leads. These students were also involved in choosing their project allocation, however there was much less scope for autonomy in the management of these projects. In fact, on the largest project, client concerns about the placement students’ early design decisions led the Studio Manager to appoint an experienced games design Masters student as project manager. This low level of autonomy in terms of project management made these other projects unsuitable as cases for our study.
A code book was developed prior to analysing the data. Codes were based on theoretical concepts from SDT: motivation, autonomy, competence and re-latedness. An a priori coding scheme (see Appendix B) was chosen to facilitate examination of RQ0. The data were examined for statements related to these theoretical concepts. Themes and stories concerning autonomy and motivation emerged from this analysis and we present these findings below.
4
Results
The first task in the interviews was to establish whether the steps taken to ensure self-determination in the students’ involvement with the project had been effective. The first interview question asked them how their involvement in the project came about.
J: when I was getting interviewed for the placement in Canalside Studios they sort of told us what different projects they might be looking at and asked which I fancied. . . . I volunteered for [the Reading Game project] and pretty much started as soon as I came in on the first day.
it in mind and sketch out my ideas. So I think she already knew that I’d be interested.
F: Me and R really liked the idea when we heard about it so we knew straight away that we were the ones that wanted to do it.
Unlike J, both R and F initially worked on a different project (a “Top Trumps” game for the History Department). In this period J investigated technical fea-sibility and implemented a rough prototype, so R and F had a good idea of what the Reading Game project was about before joining. The alternative was a larger project to develop a command training simulator, sponsored by the West Yorkshire Fire and Rescue Service, or to propose a project of their own (an option always available to the Canalside Studios placement students). Thus the students were to a large extent self-determined in their choice of project. SDT allows us to discriminate more finely the degree of self-determination by analysing the students’ various motivations.
For J and R there is evidence for intrinsic motivation:
J: I’ve got a younger brother so we went through reading with him, and I thought it might be quite cool if I managed to get it working, you know, if my friends have kids or I have kids I could actually use it in the future.
R: I’d never done anything geared to a demographic so young with such a, I guess, important goal in mind.
However all three had extrinsic motivations, too.
J: because of the way consoles are going now, and PCs, speech recog-nition is becoming a big thing, so I thought it’d be a good skill to have . . . I think the main [motivation] was the speech recognition for the CV
R: Being able to take something away to say I’ve worked on this kind of project, I got this out of it, and we demoed it to this audience and this was their feedback, this was their response. So I suppose a portfolio piece . . . I hadn’t done some nice 3-D work for a while so I was looking to make some models and stuff.
F: Well I was interested in learning how to create assets for a proper project. Being able to work together with R . . . and being able to make our art styles fit because in the past I’ve had trouble, like, getting all the designers to have the same art style for the project.
In each case the student sees the project as a way to gain a particular skill that they see as valuable to their development as a professional game developer. This is integrated regulation, the most self-determined of the four forms of extrinsic motivation, supporting RQ1-H1.
(Bosni´c et al., 2011; Herman Et Al., 2012). SDT predicts that the more self-determined an individual’s motivations the better able they are to deal with such difficulties. Three stories from the case study show the students encoun-tering and dealing with challenging problems, and illustrate how self-determined motivations helped them cope, offering some support for RQ2-H1. The first two concern technical problems with the 3D models. Four weeks into the project the students decided to implement cel shading, reporting this in their weekly report.
On the programming side we’ve added cel shading ... but again have encountered various problems, such as the original version only working on static models.
The problems continued for J, as the following weeks’ report (written by J) makes clear:
In terms of programming, this week has been quite tough due to the fact that we started the week focusing on cel shading and so spent a couple of days on that, only to later encounter many problems with it due to the fact that I [J] have no HLSL knowledge. . . . This took quite a toll on J as it meant that he’d spent 2 days working on something only to later realise it was all for nought.
J seems to have underestimated both the technical challenge and the the amount of learning required. Attempting cel shading without HLSL knowledge is am-bitious, requiring significant learning of new technical competencies. Allowing only a week to complete the task shows a lack of competence in estimation, a key project management skill for games developers.
The students’ responded to this problem pragmatically: they recognised that cel shading was a could-have, rather than a must-have, and removed the feature from the first prototype. Note that both the original decision to implement cel shading and the subsequent decision to remove the feature were taken au-tonomously, without consulting the Project Sponsor. This supports RQ2-H1 since the students were able to pick themselves up after a nasty fall; despite a significant failure they did not need reassurance from the Project Sponsor that they were responding appropriately.
The second problem with the 3D models challenged R’s technical compe-tence: although the model animations worked fine in their design software, they did not work when put into the game. In this case the first weekly report to mention the issue shows that the students are beginning to appreciate that new techniques may take time to master:
In fact, sorting out the animation problem was a significant challenge for R:
R: A lot of the times I thought I’d found a fix after maybe a few days, you know, struggling – then, you know, fantastic it’s working let’s keep the project going. And then we’d move on to a few more mod-els. The next one was fine; oh, great this is working. The next one after that animations would be playing in reverse, animations, like character’s arms, would be flailing all over the place where they’re not supposed to be.
The students’ knew they had to solve this problem if the prototype was to include any 3D animations.
R: in some instances early on I tried to recreate work because I assumed that maybe if I just started from scratch . . . if I just tried again from the beginning, maybe I did something really small along the way and that has had a knock-on effect.
This did not work, and R clearly found the experience challenging:
R: I suppose there was a bit of a morale knock-on effect as well. Knowing that I was doing all that work and it was here and on my screen was working but when you put it on that screen over there it was not working. That was frustrating.
What kept R working at the problem were their strongly self-determined moti-vations:
R: here’s a chance to do something I’ve personally not seen before; these little 3-D characters that come out of the woods and everything and listen to you tell the story. . . . it seemed like a really exciting concept, so giving up on that would have been a real shame. And . . . a need to know why this isn’t working, as opposed to just straight up dropping it. Because you don’t learn anything that way, you don’t learn anything from just, ahh this isn’t working.
This last quote from R also makes clear a determination to ensure the project succeeds, offering support for RQ1-H2.
These technical challenges raised motivational problems for individual stu-dents, occurred quite early and were resolved within a couple of weeks. The third story concerns expert input from educational professionals. The story begins in the second weekly report:
One thing that did come up in the meeting between the 3 of us was that it may be worth getting some feedback with the style and speech recognition by children, sooner rather than later so we can adjust them for the Alpha at the end of the year.
A couple of weeks ago we set ourselves the task of making a basic yet polished proof of concept which we could show in schools and to [our contact in the School of Education and Professional Development]. I am happy to report that we have managed to achieve this goal and the tasks that we set ourselves.
However, subsequent feedback from their contacts with educational experts raised two significant problems. First, when small children are learning to read they are taught to spell out unknown words. So, for example, “cat” would be sounded out as “c – a – t cat”; this confused their original speech engine. The second problem arose with their story about a bear:
J: we changed some of the stories because we got in contact with the School and they gave us a document of the different sounds [the children] were learning at that stage ... we had [a story about] a Bear which had a lot more complex sounds, which I think was Year 2 or Year 3. So one of the stories we couldn’t really use, which was a shame because we couldn’t use, really, the assets
In response to the problems identified by the educational experts F wrote a new story using an age appropriate sound. J re-wrote the speech engine, implement-ing a tricky technical solution to allow the engine to ignore the soundimplement-ing out of letters and only recognise the word itself. This technical fix worked when tested by the students. These changes meant that the students approached the actual play testing confident that they had a good product:
J: Before [the play testing] I was quite happy with what I managed to do. It was really fleshed out, we had all these different stories, there was a goal, we had the achievements side of it. It felt like there was a lot there. All it needed, then, was just more stories being added. It was pretty much finished.
They also prepared sensibly:
J: We tested the program to make sure it was working with the speech, but also the buttons and everything was working perfectly and that. Then we had a team meeting to make a note of what we wanted actually to observe
However, the play testing did not start well.
J: we could understand what the child was saying but the recognition engine just didn’t.
The report on the play test provided more detail of these issues:
this could be because the children tried to sound out the individual letters instead of the full words. . . . One more problem we saw the children facing was the UI in our game was not self-explanatory, and perhaps too complex for children of that age. This resulted in them losing their position in the story and asking the same questions (i.e. “how do I choose a story”).
The problems with the speech engine were a significant setback, as the technical fix that worked for adult speech in a quiet office did not work with children’s speech in a noisy classroom. These serious issues were likely to indicate that the project risked failing. However, the play test report also noted:
Despite these problems, we found a lot of positives to take forward into any future build of the application. The visuals of the game were very popular among the children, and they seemed to be very enthusiastic about the 3D animals. The children also really seemed to enjoy the achievement system
The problem with the speech recognition required some immediate re-thinking of the play test itself:
J: So we thought, let’s not focus on the speech anymore let’s actually see if they like the look and feel of it.
R: So we decided to make it as simple as possible. . . . We were writing down notes that day about how things needed to be refined
On a personal level the response of the students to this major setback was, in fact, very positive:
F: [Play testing in a classroom] was really interesting because we found out a lot about how [the game] would work in reality.
R: The kids were very, very interested in playing it. They all seemed to get exactly what we hoped they would get out of it. So when they were looking at the scene in motion they were very excited that, by telling the story, little characters were appearing.
J, the programmer, was most affected by the poor performance of the speech recognition, but also found positives:
J: whilst it was crushing to not have the speech work, it was a saving feature that the kids were all rushing up, that they really wanted to play it, and were pushing in front of each other to be the next one to use it. So that was nice.
R: we were even talking about just dropping the speech recognition at one point just because we were worried that the project couldn’t move forward with the speech recognition.
J: We talked and tallied up the styles that children liked and the animals they wanted to see. ... it was like a 60/40 split between the [2d and 3d]. And the elephant in the room was what we should do about the speech. Do we go back and waste time trying to tweak it? Do we get rid of the speech altogether and keep it at that age group, or actually try and focus at older kids [who] could pronounce their words better and it would be more challenging stories. Rather than teaching the kids to read it would help them enhance their reading skills and abilities.
F: First of all we had a meeting about whether or not we should continue with the voice recognition. And I think that was more like, we thought it might work in the future but were not sure yet, but we could focus on other parts of the game . . . focus on the [3D] bonfire one, since that one seemed to have more likeable feedback.
F’s willingness to consider focusing on the 3D version of the game reveals a strong commitment to the project “because I’m more of a 2-D artist I would very much have liked to work on the 2-D side”. This suggests a willingness to put the project before personal development goals, indicating integrated regulation of F’s motivation for working on the project. J was also willing to consider dropping the speech recognition to allow the team to continue with their early-years focus. This is consistent with J’s intrinsic motivation to undertake the project, noted above.
At this point the demands of a commercially run Games Studio began to impinge on the project:
J: The Fire Sim took over. [The Studio Manager] took the two designers for that so we lost them, and then, you see, because I needed them and we weren’t sure what was going to happen I was put on a different project. ... the other projects had clients, sort of people we had to try and appease, whereas this, the reading game, didn’t have a client who had either paid or expected an end product.
J actually moved to work on an augmented reality project, given high priority by the direct involvement of the Vice Chancellor. For J, the abrupt end of his involvement on the reading game project was “a bit of a shock”. R was more phlegmatic:
Although not planned for, the cancelling of a speculative project occurs com-monly enough in commercial games studios that it is a reasonable way for a placement project in Canalside Studios to conclude.
5
Discussion and future work
First, it is important to emphasise that this small, exploratory case study can only offer tentative support for the research hypotheses arising from RQ1 and RQ2. However, what emerges clearly is that the concepts of SDT do provide an effective framework for analysing the case study. The students’ discussion of their experiences provides clear evidence for autonomy and for a range of intrinsic and extrinsic motivations for their behaviours: a positive answer to RQ0. This is important since motivating computer science students, in both individual and team project work, is an ongoing concern. We believe that SDT provides a conceptual framework within which we can develop, and evaluate, the effectiveness of a range of motivational strategies.
There is evidence to offer support to both hypotheses stemming from RQ1. The students’ motivations for taking part in the project are clearly self-determined, and it seems reasonable to infer that this is due in part to their being allowed some choice over their involvement in this project (RQ1-H1). The mix of intrin-sic and the more self-determined forms of extrinintrin-sic motivation is also significant: the project captured their interestand met their personal development goals.
The response of the students to the significant problems with the game highlighted by play testing in a School provides some support for RQ1-H2. Without seeking guidance from the Project Sponsor or Studio Manager they considered quite radical changes to the project. In the case of F and J these changes would have taken the game in directions that involved them giving up on some clearly articulated personal goals. J in particular was facing the prospect of a second major technical failure, after the earlier abandonment of cel shading. Thus the self-determined motivations helped foster a commitment to the project itself; it was more than a vehicle for learning, and so a more effective source of motivation.
mentor, guiding the student towards the effective application of best practices. Putting the right degree of emphasis on the contextual support is a significant issue for further study. The Project Sponsor might have made a stronger case for preliminary research on phonics teaching while maintaining student auton-omy over the final project management decisions. This could have helped the students develop their understanding of the differences between designing edu-cational and entertainment games. A second, companion paper (Prigmore et al., Forthcoming) explores issues of relatedness, competence development and con-textual feedback arising from the changed relationship between the academics and the students that greater student autonomy entails.
Having established that SDT can form an effective framework for case stud-ies of autonomy and motivation on student projects further case studstud-ies will increase the validity of our tentative conclusions. They could also examine the differences in motivation between students on self-chosen and assigned projects, and between projects where the placement students have autonomy in terms of project management and those projects with a dedicated project manager. In particular, at Canalside Studios it would be possible to track the same student through different projects, with varying degrees of autonomy in project choice and project management. This work could also provide additional support for RQ2-H1, which is tentatively supported by the discussion of how J and R re-sponded to specific technical challenges. In addition it would be sensible to draw on other theoretical perspectives, such as theories of goal orientation, to better understand the individual motivations of the placement students, and how these interact with SDT’s three needs of autonomy, competence and re-latedness. In particular it would be useful to compare SDT with Achievement Goal Theory to see whether the findings of Liu et al. (2009b) for school-based project work,discussed earlier, carry through to higher educational settings.
These case studies would continue to use semi-structured interviews and project documentation. In this case study the value of the project documenta-tion was limited by choices made by the students: they adopted agile practices using a pin-up board to track progress rather than written documents. Sub-sequent analysis of the students’ project management decisions relied heavily on the interviews and weekly reports. A record of the use of the pin-up board would have been valuable, and ways to capture the students’ project manage-ment practices should be incorporated into future work. One option is direct observation by a member of the research team; a Masters student based in the studio would be an ideal observer.
References
Beers, E. (2010). Industry Roles in the Game Development Pipeline. Retrieved August 18, 2014, from Beers Game Law: http://beersgamelaw.com/2010/04/industry-roles-in-game-development/
Belagra, M., Benachaiba, C., Draoui , B., & Guemid , B. (2012). The Impacts of Project-based Teaching Approach on Students’ Motivation.6th IEEE Inter-national Conference on e-Learning in Industrial Electronics (ICELIE), (pp. 121-129).
Bosni´c, I., ˇCavrak, I., Orli´c, M., ˇZagar, M., & Crnkovi´c, I. (2011). Student Motivation in Distributed Software Development Projects.Proceedings of the 2011 Community Building Workshop on Collaborative Teaching of Globally Distributed Software Development (pp. 31-35). Waikiki, Honolulu, HI, USA: ACM. doi:10.1145/1984665.1984672
Canalside Studios. (2014). What is Canalside Studios? Retrieved August 18, 2014, from Canalside Studios: http://www.canalsidestudios.com/home.html
Deci, E. L., & Ryan, R. M. (1985).Intrinsic motivation and self determination in human behaviour. New York NY USA: Plenum Press.
Deci, E. L., & Ryan, R. M. (1994). Promoting Self-determined Ed-ucation. Scandinavian Journal of Educational Research, 38(1), 3-14. doi:10.1080/0031383940380101
Deci, E., Ryan, R., & Williams, G. (1996). Need satisfaction and the self-regulation of learning.Learning and individual differences, 8(3), 165 - 183.
Herman, G. L. (2012). Designing contributing student pedagogies to promote students’ intrinsic motivation to learn.Computer Science Education, 22(4), 369-388. Retrieved August 3, 2015
Herman, G. L., Somerville, M. H., Goldberg, D. E., & Green, K. A. (2012). Creating low-cost intrinsic motivation course conversions in a large re-quired engineering course.American Society for Engineering Education An-nual Conference. San Antonio, Texas. Retrieved August 3, 2015, from https://peer.asee.org/21115
Hove, S. E., & Anda, B. (2005). Experiences from conducting semi-structured in-terviews in empirical software engineering research.IEEE International Soft-ware Metrics Symposium. Como, Italy: IEEE Computer Society. Retrieved August 4, 2014
Lam, S., Cheng, R. W., & Ma, W. Y. (2009). Teacher and student intrinsic motivation in project-based learning.Instructional Science, 37(6), 565 - 578. doi:10.1007/s11251-008-9070-9
Larsen, J., Nielsen, J., & Zhou, C. (2013). On student motivation in a problem and project-based satellite development and learning environment.6th Inter-national Conference on Recent Advances in Space Technologies (RAST)(pp. 923-928). Istanbul, Turkey: IEEE.
Liu, W. C., Wang, C. J., Tan, O. S., Koh, C., & Ee, J. (2009a). A self-determination approach to understanding students’ motivation in project work.Learning and Individual Differences, 19(1), 139 - 145.
Liu, W. C., Wang, C. J., Tan, O. S., Koh, C., & Ee, J. (2009b). Un-derstanding students’ motivation in project work: A 2 x 2 achievement goal approach. British Journal of Educational Psychology, 79(1), 87 - 106. doi:10.1348/000709908X313767
Marra, R., & Wheeler, T. (2000). The impact of an authentic, student-centered engineering project on student motivation. 30th ASEE/IEEE Frontiers in Education Conference(pp. F2C-8 to F2C-13). Kansas City, MO, USA: IEEE.
Miller, P. (2013). Ubisoft digital VP on retail, mobile, and the publisher’s role. Retrieved August 18, 2014, from Gamasutra:
http://www.gamasutra.com/view/news/194306/Ubisoft digital VP on retail mobile and the publishers role.php
Myers, M. D., & Newman, M. (2007). The qualitative interview in IS research: examining the craft.Information and Organization, 17(2007), 2-26. Retrieved August 4, 2014
Prigmore, M., Taylor, R. and De Luca, D. (forthcoming) The role of the aca-demic in project based learning: a case study of unobtrusive management on a student-led game development project
Rawlings, P., White, P., & Stephens, R. (2005). Practice-Based Learning in Information Systems: The Advantages for Students.Journal of Information Systems Education, 16(4), 455-464.
selfdeterminationtheory.org. (2015). Questionnaires. Retrieved September 3, 2015, from Self-Determination Theory: http://www.selfdeterminationtheory.org/questionnaires/
Stead, R. (1997). Mentoring young learners: does everyone really need a mentor?
Education + Training, 39(6), 219 - 224.
Taylor, R. (2013). From theory to practice: Canalside Studio, a case study.
Taylor, R., & De Luca, D. (2014). Theory to practice: Canalside Studio, a case study.Higher Education, Skills and Work-based Learning, 4(1), 5-16.
Towl, M., & Senior, C. (2010). Undergraduate research training and graduate recruitment. Education + Training, 52(4), 292 - 303. doi:10.1108/00400911011050963
Dept. for Education and Skills (2007)Letters and Sounds: Principles and Prac-tice of High Quality Phonics.
University of Huddersfield. (2013). Serious Games Research
Group (SGRG) - Project Details - Tell us a story.
Re-trieved August 18, 2015, from University of Huddersfield:
http://www.hud.ac.uk/research/researchcentres/sgrg/projects/projectdetails/index.php?id=358
A
Interview script
• Introduction
– Research on the development process and experience of the develop-ers
∗ Includes my participation - I am interested in what the team felt worked well, or could be improved, in terms of my role on the project. Mention this is tricky, so is covered last.
– Is it OK to record? Is it OK to take notes?
• Questions
– How did your involvement in the project come about?
∗ What were your personal goals?
– How did the team set about initiating the project?
∗ Team building?
∗ Background / market research?
∗ Product development?
∗ Who were you building the game for?
∗ Why did you, as a team, choose these initial approaches?
– How did the initial game prototype development go?
∗ Successes?
∗ Challenges?
∗ How did you evaluate the game prototype?
– Why do you think the project didn’t proceed to a second prototype?
∗ Would you - is it a realistic product?
– Looking back, how could the management of the project be im-proved?
∗ What worked well, what can we keep / improve?
∗ What caused difficulties, how to resolve/avoid these?
– In what ways did the making of the game help you meet your personal goals?
• Wrap up
– Anything you’d like to add?
– Thank you for doing the interview
B
Coding scheme
Category Subcategory
Motivation General
Intrinsic Integrated
Identified Introjected
External
Autonomy General
Supporting motivation Supporting competence development
Source of anxiety
Competence General
Overestimating Actively developing Relatedness General
