Assessment for learning Systems Analysis and
Design using constructivist techniques
Jonathan A. Tepper
School of Science and Technology
Nottingham Trent University
Clifton Lane, Nottingham, NG11 8NS
[email protected]
http://www.ntu.ac.uk/apps/staff_profiles/staff_directory/125395-2/26/jonathan_tepper.aspx
Abstract
This paper describes an innovative approach to assessment design that enables first year undergraduate students to learn Systems Analysis and Design concepts in a way that is relevant to them. The approach is inherently constructivist from two standpoints. Firstly, common group knowledge of board games is used as a means for learning subject-specific knowledge and secondly, concept mapping is used to enable groups to visualise and evolve their understanding over time. The preliminary results reveal a significant impact on student achievement and also strong student feedback in support of the approach.
Keywords
Constructivist, games-based learning, concept maps, systems analysis and design
1.
Introduction
Good teaching practice is commonly perceived as that which creates and uses learning environments and activities that fosters deep student learning (Marton, F. & Saljo, 1976). Deep learning occurs when the student is motivated to understand and engage with the concepts taught to satisfy intrinsic curiosity and to attain higher conceptual levels of understanding (Biggs and Tang, 2011). Teaching subject areas such as Systems Analysis and Design or more generally, Software Engineering, to first year undergraduate students poses a number of barriers to promoting deep learning. Firstly, it requires students to develop a wide range of analytical and inter-personal skills such as team-working, constructing project plans, eliciting requirements, and applying systems thinking and modelling techniques to identify and solve problems. Secondly, traditional teaching approaches alone, such as lecture and seminar activities, struggle to promote such skills and therefore other approaches are needed (Oh Navarro and Van der Hoek, 2005). Finally, students initially appear to have little appreciation of the usefulness and effectiveness of software engineering and therefore it is essential to motivate them to want to engineer software (Shaw and Dermoudy, 2005).
This paper presents an innovative approach to overcoming a number of the barriers highlighted above through the use of an open-ended assessment scenario that utilises the principles of games-based learning (GBL) and concept mapping techniques.
2.
Background
Games-based learning (GBL) approaches (Prensky 2001; Champion 2009) offers an alternative to traditional teaching methodologies by applying the principles of game play to educational contexts. Prensky (2001) emphasised that GBL provides an effective medium for engaging and motivating students. Evidence supporting this has been reported by Groff et al (2010).
There have been numerous GBL approaches to teaching software engineering-based principles to undergraduate students. Oh Navarro and van der Hoek (2005) developed SimSE, an educational game-based software engineering simulation environment that simulates a series of real-world software engineering situations such as managing large teams and projects. Shaw and Dermoudy (2005) developed SimJavaSP, an interactive web-based, graphical simulation game of the software development process, again with the student acting project manager. Baker et al (2005) developed a two-player educational card game, called Problems and Programmers, that simulates the software engineering process. Students take on the roles of project leaders in the same company. The player to complete the project first wins, however, the rules of the game enforce them to follow key software engineering principles or risk a penalty. Hainey et al (2011) developed a GBL application to teach requirements collection and analysis which proved to be as effective as role-playing and more effective than paper-based case studies. Designers of digital GBL have to carefully balance the trade-off between the enjoyment and educational value provided by the game (Oh and van der Hoek, 2001). Notably, Drappa and Ludewig (2000) reported that their simulation game, SESAM, enhanced students’ motivation but failed to sufficiently improve either their learning or skills likely due to the lack of interactive feedback offered. McCrindle (2010) made an innovative shift away from ‘digital GBL’ and used the notion of developing a physical prototype of a board game to facilitate first year Software Engineering students to learn the principles of software engineering. McCrindle used this approach with 200+ first year students for over 6 years and reported much success in getting students from a variety of computing and technology degree courses to participate in the deep learning of software engineering concepts and acquiring strong inter-personal skills.
2.1
The problem of common core modules
Nottingham Trent University (NTU) currently run 10 different undergraduate courses in computing and technology, which differ in their core emphasis (ranging from general computer science to more specific courses such as software engineering and digital media technology). All 200+ first year computing and technology students have been required to pass the core 30 credit point module,
Introduction to Systems Analysis and Design. The primary focus of the module is on project management and structuring requirements using process modelling, logical modelling and data modelling techniques. Note that from 2012, the module was superseded by a core 40 credit point version. Although efficient from a course design perspective, an issue of such large common modules spanning multiple courses, is how to ensure relevance to the individual course. Previously, students had been given coursework assessments that focused on the development of a common information system as it enabled the Module Leader to provide target answers/models and ensure high degrees of assessment reliability and validity across a team a five Module Tutors. However, students subsequently complained of the perceived lack of relevance of the module to them e.g. students whose computing course focused on either games, forensics, networks or security could not appreciate the importance of designing an information system.
3.
Approach
To address this issue of relevance, the approach taken from the academic year 2011/12 was to abandon the use of convergent assessment designs (i.e. group project to design a solution for a well-specified information system) and to exploit open-ended assessment designs which incorporate some form of games-based learning. More specifically, we decided to build upon the success of McCrindle (2010) and utilize the idea of having student groups design a board game based on a systems development methodology. The approach differs from McCrindle’s model in that rather than expecting students to produce a physical board game, students are actually required to produce a project portfolio showing the requirements and system models for the board game. Also, the board game may actually have a computer-based component (e.g. DVD with media-based activities supporting game play). Clearly, if students are using an iterative methodology, such as prototyping, then they may indeed decide to produce a physical prototype as they themselves move through the process (with the NTU providing financial support) as part of their proposal (such students are typically aiming for a first class grade). Table 1 highlights the assessment brief and the required deliverables for the group project. Since 2011/12 there has been an average of 240 students per year, divided into groups of between 4-5 students (approximately 48 groups).
As indicated in Table I, student groups were also required to construct concept maps (Novak & Gowin, 1983) of their group knowledge about systems analysis and design and describe how it has evolved according to criteria developed by Kinchin et al (2000). Concept maps enable students to visualize and collaboratively reflect on their knowledge, whilst tutors can see how learning activities can change the shape of student learning (Kinchin et al. 2005) and also relate the map to assessment performances. A dual benefit of concept maps is that sharing and advancement of knowledge is concomitant.
Assessment brief Deliverables
You are required to design and model a board game which teaches managers about a systems development methodology. The methodology must be relevant to your course (e.g. agile methodology for BSc (H) Computer Science with Games Tech, spiral methodology for BSc (H) Information Systems).
The game must teach management about each of the four phases of the systems development lifecycle, namely, planning, analysis, design and implementation. In order for you to design and model the game, you will be required to move through the Planning and Analysis stages yourself.
Introduction to Team and Methodology
Team Concept Maps
Project Schedule & Risk Assessment
System Proposal
The name of your game; Game description; Motivation or inspiration; Game objective; list of functional and non-functional requirements; a summary of the Use Cases; the following three process models: Context level DFD, Logical level 0 DFD, Logical level 1 DFD for a level 0 DFD process; Either Structured English, Decision Tree or Decision Table; a set of 3NF tables with corresponding ERD
Evaluation
4.
An example game
The following briefly highlights the features of a game that was awarded a first class mark. Examples of concept maps and also game designs from lower award bands and the qualitative differences between them are beyond the scope of this paper. Key details of the game are as follows: Name:
Project eX(p); Methodology: Extreme Programming; Inspiration: Talisman board game and AtmosFear board game with DVD; Components: physical board, DVD, dice, coins, player tokens/figures, question cards, reveal card.
Each game refers to an XP project and a game timer is set to run for 45 minutes. Before starting the game, each player chooses a token and roles the dice (player with highest score starts and game proceeds clockwise). Players may start on any space in the outer level. The cards should be shuffled and placed face down beside the board. Players can move in either direction within their current level and must follow any instructions either on the space or on any cards drawn. Typically, they will pick up test and code fragments along the way – although a test fragment must be obtained before a code fragments. Different levels in the game refer to different releases of the system. Each level completed represents a completed part of the system which is working and has met the user requirements. To pass between levels the players move must pass over the ‘release candidate’ board space. The player must successfully complete an instruction before passing this space otherwise they remain on it. The DVD times the game and interrupts the players periodically to broadcast an instruction directly from the user of the system. The player targeted must follow the instructions instantly. Once the user has finished, timer is un-paused and play resumes. To win the game, a player must be the first to make it past the third release candidate and therefore arrive in the centre of the board with a completed project. If no player has made it to the centre of the board before the timer runs out (as signalled by the DVD) the player with the most test and code fragments wins. Figure 1 shows a Level 0 data flow diagram (DFD) depicting the core processes of the game.
5.
Impact and evaluation
After reviewing a total of 94 group submissions over the last two academic years, the student comments in table 2 were representative as to how satisfied the students were with the new approach. A subsequent analysis of the resulting marks with respect to a 3 year average revealed a 28% improvement in the minimum mark (42%) showing that the more challenged students were better able to engage with module. 4% and 7% improvements over the 3 year average mean and median mark respectively were also observed (61.3% mean and 62% median mark) thus indicating
most students were positively impacted by the approach.
GBL-based Learning
Pros
Helped take out basic knowledge of SAD to a higher level. We now feel we have modelling skills to carry out projects to a high standard
Helped us to learn the principles in the textbook – brought them to life
Helped us to understand the principles of SAD as it made us go through the stages ourselves to design and model the game
‘showed’ us the advantages and disadvantages of different methodologies and relevance to our programme. We can now justify the methodologies we choose
Cons
Not a good idea – we felt a system such as an on-line ordering system would have been better as we would apply to real-world situation
Proved resourceful for learning the tools we have been taught however we recommend you are actually required to build a prototype of the board game rather than just model it!
Table 2. Representative student comments with respect to the GBL approach.
Unfortunately, 8% poorer performance (at 81%) was observed for the maximum mark, however this is not entirely unexpected as previous assessment scenarios were closed rather than open-ended and this naturally placed more demand on students as they had to contextualise the assessment themselves.
6.
Conclusions
We have presented an innovative approach to overcoming a number of the barriers highlighted with courses that make heavy use of common core modules. The approach uses an open-ended assessment scenario that utilises the principles of games-based learning and concept mapping techniques. More specifically, we built upon the success of of McCrindle (2010), rather than that of the more traditional digital GBL (Prensky, 2001). The approach we’ve used is inherently constructivist from two standpoints. Firstly, common group knowledge of board games is used as a means for learning subject-specific knowledge and secondly, concept mapping is used to enable groups to visualise and evolve their understanding over time. The impact on student grades with respect to the three year average is very encouraging and a review of student feedback provides further evidence that GBL approaches to teaching systems analysis and design (or more generally, software engineering) can tangibly improve the student learning experience.
7.
References
Baker, A., Oh Navarro, E., and van der Hoek, A (2005) An Experimental Card Game for Teaching Software Engineering Processes. The Journal of Systems and Software, 75, 3–16.
Biggs, J., and Tang, C. 2011. Teaching for quality learning at university. 4th ed. Buckingham: The Society for Research into Higher Education & Open University Press.
Champion, E. (2009). Game-based historical learning. In R. E. Ferdig (Ed.), Handbook of research on effective electronic gaming in education (pp. 219–234). Hershey, PA: IGI
Drappa, A., and Ludewig, J (2000) Simulation in Software Engineering Education Education. In Proceedings of the 22nd International Conference on Software Engineering, Limerick, Ireland, ACM Press, pp. 199-208
Groff, J., Howells, C. and Cranmer, S. (2010). The Impact of Games in the Classroom: Evidence from schools in Scotland. Bristol: Futurelab.
Hainey, T., Connolly, T.M., Stansfield, M.H., and Boyle, E.A. (2011). "Evaluation of a Games to Teach Requirements Collection and Analysis in Software Engineering at Tertiary Education Level", Computers and Education, Vol. 56, Issue 1, pp 21-35.
Kinchin, I.M., DeLeij, F.A.A.M., and Hay, D.B. (2005) The evolution of a collaborative concept mapping activity for undergraduate microbiology students. Journal of Further and Higher Education, 29 (1): 1-14.
Kinchin, I.M., Hay, D.B. and Adams, A. (2000) How a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development. Educational Research, 42 (1): 43-57.
Marton, F. and Saljo, R. (1976). On qualitative differences in learning I. Outcome and process. British Journal of Educational Psychology 46: 4-11.
McCrindle, R. (2010) Software engineering –engagement through innovative and interaction. Higher Education Academy, Engineering Subject Centre, Teaching Award 2010
http://www.engsc.ac.uk/downloads/teaching-awards/case-studies-2010/210610-McCrindle-TA2010_web.pdf (accessed 05 September 2011)
Novak, J and Gowin, D. B (1983) Learning How to Learn, Cambridge University Press, ISBN 0521319269.
Oh Navarro, E., and van der Hoek, A (2005) On the Role of Learning Theories in Furthering Software Engineering Education. In H.J.C. Ellis, S.A. Demurjian, and J.F. Naveda (Eds), Software Engineering: Effective Teaching and Learning Approaches and Practices, IGI Global, 2008.
Oh, E., and van der Hoek, A (2001) Adapting Game Technology to Support Individual and Organizational Learning. In Proceedings of the 13th International Conference on Software Engineering and Knowledge Engineering, Buenos Aires, Argentina, June 2001
Prensky, M. (2001). Digital Games-Based Learning. New York: McGraw-Hill
Shaw, K. and Dermoudy, J. (2005). Engendering an empathy for software engineering. In Proceedings of the 7th Australasian Computing Education Conference (ACE2005), Newcastle, Australia, 42, 135– 144
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission.
