Paired T-Test Calculation of P and T
5.4 Knowledge Creation through Artefact Development
It is important to review the effectiveness of creating the artefact with the current study. Developing the app within the scope of the project posed several challenges in terms of working with particular time and resourcing constraints. It was conclusively shown that a functional application with a reasonable amount of original content and assets can be created within a short
105
timeframe. Currently, there is not a substantial amount of indigenous language learning applications for mobile devices that are truly gamified or involve the use of a virtual game world; as there is a lack of existing tools to evaluate; this study indicates that it is at least possible to develop and evaluate a new tool within the scope of an academic project for the purpose of creating knowledge about this combination of features. The investigation and evaluation of different development kits and the subsequent development of an efficient pipeline for creating and integrating game assets was an essential part of this process. By using an efficient workflow between image and animation creation programs and plugins for the SDK, assets could be created and optimised by one person within short development iterations. More focus on reviewing and evaluating the effectiveness of existing applications could have been more beneficial to developing the field of mobile indigenous language learning if there was a critical mass of dynamic interactive applications that utilised features of popular mobile games. Given the vast majority of existing applications did not have the features proposed to be developed and evaluated within this project, the combination of development and evaluation allowed the investigation of a relatively new type of language learning tool. The major advantage was the ability to create a an original tool as a result of the research that was being undertaken rather than evaluating a range of pre-existing tools that have not necessarily been designed within a similar conceptual framework to the evaluation. The subsequent trade-off meant that there was less depth in the evaluation than a project that worked entirely with pre-existing tools; however, this is an entirely necessary trade-off when tools do not exist.
106
6.
Conclusion
Through the Design Science Research Process this project has created a novel combination of existing features of modern games and learning tools. Through reviewing existing developments in industry and academic literature a proposal to develop the learning tool within a simulated game world was created in the initial phase of the research process.
The iterative design cycles in the development phase of the project were a practical way to create a new artefact for the purpose of knowledge creation which created the possibility to observe novel features and mechanics applied to a Māori language tool that had not previously been combined together or studied. Students were observed to be consistently engaged while using the software and commonly reported features they wished to see in the game world.
Game mechanics seemed to enhance the tool by allowing various levels of learners to find individual challenges within the structure of the game. Students who learned the vocabulary reasonably easily consistently aimed to perfect their knowledge of the vocabulary to gain three stars for the level while other users still found a sense of progression and returned later to try to improve star ratings.
The combination of a simulated game world and game based mechanics applied to indigenous language learning appears to be a good option for research. Because educational software is often made on limited budgets it means that recent video games and other interactive media developed with larger budgets will contain a wide range of features not yet trialled in educational software; this presents a continually evolving set of tools that can be tested and applied. In this project, the learning tool appeared to engage students in a similar way to more traditional games; specifically, they did not need to be taught how to use the tool or have the task instructed to them. Users definitely seemed to have a sense of virtual presence in the game which created a perception of space that is potentially instrumental in teaching locative prepositions and other concepts involve space and interaction.
107
6.1
Limitations Further Research
There are some clear limitations to this research. The focus on continually developing and testing new features rather than creating a finished software tool with less features meant that a summative evaluation of the tool was not possible. Testing was only carried out on short user sessions rather than longer term usage patterns. This meant that retention of learnt vocabulary was not tested and also that game mechanics that develop over a longer time frame were also not tested. The software was tested on a small group of students over different iterations of development and testing with observations of their usage contributing to the features developed in the next iteration; this had advantages but also meant that there was no deeper analysis of a single tool.
Further research could focus on progressing towards a finished tool in order to analyse longer term usage patterns and a more summative and quantitative evaluation of the effectiveness of the tool. Game mechanics relating to longer term usage were developed but not tested due to the impracticality of adding enough content to allow for longer term usage within the timeframe of the iterative cycles in the Design Science Research Process. During the iterative development phase, fundamental parts of the structure of the game were being tested and redesigned on a weekly basis; this meant that the whole level progression had to be rebuilt in order to incorporate new structural features to test in the next week. This was considered a strength of the research as many of these fundamental structural features like the user interface, interaction with objects, enemies, the introductory splash screen and other game mechanics were the key area of knowledge generation so they could not be studied without redesigning them regularly. In further research, if the more fundamental structures are no longer the focus, more attention can be given to investigating vocabulary retention and how longer term game mechanics like rank and currency systems affect user engagement.
Another clear area for further research is further investigating how locative prepositions can be taught with the software and further developing the conceptual framework from The Silent Way method. While locative prepositions were tested in one iteration it became obvious that they would have to be scaffolded over many levels and would require more time to learn than was typically available in classroom testing sessions. Additionally, scaffolding tasks that used more complex
108
sentences with locative prepositions, would have required a progression across many levels and creating these extra levels within the iterative development phase would have made it a lot more difficult to iteratively redesign the structure of other mechanics on a weekly basis. Teaching more complex sentences is important and something that not all language learning apps do so it is a clear area for further research; it also provides a clear structure for the progression of learning tasks from focusing on just nouns, just locative preposition, just adjectives and on to increasingly complex sentences that combine them all. A clear avenue for research would be investigating whether users improve their ability to solve these types of tasks improve their ability to construct spoken or written sentences of a similar structure. In a broader context, the connection between progression in game based learning tasks and spoken or written language skills outside of the application would be a valuable area to study.