Implications with Respect to LAVA Framework

The LAVA simulation system has been developed as part of a project aimed at allowing students to develop an understanding of practical archaeology skills. The need for a virtual excavation stems from the fact that real-world fieldwork projects can be expensive to attend and are often heavily oversubscribed, thus limiting the number of students able to be involved. In addition, there are a number of accessibility issues which can often prohibit the inclusion of real-world excavation activities in the curriculum: excavation sites are frequently in-accessible to physically disabled people and so it becomes difficult for the general curriculum to include excavation site experience without unfairly discriminating against those who would be unable to navigate around the environment. Through the evaluation activities undertaken it has been possible to determine the effectiveness with which the current implementation of LAVA fulfils these needs.

As an excavation simulator, LAVA is not designed to replace the physical processes associated with undertaking an excavation, but is instead designed to prompt users to consider the issues and problems which must be addressed when carrying out an excavation project. Through the development of the simulation based on the excavation of the Sparta acropolis basilica located in the Sparta region of Greece, LAVA provides opportunities for students to actively engage with these issues in a realistic environment. This simulation does not suffer the same physical and financial restrictions of a real world excavation. However, for the LAVA virtual experience to be meaningful it is important that students are exposed to environments in which they must consider the issues and problems associated with excavation work. In order to achieve this, the development of LAVA has followed four key objectives, with each virtual simulation aiming to provide:

• Learner Engagement: It is important for the system to engage with students and encourage

them to explore the subject matter.

• Groupwork and Collaborative Working Practices: The motivation for the LAVA system

to support collaborative learning is two fold:

a. Real archaeological excavation projects involve a high level of collaboration and so it is important that this collaboration be mirrored in LAVA simulations.

b. Groupwork is an important transferable skill which is often underdeveloped within traditional educational contexts, especially within the domain of practical archaeology.

Whenever students use the LAVA system they are organised into groups. This strategic groupwork forces students to adopt cooperative learning strategies and thus emphasises the importance of collaboration within real world excavation scenarios.

• Realism: When undertaking a virtual excavation, students should be faced with realistic scenarios which accurately reflect real-world working practices. This realism should also be extended to include realistic outcomes based on the activities undertaken by the students, thereby reinforcing their theoretical knowledge of practical archaeology. Realism is achieved in two ways:

a. LAVA is based upon a real archaeological site. All electronic resources presented to students are based on original excavation data and are organised both geographically and temporally so that learners may engage in sequenced activities, with realistic, real-world inspired results.

b. Excavation information is presented to students using a variety of complementary technologies ranging from text to provide contextual information, 2D maps to define geographical context, photographs to provide high fidelity imagery of the region, and 3D collaborative environments to provide a sense of spatial relationships.

• Accessibility: LAVA has been developed to run alongside an existing archaeology module

within the School of Classics. It is intended that groups will work with the LAVA system collaboratively within the university PC Classrooms as well as individually on private computers. Given this varied usage, LAVA has been developed to allow students to access and interact with the system from anywhere at anytime. To facilitate group access, management tools are provided by MMS to allow student groups to manage and coordinate their progress through the excavation in a distributed way.

Through the evaluation process we have identified significant levels of user engagement, with a majority of participants providing positive feedback in both the SUS and Educational Value components of the evaluation process. In addition, the observations from evaluators and domain experts point to positive learning experiences by many participants, with active engagement apparent in all evaluation sessions. Having said this, as a developing system, some aspects of LAVA still require additional work – something that has been made clear by the evaluation process. Participant feedback in relation to system interfaces and the feedback offered have been invaluable in identifying aspects of the system in need of extra development work. In many cases, this participant feedback has helped to directly determine the development priorities of LAVA.

Overall the evaluation of LAVA has been a worthwhile exercise in determining LAVA’s fitness for purpose. User feedback has broadly indicated a positive response to the system and its aims and objectives, with the deployment and use of LAVA in a classroom environment being vital in proving the technical feasibility of delivering simulated excavations to multiple simultaneous users.

Looking forward, the evaluation process has helped to identify aspects of the system which would benefit from further development work. The results of the evaluation process show initial promise, with the usability and educational value of LAVA reported as broadly positive by evaluation participants and domain experts. However, given the reliance on user opinion, the true value of the interactions that students have with LAVA can only really be determined by collecting more usage data through the deployment of the system over an extended period of time.

5.6 Chapter Summary

In this chapter we describe how the core of the LAVA system has been deployed and evaluated over three consecutive academic years. We outline how students in honours level archaeology modules have participated in virtual excavations both as part of a team and also individually. The trials, usually undertaken during class hours, have been voluntary, with students able to opt out as they wish. The uptake and completion rates of the trials have been high, with a sizeable number of volunteers completing their participation in individual evaluation sessions focussing on specific components of the LAVA system.

When evaluating the system as a whole, a two pronged approach focussing on the usability of the system and the educational benefits that it offers has been adopted. The opinions of student participants have been collected using questionnaires broken down into three sections that included questions on system usability using the SUS, educational aspects of the system and open questions designed to allow respondents to provide feedback on any aspect of the system. In addition, group observations and individual interviews have been conducted to gain more specific information about key aspects of the system.

The initial findings of the evaluation process have been positive, with responses to the educational value and SUS sections of the evaluation indicating a broadly positive response to the system which has been consistent over each of the three years of evaluation. In addition, several encouraging user responses have been obtained which go some way to identifying specific aspects of the system that participants found engaging – the graphical and realistic nature of the visualisations presented being most noteworthy.

Domain experts in the fields of archaeology and computer science agree that LAVA has been well received by students, with many showing high levels of engagement with the scenarios presented. Indeed, many student respondents have indicated that they feel that LAVA was easy to use and that it had a positive impact on the courses in which they used it.

It is promising that the initial findings indicate that the adoption of gaming methodologies can have a positive impact on the educational benefits of a course. In addition, it is important to note that the evaluation process has played a significant role in shaping the instantiation of the system, with user feedback directly affecting the priorities adopted during the development process.

Given the heavy reliance placed on user opinion and other types of qualitative data, the evaluation process acts as an indicative guide to the applicability and suitability of LAVA as a teaching tool, with the process of deploying the system providing more concrete evidence of LAVA’s fitness for purpose – the deployment and use of the system in a classroom environment proves the technical feasibility of the framework, architecture and current instantiation of LAVA.

In summary, this chapter describes a systematic evaluation of LAVA and explores areas of further research and evaluation which may be beneficial when further considering the impact of LAVA on the curriculum. The evaluation process identifies areas of positive response from participants and has helped further the development of the system. The deployment investigates and provides evidence to confirm the technical feasibility of the system, with opinions from domain experts and evaluation participants providing extensive qualitative data to support the educational value offered by the system.