Client

Each modern W3C compliant Internet browser can be used as client (of course no text based browsers such as Lynx). Coherence uses a lot of HTML 5 technology like SVG, Cascading Style Sheets (CSS) 3 transitions and animations, and web workers, so it needs to be supported by the browser. The current versions of the web rendering engines WebKit, Gecko, and Presto fulfil these demands, but Trident does not. This means that among others Firefox, Safari, Opera, and Chrome are supported – but Internet Explorer is not. Coherence has been tested to work with Safari 6.0 on Mac OS X and Firefox 10.0.2 on Linux, Mac OS X, and Windows.

5.6

Summary

This chapter has shown how the prototype that is based on the findings and concept presented in the previous chapters has been implemented. Therefore, the scope has narrowed to an application that is usable for prototypical testing and evaluation, but not a fully featured or integrated system. Authoring annotations needed for adaptive learning content can be added and modified using the Coherence Editor. The player presents the adaptive content, while the Coherence Monitor is used to monitor the internals of content presentation while playing the content.

The applications have been designed and developed as web applications and so that they can be used as online services. This makes it also easier to integrate it with common LMSs.

To generate the paths proposed to the learner, pathfinders that demonstrate the concept have been implemented. There are additional pathfinders possible to

5.7. What’s next handle other scenarios, like optimising the path to one lecturer’s content instead of optimising to the learner’s decisions. These should be designed, implemented, and tested in future research.

5.7

What’s next

After having implemented a prototype, the concept can be tested and evaluated on a working system. The following chapter describes the evaluation, using parts of real lectures, authored and presented using the Coherence Services.

Chapter 6

Evaluation of the Concept by

Research Interviews

The theoretical and practical work on this research project has been described in the preceding chapters. Within this chapter the evaluation of the outcome and the thesis is presented.

After discussing the aim of this evaluation, the expert interviews themselves described and why this kind of inquiry has been chosen. This is followed by sections that provide an in depth look at the evaluation process for this research project. Lastly the outcomes of the interviews are discussed.

6.1

Aim of Inquiry

As the objective of this PhD research project is a new concept that simplifies the authoring of adaptive learning content compared to current concepts and approaches, the main question of this inquiry can be formulated as follows:

Does this concept of structuring content have the capability of simplifying authoring of adapted eLearning content?

This can be seen as evaluating the usability of the concept for authors who, in this case, are usually lecturers.

In order to answer the main question secondary questions have been formulated that should be also discussed with the evaluation:

• Does the concept support common ways of authoring (learning) content? One of the main approaches of the concept developed in this PhD research project is supporting authors by adopting their common ways of thinking and creating content structures. It is of interest if this goal has been achieved. • Does the concept support creation of adaptive (learning) content?

As the concept of this research project should help the creating of adaptive content structures, it is crucial that it not only supports authoring but that the outcomes can be used as structures for adaptive content.

• Is the concept suitable for learning purposes?

This PhD project focuses on the learning purposes of created content structures. Although it would be excellent if this goal could be achieved for other purposes or in general, it should first be checked that the results can be used for learning settings.

• Does the adaptation support common ways of helping learners, for example in the case of comprehension problems?

This question focuses on the kind of adaptation that is supported by the concept and specifies whether it is expected to support learning in a way that learners are used to.

It is not only of interest to confirm whether the goals have been achieved but also if the underlying assumptions are correct and have been met by the concept. Whether the concept support authors in other ways than intended should also be clarified. Hence, additional secondary questions that are based on those above have been

6.1. Aim of Inquiry • How does the concept support common ways of authoring (learning) content? • How does the concept support creation of adaptive (learning) content?

• How is the concept suitable for learning purposes?

• How does the adaptation support common ways of helping students, for example in case of comprehension problems?

Some additional aspects have influenced the way the evaluation has been under- taken: The target group and the way the prototype Coherence should be used for evaluation purposes.

While a completely implemented and integrated concept would target both lecturers and learners, in both the roles of author and consumer, this research and therefore its evaluation focuses on lecturers as authors. Learners are the target group for any adaptive eLearning outcome that has been created within an authoring process designed for that. A perfect environment that might be realised based on the concept and approaches described in the previous chapters would and should ultimately help learning. As an essential preceding step before anyone can start learning with adaptive content, the content itself and its structure still need to be created. But whilst even a perfect authoring environment – of course – cannot guarantee that premium content is created, it is a requirement that the intended authors can get along with the concept, the workflows and the tools.

Evaluation of the technical feasibility of the concept has already been carried out by realising the research prototype which is also used to test the workflow by creating the content and structure for the evaluation. Additionally, it is used during the evaluation to enable the evaluators to get an impression of the look and feel of the envisaged realisation of a completed system.

However, it was necessary to make sure that the concept was evaluated, but not the prototype, because the realised tools should only demonstrate its feasibility

prototype, including its interface and the demonstration content, creating a bias in the results of the evaluation.

The following sections will describe how these aspects have been used to find a way to evaluate the concept, and how the evaluation has been performed.