eLAT User Interface C (Question-based Launchpad)

The third main user interface was more intensely concerned with personalization and the support of questions-based indicator analysis, as characteristics of AR suggest (see section 4.2.2).

Regarding personalization, every user has different interests and needs her own configuration. This is based on several factors. First of all, there are different types and sizes of courses in higher education, such as lectures, seminars, labs and individual thesis supervisions, to be supported by LA. Each course type can in turn be organized differently. In one event, for example, e-tests are offered, in the other there are no exercises at all, but in the next there are lecture videos and exemplary exam question, but no sample solutions, and another one is very collaborative. Depending on the scenario, the teaching and analytics goals are therefore different. Also, teachers do not look for specific indicators, but more naturally ask questions. This is because the teachers’ questions primarily are arising from the educational scenarios and not so much from the available data sources. For the teachers, it is therefore useful to find their own questions in the system and rather get guidance for analysis procedures; e.g., by recommendations for appropriate indicators for their specific questions.

For this reason, the interface C was mainly composed of question-based widgets, which showed the question in natural language in the title (Linden 2013). In addition, higher flexibility in terms of the adaptability of the interface was implemented within low and high fidelity prototypes, and evaluated. Basically, the former launchpad structure of eLAT User Interface B (Launchpad) remained. Appearance and placements of the indicators, however, were more individual. The new design focused attention towards the questions of a user. Each indicator was therefore associated with one or more question. A question was not necessarily associated with only one indicator. Indeed, there are several questions that can be supported by sets of indicators because, there are often several possibilities, which data to analyze and which visualization to choose.

Figure 31. ARLA's first screen.

Paper prototyping resulted in the following UI design, which is discussed in form of a use case: An LA beginner, who first logs into the system finds an example visualization in his LA dashboard, as a background image of the clearly exposed option to add his or her own questions (see Figure 30). These individual questions can then be entered in natural-language text into a search box, which appears after clicking on an “add question”-button (Figure 31, “Frage hinzufügen”). Once the first words are entered, the system matches15 them with the list of available

15 _{The matching process was predefined during the development of our prototype, but it could be}

questions and proposes items that are similar. Besides, the user has the opportunity to view the list of all available questions and search for interesting items. If this list is very long, it should be searchable and structured with appropriate categories, whereby some questions could be assigned to several categories. Therefore, each question should be associated with a category. Furthermore, some users want to add all questions to their dashboard and delete irrelevant questions later during the analysis step. So, it should be possible to add all questions to the dashboard with one click.

Figure 32. ARLA's questions selection process.

Those questions, which a user likes to have in his or her dashboard, can be added one by one or in a bulk. If a user has a question that is not supported, there is either the possility to request it from the indicator developers team16, or he or she types her question, stores it, and searches among a list of already implemented indicators for suitable matches. If suitable indicators for the new question are available, the user could map them onto the question, and also share it with other users in the overall catalogue of questions.

16 _{However, in this case, there needs to be such a team, and the provision would be associated with}

Figure 33. ARLA's popup window for adding a new question.

Figure 34. ARLA's indicator selection process.

If a user has finished compiling individual questions, he or she can start working with his or her dashboard during the semester. During this time, still more questions can be added or removed. Moreover, the questions can be arranged on the dashboard according to principles of widget standards.

The process of adding individual questions and their corresponding indicators to a dashboard is quite similar to an app store principle. Driven by mobile technology developments and their interface designs, modern platforms that lay focus on personalization and openness often provide users with the opportunity to select information and applications from larger lists (app stores) or even create their own apps and make them available for others. Hence, ‘question-based LA apps’ could also be integrated into these open platforms and allow for flexible integration of

analytics, wherever a user needs them. Hence, users would be able to arrange personalized LA dashboards within open VLEs or directly integrate LA nuggets (questions) into their personal teaching and learning environment (e.g., other modules of the VLE or mobile devices).

When monitoring the visualizations during the semester, users also want to have features to explore the data. Considering the ARLA interface described above, users can access larger presentation of the indicators just by clicking on the diagrams. This opens a more detailed analysis view (see Figure 35). This analysis view can include several options for exploring the data more intensely than just having a glance at it (monitoring it). There should be filters to decide which kinds of data should be presented in the visualizations. E.g., these filters could allow for the selection of timeframes, properties of students (like gender, field of study, etc.), specific folders or documents, and etc. Furthermore, it should be possible to make notes on thoughts and findings and capture specific parts of the analysis results for later usage or share it with other (e.g., colleagues or students). At the same time, all these features and visualizations of different indicators should be summarized regarding the particular question that had been asked in the beginning. This way it is easier to compare the outcomes and create an overall picture for the answering process.

The ARLA interface does not only provide indicators regarding one data source, but also allows for manual data import. If a teacher, e.g., would like to correlate data on how many students attended each lecture with the summarized usage data of lecture recordings, she could take notes about student participation during the actual meeting and then upload this data to the LA tool. Figure 36 shows a screenshot of the prototype on this process. The image in the background presents an indicator that is demanding new manual data input. The second screenshot below depicts an example spreadsheet for entering the corresponding data.

Figure 36. ARLA's manual data input.

The above described requirements for the interface – namely the introduction of filtering mechanisms, personalization, question-based collections of indicators, manual data input, and flexible integration into different VLEs – were based on findings of an evaluation, which is presented in chapter 7.

6.6 Conclusion

This chapter presented the design and implementation process of eLAT: an exploratory LA Toolkit that enables teachers to monitor and analyze their teaching activities. The main goal of eLAT was the improvement of teacher

support with graphical analytics, which are useful because they allow extending the audience to teachers without prior knowledge in data mining techniques. Having a mature prototype in early stages of the development process for user tests that are based in real world scenarios was a great challenge of the design process. Paper prototypes were somewhat helpful, but also in many aspects too general because they did not represent concrete data of users’ courses. Therefore, we needed LA implementations that could handle real data. With the help of the different interfaces of eLAT, teachers were supposed to be able to explore, reflect and evaluate teaching interventions based on their own interests:

• Interface A was wizard-based. It led the user through the process of parameter-selection before the outcome of the indicator-calculation was visualized. However, it was not designed for quick access.

• Interface B was designed as a launchpad. This launchpad design included a monitoring view, which united indicators of different categories on one screen, and analysis views, which provided different filtering options for each indicator.

• Interface C also adopted the launchpad metaphor, but it focused on personalization and question-based task design. Hence, users could personalize their starting page (the monitoring view) by selecting and arranging those questions that were interesting for their particular course. The impact of eLAT needed to be evaluated in real world scenarios. Chapter 7 presents the method and findings of such an evaluation, followed by the introduction of the final ARLA model and architecture (chapter 8).