Action Research and Learning Analytics

A literature review of the current situation showed that AR and LA have not yet been combined (see chapter 2). At first glance, the two approaches seem to be different, but a closer look at the goals associated with related AR or LA projects reveal similarities that call for a deeper analysis. While educational AR supports professional development by finding answers to practical questions, LA provides the tools for awareness and reflection, which also might facilitate professional development. Both approaches have the goals to foster reflection.3

Although the goals behind AR and LA are very similar, a difference can be seen in the initial trigger of related study projects. While AR projects usually start with a research question that arises from teaching practice (Altrichter, Posch, and Somekh 2005), LA projects often evolve based on observations made with regard to data, which has already been collected. While humans conduct AR, LA is often described as data-driven. Action researchers often use qualitative and quantitative methods to collect data and to generate a holistic picture of the learning situation. Current LA implementations mostly use different analytical methods from statistics and educational data mining to find information in large data sets, which are resulting from different kinds of information systems.

3 _{This section is a revised and extended version of parts of A. L. Dyckhoff, Lukarov, Muslim, et al.}

Figure 7. Current view of AR and LA.

Regarding the AR cycle and based on the interpretation of current LA definitions, certain LA steps can overlap with AR at the stages of data collection, analysis and reflection, and sometimes also in action planning or recommending activities to users (Figure 7).

In terms of LA, the creation of indicators so far has been controlled by and based on the (amounts of) data available in learning environments. Hence, the indicators might solely represent information that depends on the data sources used, e.g., by just making data visible that has been “unseen, unnoticed, and therefore unactionable” (Cator and Adams 2012).

But an important principle of AR is first to think about the questions that have to be answered before deciding about the methods and data sources (Hinchey 2008). Asking questions independently and putting aside the fact whether the necessary data is available or not, will lead to more and more relevant questions. This can provide more insightful information during the requirements analysis. This way, data and information gaps could be discovered, whose eliminations could help to improve the design of future LA tools and learning environments. These tools should not limit the possibilities of formulating own questions and pursuing individual goals.

Table 6 opposes key characteristics of AR to LA to show similarities as well as differences of both approaches; based on (Chatti et al. 2012a; Elias 2011; Hinchey 2008). Develop$ Question$ Formulate$ Research$Plan$ Collect$Data$ Analyse$&$ ReClect$ Develop$ Action$Plan$ Record$in$ Writing$ Share$ Learning Analytics

Table 6. Comparison of key characteristics of AR and LA.

Action research (AR) Learning analytics (LA)

Goals

Professional development, finding answers to practical questions, improvement of teaching, and social justice

Monitoring, analysis, prediction, intervention, assessment,

feedback, adaption,

personalization, recommendation, reflection and self-reflection

Process cycle

Develop a question – formulate research plan – collect data – analyze – develop and implement action plan – record project in writing – share

Data gathering (select, capture) – information processing (aggregate, report) – knowledge application (use, refine) and sharing

Driving

factor Human-driven: activities are centered around the person (group), who conduct the project

Data-driven: process is based on large amounts of data that promise to reveal new information

Advantages Individual, perfectly fitting

to a specific scenario, answers exactly the questions a teacher asks, open for all questions: What do I want to learn about my teaching? Methods of data collection can be adjusted creatively/accordingly

Standardized, general, suited for several scenarios, possibility to provide approved data

analysis/visualization by research experts, developed for and well suited for TEL or distance learning, data privacy issues can be handled centrally

Drawbacks Limited by time-constraints, a teacher’s workload and AR know-how, data analysis error-prone due to human error, not optimized for TEL, data privacy and permissions need to be handled by the teachers

Limited by missing data, often restricted to quantitative data collection methods, interpretation difficulties, danger to answer only questions nobody is interested in, focused on questions like: What does the data tell us? Specific question cannot be studied

Impact Effecting reflection, motivation and teaching activities

Assumed influence on users’ behaviors and reflective practice

Context knowledge

Knowledge about individual teaching situation (e.g., motives, teaching history, reasons)

Only data on teaching activities that have been recorded (e.g. log files, teacher journals, IMS learning design)

Instance Single instance (or rather

courses by one teacher) Multiple instances of the same scenario possible

Methods All kinds of qualitative and quantitative methods (e.g., surveys, interviews, video recording)

Limited to quantitative data collection methods (mostly data that can be logged automatically on different devices)

While AR is more focused on the human perspective, LA considers most aspects from the technical point of view. Both approaches have different advantages and disadvantages. This leads to the conclusion that joining both concepts can improve the impact of LA.