Notable learning resources discovery projects

Existing applications and research prototypes are another useful source of functionalities, which can be reverse-engineered to tasks. A few of them are briefly reviewed here, with the objective to highlight just their main characteristics of potential interest in this research.

4.6.1 MACE: domain-oriented visual interfaces

MACE (Wolpers et al., 2009) was a federated search system for learning resources in the architecture domain. While it is not operational any longer, the project was noticeable for providing exploratory oriented, innovative, visual interfaces to search for resources. As an example, the interface to browse its hierarchical topic classification, that played a key role in organizing the resources, is depicted in Figure 4.2. The figure shows the whole topic classification in a dynamic dendrogram, where it was possible to expand, shrink or pan interactively any area. By selecting one of the topics visualized, it was possible to navigate to the associated resources.

Figure 4.2 – MACE dendrogram (Source: http://truth-and-beauty.net/projects/mace/ 2016). The corresponding task, therefore, is to identify the resources by navigating in a topics taxonomy, which in this case was displayed graphically. Another example of similar visual exploratory oriented interface, was the possibility of navigating to the resources from a geographical map – particularly relevant in the architectural domain. The goal of MACE, indeed, was not to compete with generic search engines, but to provide a search portal with domain-oriented features.

4.6.2 OpenScout: access via a taxonomy of problems

OpenScout (Kalz et al., 2010) is a federated OERs search portal, specialized in management. It has a metadata schema based on LOM, and offers search functionalities such as keywords search, faceted search, and a rich set of social oriented functionalities.

The key aspect of interest for this doctoral research, is that its designers recognized the opportunity to move from a more traditional content-based paradigm, typical of pre-existing search systems, to a new paradigm based on competencies (formally defined abilities or skills). Therefore, the whole system is organized around a competency taxonomy (similar to an educational framework), developed by integrating elements from different models in the literature. Yet, at the same time, its designers fully recognized that many of the users would not be familiar with the concept of competency, and would not be comfortable using such a relatively complex framework. Hence, they acknowledged the need to integrate alternative mechanisms to simplify the access to the resources. They advanced two main solutions, which were specifically tailored to a group of their target users and their specific context, that is professionals in small and medium enterprises. One of the solutions proposed was to offer access to the resources via a more pragmatic oriented taxonomy of problems.

Another solution proposed was purpose-oriented tagging, where users are asked to tag resources with what they considered to be their purpose, to support a goal-oriented social search.

At the time of writing, the portal only mentions the possibility of accessing learning resources by keywords, unconstrained tags, and competencies. However, the main contributions of this project for this thesis, is the recognition that competency or educational frameworks are fundamental, but many users might not like to use them directly, and alternative techniques should be tailored to the specific needs of users and their context.

4.6.3 National Science Digital Library: exploring resource relationships

The National Science Digital Library (NSDL, 2015) provided search services based on standard aligned and curated metadata with a proprietary format. They made also use of social data and paradata (they are the ones who coined this term), to provide indicators of resource utility.

They provided a search portal including in particular an innovative search from conceptual maps. As it is possible to see in Figure 4.3, these maps let users interactively explore concepts (represented by rectangles) and their relationships (represented by arrows) in various domains. Sumner et al. (2005) argue that these active maps allow users to focus on their scientific discovery task rather than on low level aspects of the search process. The elements in the map allow one to navigate to related educational standards and learning resources. Different static and manually created maps were provided, to allow for the exploration of concepts, their relationships, and associated resources from multiple points of views.

Their search facilities have been later transferred to ISKME, so that they now share the same interface as OERCommons (Figure 4.4).

Figure 4.3 – An example of Science Literacy Map (Source: http://strandmaps.dls.ucar.edu/).

4.6.4 mEducator: repurposing

The project mEducator (Dietze et al., 2013), specialized in medicine, is strongly oriented to Linked Data technologies, that make it possible, in principle, to support the web-scale search of learning resources. It supports federated search, by automatically uplifting metadata from different traditional

standards to open Linked Data, and aims to automatically discover services providing information about resources (Yu et al., 2011).

mEducator has an unusually articulated strategy to support tasks related to the repurposing of existing resources and flexibly searching among these repurposed resources (Stefanut et al., 2012). Repurposing is a very common task for educators (de los Arcos et al., 2016), who constantly need to adapt and integrate existing resources (Dietze et al., 2013), for example to suit different pedagogical approaches, or different target people (Kaldoudi et al., 2011). mEducator’s rich ontology supports tasks such as the identification of resources that a given resource is derived from, or which were derived from it, but it also allows one to identify the motivations for repurposing, and reconstruct the whole history.

4.6.5 OERCommons: approximate alignments to educational standards

OERCommons (2016) provides a comprehensive interface and articulated information about its resources. Metadata in OERCommons are collected from authors, but also crowdsourced from users (third party or non-authoritative metadata). However, all metadata undergo a well-organized curation process, to guarantee their quality (Campbell and Barker, 2014). The metadata schema used by OERCommons is based on LOM with some extensions. Hence, as it can be seen from Figure 4.3, it is possible to identify resources using metadata such as Subject Area, Educational Level, or Educational Use, as well as alignments to educational frameworks such as the Common Core State Standards.

Figure 4.4 – OERCommons search interface (Source https://www.oercommons.org/). Notably, OERCommons provides the possibility of specifying the “degree of alignment” of resources to educational standards. This makes it possible to search for resources which are not necessarily perfectly aligned to a standard, but just approximately aligned. This can be useful when it is not

possible to state that a resource is perfectly aligned to a standard, or to obtain additional results when there are not enough resources perfectly aligned to a standard.

A similar feature was already introduced, in the nineties, in the bibliographic cataloguing format MARC (Machine Readable Cataloguing) with the CEMARC (Curriculum-Enhanced MARC) extension, via the so called MARC 658 tag – curriculum objectives (Murphy, 1995). Yet, even the recent schema.org / LRMI only foresees, at the moment, an exact alignment of a resource to a standard.

4.6.6 SocialLearn: multiple recommender systems

Other projects in the area of OER search (and beyond) recognize the value of learning analytics, that is the process of exploiting the traces that students leave when they interact with the educational material or their educational setting. Examples of these traces are the time spent using a resource, or the performances obtained in a test. These data can be conveniently used to improve teaching and learning (Sclater et al., 2016). SocialLearn was a project focusing on social learning analytics (Ferguson and Buckingham Shum, 2012). This project, grounded on the firm belief that the value of OERs can only be maximized in the context of a social learning space, draws upon social learning analytics to support learners and teachers activities. It suggests, in particular, a wide range of innovative recommender systems, exploiting different classes of social learning analytics. The analysis of students’ interactions in forums (social learning discourse analytics) could be used to understand their attitude towards the topic under discussion, and suggest OERs encouraging alternative approaches to a given subject, or challenging their point of views. Information about the students’ disposition towards learning (social learning disposition analytics), possibly collected through profiling questionnaires, could be used to suggest educational resources to strengthen motivation.

4.6.7 OpenEd: personalized remediation activities

As previously discussed, the literature suggests to shift the attention from isolated search tasks to the context where they are originated, hence focusing on directly supporting high / WC-level tasks. This suggestion is embraced by portals that offer integrated search within Learning Management System (LMS) oriented environments. OpenEd (2016a) was a portal of this type, innovative for its commercial-oriented business model and its “formative assessments” educational model. OpenEd offered teachers a simple LMS online environment, focused on the cycle assess, analyse, and personalize teaching. Lesson plans (Figure 4.5) were basically playlists or annotated bookmarks, where it was possible to include tests retrievable from a large data bank, precisely aligned to educational standards. This made it possible to associate them with learning resources, which were recommended to students according to their test results. OpenEd supported educators to assemble a set of correlated resources (addressing complementary instructional strategies) with personal annotations in a simplified lesson plan, deliver the lesson, collect data about students’ performance for formative evaluation, and advise personalized remediation activities.

Therefore, search was embedded in the WC-level tasks of planning and delivering personalized teaching, and the whole mechanism was based on the alignment of the resources to educational frameworks.

Figure 4.5 – OpenEd: editing a Lesson Plan (Source: https://www.opened.com).

4.6.8 Gooru: supporting different delivery strategies

Gooru (2016a) is another portal aiming to address directly high / WC-level tasks, supporting the paradigm of integrated search in a simplified LMS. Interestingly, they motivated their approach of integrating search within a wider context, as an attempt to make the whole mechanism sustainable, exploiting the synergy between searching and using. Gooru, therefore, supports teachers in planning as well as delivering their lessons. Concerning planning, educators can organize resources within collections: “playlists of multimedia resources in a variety of formats such as videos, interactives, websites, images, and more” (Gooru, 2016b). Concerning delivery, it supports the flexible organization of different delivery strategies, letting educators assign resources to students (classes), whose progress can be monitored via a Collection Progress dashboard displaying rich analytics. Within the context of these high-level tasks, Gooru adopts (Campbell, 2014b) a rich set of descriptive metadata fields and plenty of usage data and social data, to power their search, ranking, and recommendation algorithms. Their metadata schema may be considered a variant of LOM, and includes traditional metadata such as Age Range, Educational Use, and Interactive Type. They even include metadata such as Mobile Friendliness, or 21st Century Skills, where content is categorized based on detailed skills such as Reasoning and Argumentation, Environmental Literacy, or Building of Persistence. Social and usage data include search, rate, views, clicks, assessment results, as well as student reactions such as “I can explain”, “I need help”, “I don’t understand”. Most (95%) of the resources, over 18 million, are tagged with LRMI too.

Until 2014, Gooru attempted to solve the problem of multiple existing educational standards, by having its resources aligned by curator experts, to a proprietary subject taxonomy, visualized with a

collapsing tree of 6 levels. This proprietary taxonomy would then be aligned to educational standards by users, therefore fostering, they claimed, collaboration across people using different standards. This idea looked promising in principle, yet, at the time of writing, there is no more mention of that proprietary taxonomy. Instead, as it can be seen in Figure 4.6, Gooru too has adopted existing educational standards such as Common Core and Next Generation Science.

Figure 4.6 – Gooru: searching from educational standards (Source: http://www.gooru.org).

4.6.9 Achievement Standards Network: educational frameworks

The Achievement Standards Network (ASN, 2016), whose search website is described as a “web- scale search service for learning outcomes”, offers metadata and services related to the alignment of learning resources to achievement standards, which can be very useful in the context of OER discovery.

ASN offers access to machine-readable representations of standard educational frameworks, including globally unique Universal Resource Identifiers (URI), where educational resources can be aligned to, for example via LRMI/schema.org. While the (K12) CCSS plays a prominent role among the standards considered, they support, or plan to support, other educational, skills, and professional standards, including from other countries in addition to the USA.

ASN offers services to search and browse the supported educational standards, to obtain different representations of the same standard, and to support standards crosswalking – that is finding corresponding alignments among different educational frameworks (Sutton, 2008). This is a mechanism that can be used to allow a user searching for OERs aligned to a given educational standard, to discover other OERs aligned to different but equivalent standards.

ASN recognized the opportunity to specify the degree of equivalence between standards (as well as the degree of alignment between resources and standards) (Sutton, 2008). Hence, they specified various semantic relationships to model that two standards are partially equivalent: minor, major,

narrow, and broad, in addition to exact equivalent. A minor partial equivalence, for example,

indicates a minimum overlap among two competency standards, while a broad partial equivalence indicates that a competency standard covers all the relevant concepts of another standard and

additional ones. They also model prerequisite alignment, which specifies that a competency is a prerequisite for another. This type of alignment between standards, could be used to identify educational resources that are prerequisite for other resources, with no need to specify this type of relationship between the resources themselves.

4.6.10 Summary of the projects reviewed

Table 4.1 summarizes the mentioned projects, indicating their main characteristics of interest in the context of this research.

Portal Specific characteristics of interest MACE

Strongly domain-oriented, focus on exploratory-search functionalities with innovative visual interfaces. Access to the resources via a topics taxonomy.

OpenScout

Adopted a competency-oriented framework, rather than a traditional content-oriented paradigm. Recognized the difficulty for some users to exploit these complex frameworks, and the need to provide additional alternative search techniques, tailored on users and their context.

National Science Digital Library

Exploration of resource relationships via multiple static cognitive maps. Maps aim to allow users to focus on their scientific discovery task rather than on low level aspects of the search process.

mEducator

Linked data oriented. Articulated support for repurposing, and flexibly searching among repurposed resources.

OERCommons Particularly curated data about resource quality. LOM oriented metadata. Alignments

to standards, including the possibility of specifying the degree of alignment.

SocialLearn Articulated recommendation systems based on social learning analytics.

OpenEd Search integrated in the WC-level tasks of planning and delivering personalized

teaching. Based on the alignment of resources to standard educational frameworks.

Gooru

Search integrated in a simple Learning Management System, aiming to improve sustainability by supporting the whole life cycle of resources in education. Focus on supporting alternative delivery strategies. Recently shifted from a proprietary subject taxonomy to alignments to standard educational frameworks.

Achievement Standards Network (ASN)

Metadata and services related to the alignment of learning resources to achievement standards. Crosswalking between different standards, modelling degree of equivalence between standards.

Table 4.1 – Summary of the main characteristics of interest for the mentioned projects.