INFORMATION SOCIETY TECHNOLOGIES PROGRAMME
TECHNOLOGY ENHANCED LEARNING
Annex to D8.3
Final Evaluation Report (annex)
Project Acronym : UNFOLD Contract No : IST-2003-507835 Delivery Date : 21st February 2006
Project Partners
Partner Org. Contact Person Tel Fax Email Address
UPF Dai Griffiths + 34 93 542 2173 +34 93 5422517 [email protected] Bolton
Institute
Oleg Liber + 44 1204 903660 +44 1204 399074 [email protected]
OUNL Rob Koper + 31-455762317 + 31 455762802 [email protected]
Table of Contents
Table of Contents...3
Deliverable Identification Sheet... 4
Executive Summary... 5
Log file analysis...7
Evaluating the context of UNFOLD results...18
Evaluation reports on UNFOLD events...29
Deliverable Identification Sheet
Project ref. no. IST-2003-507835
Project acronym UNFOLD
Project full title
Understanding New Frameworks of Learning Design
Distribution level PP
Contractual date of delivery 14th February 2006
Actual date of delivery 21st February 2006
Deliverable number 8.3
Deliverable name Final Evaluation Report (Annex)
Type Report
Status & version Final
Number of pages 132
WP / Task responsible UPF
Other contributors OUNL, Bolton, EUCEN
Author(s) Dai Griffiths, Josep Blat, Nidia Berbegal, Ayman Moghnieh, Fabien Girardin, Jose Luis Santos, Sergio Sayago, Daniel Burgos, Colin Tattersall, Chris Kew, Ana Dias
EC Project Officer Marco Marsella
Abstract This is the annex to UNFOLD deliverable 8.3, Final evaluation report. It contains details of evaluation actions carried out in the final six months of the project.
For details of evaluation actions carried out in earlier periods of the project, and the evaluation plan, please see project deliverables 8.1 and 8.2.
Keywords UNFOLD, Learning Design, IMS LD, evaluation, specification
Circulated to partners
21
stFebruary 2006
Executive Summary
This annex contains details of evaluation actions carried out in the final six months of the project.
For details of evaluation actions carried out in earlier periods of the project, and the evaluation plan, please see project deliverables 8.1 and 8.2.
Evaluation actions in this period were guided by the UNFOLD evaluation plan, submitted as an annex to UNFOLD deliverable D8.1, Periodic Evaluation Report 1. All the scenarios identified in the plan have been addressed by UNFOLD evaluation actions.
The actions caried out in this period were as follows.
Log file analysis
Studies were carried out of the project website at www.unfold-project.net, and of the Learning Network for Learning Design (LN4LD) site which hosts the UNFOLD forums and activity nodes, at
moodle.learningnetworks.org.
For the UNFOLD project website the most significant results of this analysis indicate are that:
• · The busiest month was December, showing that project activity was on an upward trend right up to the close of the project.
• · The heaviest use of the server was in the week, especially on Monday and Tuesday. While activity was low on Saturday, as might be expected, there was significant activity on Sundays, more than on Wednesdays. The figures for hourly analysis show two periods of intense activity, 2 - 4 am, and 9am to 1 pm. It is suspected that the high activity in the night is due to web crawlers which take advantage of periods of low Web activity to catalogue the site (although visitors from other parts of the world no doubt also accessed the site). The mid day spike indicates that most users were accessing the site during their work hours. There was also, however, substantial activity on Sunday. This suggests that some users were investigating IMS LD because they wished to use it in the future, but that their current work did not support this.
• · The most frequent query words used were unfold, learning, ims, design and ld. This indicates that a large number of accesses were by people looking for basic information about the specification, backing up the findings for the LN4LD site.
• · The most frequently accessed documents in this period were the motplus guide and the collage presentation. These both contain practical information about graphical Learning Design editors. This confirms feedback from members that the need for easy to use editors is the most urgent challenge facing IMS LD, and also the findings for the LN4LD site.
• · The UNFOLD server responded successfully to most requests for documents. There was, however, a substantial incidence of “document not found”. Investigation of this issue indicates that it is largely due to users following out of date links from search engines to documents which had been moved following site reorganisation.
The overall picture of the use of LN4LD is of a steep increase in early 2005, and a maintained level of activity in the from March to December 2005. The slight decrease in activity in the final period is no doubt due to a marked slowdown in activity in the holiday period in August. The most used areas of the server are those which offer an introduction to the specification and support in the basic use of IMS LD tools.
Usability test for UNFOLD site http://www.unfold-project.net
In general the participants in this evaluation liked the site, its content and its design. Its structure and navigability were the most problematic aspects.
• Most of the problems were found in the navigation, because, as most of them said “the menu has too many options”, and the feeling was that they did not want to spare too much time reading all of them, that they would have preferred less options and deeper profundity.
• The usability test shows that the task that could be accomplished with the first level of the menu, were more successful than those that required the user to go to a deeper level.
• The double menu, is difficult to understand at first, but after a minutes of navigating thought the site it becomes useful because it provides shortcuts to some of the site’s functionalities.
• Some users did not find the communities of practice on the site. It is not clear if this is because they were not involved in the project and they did not know about their existence, or because the menu is not clear enough.
• It is difficult to find all the ways to be in touch with other people because they are not grouped, not even in the same place. For instance, the mailing list, is very difficult to find, in the test, none of the participants noticed it.
• The readability is one of the most well-rated aspects of the site, followed by the information about the events and the clearness, sufficiency and quality of the information offered.
• On the other hand, the navigability, the content distribution and the easiness to find the information the users look for are the least positively rated.
In response to this evaluation the site has been radically simplified in the showcase version, and the navigation menu to the left of the page completely eliminated.
Evaluation reports on UNFOLD events Evaluation was carried out at the following events
• Braga CoP meeting
• Valkenburg UNFOLD-Prolearn converence
• Berlin UNFOLD CoP meeting
• Rome EUCEN Workshop
• Madrid Complutense Workshop
The results were collated with the results of earlier events, and the overall results of the project analysed. This information was contrasted with other sources, such as the logfiles, site membership, and records of
Log file analysis
LN4LD
Overall levels of activity on the LN4LD server
The number of actions recorded on the LN4LD server was analysed. The three periods of analysis are of unequal length, and so they were normalised by calculating the average number of actions per day.
0 50 100 150 200 250 300
15 Oct 04 - 21 Mar 05 21 Mar 05 - 27 Jun 05 27 Jun 05 - 20 Dec 05
Average daily number of actions on LN4LD server, per period
As can be seen from the above table, use of the LN4LD project server increased markedly in 2005.
Activity recorded by Activity Node
The LN4LD server is built around activity nodes, which can be anything that is available to support learning, such as a course, a workshop, a conference, a lesson, an internet learning resource. Online synchronous events are not included in this analysis, as they were hosted on a different server.
Analysis was carried out of the activity on the Activity Nodes of the LN4LD server, as shown in the table on the following page. There were 25 activity nodes on the site in this period. Some of these experienced a drop in activity, and those were the activity nodes relating to in person events which were some months in the past. Similarly those nodes related to events in this period increased in activity.
We note the following.
• There is one activity node for an in person event in the past which increased activity. This is Activity Node 16, UNFOLD hands on meeting in Valkenburg 2005. The explanation for this increase is that resources are available here for Reload and Coppercore, which have been the most widely used applications for creating and running Units of Learning.
• There is also an increase in activity in Activity Node 10, Runnable LD Example Units of Learning. This again shows the interest of users in obtaining resources and information which help them to use CopperCore to work with Units of Learning.
• There is an increase in Activity Node 25 Getting started with the IMS LD Specification. This shows that these introductory materials met a need for users who wanted help to get to grips with the specification.
Activity on LN4LD activity nodes, 27 June to 31st December 2005
No. Activity node Actions Change in
this period
1 Advanced Issues with IMS Learning Design 679 2 Booklet for Berlin 2005 (private working group) 1102 3 Change Proposals IMS LD Specification 1135
4 Discussion forums on IMS LD 5788 More active 5 Experience a running Unit of Learning 2103
6 How to modify a Unit of Learning 2403
7 IMS Learning Design and Metadata 1202
8 On line chats discussions 670
9 Online Educa Madrid Mayo 2005 (en castellano) 337
10 Runnable LD Example Units of Learning 7508 More active 11 Understanding the basics of IMS Learning Design 1496
12 UNFOLD CoP Berlin, Nov 28th-29th, 2005 336 13 UNFOLD CoP Meeting in Barcelona April 2005 1100 14 UNFOLD CoP Meeting in Braga (Portugal) June 2005 338 15 UNFOLD CoP Meeting in Glasgow, October 12th-14th 139
16 UNFOLD hands-on meeting in Valkenburg 2005 4104 More active
17 UNFOLD Paris Workshop March 2005 80 Less active
18 UNFOLD Presence at Alt-i-lab June 2005 54 Less active
19 UNFOLD Presence at Campus Virtual June 2005 (en castellano) 44 Less active
20 UNFOLD presence at the Online Educa Berlín 2004 343 21 UNFOLD session at the EADTU 2004 conference 422 22 UNFOLD Workshop at EUCEN Conference 2004 159
23 UNFOLD/Complutense Workshop , Madrid Dec 12-14, 2005 3963 More active 24 UNFOLD/ProLearn Valkenburg September 2005 1183
25 Getting started with the IMS LD Specification 4800 More active
Total Activity Nodes 41488
Total activity in Activity Nodes + users who logged-in
and lurked 47351
Active users, once registered and logged-in 892
Log analysis of use of UPF-UNFOLD server
D
ESCRIPTIONTitle Log analysis of use of UPF-UNFOLD server
Authors Nídia Berbegal, José Luis Santos, Sergio Sayago (UPF) Date January, 2006
Abstrac t
This document shows the results of the Log analysis of use of UPF-UNFOLD server during the period between August, 2005 and January, 2005.
This evaluation action corresponds to action 7 of the UNFOLD evaluation action plan 1.
The evaluation procedure consisted in an automatic analysis of UNFOLD log files. The results of this analysis indicate that the busiest month was December. The results also suggest that the majority of the accesses are done during the week, especially on Monday and Tuesday, during the morning. The results also reveal that the UNFOLD users find the UNFOLD web site introducing the following query words: unfold, learning, ims, design and ld. The most consulted documents, during this period of time, were the motplus guide and the collage presentation. The results also point out that the UNFOLD server seems to reply to the majority of the requests.
Executive summary
This report shows the analysis of the UNFOLD servers log files. This evaluation action corresponds to the D3 Evaluation Plan, action 8 “Log analysis of use of UNFOLD servers”.
This document contains the UPF-UNFOLD server analysis. This evaluation has covered the log files corresponding to the period between 1th August, 2005 and 30th January, 2006 (both included). The issues which have been analyzed are:
Activity in each month. Daily summary
Hourly summary Search word Operating systems Status code report File type
To carry out this analysis, it has been used the Analog1 6.0 tool. Analog 6.0 is a free web log analyzer,
multiplatform and implemented in Perl.
The results of this analysis point out that the busiest month was December. The results also suggest that the majority of the accesses are done during the week, especially on Monday and Tuesday, during the morning. The results also reveal that the UNFOLD users find the UNFOLD web site introducing the following query words: unfold, learning, ims, design and ld. The most consulted documents, during this period of time, were the motplus guide and the collage presentation. The results also point out that the UNFOLD server seems to reply to the majority of the requests.
Analysis of use of the UPF-UNFOLD server
This section shows the analysis of the:
Activity per month. Which is the month with the highest activity? Daily summary. Which is the day with the highest activity? Hourly summary. Which is the hour with the highest activity?
Search word. Which type of queries do our users write in order to find the UNFOLD web site? Operating systems. Which operating systems do our users use?
Status code report. Which is the availability of the UNFOLD server ? File type. Which are the most consulted documents?
All of the results are figured out as the number of requests for pages. Each of the following diagrams shows the total number of requests (‘reqs’ column) and the number of requests for pages (‘pages’ column).
Activity per month
Image 1: Activity in each month
This diagram represents the activity in each month. According to the analysis, the busiest month was December.
Daily summary
Image 2: Daily summary
Hourly summary
Image 3: Hourly summary
This diagram represents the total activity for each hour of the day. According to the analysis, the highest level of activity takes place two periods from 2 to 3 am and from 8 to 13 am.
Image 4: Search word
This picture shows the list of the top 30 query words by the number of requests, sorted by the number of request. According to the analysis, the most used query words are: unfold, learning, ims, design and ld.
Image 5: Operating systems
This diagram represents the operating systems used by the visitors. According to the analysis, the most used operating system is Windows (Windows XP and Windows 2000). Comparing Windows to Unix, the rate is nearly 21:1.
Image 6: Status code report
This diagram represents the HTTP status of the requests. The most interesting HTTP status which have been analysed are: 200, OK; 302, document found; 404, document not found.
According to the analysis, the majority of the HTTP requests have been OK, but, on the other hand, it is worth mentioning that there is a fair number of requests corresponding to “Document not found”.
Image 7: File type
This diagram represents the extension of the files related to the amount of traffic. According to this analysis, UNFOLD visitors accessed PDF more often than WORD or PPT files, which in part reflects project policy to favour the use of open formats. The most consulted PDF and WORD documents are:
# req. Date File
907 30/01/06 /unfold/general_resources_folder/tools/mot/english/guide_motplus.pdf 620 30/01/06 /unfold/about_folder/events/cops/glasgowcop/presentations/collage.pdf 566 30/01/06 /unfold/about_folder/events/online/frenchat/models for reuse1.pdf 464 26/01/06 /unfold/general_resources_folder/tools/mot/english/guide_motplus.pdf
421 26/01/06 /unfold/about_folder/events/general/unoldfprolearn/agenda_120905_published.pdf
257 30/01/06 /unfold/virtualhostroot/siteresources/archived/projectupdates/general_resources_folder/introsld/griffiths_final.pdf/download
195 29/01/06 unfold/virtualhostroot/general_resources_folder/tools/mot/motplusld.pdf
165 27/01/06 /about_folder/events/cops/glasgowcop/presentations/griffiths_glasgow.pdf
157 27/01/06 /unfold/virtualhostroot/about_folder/events/cops/portugal/pres/pacurar_braga.pdf
156 27/01/06 /unfold/unfold/about_folder/events/valk/route description kasteel oost.doc
The most significant results of this analysis indicate are that:
• The busiest month was December, showing that project activity was on an upward trend right up to the close of the project.
• The heaviest use of the server was in the week, especially on Monday and Tuesday. While activity was low on Saturday, as might be expected, there was significant activity on Sundays, more than on Wednesdays. The figures for hourly analysis show two periods of intense activity, 2 - 4 am, and 9am to 1 pm. It is suspected that the high activity in the night is due to web crawlers which take advantage of periods of low Web activity to catalogue the site (although visitors from other parts of the world no doubt also accessed the site). The mid day spike indicates that most users were accessing the site during their work hours. There was also, however, substantial activity on Sunday. This suggests that some users were investigating IMS LD because they wished to use it in the future, but that their current work did not support this.
• The most frequent query words used were unfold, learning, ims, design and ld. This indicates that a large number of accesses were by people looking for basic information about the specification, backing up the findings for the LN4LD site.
• The most frequently accessed documents in this period were the motplus guide and the collage presentation. These both contain practical information about graphical Learning Design editors. This confirms feedback from members that the need for easy to use editors is the most urgent challenge facing IMS LD, and also the findings for the LN4LD site.
Evaluating the context of UNFOLD results
Introduction
The IMS Learning Design specification represents something of a leap into the future of Learning
Technology due to its potential to revolutionize the way we do e-learning. At the point of its inception, it was envisaged that e-learning systems and their associated environments would conform to the specification and, in so doing, would add a new dimension to the single user click and test model which has been the mainstay of learning technology to date. The extent to which this has been achieved in the period following the initial release of version 1.0 of the specification in February 2003 forms the basis of this retrospective.
This report gathers together information gathered in the course of project activities, and through contacts with members. It is not, however, restricted to projects and researchers who have worked with UNFOLD, and web searches have been carried out in order to establish the wider picture.
Any insights afforded by this study will be instrumental in providing a point of departure for the support and promotion of further adoption.
A brief history: from EML to IMS LD
The history of the IMS Learning Design specifications is in part the history of the Educational Modelling Language developed by Rob Koper at the Open University of the Netherlands.
1997 marked a key turning point in the OUNL’s course delivery strategy as a result of its decision to propose e-learning to students. In order to provide an effective online learning experience, it was necessary to account for the various pedagogical approaches already in use at the university. This led to an in-depth survey of the pedagogies in use at the university (numbering over a hundred in total) followed by a process of abstraction intended to identify the characteristics common to the different pedagogies to create a pedagogical meta-language that could describe any and every teaching approach.
By the time IMS had turned its sights to the development of a specification that addressed the need for an activity centred description of learning processes early in 2001, Educational Modelling Language was well established as a well rounded specification and it was not long before it came to the attention of the IMS Learning design Working Group (LDWG) which was tasked at the time with creating a specification (Learning Design) which would account for learning processes. Due to the extent of the IMS project, it was considered necessary to base the specification on existing work namely EML, which was submitted to the LDWG in 2nd Quarter of 2001.
The IMS Learning Design specification was almost two years in the making, and after a series of
modifications version 1.0 was approved in February 2003 thus bringing the core of EML in to play with the need for functional interoperability and re-use in e-learning.
Architectures:
In the course of its lifespan, the Learning Design specification has inspired the design of three architectural models which have formed the basis of the development of a number of runtime and authoring tools. Following the time honoured tradition of international political treaties, these reference models have come to be known by the name of the town in which they were formulated hence the Valkenberg, Dagstuhl and Berlin reference architectures.
Valkenberg
In response to the complexity of the IMS Learning Design specification, the Valkenburg Group of developers was established by the OUNL in March 2002, the very first EML/LD related meeting, as a means of
designs and associated objects with the focus very firmly set on authoring, discovery and content
management. The "Valkenburg architecture" was drafted during this meeting in the form of a set of UML diagrams conceived by Jos Warmer of the OMG architecture to account for both the runtime and authoring aspects of the specification.
Dagstuhl
Up until the latter stages of 2003 access to the work achieved in Valkenberg had been more or less limited and so it was decided to host a follow-on meeting in Dagstuhl, Germany prior to the start date of the UNFOLD project early in 2004 with a view to writing a number of chapters for the Learning Design Handbook as part of a wider dissemination effort. Included in those chapters was the "Dagstuhl diagram" drafted by Scott Wilson, assistant director of CETIS. This diagram set out to provide a composite picture of a number of previously disparate diagrams that made up a collection of earlier work that centred on authoring and content management processes. Complete with accompanying narrative, the diagram was also intended to provide a more straightforward explanation of the original Valkenberg model that would be more readily accessible to those less familiar with UML.
Berlin
Towards the end of 2005 the UNFOLD project hosted it’s sixth and final CoP meeting in Berlin during which Wilson drafted a third model that was derived from a part of the Dagstuhl model that had not been included in the book chapter and which, unlike the previous two models, was concerned solely with runtime behaviour. Originally conceived by Wilson and Bill Olivier in Dagstuhl, this new design was intended as a way of explaining the conceptual model of what was then the new CopperCore engine proposal. This work was further developed in Berlin by Wilson in an attempt to harmonize it with work carried out on tools and systems such as SLeD/CCSI, PLEs, Web 2.0, and with non-LD workflow management systems all of which had been developing in the interim.
Communities
Any attempt at charting the life of the specification would be incomplete without reference to the project and its Communities of Practice (CoPs). With this in mind, the following sets out to give a brief overview of the project in relation to the various CoPs before turning to look in more detail at the individual groups and communities of which they are composed.
Communities of Practice
Current Research groups and LD communities
Following the official IMS publication of the Learning Design specification the UNFOLD project was set up with funding from the Technology Enhanced Learning Programme as a means of promoting the adoption, implementation and use of LD. The project was set in motion a year after the release of the specification in January 2004 and during its two year lifespan it has sought to disseminate information on Learning Design through a number of initiatives designed to help cascade information down to interested parties ranging from software developers through to the authors of learning materials, educational researchers and teachers. Such initiatives have included the creation of web-based portals, and the running of numerous face-to-face meetings and workshops in the context of a support system for three Communities of Practice set up in July 2004. These Communities of Practice included a community of system developers, learning designers and teachers respectively and were conceived of as a way of bringing interested parties together to work with the specification.
UNFOLD communities of practice not withstanding, the UNFOLD project has actively sought and attracted participation from a number of different organisations, groups and communities from across Europe
(particularly IST funded projects) and, to a lesser extent, from countries outside the continent. The fact that so many different groups have sought to get involved in Learning Design, and, by extension, in the work carried out by UNFOLD, testifies to the interest aroused by the specification as well as to the success of the
UNFOLD project in raising its profile and disseminating the necessary information.
1) Work groups seeking to modify or improve specifications including IMS LD
Such groups include CETIS (www.cetis.ac.uk) which represents Further and Higher Education in the UK on international educational standards initiatives, including the IMS Global Learning Consortium. The group is funded by the UK’s Join Information System Committee (www.jisc.ac.uk) which supports the use and implementation of learning technology in Further and Higher Education through consultation. The TELCERT project (http://www.opengroup.org/telcert/) is an IST funded initiative seeking to establish a conformance testing suite of tools for IMS related specifications.
2) Groups using and implementing the Learning Design specification for a specific purpose
Spain boasts a number of initiatives in this category including the University of Vigo ( http://www-gist.det.uvigo.es/~mcaeiro/thesis.html) which is home to the Telecom Systems Engineering group (Ingeniería de Sistemas Telemáticos) who are currently exploring reusability and interoperability of Units of Leaning written in IMS LD. The DEI laboratory at the University Carlos III are working on ways to enable shared authorship of learning resources by affording access to the XML code for modification and
commentary as part of the CASLO project (http://caslo.dei.inf.uc3m.es) and the department of
Telecommunications Engineering at the University of Valladolid are heavily involved in flow patterns for computer based collaborative learning through their COLLAGE project (http://gsic.tel.uva.es/collage). In addition, the Group of Interactive technologies (www.tecn.upf.es/gti) at the University Pompeu Fabra in Barceloana are involved in the development of software and web services and in many ways can be considered the driving force behind the UNFOLD project.
Canada is also home to similar initiatives. The Simon Fraser University in Canada hosts a research group, the Laboratory for Ontological Research (http://lore.iat.sfu.ca) which specialises in knowledge management and modelling online learning systems. Their latest project includes the development of a repository for
interoperable learning objects. Another Canadian enterprise includes the T5 instructional design model (Tasks, Tutoring, Teamwork, Topic resources, Tools) at the LT3 Centre (http://lt3.uwaterloo.ca) at the University of Waterloo. The model is included as a gateway to Learning Management Systems to provide opportunities for collaborative online learning. The theory behind the model is applied to the LearningMapR application, a pedagogical design tool intended to help teachers choose designs from learning design repositories that are most appropriate to their needs. The tool is a step on the way to creating an authoring system that is IMS-LD compliant.
Salerno University (www.unisa.it) in Italy are also involved in Learning Design via the ELeGI Project for which they construct Units of Learning tailored to students needs. They are also involved in the development of computer applications designed to verify interoperability in e-learning systems and content.
3) Projects and virtual communities seeking to make the specification more widely accessible through various profile raising initiatives.
Projects seeking to disseminate information on the specification includes initiatives such as UNFOLD (www.unfold-project.net), a Framework Six program funded by the European Union. The project focuses on the implementation and use of e-learning standards in terms of individual and multi-user activities and open source applications and is central to the dissemination effort relating to the IMS Learning Design
specification. Similarly the Canadian R2R (http://commons.ucalgary.ca/weblogs/learningdesign) Repository to Reality (R2R) project also works with virtual communities with an interest in IMS LD. The aim of the project isto increase and improve contributions in tools, best practices and recommendations made in relation to the specification. This is done through the capture and dissemination of information relating to the
standard.
Learning Design, without explicit reference to the specification, and can be considered a transitional phase between writing a narrative and devising a template.
Ladie (www.elframework.org/refmodels/ladie) is yet another JISC funded inititiative coming out of the UK. Standing for “Learning Activity Design in Education”, LADIE is a reference model which supports “the design and construction of learning activities and the discovery, specification, sequencing and packaging of content”. Moreover, the model also supports the realization of the environment where the learning activities are played out. LADIE, in conjunction with the DialogPlus (www.dialogplus.org) project, also combines pedagogy and didactics into the process.
“Digital Libraries in Support of Innovative Approaches to Learning and Teaching in Geography” is a sister project to LADIE, also financed by the JISC which develops learning objects for reuse through the Alexandria Digital Library. The project aims to provide a technical infrastructure that gives access to Geographic digital resources and learning and teaching tools in both the UK and the USA.
Pool (http://www.edusplash.net), or the Portal for Online Objects in Learning is a consortium of diverse public and private organisations in the education sector who are developing an infrastructure for learning object repositories. To do this the group has researched the necessary architectures taking into account issues relating to metadata and software and hardware. Thanks to the contributions of this virtual work group, the SPLASH tool was developed using peer-to-peer protocol as a means of categorizing learning objects. Although the tool was finalized in 2002 it is compatible with eduSource, another Canadian project (http://www.edusource.ca) whose goal is to set up and implement a network of repositories that comply to international standards. The project also includes a suite of tools for locating, referencing and using learning objects with educational technology.
Iclass (www.iclass.info, Europa)
“Intelligent cognitive-based open learning system for schools” is a project which is developing an intelligent cognitive-based open learning system that adapts to the needs of the individual learner. The aim is to produce a system whose architecture is founded on an ontology which will help sequence and adapt knowledge in the form of learning objects according to the learner’s level of understanding and to promote collaborative work. Lornet (www.lornet.org, Canadá)
Central to the resusability of learning objects over the internet, LORNET attempts to define the mechanisms of interoperability which enable searches and allows for the adaptation of the final use context. This involves the construction of various prototypes based on the aggregation of multi-lingual and multi-media learning object repositiories. LORNET uses a variety of e-learning specifications (mainly IMS) to carry out critical analyses.
.Groups and projects that use diverse e-learning specifications for inspiration and support without taking anything away from their own modelling systems
The Modular Object-Oriented Dynamic Learning Environment or Moodle (http://moodle.org) is an open source course management system (CMS) developed in PHP which is based on the social constructivist precepts of a virtual and highly diverse educational community throughout the world. It was developed under the GNU (Free Software Foundation 1989) which allows any interested party to modify and adapt the application code for their own purposes. Moodle has been translated into many languages and its virtual community is one of the most active among educators and learning designers. Interoperability is currently being established between the LAMS (learning Activity Management System) system and the IMS Learning Design specification. The LAMS foundation (www.lamsinternational.com) based at Macquarie University in Australia, is heavily based on a pedagogical view of the construction of educational materials as opposed to a purely technical view. Teachers appreciate the simplicity involved in designing for lessons and courses. The system is Learning Design “inspired”, the next version of LAMS will be open source and will develop interoperability of learning objects with other systems.
in Moodle thus supporting implementation and support within virtual communities. An implementation of IMS LD has now been released for .LRN (http://strauss.gast.it.uc3m.es/).
The Boddington Open Source Virtual Learning Environment has also expressed interest in becoming IMS LD compliant, and the fact that it is Java based might facilitate integration of CopperCore.
A number of French language and bilingual French/English projects also fall in this category. The University Pierre et Marie-Curie Paris 6 hosts a research group headed by Monique Baron which has designed an editor and simulator based on the EML specification (OUNL, 2000) the precursor to IMS Learning Design. The main objective is to provide the educational community with a tool capable of modelling units of teaching and using learning objects in a simple manner and adapted according to the needs of each teacher.
Another contribution from the French speaking world includes LICEf, a research group based at the University of Quebec (www.licef.teluq.uquebec.ca/francais/real/mot.htm) which is headed by Gilbert Paquette. In response to the emergence of e-learning standards, the group developed the MOTPlus
application (available in both French and English) which provides a method of creating pedagogical materials and which holds many similarities with IMS Learning Design without being completely compatible with it. The application produces a graphical representation of any given didactic model used in the development of a unit of learning. The editor is similar to ASK LDT in its graphical representations of learning design but, as with ASK LDT it does not allow imports of resources developed in other applications.
Labset, of the University of Liège (www.labset.net) is another French language initiative which has overseen the development and implementation of the Eight Learning Event (8LEM) methodology headed by
Dominique Verpoorten. This model generates more than 80 typical pedagogical scenarios which can be adapted and personalised to the needs of individual clients. The tool is designed to help teachers adapt online teaching methodologies through the creation of templates which identify the learning process used without any prior knowledge on the part of the teacher. Work is being carried out in the iClass project to implement the 8LEM using IMS LD.
The WebCT / Blackboard “BlackCT” corporation is clearly the key player in the VLE market, despite the progress of OpenSource alternatives. Blackboard has a memorandum of understanding with the Open University of the Netherlands to use their Edubox EML player as the basis for an IMS LD player. This involves developing stylesheets to transform IMS LD UoLs to EML so that they can be played. Blackboard have not been willing to provide information on progress in this.
Another industrial development is the Elive LD Suite (www.elive-ld.com) which is under development by GmbH in Germany. This application enables editing of learning scenarios in a graphical format. It is based on the IMS LD specification and will allow the export of Level A units of learning. The final objective of eLive is to provide a visual tool which can generate concept maps, structures and methods in accordance with pedagogical models selected by the learning designer or teacher. This development project seems to have gone into hibernation.
This has been the pattern of industrial development (see section below). However a positive sign is that the German company theCode AG has released SchoolCMS a web-based authoring tool intended for the production of online courses with the ability to handle and export Level A compliant Learning Designs.
4) Groups which are involved in the development of computer applications which interpret the specifications and which allow for simple and effective use by different interest groups.
immediate way to run previously designed Units of Learning. The IMS LD editor works with all three levels of the specification although level C has not been fully trialled to date.
The University of Duisburg
(http://www.unfoldproject.net/general_resources_folder/cosmos_tool.zip) are developing a similar product in the form of the Duisburg Collaborative Learning Authoring System which allows for the design of units of learning through its own system of XML notation which adds conditions and properties to concrete events. The similarity with IMS LD and the need to associate the system with an existing specification has led its creators, Young Wu Miao and Kai Hoeksema, to adapt the system to level B to create COSMOS.
Advanced e-Services for the Knowledge Society (ASK) (www.certh.gr/&en/home/index.htm) is a group based in the Institute of Information Technology and Telecommunications at the University of Piraeus, Greece. Demetrios Sampson heads ASK, a research unit which focuses on the creation of educational software based on standards and metadata. They are developing an authoring tool called ASK LDT (www.ask.iti.gr) which enables the creation of IMS LD units of learning that are also compatible with ASK LOM-RM which manage learning object repositories.
Cepiah (www.hds.utc.fr/cepiah) is a research group based at the Université de Technologie de Compiègne -Centre de Recherches de Royallieu and focuses on the development of modules which enables the
improvement and development of design, and evaluation of teaching models within an educational website. netUniversite consists of an editor which generates web pages and which includes an administrative module that aims to help teachers produce LD compliant lessons without burdening them with too much technical talk. As a result, this project attempts to work through the entire cycle of design for online learning scenarios from design time to runtime.
5) Multidisciplinary groups which carry out some or all of the aforementioned tasks.
This category is made up of two major centres for Open and distance learning namely the OUNL (Open University of the Netherlands) and the OUUK (open University of the United Kingdom). Based in the Netherlands at the OUNL, the Educational Technology Expertise Center, OTEC
(www.ou.nl/eCache/DEF/5/071.html) focuses on educational technology. More specifically the
Department of Development develops standards based projects and it was the creation of EML (Educational Modelling Language) which served as the foundations for the implementation of the IMS Learning Design specification. The group also works on the development of specifications based runtime engines
(Coppercore, www.coppercore.org), and editors (Copperauthor, www.copperauthor.org), virtual learning environments (Alfanet, http://alfanet.ia.uned.es/ and Edubox (http://ldplayer.sourceforge.net/),
The centre also maintains groups seeking to refine and further develop IMS specifications, interoperability (with SCORM, Moodle and LAMS), development of dissemination projects (UNFOLD, Prolearn, Scope), conferences (Online Educa Berlin, Online Educa Madrid, SIGOSSE, ICAL), publications (JCAL, ILE, JIME, ET&S) courses and workshops (Campus Virtual, Eucen…), oficial regulatory bodies (IMS, IEE) etc. The main focus of research currently lies in development, debate and dissemination in relation to educational technology.
Publications
Much has been written on the subject of Learning Design over the course of its short life in journals,
magazines and on websites, but there have been three landmark publications which deserve a special mention by virtue of their comprehensiveness. These include the Springer-Verlag book on Learning Design “Learning Design: A Handbook on Modelling and Delivering Networked Education and Training” which was published in early 2005. The book includes contributions from 38 authors and consists of three major lines of enquiry: Specification, architectures and tools, Designing E-Learning Tools and Experience.
Subsequent online discussion of the book gave rise to a second publication, the special August 2005 issue of the Journal of Interactive Media in Education (JIME) (
http://www-jime.open.ac.uk/)
The journal includes a total of seventeen articles each focussing on different aspects of the specification which have given rise to a good deal of debate on how best to take the IMS LD specification forward on all levels.The UNFOLD project produced a booklet to inform people both about the project as well as the specification. The booklet provides an easy to read non-technical overview of the specification making it accessible to newcomers with an interest in activity-centred eLearning. Five hundred copies were published and strategically circulated free of charge in time for the New Year of 2005/2006.
Implementation with learners
Despite the efforts of the UNFOLD project and the interest surrounding the specification, concrete
implementations of IMD LD have been slow to emerge due to a large extent to the relative immaturity of the tools available in the first part of the project lifecycle. However, with the acceleration of tool development in the latter part of the project, Liverpool Hope University came into view as one of the only institutions to have implemented the Learning Design specification under the supervision of Mark Barret-Baxendale and his team Paul Hazlewood and Amanda Oddie at the Deanery of Business and Computer sciences.
Implementation of the specification took place over a single term (autumn 2005) in the context of a second year Higher Education module entitled “Design and Implementation of Multi-media.” In practical terms implementation involved the use of levels A and B of the specification and required the installation of Coppercore and SLED (see Tools) to run courses developed with Learning Design by staff members at Liverpool Hope.
Given the unprecedented nature of the trial, the specification proved remarkably resilient to the rigours of implementation in a real educational context and some valuable feedback was gained as to the needs of end users (both learners and staff) complete with suggestions for future attempts at implementation. These included the proposal that students be shielded from direct contact with the specification and that they should not be confronted with tools that require explicit knowledge of the specification. In addition, a number of issues were identified in relation to usability of the Learning Design player, ways to set up Learning Designs without over complicating things (i.e. setting up level B properties was easier to do in XML than in
RELOAD.), handling group work, the apparent lack of flexibility in an LD run, the granularity of Learning design, performance issues and file formats in SLeD.
Nevertheless, the general consensus among the users at Liverpool Hope was that Learning Design provided a valuable means of capturing traditional practice for subsequent execution in a virtual environment. In addition, the specification was praised for its ability to allow students the option of working and progressing at their own individual pace. It was also considered conducive to collaboration in blended learning situations.
Industry
counts, not least the desire to produce a system complete with an integrated player capable of realizing the run of a UoL for users who are not necessarily conversant with the LD specification. A further concern of the group is to introduce Learning Design into industrial and commercial products to complement the work and implementations that are usually only found in use in the field of Higher Education.
Tools
The number of available Learning Design tools provides an indication of the degree of interest in the specification as well as the extent to which it is being implemented. The first major inventory of tools carried out in the summer of 2004 (Olivier 2004), (Britain 2004), charted the development of some of today’s most well established tools. At a glance the tools at the time included a significant number of open source initiatives including both general purpose tools needed to ensure the functionality of the Valkenberg Group Reference Architecture and more specialised tools needed to build and deliver Units of Learning. Examples of open source tools (general and specific) included Coppercore, RELOAD, Eduplone and MOT+, all of which are available as stabilised downloads today. A number of commercial implementations were also detailed in the survey including, Perot Systems Edubox, LAMS, eLive LD suite, and GTK Press.
By the time of the second major inventory, December 2005, a significant number of new additions to both the open source and commercial tool suites had been documented. All of the tools developed to date are included in the tables below and are categorised according to their intended purpose and use as editors, runtime players or repositories.
Learning Design editors
Higher level editors designed to facilitate the process of XML coding currently include:
Name Producer Purpose IMS LD Level
Ownership
MOT+ LICEF,
Université de Quebec
General purpose graphical editor
A,B and C Open Source
eLive eLive GmbH
Germany
General purpose graphical editor
A and B Proprietary
EduPlone
Sequencer EduPlone specialised template based editor
A Open Source
EduCreator
Editor Chronotech General purpose editor
A Proprietary
LAMS – LAMS
Foundation
Learning Activity Management System with
IMS LD
Level A
export NOTE: Not fully
compliant with IMS-LD
A Open source
COLLAGE University of
Valladolid High-level specialized Learning
Design authoring
tool for
collaborative learning
Examples of low level editors affording technicians direct access to the code are listed as follows:
Name Producer Purpose IMS LD
Level Ownership
aLFanet aL.Fanet
project
General purpose tree editor
A Open source
ASK-LDT Editor EU project ICLASS: Informatics and Telematics Institute. (CERTH) Greece Graphical
editor. A, B Freeware
CopperAuthor Open
University of the
Netherlands
Tree based
editor. A Open source
Komposer GTK Press Tree based
editor A Proprietary
COSMOS Editor
Yongwu Miao, University of Duisburg
General Tree based editor
A, B and C Open source
RELOAD Learning Design Editor
RELOAD
Project (JISC) General Purpose tree
based editor
Runtime players
The table below includes the latest list of runtime players currently available:
Name Producer Purpose IMS LD
Level Ownership
Coppercore Learning Design Engine
OUNL Core of
Learning Design player
A, B and C Open source
RELOAD
Player RELOAD project Viewer for integration with
RELOAD editor
A,B and C Open Source
SLED Player OU UK and
OUNL Service based LD player built on
CopperCore
A,B and C Open source
It should be noted that as interest in the specification grew so did the number of secondary tools intended to provide additional functionality to the specification. These include the DialogPlus Toolkit which, developed as part of the DialogPlus project, aim to produce taxonomies of learning for possible use in the learning design process. Similarly, the CASLO project of the University Carlos III in Madrid provides an environment for collaboration in development of learning objects.
Units of Learning (UoLs)
The combination of tool development and the various dissemination and training efforts carried out under the UNFOLD banner can be said to have culminated in the production of a number of Units of Learning. A Unit of Learning (UoL) as understood in the context of the UNFOLD project entails an IMS LD compatible lesson or course created in XML code. The UoL must always include an IMS manifest file and will also carry a resource folder complete with study support files (i.e. HTML pages, text documents, audio and video files etc.). As well as being an end user product, the unit of learning helps provide proof of concept by enabling users to see, use and experience the specification through the agency of an IMS LD compliant tool.
By January 2005 a number of units of learning, ten in total, had been created by members of the OUNL and were made freely available for download from the Moodle based LN4LD website
http://moodle.learningnetworks.org/course/view.php?id=20.
These UoLs were, for the most part, Level A examples and included only a couple at level B. Today however that number has more than tripled to 37, a sizeable increase, and all three levels of the specification are represented (Level A: 21, Level B: 13, Level C: 1 and Level 0: 2) thanks to the active participation of UNFOLD members from various institutions throughout Europe and beyond.
Themes of the UoLs range from “The most simple UoL” (level A) through to the more technical “Use case to show an email sent by the system to the teacher when a learner ends an activity. Need of setting-up a SMTP server”.
Whilst the majority of UoLs are intended for an Anglophone audience, a number have been created in Spanish.
concrete terms. In conjunction with the various editors and players available, they have formed the basis of a number of successful UNFOLD/Learning Design workshops and meetings throughout the period from January to December 2005.
It is anticipated that the number of available UoLs will continue to grow in tandem with the continued development of the specification.
Conclusion
Over the last two years the IMS Learning Design specification has courted the attention of a host of interested parties from developers to teachers, Industry to standards and specifications watchdogs. It has spawned the development of many specialised tools and has been adopted by leading Open Source eLearning initiatives. The specification has also been the subject of much discussion and debate thanks largely to dissemination programmes such as the UNFOLD project. Whilst the UNFOLD project in its current form has finally run its course after two years, it is expected that the work achieved to date will be carried forward by other projects with an interest in IMS LD namely TEN Competence and ProLearn providing more opportunities for the adoption, implementation and possible development of the specification.
Evaluation reports on UNFOLD events
Braga - UNFOLD Communities of Practice meeting
Questionnaire
This questionnaire will help us improve the UNFOLD Communities of Practice meetings.
The questionnaire is divided into five parts: (i) geographical spread and impact of adoption,
(ii) workshop information and organisation, (iii) usefulness of the meeting, (iv)
participation and (v) suggestions.
All the data which you provide will be kept strictly private, and used solely to evaluate the UNFOLD project dissemination activities. The results of this questionnaire will be used exclusively to prepare an evaluation report, which will be used to improve the performance of the project, and will be submitted, to the European Commission.
We estimate that you can complete this questionnaire in less than 10 minutes.
Thank you in advance for your collaboration.
Part 1: Geographical Spread and Impact of Adoption
1. Where are you from?
In your country:
2. How would you rate the adoption of IMS-LD with learners?
High 1 – 2 – 3 – 4 – 5 Low
3. How would you rate the adoption of IMS-LD regarding the development of tools?
High 1 – 2 – 3 – 4 – 5 Low
4. How would you rate the support which UNFOLD offers in order to open up new opportunities to use IMS-LD?
High 1 – 2 – 3 – 4 – 5 Low
In your company / institution:
6. How would you rate the adoption of IMS-LD with learners?
High 1 – 2 – 3 – 4 – 5 Low
7. How would you rate the adoption of IMS-LD regarding the development of tools?
High 1 – 2 – 3 – 4 – 5 Low
8. How would you rate the support which UNFOLD offers in order to open up new opportunities to use IMS-LD?
High 1 – 2 – 3 – 4 – 5 Low
Part 2: Meeting organisation and quality of the information
9. The information provided about the UNFOLD meeting on the Web was
Clear 1 – 2 – 3 – 4 – 5 Unclear
Sufficient 1 – 2 – 3 – 4 – 5 Insufficient
Easy to read 1 – 2 – 3 – 4 – 5 Difficult to read
10. The quality of the information offered by the speakers was
Excellent 1 – 2 –3 – 4 – 5 Very poor
Part 3: Usefulness of the meeting
11.
How often
do you
work with e-learning Standards
(read a document about the spec,
work on developing a compliant application, design UoLs, work with learners on UoLs,
etc.)
every working day
most working days
every week
every month
never
12. Have you used
IMS Learning Design with Learners
?
No
13. Do you hope that UNFOLD will help
open up new opportunities
to use IMS LD?
Yes. How?
No
Part 4: Participation
14. Do you
participate
regularly in any
on-line forum
?
Yes. Which is your motivation to take part?
No. What would motivate you to take part?
15. Do you
prefer e-mailing list
debate to
forum debate
?
Yes. Which are the advantages you see in e-mail list debates?
No. Which are the advantages you see in forum debates?
16. Are you
willing to participate
in the different UNFOLD Communities of Practices?
Yes.
No.
17. Have you participated in some of the UNFOLD online events?
Yes.
No.
Please, specify why:
18. UNFOLD provides two types of participation: face-to-face and online events. Which type of participation do you think are the most valuable for you?
Face-to-face meetings Online events. Please, specify why:
Part 5: Suggestions
19. Which aspects did you find most valuable about your participation in the meeting?
20. What aspects did you find least valuable about the meeting?
Results summary
Objectives
This evaluation aimed to assess the support which UNFOLD offers to the Communities of Practices. As stated in the first UNFOLD Evaluation Deliverable, this evaluation builds upon the previous evaluations in order to address: (i) the relationship between the geographical spread and the impact on adoption, and (ii) the lack of participation in online discussions.
Consequently, this evaluation was designed to cover the following aspects:
(i) the relationship between geographical spread and the impact on adoption
(ii) meeting organisation and quality of the information
a. organisation and quality
b. usefulness
c. participation
d. suggesions
This evaluation action corresponds to the UNFOLD Evaluation plan Scenario 1, “Scenario 1. Effectiveness of awareness raising material and Web”, Scenario 2, “Scenario 2: Resources for each CoP”, Scenario 5,
“Scenario 5: User group satisfaction with f2f meetings/workshops”, Scenario 6, “Scenario 6: The usefulness of the CoPs to their participants and organizations”.
Procedure
The procedure consisted of a structured questionnaire. This questionnaire was filled in by 26 respondents. The questionnaire was divided into the following sections:
Geographical spread and impact on adoption. This section gathers information about the impact on adoption of tools and IMS LD implementations as a function of the geographical spread in countries and institutions. Meeting organisation, quality of the information and usefulness of the meeting. This section gathers information about the quality of the information provided by the speaker and the usefulness of workshop in relation to the attendee’s professional activity.
Participation. This section gathers information about which type of participation is the most convenient for attendees and why.
Geographical spread and impact on adoption
UNFOLD is an interesting source of information, all the attendees to this meeting came from the European Union, only one come from broader, from Canada.
In general, the rate of adoption of IMS-LD with learners in countries and institutions is very low. For example, 72% of the attendees rated the adoption of IMS-LS with learners very low in their country.
The adoption of IMS-LD regarding tools is quite better than the adoption of IMS-LD with learners. For instance, only 34.6 % of the attendees rated the adoption of IMS-LD regarding tools as low in their country, the same percentage that rated this adoption as normal. Only 19.2 of the attendees rated it as very low. Bulgaria and the Netherlands hold the best adoption of IMS-LD regarding tools, Belgium the lowest. Regarding institutions, Spain and the Netherlands seem to have the best adoption, Belgium, again, the lowest. The results are more positive for the opportunities which UNFOLD offers to the adoption of IMS-LD, 50% of the attendees rated it as high regarding countries, and 32% rated it as normal regarding institutions. However, it is worth mentioning that the level of support, especially regarding institutions, is a bit lower in England. Regarding countries it is lower in Belgium and France. Canada and Italy, on the other side, have the best ratings.
Meeting organisation and quality of the information
The meeting was very good organised, the information available at the web site was clear and easy to read, and the quality information provided by the speakers was very good.
The results clearly point out that most of those attendees rated excellent or good the information provided about the UNFOLD meeting on the web. For example, more than 60% of those attendees rated that the information was clear, and 50% said that it was easy to read. The results also reveal that the quality of the information offered by the speakers was good.
Usefulness of the meeting
Another interesting result is that the highest frequency of use of IMS corresponds to every week (42%), followed by every month and most working days (23%) and every working day (8%). IMS Learning Design is clearly not being used with learners. Indeed, our results show that more than 90% of the attendees do not use IMS-LD with learners. Another 90% expect Unfold to open new opportunities.
Participation
First of all, the results clearly show that the 91.7% of the people are willing to participate in the UNFOLD Communities of Practice. Nevertheless, the results also point out that only 56% of those attendees did participate in online events. Indeed, the results reveal that nearly 50% of the people prefer face-to-face meeting to online events.
• The direct contact with people is extremely valuable.
• The people have the opportunity to meet the specification and tools creators.
• More time to discuss (by e-mail is somewhat more difficult than f2f).
However, the general sensation is that both types of events are needed and important in a combined way, online events to stay up to date and from time to time intensive meetings face to face seems to be the best option.
Suggestions
First of all, there were 30 comments for the most valuable aspects and 23 for the less ones, 5 of them pointing out that there was not any less valuable aspect
Most valuable things:
• The most valuable aspect for the participants was the chance for discussing and meeting people with the same research interests.
• It was also valuable the information offered about IMS LD. Less valuable things:
• There were not really extensive hands-on sessions.
Part 1: Geographical spread and impact on adoption
Geographical Spread
Next table shows the number of those attendees to the UNFOLD CoP meeting in Braga, Portugal. Chart 1: Geographical spread
0 1 2 3 4 5 6 7 8 Belgi um Bulga ria Cana da Engla nd Fran ce Italy Portu gal Spain The Neth erlan ds
All the attendees, except one -who came from Canada-, came from the European Union. Of these attendees, there were 8 participants from Portugal, 6 who came from England, 3 from Spain, 2 from Belgium, France and the Netherlands, and 1 from Bulgaria and Italy.
Impact on adoption in your country
Adoption of IMS-LD with learners
1 2 3 4 5
High 0 % 0 % 4 % 24 % 72 % Low
4.5
In general, the results of the adoption of IMS-LD with learners, point out that it is very low. Next table shows the adoption of IMS-LD with learners as a function of the geographical spread:
Table 1: Adoption of IMS-LD with learners in countries – geographical spread
Geographical spread
#
attendees
Adoption
Belgium 2 5
Bulgaria 1 4
England 6 4,83
France 2 5
Italy 1 5
Portugal 8 4,75
Spain 3 4
The Netherlands 1 4
As can be observed in Table 2, the adoption of IMS-LD with learners in Bulgaria, Spain and the Netherlands, is quite better than in other countries, despite there was only an attendee from Italy and Bulgaria and this result may be not significant. This adoption is very low in Belgium, Canada, France and Italy.
Adoption of IMS-LD regarding the development of tools
1 2 3 4 5
High 0 % 11.5% 34.6% 34.6% 19.2% Low
3.615
In general, the results show that the adoption of IMS-LD regarding the development of tools is a normal. The results are clearly better than the adoption of IMS-LD with learners. Next table shows the adoption of IMS-LD regarding the development of tools as a function of geographical spread:
Table 2: Adoption of IMS-LD regarding tools - country
Geographical spread
#
attendees
Adoption
Belgium 2 5
Bulgaria 1 3
Canada 1 4
England 6 3,17
France 2 4,5
Italy 1 4
Portugal 8 3,63
Spain 3 3,33
The Netherlands 2 3
This table suggests that Bulgaria and the Netherlands have the best adoption of IMS-LD regarding tools, in the other side Belgium is the countries where the adoption of tools is poorer. Likewise, there are 5 of the countries below the average.
1 2 3 4 5
High 23.1 % 50 % 23.1 % 3.85 % 0 % Low
2.077
In general, the results clearly point out that UNFOLD offers a number of opportunities to the adoption of IMS-LD. Next table shows the rate of expectations as a function of the geographical spread:
Table 3: Open up new opportunities - precedence
Geographical spread
#
attendees
Adoption
Belgium 2 3
Bulgaria 1 2
Canada 1 1
England 6 2,17
France 2 3
Italy 1 1
Portugal 8 2,13
Spain 3 1,33
The Netherlands 2 2
This table suggests that the support which UNFOLD offers is higher or higher appreciated in Canada and Italy, followed by Spain. Belgium and France are the counties where this support is poorer.
Institutions
• 14 of the attendees came from the university and research sector.
• 1 of them was a content provider.
• Another 1 was a content developer.
Impact on adoption in your institution
Adoption of IMS-LD with learners
1 2 3 4 5
High 0 % 3.85 % 0 % 19.2 % 76.9 % Low
4.692
Table 4: Adoption of IMS-LD with learners - institution
Geographical spread
#
attendees
Adoption
Belgium 2 5
Bulgaria 1 4
Canada 1 5
England 6 5
France 2 5
Italy 1 5
Portugal 8 4,75
Spain 3 3,67
The Netherlands 2 4,5
This table suggests that Spain has the best adoption of IMS-LD with learner in institutions or companies. Belgium, Canada, England, France and Italy, are on the other side.
Adoption of IMS-LD regarding the development of tools
1 2 3 4 5
High 4 % 16 % 32 % 28 % 20 % Low
3.346
In general, the results show that the adoption of IMS-LD regarding the development of tools is quite low. However, the adoption of IMS-LD regarding the development of tools is clearly better than the adoption of IMS-LD with learners. Next table shows the adoption of IMS-LD regarding the development of tools as a function of countries:
Table 5: Adoption of IMS-LD regarding tools - institutions
Geographical spread
#
attendees
Adoption
Belgium 2 5
Bulgaria 1 3
Canada 1 4
England 6 3,67
France 2 3,5
Italy 1 3
Portugal 7 3,43
The Netherlands 2 2
This table suggests that there are countries where this adoption is very low, like in Belgium. Otherwise, there are some others were it is quite good, like in Spain and, specially, the Netherlands. Bulgaria, France, Italy and Portugal are in the average.
The support which UNFOLD offers to open up new opportunities
1 2 3 4 5
High 24 % 28 % 32 % 4 % 12 % Low
2.52
In general, the results clearly reveal that the support which UNFOLD offers to open up new opportunities is regarded rather well within the context of companies and institutions, however it is quite lower than the results obtained for the countries. Next table shows the support which UNFOLD offers as a function of the geographical spread:
Table 6: UNFOLD opportunities - institutions
Geographical spread
#
attendees
Adoption
Belgium 2 2,5
Bulgaria 1 1
Canada 1 1
England 6 3,17
France 1 2
Italy 1 1
Portugal 8 2,88
Spain 3 2,33
The Netherlands 2 2
This table suggests that the support which UNFOLD offers to open up new opportunities is lower in England and Portugal, and excellent in Bulgaria, Canada and Italy.
PART 2: Meeting organisation and quality of the information
A) ORGANIZATION AND QUALITY
Clear
1 2 3 4 5
Excellent
61.5 % 30.8 % 0 % 3.85 % 3.85 % VeryPoor
1.577
Sufficient
1 2 3 4 5
Excellent
46.2 % 42.3 % 3.85 % 3.85 % 3.85 % VeryPoor
1.769
Easy to read
1 2 3 4 5
Excellent
50 % 34.6 % 7.69 % 3.85 % 3.85 % VeryPoor
1.769
The results clearly point out that most of those attendees rated excellent or very good the information provided about the UNFOLD meeting on the web.
The quality of the information offered by the speakers was
1 2 3 4 5
Excellent
26.9% 53.8 % 15.4 % 0 % 3.85 % VeryPoor
2
The results clearly reveal that most of those attendees rated excellent or very good the quality of the information offered by the speakers.
How often do you work with eLearning standards?
The results suggest that most of those attendees work with eLearning standards frequently. The highest frequency corresponds to every week, followed by most working day and every month. Only 4% of the attendees never work with eLearning standards.
23%
42% 8%
23% 4%
Every month Every week Every working day Most working days Never
Have you used IMS Learning Design with Learners?
No Yes
92 % 8 %
The results clearly show the IMS Learning Design is not being used with learners. Previous results might shed some light on this issue:
• the number of available tools is quite poor
•
