Instructional Design Document: Sustainable Reusable Learning Object Training

Instructional Design Document: Sustainable Reusable

Learning Object Training

EDIT 705, Spring 2016

Problem Definition

Starting in March of 2016, content created by Management Concepts’ Learning Development (LD) division has been developed exclusively using the company’s new reusable learning object (RLO) structure in Xyleme, the company’s learning content management system. This mandate comes directly from the president of Management Concepts, Steve Maier. To prepare for this shift, current instructional design (ID) staff received training on how to implement the RLO strategy in a large group workshop in February 2016. While the workshop was a quick and efficient way to train current staff, it is not a model that can be implemented for new IDs that join the LD team after this date. It also does not support the refresher training needs of current IDs who may not be assigned to a new development project for several months after the workshop ends.

Organizing course content in an RLO structure has numerous benefits. IDs will build content in a consistent, modular manner that facilitates its reuse for both client and off-the-shelf delivery. Content is organized in a database that allows IDs to quickly assess whether a developed module already exists for the subject in question. This method will save time, avoid duplication, and give IDs greater flexibility when designing custom content.1

We recommend the development of a long-term internal training program that can be used to explain to new IDs what a RLO is; why it is important at Management Concepts; and how it is used to develop content for different modalities, including instructor-led training (ILT), remote experience (RE), and virtual live-classroom (VLC). The program will also support the on-the-job training needs of current IDs that require refresher knowledge. Through our efforts, we hope to effect both a smooth transition to the new RLO structure, as well as a lasting process for maintaining the skills of individual designers.

Learner and Context Analysis

The primary audience for the RLO Implementation training consists of current and future IDs within the Learning Development division of Management Concepts. The new RLO structure within Xyleme is set to be a central part of the work performed by IDs. The proposed training will have a major impact on the success of the RLO initiative in general.

1 _{Barritt, Chuck and F. Lee Alderman Jr, Creating a Reusable Learning Object Strategy: Leveraging Information and}

Learner Analysis Survey

To further understand our learners, a learner analysis survey was distributed to all members of LD which will help identify their general ability levels and additional group characteristics. The results of this survey will help us to further understand the audience’s familiarity with Xyleme, their attitude toward the RLO initiative, and their current level of knowledge on both topics. As a result of this wide range in skill levels, the RLO implementation training will need to be designed in a way that allows for differentiation of content.

Learner Survey Results

The survey was sent out to LD staff on 2/17/16 with instructions to complete it by 2/22/16. The survey asks for the participant’s age range; length of time working in Xyleme; comfort level working in Xyleme; and familiarity with the RLO initiative.

Learner Characteristics

The potential learners can be divided into two basic categories: current IDs and potential new IDs. Xyleme Experience <6 months 6-12 months 1-3 years 3+ years

Familiarity with Xyleme

Low Medium High Very High

Age Range

25-34 35-44 45-54

Familiarity with RLO

Current Instructional Designers New Instructional Designers

Have already taken some form of Xyleme

training prior to learning about the RLO strategy

Will require basic Xyleme training before beginning RLO specific training

Have, at minimum, a basic understanding of how to use Xyleme to publish material

Little to no knowledge of the RLO strategy Limited time in learning the RLO strategy as most

training will occur on the job and while completing projects

More time dedicated to the initial learning of the RLO strategy

Orienting Context

The learners’ goals in taking this course are twofold. First, new IDs will be learning how to use the RLO structure as part of their initial training. Additionally, current IDs will be able to refresh their memories about what they learned during the initial training to be able to use the RLO structure when the need arises. In doing this, learners will perceive the training as useful in allowing them to be able to do their jobs. They will also be accountable for being able to use the RLO structure in getting their work done.

Knowledge of Learners

Current and future IDs will possess the necessary computer skills to be able to use the Xyleme software.

Motivation to Learn

The need to be able to perform daily job responsibilities as instructional designers is the primary motivation to learn for our target audience.

Instructional Context

Figure: Steps for Creating New Content in RLO Structure

Figure: Steps for Converting Content to RLO Structure

The differences in these two tracks of RLO projects led to the creation of the two major modules in our course design. Module 1 addresses the steps needed to learn the basics of the RLO structure and how to create new content in it. Module 2 focuses on the conversion of existing content that needs to be restructured for the RLO strategy.

To accommodate the diverse training needs of all instructional designers we will provide:

knowledge with novice instructional designers

 Exercises that require the use of critical thinking and problem solving

 Additional concept-related opportunities that allow the learner to apply the skills learned through training

Employees will take this training at their typical work station, either at MC headquarters or at their home office if they are remote employees. Employees will need access to a computer, the Internet, and a registered Xyleme account in order to complete the training.

Transfer context

As with any new skill, IDs at Management Concepts must have chance to apply their RLO training to on-the-job assignments in order for the training to truly stick. Since work assignments are based on the prioritization of projects outlined in the 2016 product portfolio, following the February 2016 workshop, it is anticipated that not all current IDs will immediately be given an assignment that calls for either the development of new content using the RLO structure or the revision of current content into the new RLO structure. Therefore, current IDs that attended the workshop will be encouraged to take the online refresher training prior to starting any project that calls for the implementation of the RLO strategy if that new project is scheduled to start more than one month after completing the workshop. New IDs that join the company after the February 2016 workshop will take the comprehensive RLO training and will be given opportunities to practice implementing the RLO strategy during their training through application exercises. We will encourage managers to assign RLO-related work assignments immediately following RLO training so that new hires can immediately gain more practice implementing the strategy. However, if such assignments are not available, then we recommend that these new employees participate in refresher training if they are assigned a project implementing the RLO structure more than one month after completing the comprehensive new hire training.

template, the Learning Object Strategy team does anticipate additional design and development hours for IDs, graphic designers, learning technologies and subject matter experts since the structure of legacy content varies by domain and does not align consistently with the RLO structure. The team is working with the LD Learning Management Office on how to account for this increase in time in the project budget estimates. However, as LD continues to develop its RLO library over time, the Learning Object Strategy team anticipates considerable savings in time and cost when developing new content and tailoring and configuring custom courses for clients since these products can be built using existing RLOs.

Task Analysis

Task Analysis Methods Used

To complete this analysis our team used a variety of techniques to determine what content should be included to address our instructional problem. We began the process using the topic

analysis method. This helped us to identify the content and then begin structuring its components to

cover the definition and purpose of RLOs. Once our main topics were identified, we used the

procedural analysis technique to determine the specific steps required when building RLO in Xyleme,

as well as changes to the authoring process. When interviewing our SMEs, we used the critical

incident method of analysis in order to gather information that addressed the way RLO supports our

platform-agnostic initiative and to identify the design implications the new structure poses. Through the use of multiple methods of analysis, we hope to have a strong focus and thorough understanding of the tasks to be included in our instruction.

1. What is a Reusable Learning Object (RLO)?

 Method of formatting course content by objective

o One module that contains multiple lessons

 Lessons support the terminal learning objectives

o One lesson for each enabling objective contains. Each lesson contains a combination of the following supporting materials:

 Interactivity  Assessments

 Application exercises  Examples

 Content formatted this way can be reused in multiple places 2. Why do we need RLOs?

 The RLO structure cuts costs and increases efficiency. o It facilitates reuse of content across the curriculum

o System of indexing helps avoid redundancy and duplication o Consistent design and development process

 The RLO structure makes building and maintaining courses easier. o RLOs are modular and self-contained.

o Lessons can be updated individually.

 Avoids problem of updating interdependent course pieces

 All courses containing a specific lesson are updated simultaneously o Small building blocks simplify creation of custom courses

 Decreases time billed to clients

 Increases consistency and eliminates re-work

 The RLO structure is consistent

o No need to reformat items taken from different parts of the content library o Benefits of using RLOs at Management Concepts

 Reduces new COTS content development costs  Reduces tailoring costs

 Reduces updating and maintenance costs  Increases sales opportunities

 Enables a one SSP approach

o RLOs are platform-agnostic (ILT, virtual, remote experience, and eBook all in one SSP); no need to redevelop for multiple platforms

 Use platform agnostic language when possible

 Make sure directions clearly identify what platform they address if platform agnostic language is not possible

3. What type of content makes up an RLO at Management Concepts?

 Module (contains the terminal learning objective)

o Lesson (contains the enabling learning objective and corresponding topic(s))  Required elements: Text and Visuals

 Optional elements (but must contain at least one):

 Interactivity (e.g., discussion)

 Application (e.g., exercise or case study)

 Examples (e.g., vignettes, stories, agency sample documents, links to websites with more information)

 Assessments (exam and quiz questions)

4. Key authoring changes for RLOs

 Exercises

o Authored in topics rather than title blocks

o Template accounts for the 3 different types of exercises  Exercises that require discussion only (no writing)

 May stay in guide

 Exercises that use the Question Block object (except for Essay Questions)

 May stay in guide

 Acceptable Question Block children

o Fill-in Activity, Matching Activity (with some restrictions), Matrix (with some restrictions), Multiple Choice, Sequencing Activity (drop-down only)

 Exercises that use the Essay Question Block because they require writing

 Include in companion workbook

o Only content authored at the exercise Topic level is a custom note that refers students to a separate Workbook to complete the exercise

 The topic that is authored as part of the Supplement is the topic that is linked to the Workbook

 The Workbook is created in a separate SSP outside of the RLO

 Course resources

o PDF resources

 Author as an internal link within the RLO template (Design Data > Supplements > PDF Appendix Supplement (referenced in lesson))

 Upload the PDF document from the media library

 At the course level, add a PDF Appendix (Chapter) template

 Copy and paste the media drive file location from the internal link object to the PDF reference object

 Include the “startPageNumber” and “endPageNumber”

o Authored course resources

 Author as a Topic-Based Supplement within the RLO template (Design Data > Supplements)

 At the course level, add an Authored Appendix (Chapter) template

 Copy the Authored Appendix topic from the supplement within the RLO

template and link it to the topic within the Authored Appendix Chapter

 Action Plan

o Do not author action plan in the RLO template

 Template contains boilerplate language in the Module Summary Block referring the student to the workbook

o For each course, create a Workbook SSP using the template

 The action plan template is in the Workbook SSP – do not change, this is standard for all courses

 The process to build a course within the RLO structure is the same as the process to convert an existing course in Xyleme. Templates have been developed in the RLO format so that content can be easily applied to the structure. These templates also ensure that each module in the course includes all required components. To access these templates and begin inputting course content within the RLO structure, the following steps should be followed:

Open Xyleme > Go to File > New XML Document From Template > Information Architectures > ISD Lesson Template > Lesson Template for RLOs

 This will result in the creation of a new SSP with a blank RLO template which is ready for content to be inputting as appropriate. Each element of the template includes a description and, beginning at the lesson level, there are elements that can be deleted if necessary. 6. How does RLO solve our problem of “One SSP” platform agnostic development?

 Platform-agnostic development (ILT, virtual, remote experience, and eBook all in one Single Source Project (SSP)) is important

o Keeping content in one SSP reduces confusion on course materials needed to be sent to students for classes

o Instructors only need one set of materials to be ready to teach in any platform  Instructors can be called on to teach last minute and there is a higher chance

they will have the necessary materials regardless of modality the need to teach the course in

o Always for easy upload of updated materials to Corner Stone on Demand (CSOD)  One zipped folder uploaded that works for all modalities

 Create content that can be used in multiple platforms o Use platform agnostic language when possible

 Eliminate in-class specific wording (table groups, flip chart paper, etc.)

o Make sure directions clearly identify what platform they address if platform agnostic language isn’t possible

o RLO structure allows for the ID to author specific guidance for each modality as necessary

o Minimal use of margin notes getting authored in Xyleme

 Margin notes used only for short guidance (i.e., Show Slide)  Focus on the use of instructor custom notes instead

o Allows for the “design once” mindset

 Design new courses in the RLO format that meet all modalities (remote, in-class, VLC) and all platforms (print and eBook)

7. Analyzing existing content in a course for conversion to RLOs

 Identify the different versions of the course that are currently available (ILT, virtual, eBook, etc.)

 Integrate all versions into one SSP

 Complete content analysis chart

 Analyze alignment of TLOs, ELOs, and course topics

 Determine inclusion of the following objects: o Visual o Slide o Content o Examples o Interactivity o Exercise o Assessment Instructional Objectives  Define an RLO.

 Describe the advantages of the RLO approach.

 Identify required and optional elements of an RLO.

 Determine when content needs to be published to the workbook versus the guide.

 Demonstrate how to author exercises, appendices, and the action plan using the RLO

template.

 Describe how to integrate content into a single SSP that encompasses all platforms.

 Complete a content analysis chart that analyzes LO alignment and inclusion of required objects.

Instructional Approach

The RLO training will use a flipped classroom approach. IDs will take each module

asynchronously. The modules will consist of presentations with voice-over narration as well as demos that explain how to do something specific in the RLO template as appropriate. At the end of each lesson in the module, IDs will have the opportunity to take a brief assessment to determine how well they mastered the objective, either in the form of knowledge-check questions or application

exercises. For lower Bloom’s-level objectives (1 and 2), this assessment will consist of knowledge check questions. For higher Bloom’s-level objectives (3 and above), this will also consist of an exercise where IDs must demonstrate their mastery of that objective. After specific lessons, IDs will meet one-on-one (or in small groups if multiple IDs are training at the same time) with their training teacher to review their work and ask questions about the process. After students complete a module, they will then be assigned to a project that will allow them to apply all the skills they learned in that module. Throughout the duration of this project, they will have at least weekly one-one-one meetings with their teacher to go over their work and get feedback. IDs new to the company must complete Module 1 before taking Module 2. Current IDs that participated in the RLO workshop may take either Module 1 or 2 as a refresher, or they may take individual lessons within each module to meet their immediate knowledge needs. The assigned teacher will be an expert in RLO implementation. At first the teachers will be comprised of members of the RLO team that developed the RLO strategy and template. As more IDs become proficient in RLO design and development (e.g., have completed at least two RLO projects successfully), they can then become teachers in this instructional approach. The teacher will continue to work with the student beyond the completion of the training and will serve as a mentor on the student’s first RLO project.

Detailed Design

Module 1: Implementing the RLO Strategy for New Content

TLO(s):

Learning Objectives

Topics and Subtopics Instructional Strategy

Assessment Strategy Platform

Considerat ions Content Source Estimated Timing

Define a RLO Method of formatting

content by objective  Modules  Lessons containing objectives Reusability Narrated slide presentation showing basic structure of RLO Discussion with teacher: Introduction and Q and A

Knowledge Check: short answer Sample question:

List at least three of the required elements of an RLO.

Answers: Textual content, visual content, interactivity,

assessments/application exercises, examples

N/A N/A 5 minutes

Module 1: Implementing the RLO Strategy for New Content Describe the advantages of the RLO approach Lower costs/higher efficiency  Reusability  Indexing  Consistency

Simpler course creation and maintenance  Modularity  Building blocks  Custom courses Consistent structure  Formatting  Benefits of RLO at MC  Platform agnosticism Exercise- brainstorm what you think the advantages of RLO are Narrated presentation: Advantages of the RLO Approach Discussion: How will RLO change the way you do your job?

Knowledge Check: short answer Sample question:

RLOs are __________, meaning that content created in the RLO format can be used across multiple platforms.

Answer: platform-agnostic

N/A N/A 5 minutes

Module 1: Implementing the RLO Strategy for New Content Identify required and optional elements of an RLO Required elements: Text and Visuals Optional elements (must contain at least one):  Interactivity (e.g., discussion)  Application (e.g., exercise or case study)  Examples (e.g., vignettes, stories, agency sample documents, links to websites with more information)  Assessments

(exam and quiz questions)

Job aid with required/option elements graphic (from RLO

workshop)

Knowledge check: multiple choice Sample question:

Which of the following is an optional element of an RLO? a. Text

b. Visuals

c. Learning objectives

d. Examples (correct answer)

N/A N/A 5 minutes

Module 1: Implementing the RLO Strategy for New Content Determine when the content needs to be published to the workbook versus the guide Exercises:

Discussion-Only (in the guide)

Question-Block Children that Don’t Require Writing (in the guide) Question-Block Children that Do Require Writing (in the workbook) Action Plan (in the workbook) Narrated slide presentation showing how to author different types of approved exercises and the action plan

Knowledge Check: multiple choice Sample questions:

Which of the following question types must be published to the workbook?

a. Multiple-choice b. Matching activity

c. Essay activity (correct answer)

d. Matrix The action plan:

a. is a template that is published with the workbook. (correct answer)

b. is published as a separate handout.

c. is an optional object that an ID can choose whether or not to add to an SSP. d. is published as an

appendix at the end of every course student guide.

N/A N/A 10 minutes

Module 1: Implementing the RLO Strategy for New Content Create the RLO structure within a blank SSP and describe its components

Accessing the template Overview of template components

Narrated demonstration showing how to apply the RLO template in Xyleme and a step-by-step walk-through of the template that discusses all the components that could be populated Application exercise One-on-one debrief with teacher

Knowledge Check: multiple choice quiz

Sample question:

Where would you author a walk-through or demonstration of how to use a featured website in a course?

a. Text & Tables plus Visual (Information) parablock b. Interactivity parablock

c. Example parablock (correct answer)

d. Visual parablock

Application exercise: students will create a new SSP in Xyleme and add the RLO template

N/A N/A 15 minutes

Module 1: Implementing the RLO Strategy for New Content Author and publish content in the RLO template Authoring Standards Publishing content Job aid of authoring standards Application exercise One-on-one debrief with teacher

Knowledge Check: multiple choice Sample question:

A level-one list item is

represented by which type of list marker:

a. M-dash b. Disk c. N-dash

d. Bullet (correct answer)

Application exercise - students will be given source content and must recreate the content in the RLO structure in Xyleme using the template. Afterwards, they must then publish all the associated outputs.

N/A N/A 5 minutes

(narrated video) Other activities are self-paced

Total Time: 45 minutes

(narrated videos only) Performance Support Tools: Job aid with required/option elements graphic; Job aid of authoring

standards

Module 2: Converting Legacy Content into RLOs

TLO(s):

Learning Objectives Topics and Subtopics

Instructional Strategy Assessment Strategy Platform

Considerations Conten t Source Estimated Timing Describe how to integrate content into a single SSP that encompasses all platforms What is agnostic language? Dos and Don’ts of one SSP

Narrated slide presentation: go over what it means to be agnostic, show examples of agnostic vs. not.

Presentation of dos and don’ts (workable practices for one SSP)

Job aid: checklist for the dos and don’ts provided for platform agnostic/one SSP One-on-one debrief with teacher

Application: students will be given source files that are not platform agnostic and will rewrite the wording to make agnostic.

This application exercise will be reviewed and feedback provided by the mentor. A solution with suggested answers for the mentor to use will be provided.

N/A N/A 15 minutes

Module 2: Converting Legacy Content into RLOs Complete a content

analysis chart that analyzes learning objective alignment and inclusion of required objects What is a content analysis chart? Why do we need a content analysis chart? (also covers when a content analysis chart is necessary to complete)

Narrated slide presentation: overview of the content analysis chart, each column of the chart, and why/when we need to complete the chart.

Job aid: content analysis chart with a few rows filled in, content analysis chart (blank)

One-on-one debrief with teacher

Application: students will complete a content analysis chart based on source files given in preparation for a conversion to RLO structure

This content analysis chart will be reviewed and feedback provided by the mentor.

N/A N/A 5 minutes

(narrated video) Other activities are self-paced

Total Time: 20 minutes

(narrated videos only) Performance Support Tools: Job aid of a checklist for the dos and don’ts provided for platform

agnostics/one SSP; Job aid of the content analysis chart

Limitations and Constraints

To avoid resource limitations, the training team will need to make it clear to management that instructional designers must have time to complete the training in addition to their regular

responsibilities. Some access to course development technological tools, such as Adobe Presenter, will also be necessary. Constraints are few, but technical issues with Xyleme, the company's content management system, occur frequently. This could hinder efforts for both creation and execution of the training.

Instructional Materials

To successfully complete this training, the following materials are required:

 Computer with Internet, Microsoft Word, and Adobe Acrobat

 Xyleme LCMS user account

 Cornerstone On Demand (CSOD) user account

 Job aids, which will be available electronically on the LD SharePoint site

 Recommended solutions to exercises for the teacher to use to evaluate performance

Formative Evaluation Strategy

The RLO training will be assessed continually through the use of focus groups and SME consultation during its development. The focus group members will consist of new or current IDs (depending on the module being delivered) and will be active participants in the training. The SMEs involved in the consultation will consist of the RLO administrators who were responsible for

consultations. During these consultations we will discuss the best ways to implement focus group feedback into revisions; listen to SME feedback and suggestions; and gather the most up-to-date information about changes to RLO usage and the RLO structure. The use of these formative

assessment strategies ensures that our training remains relevant to learners and provides them with the most current RLO information possible.

Summative Evaluation Strategy

Our summative assessment strategy focuses on three sources of data:

1. Knowledge Checks and Application Exercises: After completing certain lessons throughout the training, students will be prompted to take a knowledge check quiz which will consist of multiple-choice and short answer questions that will test their understanding. After some lessons, students will also be required to complete an application exercise to demonstrate how well they have mastered an objective and can apply that skill to a job-related task. Examples of the knowledge-check questions and descriptions of the application exercises are provided in the detailed design document above.

2. Teacher Check-ins: After students have completed an application exercise, they will not be able to progress further along in the training until the teacher has the opportunity to review the student’s work and provide feedback. The teacher will be provided a school solution for each application exercise, where appropriate, and will evaluate the student’s performance against the school solution. If the teacher feels that the student is not ready to move forward, he or she will request that the student continue to practice the skill by preparing a

comparable, alternative activity.

3. Student Evaluation: At the end of the course, students will be given a short evaluation survey to provide feedback on how well the training met the stated objectives and improved their ability to develop content in the RLO template. The following questions are derived from our

corporate Metrics that Matter end-of-course student survey:

(Course Design and Materials)

Strongly

Agree

Agree Neither Agree

nor Disagree

Disagree Strongly

This course met the stated learning objectives.

The course materials are relevant and support the learning objectives. The course materials are accurate and up-to-date.

The course length was appropriate relative to the subject material.

(Teacher Feedback)

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree The teacher was knowledgeable about

course topics.

The teacher was prepared and organized. The teacher was responsive to questions.

(Learning Effectiveness)

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

I increased my knowledge/skill level by completing this course.

(Job Impact)

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

I will apply the new knowledge and skills to my job.

This training aligns with the business priorities and goals identified by my organization.

Confirmative Evaluation Strategy

To assess the monetary value of the RLO training, we will compare time spent on