Course Approval Application Packet

(1)

For General Contractors



Application

Process



Course

Development

Guidelines



Course Approval

Application Form

with Attachments

Course

Approval

Application

Packet

(2)

Course Development Guidelines ………..………….… 4 - 37

Basic Requirements for All Elective Courses ………...….. 4 - 6

Requirements for e-Learning/Asynchronous Courses for 2021 CE Year …………...… 6 - 7

Requirements for e-Learning/Asynchronous Courses for 2022 CE Year ……….….… 8 - 37

Sample Timed Outline ………... 38 - 39

Application for Course Approval ……… 40 - 45

Content Use Authorization Form …...………...…… 46

Application Process ……...……… 3

Reviewer Checklist Document ... 8 - 15

Review Criterion ... 16 - 37

(3)

Application Process for Course Approval

1. The applicant completes the Application for Course Approval and submits the

application to the Board. (Please send only the application, not the entire Course Approval Application Packet).If the applicant does not own the course content, the applicant must also submit a properly completed Content Use Authorization Form. Failure to do so will result in a denial. Completed applications and forms should be emailed to

[email protected]. *Please do not drop off or mail hard copies of course materials to the NCLBGC office - all materials must be submitted electronically.

2. Applicant submits course review fee to the Board by calling the Education Department and providing credit card information over the phone. (Please note: checks are no longer accepted as payment for CE-related fees effective February 1, 2021). The fee is non-refundable. Course review will not begin until payment has been successfully submitted. Initial course review fees are $25 per credit hour requested.

3. The Board’s Education Committee will review applications and forms for completeness and compliance with continuing education requirements and will recommend approvals to the Board.

4. If corrections are needed or more information is required, the Board’s education staff will contact the applicant using the contact information provided in the application. Email is the preferred method for communication and for submission of corrections.

5. Once the Board gives final approval, the application is approved, and the applicant is assigned a course number.

(4)

Course Development Guidelines

This document contains specific recommendations for elective course development and

approval; however, it does not contain all rules, requirements, and information affecting

course providers and instructors. It is not, nor is it intended to be, a replacement or

substitute for reading the Board’s continuing education rules. It is essential that all

prospective providers, instructors, and elective course developers thoroughly read the

complete rules for continuing education before attempting to use the information in this

document. This material is provided to supplement the Board’s rules concerning elective

course subject matter and criteria for elective course approval. The subject matter

guidelines are intended to assist prospective course providers in determining whether

the content of a particular course (or proposed course) is likely to be found acceptable

by the Board. The course development guidelines contain suggestions on developing

acceptable courses.

Basic Elements for Elective Course Subject Matter:

1. A proposed elective course shall cover subject matter that is deemed relevant

statewide, and provides information that will increase a qualifier’s knowledge,

professionalism, and competency in the practice of general contracting as

determined by the Board.

2. As a general rule, it is best to select a fairly specific topic and address that

topic in depth, or to select the most important points relating to a topic and focus

on those points.

Minimum Requirements for Approval

For All Elective Courses:

1. Should include substantive information relevant to the practice of general

contracting which shall consist of two hours or four hours of instruction, offering

two or four CE credit hours, respectively (one credit hour is equal to 50 minutes

of instructional time).

2. The application should include:

a.

Course Objective – What will the student know or be able to do upon

completion of this course? What is the intended outcome? Do students

only need awareness or recognition of the information presented? Or

(5)

will they need to be able to apply the information/solve problems? The

objective statement should be brief, clear, and address what the

course should accomplish and should define the standard of

acceptable performance of the students upon completion. Course

objectives are the keystones for the design and development of the

course. The design should be based on principles of adult learning,

and the presentation should reflect the goals of the learning objectives.

Objectives should be attainable in the allotted time session.

b.

A narrative of the substantive information – Key points to be made/

discussed must be provided in sufficient detail to demonstrate the

information is current, complete, and otherwise appropriate. Information

should follow a systematic approach, ensuring all elements

interconnected. Topics should each build upon the other, moving from

simple to complex.

c.

A timed outline, including any breaks or meals – total instructional

time should be equivalent to the number of CE credit hours requested.

(One credit hour is equal to 50 minutes of instructional time). A brief,

topic outline is not sufficient – be as detailed as possible, clearly stated,

using full sentences to describe each topic, as well as its relevance to

the course objectives (See Sample Outline in this Packet). The timed

outline should easily map to the course content provided. For example,

the topics listed in the timed outline should tie back to the applicable

slides in the accompanying powerpoint presentation if your course will

be taught utilizing slides and lecture. The time (in minutes) devoted to

each topic listed should be clearly stated. If slide presentations are not

used to deliver information to students, then detailed instructor notes

describing the content to be discussed, the flow of that content, and the

number of minutes that will be devoted to each segment must be

provided and must clearly align with the timed outline.

If you are

submitting an e-Learning (Asynchronous) elective course, your timed

outline should clearly reflect not only the topics covered in detail, but

also clearly lists the onscreen run time that indicates where each topic

can be found within the e-Learning course content (i.e. 0:00 - 1:03)

d.

Instructional methods/aids to be employed – (lecture,

question/answer, student handouts, case studies, demonstrations,

group discussions, role plays, games, etc.)

e.

Teaching tools to be used for delivery – (PowerPoint slides, props,

hands-on equipment, videos, flipcharts, whiteboards, computer, etc.)

(6)

4. Courses shall not be designed to promote specific products, services or markets.

5. Course reviews include reviews of both course content and delivery method.

For Instructor-led, Classroom Setting Elective Courses

(Students and

Instructor are not separated by time but may be separated by distance)

:

In addition to meeting the minimum requirements for approval listed under “All

Elective Courses”, the following requirement must also be satisfied:

1. If the instructor is not physically present in the classroom (for example, the

instruction is streamed in live from a location apart from the students), then a

proctor must be physically present with the students in the classroom to

monitor attendance and participation.

3. If the provider seeking course approval does not own the course content:

o

the provider must also submit a completed

Content Use Authorization

Form, along with the Application for Course Approval. This form, found

on the Board’s website, must be properly completed and signed by the

course content owner, and acknowledges that the content owner gives

the provider seeking course approval permission to use the content for

the purpose of offering the course for continuing education credits through

the North Carolina Licensing Board for General Contractors. Content

includes all images, illustrations, and videos.

Requirements for 2021 CE Year: In addition to meeting the minimum requirements for approval listed under “All Elective Courses”, the following requirements must also be satisfied to obtain *conditional approval for 2021:

*Approvals for the 2021 CE year will require another full review under the expanded guidelines found in the Reviewer Checklist Document on page 8 of this packet to be considered for approval for the 2022 CE year.

1. All asynchronous courses shall be fully developed prior to submittal to the

Board for review.

2. A set of log-in credentials shall be provided to the Board at no cost to the

Board at the time of submission for review of the online course. The Board

will need 5 accounts created for accessing both the gated and ungated

versions of the course: one for the Education Department, one for the

e-Learning validation service, and 3 for the Review Committee members.

The log-in procedure should not require any committee member to put in a

valid name, email address nor a valid qualifier number to access the course.

All accounts and credentials should be unique but generic in nature. Log-in

For e-Learning (asynchronous) elective courses

(students register and log-in

through the internet for education delivered through e-Learning modules - no live

instructor):

(7)

credentials shall also be made available at no cost at any time when

requested by the Board in its sole discretion for audit purposes.

3. Providers of asynchronous courses shall clearly state to students the technical

requirements necessary for accessing the course, provide a technical support

phone number or email address available to those students who require

technical support for accessing and/or completing the course.

4. Providers shall implement measures to verify student identity. At a minimum,

the provider shall collect the student’s qualifier number at the time of

registration for an asynchronous course. The student shall be required to enter

the qualifier number each time he/she accesses the course, and this number

shall match exactly the qualifier number provided during registration.

Otherwise, the student shall not be permitted to access the course.

5. Students shall complete all asynchronous elective courses 30 days from

course registration or no later than November 30

th

_{of each year.}

6. All content in asynchronous courses must be gated (the learner is required to

listen, read, or engage with all course content) to ensure that courses cannot

be completed in less time than the approved credit hours. A non-gated version

of the course for quick content review should also be included with the course

submittal.

7. Asynchronous continuing education courses shall consist of e-learning

modules that will include no more than fifty minutes of curriculum.

8. Each module shall include a knowledge check consisting of at least 5

questions per 50 minutes of online instruction. The learner must select the

correct answer for each knowledge check before being able to progress ahead

in the course. These knowledge checks count as instructional time and each

question should be allotted no more than 20 seconds. A separate document of

all knowledge check questions and correct answers should be included with

the course submittal.

9. A summative assessment quiz at the end of the final module which will consist

of twenty (20) questions and includes questions pertaining to each module. A

passing score of 70% is required to pass the course and to earn a certificate of

completion. Quizzes count as instructional time. Each quiz question should be

allotted no more than 20 seconds for completion, counting towards the required

100 minutes of documented on-screen instruction for a 2-hour course, or 200

minutes for a 4-hour course. A separate document of all summative quizz

questions and correct answers should be included with the course submittal.

10. A certificate of completion will be provided to each student and the continuing

education hours and all payments will be uploaded to the Board per

21 NCAC

12B .0204.

(8)

NCLBGC eLearning Elective Course Evaluation Instrument 1

eLearning Continuing Education Credit Courses

Functionality, Content, Instructional Soundness, and Compliance Evaluation Instrument

The purpose of the following evaluation instrument is to be able to effectively and consistently assess continuing education credit content presented to the North Carolina Licensing Board for General Contractors (NCLBGC) for certification.

Presented courses will be assessed based on the following criteria: • Functionality/Usability

How the course performs in relation to navigation, general functions, and the student experience

• Content

Suitability of content for the intended audience • _{Instructional Soundness}

Presented content conforms to acceptable standards of adult learning theory and best practices

• _Compliance

Content complies with the guidelines, policies, and/or rules regarding continuing education credit content as defined by the NCLBGC

Each section of this evaluation contains a series of criterion and free-form comment areas that the reviewer should use to capture the results of his/her review.

The results of the review will be used by the Board to determine if a course is eligible to be a continuing education credit eLearning course. Each course will be assessed against every criterion and noted as either Meets or Does Not Meet the standard.

Complete your review and return the results to the designated person by the assigned date below. Name Sonya McLamb

Date

For questions about the review process contact Sonya McLamb at [email protected] (919) 571-4183, ext. 119.

Page 8

(9)

Reviewer Information

Initials Date

General Course Information

Course Name

Number of Credit Hours

Course Review Information

Information to consider while conducting the review Timed outline is provided

Answer key to assessment questions is provided Gated and ungated/slide deck version of course provided

Information to access the course URL for provider website

URL for course access

Username and password for reviewer Additional account information for reviewer

(10)

NCLBGC eLearning Elective Course Evaluation Instrument 3

Section 1 - Functionality & Usability

This section assesses how the course performs in relation to navigation, general functions, and the student experience

Validate the course/platform either MEETS or DOES NOT MEET

the given statement. Meets

Does Not

Meet Notes

1 Navigation is simple, easy to use, and intuitive (including the ability to move back and forward within the course)

2 Instructions are provided with how to navigate

3

Participants can bookmark their place in a course and return to their place without losing previously

completed content, activities, knowledge checks, assessments, and time spent

4 Information about how to interact is provided for _{interactive screens} 5 All course functionality works as labeled and/or _{described and is free of technical malfunctions} 6 The look and style of the course is appropriate, _{professional, and engaging} 7 Any fonts used are easy to read

8 Pictures, illustrations, and graphics are easy to see and _comprehend 9 All supporting visuals directly align to the content

10

If supplemental assets (quick reference handouts, step-by-step instructions, detailed illustrations, and so on) are provided, they should be printable, download-able, or electronically send-able (for example, via email)

11

Assessment criteria is defined (including which questions are scored, how scored questions relate to a student’s ability to successfully pass the course, how many times an assessment can be taken, and so on) 12 The summative assessment quiz can be retaken if not _passed 13 The consequences of unsuccessful course completion _{are defined}

(11)

Use the space provided below to capture any observations and/or specific recommendations regarding course functionality and/or usability.

Section 2 - Content

This section assesses the suitability of presented content for the intended audience Validate the course/platform either MEETS or DOES NOT MEET

the given statement. Meets

Does Not

Meet Notes

1 Near the beginning of the course, the number of eligible _{credit hours is stated} 2 Near the beginning of the course, estimated time to _{complete the entire course is stated}

3 Each module provides an estimated time to complete (either in the course outline, or at the beginning of a module)

4 The amount of content presented on screen is not too _{sparse and not too crowded} 5 Written content is clear and concise, without spelling or _{grammar errors.}

6

Any industry acronyms and/or jargon are explained and/or spelled out on the first occurrence (this can be achieved in a variety of methods such as within the content presented on the screen or by hovering over the acronym or term, and so on)

7 Content is not sexually, culturally, racially, or religiously, offensive, images are inclusive and representative of General Contractors in North Carolina

NCLBGC USE ONLY 8

Content covers subject matter that is deemed relevant statewide and provides information that will increase student knowledge, professionalism, and competency in the practice of general contracting

(12)

NCLBGC eLearning Elective Course Evaluation Instrument 5 10 Content is focused on the intended topic area and does _{not become tangential}

11

The amount of presented content is commensurate with either 2 or 4 hours of instruction (as appropriate for the content topic and instruction required to present the topic)

Use the space provided below to capture any observations and/or specific recommendations regarding course content.

Section 3 - Instructional Soundness

This section assesses presented content in relation to how it conforms to acceptable standards of adult learning theory and best practices

Validate the course/platform either MEETS or DOES NOT

MEET the given statement. Meets

Does Not

Meet Notes

1 The course has an overall, clearly stated goal

2 Course modules have clearly stated learning objectives 3 Near the beginning of the course, a course outline is _{presented to student}

4

Tone and language used to present content is written in a way that reduces barriers to comprehension, is easy to understand, uses headings to convey meaning and structure, provides clear instructions, and keeps content clear and concise

5 The course gives opportunities to engage or interact with the content, verses passive observing or using basic navigation (in addition to assessment questions)

6 Passive delivery of content (watching videos, reading text) are kept to no more than 15 minutes in duration before the student needs to interact with the course. 7 The presentation of content is done in sufficient detail that enables students to successfully accomplish stated

learning objectives in real world applications

8 The presentation of content is chronological, systematic, _{or process oriented} 9 The presentation of content is broken down into easily _{understandable learning objects}

(13)

10 The student is provided with opportunities to apply the content and concepts in case studies/scenarios or application-level questions

11 Assessment questions are directly related to learning _{objectives and presented content}

12 Knowledge checks provide correct or incorrect feedback with remediation, or allow student to correctly answer before progressing

13 The summative assessment questions do not provide _{correct/incorrect responses and no remedial explanation} 15 The summative assessment does not include more than _{five true or false questions.}

Use the space provided below to capture any observations and/or specific recommendations regarding instructional design soundness and best practices.

Section 4 - Compliance

This section assesses how content complies with the guidelines, policies, and/or rules regarding continuing education credit content as defined by the NCLBGC

Validate the course/platform either MEETS or DOES NOT

MEET the given statement. Meets

Does Not

Meet Notes

1 The platform includes the ability to collect a student’s Qualifier ID and the Qualifier’s name on file with the Board

2

Effective July 1st_{, 2021 - Within the first three pages or}

slides of the course materials: “THE NORTH CAROLINA LICENSING BOARD FOR GENERAL CONTRACTORS HAS APPROVED THE COURSE ONLY AS TO ITS RELEVANCE TO THE PRACTICE OF GENERAL CONTRACTING IN NORTH CAROLINA. THE COURSE PROVIDER AND INSTRUCTOR ARE RESPONSIBLE FOR THE ACCURACY OF THE CONTENT AND COMPLIANCE WITH ALL STATE AND FEDERAL LAWS DURING THE ADMINISTRATION OF THE COURSE”.

3 The student is given instructions about how to search and confirm their Qualifier ID and Qualifier’s name on file with the NCLBGC Website

(14)

NCLBGC eLearning Elective Course Evaluation Instrument 7 200 minutes of student seat time (including all

assessments)

5

Prior to the start the course, technical system requirements are stated (including connectivity requirements, operating system version, impact of ad blockers, whether access to speakers or a printer is required, and so on)

6 Prior to the start of the course, a technical support _{phone number and/or email address is provided} 7 The course does not promote specific products, services, _{or markets}

8

For every 50 minutes of learning content (differentiated by sections, modules, chapters, or other appropriate content organizational methods), there are 5 knowledge checks

9 The course has a final summative assessment consisting _{of 20 questions} 10 The summative assessment passing score is set to 70%

11

The course is “gated” providing no options (i.e., scrubbing through a video, or navigating past content) for students to earn credit without completing the prescribed course content and by passing the final assessment

12

A certificate of completion is provided to the student once the student successfully completes the course, or instructions accurately state how a certificate can be obtained.

13

Prior to the start of the course, it is clearly stated that students will have 30 days to complete a course from initial enrollment (Students who do not complete the course within 30 days will need to complete the entire course if reenrolled)

14 Prior to the start of the course, it is clearly stated that the course will not be available to purchase, start, or complete after November 30th_{, and during December}

Use the space provided below to capture any observations and/or specific recommendations regarding compliance with NCLBGC guidelines, policies, and/or rules.

(15)

Assets Usage Yes No Notes

1 Except for the use of stock assets, the course contains images, videos, or other assets that appear to require written authorization for use in this course.

(16)

Created for the North Carolina Licensing Board for General Contractors

Review Criterion of

eLearning Elective

Courses

(17)

Overview

The purpose of this document is to provide the criterion by which all e-Learning elective courses will be evaluated to ensure consistency across all reviews. This document will be of value to the provider when developing an e-learning elective course for the first time, or, when seeking further clarification as to why a course does not meet a certain criterion following a review.

Presented courses will be assessed based on the following criteria:  Functionality/Usability

How the course performs in relation to navigation, general functions, and the student experience

 Content

Suitability of content for the intended audience  Instructional Soundness

Presented content conforms to acceptable standards of adult learning theory and best practices

 Compliance

Content complies with the guidelines, policies, and/or rules regarding continuing education credit content as defined by the NCLBGC

The results of the e-Learning review will be used by NCLBGC to determine if a course is eligible to receive a content review. Each course will be assessed against every criterion and noted as either Meets or Does Not Meet the standard. A course must satisfactorily meet all e-Learning standards to receive a content review. Once all standards are met, the course will receive a content review. In order for the Board to approve an elective e-Learning course for the purpose of satisfying continuing

education requirements, the Board's Review Committee must also recommend approval of the content.

(18)

3 | P a g e

Section 1 – Functionality & Usability

Overview: This section assesses how the course performs in relation to navigation, general functions, and the student experience.

1.1 Navigation is simple, easy to use, and intuitive (including the ability to move back and forward within the course)

Ensure that the course functions appropriately by clicking and interacting with all buttons, both on the slide, in the player, and in the LMS. Look to see if the buttons make sense to you in terms of their appearance and what they are expected to do.

What “meets?” What “does not meet?”

 Buttons are clearly labeled with print (e.g., say “back” or “next”) or with a universal and clear sign or icon (e.g., arrows or a question mark)

 All buttons function as intended. For example, if the button says “back,” it goes to the previous slide

 Buttons are confusing or mislabeled  There are too many buttons or navigation

that it inhibits course navigation  There are buttons that lead to incorrect

sites or slides, or do not function at all when clicked

1.2 Instructions are provided with how to navigate

Look for a description of how to use the course navigation buttons or how to maneuver in the course or LMS. These instructions could appear on the LMS, in the welcome screen of the course, or in the course itself.

 Instructions of how to move in the course are provided in written, video, or audio format. This can occur in the LMS, welcome screen, or course itself

 No explanation is found anywhere about how to navigate the course

1.3 Participants can bookmark their place in a course and return to their place without losing previously completed content, activities, knowledge checks, assessments, and time spent

To ensure a course meets this criterion, keep track of what you have completed already, exit the course, and then re-enter the course. You may also want to fully sign out of the LMS and then sign back in to see if the course has kept your information. Do this at multiple points of the course.

(19)

 After exiting and re-entering the course, the course either has saved your place on a countdown timer or your slide location  If you have already completed a quiz or

knowledge check, the course should save your completion of it

 After exiting and re-entering the course, the course has not saved your place on a countdown timer or your slide location  You must restart at the beginning of the

course each time you enter  You must retake knowledge

checks/quizzes after re-entering the course

1.4 Information about how to interact is provided for interactive screens

There should be explicit instructions on a slide or page that has an interactive section (e.g., a slide full of clickable tabs or a video interaction). Students should not have to guess how to use an interaction.

 Text instructions about how to use the interaction are given on any interactive slide or page

 There are no text instructions explaining how to interact with an interactive slide or page

1.5 All course functionality works as labeled and/or described and is free of technical malfunctions This is an overarching criterion and addresses the functionality of the course from beginning to end. Reviewers should check that the following are working correctly:

- Navigation

- Knowledge checks

- Any audio or visual component of the course - Interactive slides

- Final assessment

- Resource documents and outside links

 Navigation, knowledge checks, any audio or visual components of the course, interactive slides, the final assessment, and any resource documents or outside links are working correctly

 Navigation, knowledge checks, audio or visual components of the course, interactive slides, the final assessment, and/or any resource documents or outside links are working correctly

(20)

5 | P a g e

The look and style of the course should not distract or detract from a student’s comprehension of the material. Color palette, fonts, images, slide templates, and audio/visual components should both complement each other and create a distraction-free environment for the content.

 The look and style of the course does not distract or detract from a student’s comprehension of the material  The color palette, font, images, slide

templates, and audio/visual components complement each other and share a similar theme

 The look and style of the course distracts or detracts from a student’s

comprehension of the material  The color palette, font, images, slide

templates, and/or audio/visual components do not complement each other or do not share a similar theme

1.7 Any fonts are easy to read

Fonts chosen by the course provider should be clear, readable, and accessible to students. Excessively stylized, embellished, or script-like fonts should be avoided because of their lack or readability. Font size should remain relatively constant and should not vary widely on the same slide or learning object.

 Fonts used in the course are readable

because of font choice and font size  Excessively stylized, embellished, orscript-like fonts are used that affect comprehension

 Excessive amount of font sized used to a degree it affects comprehension

1.8 Pictures, illustrations, and graphics are easy to see and comprehend

Pictures, illustrations, and graphics should be a part of any eLearning course. See below for an example of a clear picture that achieves this criterion.

 All pictures, illustrations, and graphics are of an appropriate size

 All pictures, illustrations, and graphics are clear and not pixilated

 Any picture, illustration, or graphic is too small to read or understand

 Any picture, illustration or graphic is highly pixilated to a degree that affects comprehension

 Any picture, illustration, or graphic is cropped in an odd way that affects comprehension

(21)

1.9 All supporting visuals directly align to the content

The visuals that a provider uses should directly relate to the content that is being presented. It should enhance learning and not distract from student from the content in the course. To better assess this criterion, ask yourself:

- How does a student’s comprehension of the course benefit from this visual? - Does this visual distract from the student’s understanding of the course? - Is it clear what a student should learn from this visual?

 Photographs, illustrations, diagraphs, and other visuals clearly connect with the course content

 Photographs, illustrations, diagrams, and other visuals do not distract from

students’ comprehension of the course

 Photographs, illustrations, diagrams, and/or other visuals distract from students’ comprehension of the course  Photographs, illustrations, diagrams,

and/or other visuals show no connection to the course content

1.10 If supplemental assets (quick reference handouts, step-by-step instructions, detailed illustrations, and so on) are provided, they should be printable, download-able, or electronically send-able (for example, via email)

Supplemental assets are sometimes used in eLearning courses. Note that if a course does not provide supplemental assets, they simply are marked as “not applicable” in this checklist criterion.

 If supplemental assets exist, they are provided to the student in printable, downloadable, or electronically send-able format

 Supplemental assets exist but are challenging to print, download or send via email

1.11 Assessment criteria is defined (including which questions are scored, how scored questions relate to a student’s ability to successfully pass the course, how many times an assessment can be taken, and so on)

The only assessment questions to be scored are the summative assessment questions at the end of the course (also known as the final quiz or final test).

The course should lay out the following criteria: 1. The final assessment will have 20 questions

2. The final assessment will be scored as a percentage, with a passing score of 70% 3. The final assessment can be taken as many times as the student wants

4. The final assessment must be passed with at least a 70% score for students to pass and finish the course

(22)

7 | P a g e

Knowledge check questions (quiz questions) do not need to be scored and their answers do not factor into a student’s passage of the course. The final assessment criteria should be listed in the LMS website or the course itself.

 Text explaining the final assessment criteria (as listed above) is listed on the LMS website or the course

 There is no text explaining the final assessment criteria (as listed above)

1.12 The summative assessment quiz can be retaken if not passed

The summative assessment can be taken an unlimited amount of times. This must be tested and verified by the provider. To do this, follow the assessment protocol as laid out in 4.8. The reviewer will fail the course during the first take. If the course allows you to take the test again after the failure, the course passes this criterion.

 The summative assessment can be taken

more than once until it is passed  The course only allows the student totake the summative assessment once (or a certain amount of times not based on passage of the assessment)

1.13 The consequences of unsuccessful course completion are defined

The course provider must explain that if a student does not pass the summative assessment, they will not receive credit for the course. This can be explained on the LMS, on the introduction page of the course, the introduction to the course, or the introduction to the summative assessment.

 The provider has explicitly stated that the student will fail the course if they do not pass the summative assessment

 The provider has not stated anywhere that the student will fail the course if they do not pass the summative assessment

(23)

Section 2 – Content

Overview: This section assesses the suitability of presented content for the intended audience. 2.1 Near the beginning of the course, the number of eligible credit hours is stated

Elective courses can either be worth 2 CE credit hours or 4 CE credit hours. The number of eligible credits hours can be stated in any of the following places: the LMS, the title of the course, the course webpage on the LMS, or the introductory slides of the course.

 The number of eligible credit hours is stated in either the LMS, the title of the course, the course webpage on the LMS, or the introductory slides of the course

 The number of eligible credit hours is not stated anywhere in the course or LMS

2.2 Near the beginning of the course, estimated time to complete the entire course is stated

Students should be told an estimated time completion of the course. For a 2 CE course, the estimated time completion should be 100 minutes. For a 4 CE credit course, the estimated time completion should be 200 minutes. The estimated time to complete the course should be stated in either the LMS, the course webpage on the LMS, or the introductory slides of the course.

 The estimated time to complete the course is stated as either 100 minutes for a 2 CE course or 200 minutes for a 4 CE course

 The estimated time to complete the course is stated in either the LMS, the course webpage on the LMS, or the introductory slides of the course

 The estimated time to complete the course is not stated as either 100 minutes for a 2 CE course or 200 minutes for a 4 CE course

 The estimated time to complete the course is not stated anywhere in the course or LMS

2.3 Each module provides an estimated time to complete (either in the course outline or at the beginning of a module)

Not only should the estimated time to complete the entire course be stated, but the course provider should also list the approximate time to complete each module. This can take place at the beginning of each module or in the course outline.

(24)

9 | P a g e

 The estimated time to complete each module is stated at the beginning of each module, in the course outline, or in the introduction to the course

 The estimated time to complete each module is not stated anywhere in the course or LMS

2.4 The amount of content presented on screen is not too sparse and not too crowded

This course criterion is similar to criterion 3.8. Strong eLearning courses “chunk” material. Chunked material is material that has already been broken down into smaller concepts. Small concepts are much easier to retain and connect with. It is the job of the course provider to break down concepts into manageable pieces for the student. Visually chunked content means that there is not too much information or too little information on the screen.

To better assess this criterion, ask yourself:

- Is there more than one large concept on the screen?

- Does this screen or course read more like a book than an eLearning course? - Is it hard to figure out what the most important part of the screen or slide is? - Would a student be overwhelmed by the amount of information on the screen?

- Is there so little information on the screen that a student would struggle to learn anything?

 There is not more than one major concept on a screen or slide

 There are less than 150 words each slide or visual element

 Photographs and illustrations have descriptive text and/or are provided context for their appearance on the screen

 There is more than one major concept on a screen or slide

 There are consistently more than 150 words on each slide or visual element  Photographs and illustration appear with

no context or explanation

2.5 Written content is clear and concise, without spelling or grammar errors

Courses should be free of spelling or grammar errors because these errors detract from students’ comprehension of the course.

 Course content does not contain spelling

or grammar errors  Course content contains spelling and/orgrammar errors

2.6 Any industry acronyms and/or jargon are explained and/or spelled out on the first occurrence (this can be achieved in a variety of methods such as within the content presented on the screen by

(25)

Specific vocabulary that is taught and used in a course must be clearly defined to a student. This is best done by giving a definition to the student before they encounter the word in the course, when they encounter the word in the course, or in some type of supplemental asset like a glossary.

 Specific vocabulary, like industry acronyms, are defined for the student

 Specific vocabulary is not defined for the student

2.7 Content is not sexually, culturally, racially, or religiously offensive, images are inclusive and representative of General Contractors in North Carolina

Courses should not suffer from explicit or unconscious biases in terms of gender, culture, race, or religion. Course providers should ensure the use of inclusive images.

 Content, including images, are inclusive of gender, culture, race and religion  Content, including images, are not

sexually, culturally, racially, or religiously offensive

 Content, including images, are not inclusive of gender, culture, race and religion

 Content, including images, are sexually, culturally, racially, and/or religiously offensive

(26)

11 | P a g e

Section 3 – Instructional Soundness

Overview: This section assesses presented content in relation to how it conforms to acceptable standards of adult learning theory and best practices.

3.1 The course has an overall, clearly stated goal

A course goal is usually an overarching, broad statement of what a learner will achieve by the end of the course. Goals are not usually measurable or specific; instead, they focus on the big picture of what learners will be able to do when finished with the course. This overall course goal should appear during the beginning of the course or in the course information page on the LMS.

Some examples of course goals include:

- Learners will explore and understand weatherization techniques of residential homes. - Students will learn the basics of writing a business contract.

 Course provides in writing a goal that matches the course content

 The goal is overarching, clearly stated, and broad

 The goal describes what a learner will achieve by the end of the course

 There is no goal present

 The goal does not match the course content

 The goal does not describe what a learner will achieve by the end of the course

3.2 Course modules have clearly stated learning objectives

Learning objectives are specific, measurable, and achievable statements of what students will gain or achieve during a course. They are similar to goals but are not as broad or overarching. Objective should be specific to each module.

Some examples of learning objectives include:

- Students will be able to name three common OSHA violations found on construction sites. - Learners will be able to distinguish between illegal and legal uses of their license.

 There is at least one objective per module in the course

 Each objective is specific, measurable, and achievable

 Each objective matches the course content

 There are no objectives

 Objectives are not stated for each module

 Objectives do not match the course content

 Objectives are not specific, measurable, and achievable

(27)

3.3 Near the beginning of the course, a course outline is presented to student

A course outline should concisely lay out the progression of the course. It can simply consist of the names of the modules or a narrative of what the student will experience in the different modules.

 The course outline is presented at the beginning of the course

 The course outline presents the entire course in a quick, readable format

 There is no course outline

 The course outline is presented in a confusing manner (e.g., in a narrative format)

3.4 Tone and language used to present content is written in a way that reduces barriers to

comprehension, is easy to understand, uses headings to convey meaning and structure, provides clear instructions, and keeps content clear and concise

Language used in the course should follow Standard American English rules. Text, audio, and video should be free of colloquialisms, verbiage, and ungrammatical statements. When text is on a slide or page, it should be organized into headings, bullets, or other clear method of organization.

 Written and oral language used is Standard American English

 Text on a page is organized into heading, bullets, or other clear method of

organization

 Language is free of colloquialisms, verbiage, and ungrammatical statements

 Written and oral language does not follow Standard American English rules  Text on the page is disorganized and

confusing

 Language is overly colloquial, verbose, or ungrammatical

3.5 The course gives opportunities to engage or interact with the content, versus passive observing or using basic navigation (in addition to assessment questions)

An eLearning course can engage students in a variety of ways. Some ways providers can chose to engage include:

1. Physical Interaction Examples: Students engage with content and explore content by clicking, hovering, dragging, scrolling, or swiping on screen.

2. Mental Interaction Examples:

a. Give opportunities to reflect. Give students time to think and write about how the content relates to their own lives and work.

b. Include decision making scenarios. Have students explore a scenario by choosing options based on what they have learned so far.

c. Give students choice. Allow students the choice of how they proceed through content (e.g., they can choose which module they want to do first, second).

(28)

13 | P a g e

 The course has physical and mental

interactions as listed above  The only interaction with the course is topress “next” or “forward” and participate in knowledge checks/quizzes

3.6 Passive delivery of content (watching videos, reading text) are kept to no more than 15 minutes in duration before the student needs to interact with the course.

The delivery of content should not be a completely passive experience. For example, content should not be delivered in 50-minute videos. This allows a student to press play and walk away from the computer for the entire time.

Interaction can take the form of: - Clicking next on a slide - Taking a knowledge check - Using an interactive slide - Pressing play on a video

 The course ensures students interact with the computer at least every 15 minutes

 Students do not need to interact with the computer every 15 minutes

3.7 The presentation of content is done in sufficient detail that enables students to successfully accomplish stated learning objectives in real world applications

Content that is presented in a course should be both specific and relatable to a student. While reviewing a course, think of whether a student can use anything tangible from this course.

Questions to ask yourself while assessing this criterion include:

- Is the course content detailed enough or does it offer vague or overly-broad information that a student may already know?

- Does this content directly relate to what a general contractor does in the field?

- Would a student be left with questions about implementing what they have learned in this course?

- Are the course objectives too broad? Does the course try to cover too much material and therefore does not cover enough for students to apply their knowledge?

 Course objectives and course content directly to a general contractor’s work in the field

 Content is relevant to general contractors

 Course objectives and course content do not related to a general contractor’s work in the field

(29)

 Content is specific and actionable  Content is not relevant to general contractors

 Content is vague and overly-broad or contains common-sense, well-known information

3.8 The presentation of content is presented in a chronological, systematic, or process oriented manner

This checklist criterion speaks to the organization of the content and presentation. Depending on the type of course and type of content, it will be organized in a different way (either chronologically, systematically, or in a process-oriented manner).

 Content is presented in a chronological,

systematic, or process-oriented manner  Content is disorganized Content presentation hampers comprehension of the course  Content is not presented in a

chronological, systematic, or process-oriented manner

3.9 The presentation of content is broken down into easily understandable learning objects

Strong eLearning courses “chunk” material. Chunked material is material that has already been broken down into smaller concepts. Small concepts are much easier to retain and connect with. It is the job of the course provider to break down concepts into manageable pieces for the student. This can appear in a variety of ways. For example, an eLearning course can have many slides with only one concept on each slide. Or, material is presented slowly, uncovering one idea at a time instead of putting all the concepts on a slide at once.

 Course content is chunked into smaller concepts

 Material is presented in manageable bits using tools like bulleted lists, interactive tabs, short videos, slide layers, or other similar tool

 Multiple ideas or concepts appear on one page or slide with no attempt to break down content

 All content is presented on one page

3.10 The student is provided with opportunities to apply the content and concepts in case studies/scenarios or application-level questions

(30)

15 | P a g e

Strong eLearning courses provide not just a presentation of content but also an opportunity for students to learn how to apply their knowledge and practice applying their knowledge in the course. Scenarios are often the most common way to have students apply their new-found knowledge.

 The course provides application level-questions in either the knowledge checks or summative assessment, or

 The course provides scenarios/case studies with questions for the student

 There are no application-level questions in either the knowledge checks or summative assessment

 There are no scenarios or case studies with questioning

 Only remembering/understanding level questions are asked

3.11 Assessment questions are directly related to learning objectives and presented content

Students should be assessed on what they have learned in the course and the stated objectives set forth by the course. While reviewing the assessment, ensure the questions come from material covered in the course and connect to the objectives stated at the beginning of the modules. Students should not be expected to google or search outside the course material or their notes to answer assessment questions.

 Each question in the summative

assessment is based on material directly from the course

 Each question in the summative

assessment is connected to at least one of the objectives from the course modules

 Answers to summative assessments cannot be found in the course materials  Summative assessment questions do not

connect to any of the objective from the course modules

3.12 Knowledge checks provide correct or incorrect feedback with remediation, or allow student to correctly answer before progressing

Knowledge checks (sometimes called quizzes by providers) are a learning tool for a student. Students can assess whether they are understanding the material and it serves as a checkpoint in the course. When a knowledge check provides feedback, it is letting the student know if they got the question correct or incorrect. Remediation goes a step further and either explains why the question is right or wrong or where the student can find this information. If a knowledge check does not provide feedback, a student should be allowed to go back and correctly answer an unlimited amount of times.

 Knowledge checks provides correct or

(31)

 Knowledge checks allow student to

correctly answer before progressing  Does not allow student to correctlyanswer before progressing

3.13 The summative assessment questions do not provide correct/incorrect responses and no remedial explanation

The summative assessment occurs at the end of the course. This should be a test of the students’

retention of the knowledge. When a student answers a question and submits it, they should only see the next slide. Their score should come at the end of the assessment. If correct/incorrect responses and remediation are given, it allows the student to quickly answer the assessment correctly without going back and learning the content.

 When a student answers a question on the summative assessment, there is no feedback and the course simply navigates to the next question

 The score for the summative assessment is only given at the end of the assessment after the student has answered all questions

 After answering a summative assessment question, the course indicates if the answer was right or wrong

 After answering a summative assessment question, the course gives the student the right answer

 After answering a summative assessment question, the course explains why the answer is right or wrong

3.15 The summative assessment does not include more than five true or false questions

True or false questions do not provide students opportunities to think deeply and show their learning, especially in application-type questioning. True or false questions should be limited in the summative assessment since they usually only test a basic level of understanding.

 The summative assessment includes five

(32)

17 | P a g e

Section 4 – Compliance

Overview: This section assesses how content complies with the guidelines, policies, and/or rules regarding continuing education credit content as defined by the NCLBGC.

4.1 The platform includes the ability to collect a student’s Qualifier ID and the Qualifier’s name on file with the Board

The Board tracks completions with students’ Qualifier ID numbers. Providers can collect ID numbers at in the LMS or in the course itself.

 The platform clearly collects a student’s

Qualifier ID in either the LMS or course  The platform does not collect a student’sQualifier ID anywhere

4.2 Effective July 1st_{, 2021 – This text is within the first three pages or slides of the course materials:}

“THE NORTH CAROLINA LICENSING BOARD FOR GENERAL CONTRACTORS HAS APPROVED THE COURSE ONLY AS TO ITS RELEVANCE TO THE PRACTICE OF GENERAL CONTRACTING IN NORTH CAROLINA. THE COURSE PROVIDER AND INSTRUCTOR ARE RESPONSIBLE FOR THE ACCURACY OF THE CONTENT AND COMPLIANCE WITH ALL STATE AND FEDERAL LAWS DURING THE ADMINISTRATION OF THE COURSE” Note that this is criterion is effective July 1st_{, 2021. If completing a checklist before this time, leave the}

checklist item blank and write “not applicable” in the notes.

 This text appears in the first three pages

or slides of the course materials  This text does not appear anywhere inthe course  This text appears after the first three

pages or slides of the course materials

4.3 The student is given instructions about how to search and confirm their Qualifier ID and Qualifier’s name on file with the NCLBGC Website

Explicit instructions are needed for students to look up their Qualifier ID and Qualifier name on nclbgc.org/qualifier-search.

 The student is told to go to the NCLBGC website and search their Qualifier ID and name

 There is no mention of how a student can find their Qualifier ID and name

(33)

4.4 A 2-hour CE elective course consist of 100 minutes of student seat time, a 4-hour CE elective course consist of 200 minutes of student seat time (including all assessments)

This is a mandatory requirement as outlined by the Board. To measure this, keep track of the time you spend in the course.

 The course, including assessments, takes approximately 100/200 minutes to complete

 The course, including assessments, takes significantly more than 100/200 minutes or significantly less than 100/200 minutes to complete

4.5 Prior to the start the course, technical system requirements are stated (including connectivity requirements, operating system version, impact of ad blockers, whether access to speakers or a printer is required, and so on)

Providers should give information on technical requirements somewhere on the LMS or inside the course. These technical requirements should be stated before the start of the course or near the

beginning of the course. This allows students to ensure they meet all requirements before beginning the course. If a student is using a computer that does not meet the technical system requirements, they may have difficulty completing the course.

 Language about system requirements is clearly laid out in either the provider website, LMS, or introduction to the course

 There is no language about system requirements on the website, LMS, or introduction to the course

4.6 Prior to the start of the course, a technical support phone number and/or email address is provided Students that have technical issues should be able to get them resolved through a phone number or email address that is maintained by the provider. The phone number or email address should be easy to find and clearly labeled as a technical support number for the eLearning course.

 A technical support phone number or email address is provided on the website, LMS, or introduction to the course  The phone number or email address is

explicitly labeled as a technical support number

 No phone number or e-mail address for the provider can be found on the course website, LMS, or introduction to the course

 A phone number or email address is found for the provider but it is not labeled as technical support number

(34)

19 | P a g e

4.7 The course does not promote specific products, services, or markets

The course should not function as a 2 or 4-hour commercial for a product or service. The provider should not recommend specific products or use the course to promote their service to contractors.

 No specific products, services, or markets are promoted or recommended in the course

 Product(s), service(s), and/or market(s) are promoted or recommended in the course

4.8 For every 50 minutes of learning content (differentiated by sections, modules, chapters, or other appropriate content organizational methods), there are 5 knowledge checks

Knowledge checks help students organize the knowledge that they have learned and provide a check to ensure they are understanding the content. Sometimes providers call these knowledge checks “quizzes.”

 Knowledge checks occur approximately every 50 minutes of learning

 There are 5 knowledge check questions per 50 minutes of learning (10 for a 2-hour course, 20 for a 4-2-hour course)

 There are no knowledge check questions  Knowledge checks are unevenly

distributed (occurring all at the beginning or all at the end of the course)

 There are less than 10 knowledge checks for a 2-hour course and less than 20 for a 4-hour course

4.9 The course has a final summative assessment consisting of 20 questions

The summative assessment can be thought of as the “final exam” of the course. It should contain exactly 20 questions

 There are 20 questions on the summative

assessment  There are fewer than 20 questions ormore than 20 questions on the summative assessment

4.10 The summative assessment passing score is set to 70%

To ensure that the course has a passing score set to 70, take the test twice in the following manner: (1) The first time you take the course, get questions 1-13 correct and questions 14-20 incorrect.

(35)

(2) The second time you take the assessment, get questions 1-6 incorrect and 7-20 correct. This should yield a score of 70%. Ensure that the course passes you for this attempt.

By taking the assessment this way, you will also be able to check if the answer key is correct. Note any question that has marked you incorrect when in fact you were correct, and vice versa.

 The summative assessment passing score is 70% (must answer at least 14 out of 20 questions correctly)

 The summative assessment answers match the answer key given by the provider

 The summative assessment passing score is not set to 70%

 The summative assessment answers do not match the answer key given by the provider

4.11 The course is “gated” providing no options (i.e., scrubbing through a video or navigating passed content) for students to earn credit without completing the prescribed course content and by passing the final assessment

If a course is gated, it means that a student is led through the course in a specific way. The next part of the course is made accessible to the student only if certain conditions are met. Courses can be gated in a few different ways, but mainly you will find courses that are gated via time. Courses can have a

countdown timer on a slide or page, and students may not move to the next slide until the timer is completed. Or, students must interact with a slide in a certain way and the next page will unlock after that. It must be ensured that a student cannot “fast-forward” through a course and reach the final assessment.

 Gating is provided either by a countdown timer or interactivity that limits

movement through the content to provide the appropriate amount of seat time as dictated by the credit hours of the course

 The course allows students to skip content and jump to the summative assessment

 The course allows students to quickly page through content faster than the seat time required (e.g., pressing next on slides rapidly without reading them)

4.12 A certificate of completion is provided to the student once the student successfully completes the course, or instructions accurately state how a certificate can be obtained

Students must be given the opportunity to get a certificate of completion from any course they take. Keep in mind that the certificate is not necessary for course completion and submission to the Board, but it can be a useful tracker for a student as they complete their continuing education credits. Very often courses put directions for finding and printing a certificate at the end of the course or in the LMS page. However, there is no mandatory way a certificate needs to be delivered to a student.

(36)

21 | P a g e

 A certificate of completion is provided in PDF or other printable format

 Instructions on how to obtain a certificate of completion is provided

 No mention of a certificate of completion  A certificate of completion is mentioned

but no instructions or there is no way to access the certificate

4.13 Prior to the start of the course, it is clearly stated that students will have 30 days to complete a course from initial enrollment (Students who do not complete the course within 30 days will need to complete the entire course if reenrolled)

This is a mandatory requirement as outlined by the Board. While it is not possible to assess if a course actually expires in 30 days, a reviewer can instead look for language on the website, LMS, or course that states this criterion directly.

 Text on the provider website, LMS, or course states students have 30 days to complete the course from initial enrollment

 No mention on the provider website, LMS, or course of the 30 day limit  There is a mention of a different time

limit (e.g., 90 day or 6 month limit)

4.14 Prior to the start of the course, it is clearly stated that the course will not be available to purchase, start, or complete after November 30th_{, and during December}

This is a mandatory requirement as outlined by the Board. Assessments of courses may occur at any time of year. Therefore, to assess this look for language on the website, LMS, or course that states this criterion directly.

 Text on the provider website, LMS, or course states that courses will not be available to purchase, start, or complete after November 30th_{, and during}

December

 No mention on the provider website, LMS, or course that courses will not be available to purchase, start, or complete after November 30th_{, and during}

December

 A different time period than the one mentioned in this checklist criterion is on the website, LMS, or course

(37)

Glossary of Terms

Bookmark/Bookmarking: The ability to stop, leave a course, and begin in the same place. Course Outline: A bulleted or snapshot overview of the different modules or topic of the course Gated Course: A gated course restricts navigation of the student.

Goal: An overarching main idea and purpose for a course. A goal is broad and not specific.

Knowledge Check: This is a quiz, graded or ungraded. It can provide remediation (an explanation of the answers) and can be taken an unlimited number of times.

LMS: LMS stands for learning management system. It is a software that administers, tracks, reports, and delivers educational courses. Students typically interact with it through a website.

Module: This is a specific section or lesson of the course usually around 50 minutes long. Longer courses are usually broken up into modules.

Objective: A specific, measurable, and attainable learning outcome for the course.

Summative Assessment: This is the final exam that occurs at the end of the course. It is made up of 20 questions and must be passed with at least 70% correct. This assessment can be taken an unlimited amount of times.

(38)

**Sample Timed Outline - *2-Hour Elective Course**

*This sample course will be delivered through lecture, supporting Powerpoint

slides and includes a case study activity

5 min Welcome and Introduction

• Procedures/announcements/facilities

o Attendance requirements, breaks, restroom locations, emergency procedures, etc.

• Course objectives

o Upon completion, students will understand … What will your students learn during this class? Touch on the main points of your course.

• Instructor intro/credentials o Instructor bio 5 min Topic #1 (Slides 1-2)

• Subtopic #1 (include topic to be discussed + important points to be made about the sub-topic)

• Subtopic #2 • Etc.

10 min Topic #2 (Slides 3-6)

10 min Topic #3 (Slides 7-10)

15 min Topic #4 (S11-18)