The DLM Assessment System: Planning for Implementation. November 13 & 14, 2014

The DLM Assessment System:

Planning for 2014-15

Implementation

Today’s Topics

1. Overview of the DLM system and year end testing model

2. 2014-2015 from the teacher’s perspective

3. District support for implementation 4. Other topics as requested

OVERVIEW OF ASSESSMENT

SYSTEM

Learning

Map

Claims Conceptual Areas Essential Elements

DLM Essential Elements

• Are the target for the grade level

• Reduced depth, breadth, complexity • Provide appropriate level of rigor and

challenge

• Focus on the skills (with multiple means of demonstration)

• Are not functional or pre-K skills or instructional descriptions

Example for English Language Arts

• RL.6.2 Determine a

theme or central idea of a text and how it is

conveyed through particular details;

provide a summary of the text distinct from

personal opinions or judgments.

• EE.RL.6.2 Determine the

theme or central idea of a familiar story and

identify details that relate to it.

Common Core State

Identify two related points the author

makes in an informational text

Testlets in Linkage Levels

Initial Precursor

Target

Successors

Connect the map…

Behavior Testlet _IP

…to the items developed.

Distal Precursor Proximal Precursor Behavior Behavior Behavior Behavior Testlet DP Testlet PP Testlet T Testlet S

Fractions

M.EE.3.NF1-3 Differentiate a

fractional part from a whole

Assessment Design

• Testlets

– Usually assess 1 EE / linkage level – 3-5 items

– Engagement activity

• ELA: Text or writing task

• Math: Context at beginning, carried

throughout

– Teacher-administered or

• Special user interface designed for DLM

• Customized presentation through

Personal Needs & Preferences (PNP) profile

Sample Testlets

ELA: RI 5.8: Identify the relationship between a specific point and supporting reasons in an

informational text

• Target Level

• Node: Same as EE

Math: 7.NS.2.a: Solve multiplication problems with products to 100

• Initial Precursor Level

• Nodes: Recognize set, recognize

Two Levels of Writing Assessments

• Conventional Writing

– Using orthography to communicate – Selecting topics

– Generating and communicating facts, ideas and

details in writing

• Emergent Writing

– Behaviors that precede conventional writing – Communication about choices and information – Cognitive processes that are precursors to

Example Task/Rating in an Emergent

Writing Testlet – Screen 1

SAY: What do you want to write about?

Choose one.

WAIT AND OBSERVE: Give the student

Example Task/Rating in an Emergent

Writing Testlet – Screen 2

Choose the highest level that describes your observation.

• Makes a choice between objects.

• Attends to objects but does not make a choice. • Seeks objects but does not make a choice.

• Attends to other stimuli. • No response.

Example Task/Rating in an Conventional

Writing Testlet – Screen 1

SAY: What other facts, ideas or details can you write about [topic]?

WAIT: Give the student time if needed to think

about additional facts, details or other information about the topic.

SAY: Write about them.

WAIT AND OBSERVE: Give the student time to

Example Task/Rating in an Conventional

Writing Testlet – Screen 2

Choose the highest level that describes your observation.

• Independently wrote 3 or more facts, ideas or

details related to the topic

• Independently wrote at least 2 facts, ideas or

details related to the topic

• Wrote one fact related to the topic

• Wrote multiple facts, ideas or details unrelated to

the topic.

• Wrote a single fact, idea or detail unrelated to the

Accessibility

Accessible Content Technology Personal Learning Profile

Accessible Content

• Testlet levels

• Vocabulary

• Multiple and alternate pathways • Special item writing guidelines

– Careful attention to unnecessary

Personal Learning Profile

Personal Needs and Preferences (PNP)

• Display

• Language & Braille • Audio & Environment

First Contact • Communication • Academics • Sensory characteristics • Motor characteristics • Computer access

• Special user interface designed for DLM

• Dynamic routing

Accessibility Manual

1. Include Eligible Students in the DLM Assessment

2. Learn About the DLM Accessibility Features: What Does DLM Provide?

3. Discuss and Select Appropriate Supports and Tools: Considerations for IEP Teams

4. Enter Appropriate Supports into the DLM System

5. Prepare for the Assessment: Using the Chosen Accessibility Features

6. Evaluate the Accessibility Features Used After the Assessment

1. Supports activated by PNP

A. Magnification

B. Invert Color Choice C. Color Contrast

D. Color Overlay

E. Read Aloud with highlighting – Text to Speech text only

text & graphics graphics only nonvisual

2. Supports Requiring Additional

Materials/Tools

F. Uncontracted Braille

G. Single-switch system/PNP enabled H. Two switch system

I. Administration via iPad

J. Adaptive equipment used by student K. Individualized Manipulatives

3. Supports Provided

Outside the System

L. Human Read Aloud

M. Sign interpretation of text N. Language translation of text

O. Test administrator enter responses for student P. Partner Assisted Scanning

Personal Needs & Preferences

(PNP) Profile

• Display Enhancements • Language & Braille

• Audio & Environment • Other Supports

Students with Blindness/Visual

Impairments

• Alternate forms

– Content would normally be represented

on computer but translated into teacher-administered testlet

• Braille

Braille

• Very small number of DLM students use Braille

• Forms will be available:

– Beginning spring 2015 operational

window

– Only in upper linkage levels

• Delivery method still under evaluation by state partners

Hardware/Software

• No software costs (KITE is free)

– PNP features built in

• Use devices already available

(interactive whiteboard, 1-switch or 2-switch systems, headphones, etc.) • Option for test administrator to enter

Year End Assessment Model

Optional

Instructionally-Embedded Spring Testing Required

Optional Instructionally Embedded Testlets can be delivered for select Essential Elements

System recommends linkage level; teacher can accept or override

Student performance does not contribute for summative

purposes

Testlets cover the whole blueprint

Dynamic routing supports adaptivity between testlets (system determines linkage levels)

Scores are used for summative purposes

Content of the Assessment

• Grades 3-8 and high school

– English Language Arts (ELA) – Mathematics

• Blueprints:

– Consortium approved a subset of

Essential Elements

– Consortium set minimum requirements

Test Delivery

Optional Instructionally Embedded

• One testlet per EE

chosen in ITI

– Teacher chooses linkage

level

• Each testlet assigned

separately

• Test administrator

schedules sessions within testing windows

Spring (End of Year)

• 4-7 testlets per subject

depending on grade

– Entire blueprint covered – System chooses linkage

levels

• Each testlet assigned

separately

• Test administrator

schedules sessions within window

Testlet Delivery

Part

1

IP

DP

PP

T

S

Part

2

IP

DP

PP

T

S

Part

3

IP

DP

PP

T

S

Part

4

IP

DP

PP

T

S

System has testlets available at all 5 linkage levels for

every part of the test Students take one

testlet from one level for each group

System determines which level to deliver to the student

Testlet Delivery

Part 1

IP DP PP T S

Part 2

IP DP PP T S

Part 3

IP DP PP T S

Part 4

IP DP PP T S

Testlet Delivery

Part 1

IP DP PP T S

Part 2

IP DP PP T S

Part 3

IP DP PP T S

Part 4

IP DP PP T S

Testlet Recommendations &

Assignments

Performance Profile

Part 1: Overall Results

Performance Profile

Performance Profile

Part 3: Narrative

• Installed on testing devices • Student login credentials

• Website access (use Firefox) • Staff login credentials

2014-15 FROM THE TEACHER’S

PERSPECTIVE

Key Steps

1. Training & PD

2. Eligibility and accessibility decisions

3. Manage student data in Educator Portal 4. Select content; provide instruction

5. Optional: prepare for & deliver

instructionally embedded assessments 6. Prepare for & deliver testlets during

spring window

Required Training vs.

Professional Development

Training

• Multiple modules

• Available in self-directed

and facilitated formats (states/districts decide which to use)

• Covers critical content for

managing and delivering the assessment

• Required for all test

administrators

– No access to student logins without it

PD

• Multiple modules

• Available in self-directed

and facilitated formats (states/districts decide which to use)

• Covers a variety of topics

to support instruction in academics

• States determine what is

Other Resources for

Test Administrators

• Test Administration Manual

– With checklist

• Accessibility Manual

• Information about EEs and levels • Access to released testlets and

practice activities

• Educator resource page with supplemental info & videos

Management of Student Records

• Accuracy of roster

– Correct students, correct information

• First Contact

• PNP: Accessibility features discussed with IEP teams

– Documented in IEP

DISTRICT SUPPORT FOR

IMPLEMENTATION

DLM-Defined Roles

• Data Steward

• Technical Liaison

• Assessment Coordinator

• Test Administrator (teacher, educator) • Individuals may fill more than one

Some States Have Other Roles

• Regional technical assistance provider • Mentor/facilitator

• Vendor (other support services) • Others?

Data Steward Responsibilities

• Gather, validate, and upload teacher,

student, and roster data into Educator Portal

Technical Liaison Responsibilities

• KITE™ Client

• Troubleshooting Educator Portal • Troubleshooting CSV files

• Whitelisting

– *.cete.us

Assessment Coordinator

Responsibilities

• Manage district/school staff roles

• Develop and implement training plan • Develop and implement

communication plan

• Monitor / support test preparations and administration

Checklist

• Categories:

– Before beginning assessment

• Information/resources • Steps to take

– During (Optional) instructionally

embedded assessment

– Spring assessment

Before Assessment

Manage district/school staff roles

and responsibilities.

• Identify test administrators/examiners • Identify a Technical Liaison

• Share roles/responsibilities and timelines with staff

Before Assessment

Prepare for assessment implementation

• Disseminate Technical Liaison contact information • Encourage sign up for test notifications

• Establish communications between Data Steward,

Technical Liaisons, and DLM.

• Develop communication channels with district

personnel (updates, etc.)

• Monitor creation of Educator Portal user accounts

• Follow up with test administrators • Establish a test security plan

User Account Creation

Common Roles in Educator Portal

• District Test Coordinator • Teacher

• Building Test Coordinator • Building & District Users

Assigning Permissions in EP

State

District

Building

Unusual Cases

• Contract teachers

• Out of district students • Charter schools

Before Assessment

Develop and implement a comprehensive training plan OR implement your state’s training plan for district staff and test administrators.

• Review the checklists (TL & TA)

• Encourage Technical Liaisons to complete training • Deliver required test administration training

• Monitor educator completion of required training • Identify additional training needs

Monitoring Completion of RT

Now

• Participant can take screen shot of

transcripts area

• State PD administrator can look up

individuals

• State PD administrator can run PD

completion report, share with districts Starting in January

• New District PD administrator role – look up

Before Assessment

Monitor and support test preparations.

• Remind Technical Liaisons and Test Administrators

about important deadlines.

• Monitor identification of students for DLM

• Ensure every student in the system is assigned to a teacher

• Monitor completion of Access (PNP) Profile and First Contact

• Assist test administrators with technology

preparations

• Support test environment preparation

Before Assessment

Develop a plan to facilitate communications with parents and district staff.

• Consider what parents and district staff will

need and want to know about DLM assessments in 2014-15.

• Determine the frequency of communications

to various audiences.

• Deliver communication to various audiences

on the timeline in your communication plan.

• Relay new information from DLM and your

state and district to appropriate audiences as needed.

Before the Spring Window

• Support Test Administrators in

rechecking student enrollment, demographic information, PNP

Checklist

• Categories:

– Before beginning assessment

• Information/resources • Steps to take

– During instructionally embedded

assessment

– Spring assessment

Spring Window

• Monitor and support test administration.

• Facilitate communications between DLM staff

and district staff during testing window(s).

• Obtain and deliver Braille forms as needed.

• Monitor test activity, including test completion

prior to end of testing window.

• Serve as the first point-of-contact for teachers,

answering questions about the assessment. Troubleshoot issues as they arise.

• Serve as the point-of-contact to the DLM help

Monitoring

Available in January:

• Report showing number of testlets

completed and in progress vs. what is intended

Checklist

• Categories:

– Before beginning assessment

• Information/resources • Steps to take

– During instructionally embedded

assessment

– Spring assessment

Resources & Next Steps

• Bookmark your state’s page on the DLM

website. Use the resources on your state’s page.

• Activate your Educator Portal account by

following the instructions in the KITE activation email.*

• Complete the Security Agreement in Educator

Portal profile.*

• Sign up for DLM test notifications during the

Help Desk Support

• Three tiers of support for local users • Information published in the front of

all manuals

• Toll-free phone number

– Extended hours during testing windows

• Email with ticketing system

• Answers customized per testing model & for unique state policies

Communication

KITE system • Status on website • Testing updates subscription • Special email to states Other issues • Special email to states • Additional postings, announcements as warranted

THANK YOU!

For more information, please contact:

[email protected] or

Go to: www.dynamiclearningmaps.org

For Professional Development, contact:

[email protected]

The present publication was developed under grant 84.373X100001 from the U.S. Department of Education, Office of Special Education Programs. The views expressed herein are solely those of the author(s), and no official endorsement by the U.S. Department should be inferred.