Introduction to the SIF 3.0 Infrastructure: An Environment for Educational Data Exchange

Introduction to the SIF

3.0 Infrastructure:

An Environment for

•

Goals of the Release

•

Environment types & Service types

•

Registration & Security

•

Basic Architectural Framework

• Service Scoping

• REST and REST “extensions”

• Query options

• Queues & Events

•

Getting Started

• REST Developer Sandbox (SIF-RS)

• Documentation

A high level overview of the

SIF 3.0 Infrastructure

• Independence from Payload Contents

– Secure, scalable transport

– Single infrastructure for all locale DMs, special Profiles

• REST Transport / Design Paradigm

– No “SIF-unique” technology where industry standards exist – “Basic+” URL access to “REST Resources”

– XQuery script support

• Direct Consumer   Service Connection Option

– Middleware optional in SIF v3.0 solutions!

• Vertical Organization Scalability Extensions

– Multiple v2.x Zones in single v3.0 Consumer “Environment” – Multiple “Contexts” for same Object Type in given Zone

Goals of the Infrastructure

release

A Comparison of Environment Types

•

Central Data Service

• Connects Consumers to its Services (no middleware) • SIS, LMS, Data Store

• No externally implemented Object Services are available

•

Supports multiple Consumers

• Each Consumer is the “only” one in its Environment • Consumers can see Events generated by others

•

Supports mobile Consumers

• Tablet Dashboards, Smart Phone Queries

•

Provides Clean API Subset of Brokered

Environment

– All Direct Consumers run in Brokered Environments

SIS providing a Direct Environment to

multiple Consumers

•

Central Message Broker

• Connects multiple Service Consumers and Providers • Enterprise Service Bus + other middleware components • Modular (Queues, Connectors, Registries) vs. SIF v2 ZIS

•

Provide Clean API Superset of Direct Environment

– All Direct Consumers run in Brokered Environments – Brokered Consumers can become Service Providers

•

Support for Administrative Interfaces

• Centralized Enforcement of Site Security Policy • Registries & Service Governance

• Preventative Maintenance & Diagnostics (“quiet apps”)

•

Highly scalable

SIS as a Service Provider in a Brokered

Environment

•

Standard REST Service Request Framework+

• All or subset of Query, Create, Update, Delete • Publish Events

•

Data Object (ex: SIS)

• Data Model (ex: SIF US 3.2) defines message payloads

•

Functional (ex: SRE)

• Encapsulate “behavior” as well as data • Awaiting DM POC

•

Infrastructure (Connectors, Queues,

Self-Provision)

• Core component of Infrastructure, URL Addressible

•

Utility (Alert, Registries)

• “Normal” service concerned with infrastructure

•

SIF v2.x Zone

• One Object Provider Service (SIS) per Object Type in Zone

•

SIF v3.0 Environment spans all known Services

• Available Services divided into Zones

• Single Zone typical of Direct Environment Provider

•

Contexts

• One or more “Contexts” specific to Service Type • Ex: “Current” or “Archived” Students

 One Service Provider / Context / Service Type / Zone

 A Consumer Registers & Provisions once

Student Services in an Environment

High School A Zone Student Service (Context: Archived) Student Service (Context: Current) Student Service (Context: Current)

Environment

High School B Zone Client

Application

Service Consumer

•

Encryption

– X.509 Certificates via Transport Level Security (SSL)

•

Authentication

– Shared Secrets, supported by HMAC SHA256 – Session Tokens

– Client Certificate Management reuse where present

•

Authorization

– Consumer Registration maps to Consumer Provisioning – Application Key, Instance and / or User Key (ID, Passwd) – XML Filters applied via “XQuery Reports”

•

Service operations: Get/Post/Put/Delete

– Every Service Provider is a REST Resource

– “Class” is URL, “Object” is URL/RefId (students/1234) – Responses are immediate

– HTTP Message payload is pure data

•

HTTP Error Codes

•

Infrastructure values in:

– HTTP Header fields – URL Segments

– URL Matrix and / or Query Parameters as appropriate



But there’s more ...

Service

Consumer

Environment

Provider

Basic Consumer / Provider Interface

(all operations single or multi-object)

CRUD Request

CRUD Response

API

Service

Consumer

Environment

Provider

Immediate and

Delayed

Responses

CRUD HTTP Request “Accept” HTTP Response

API

Q

Data Query Packaging Options

Query Retrieval Option

Provider Response if successful

Unqualified Query Returns all data for all objects held by the Provider. Ex: students

Query by RefId Returns all data for the object corresponding to the specified RefId Ex: students/1234

Paged Query The XQuery response is limited to a selected “Page” of object results. Each page of

results is individually requested by Page # and limited to Size of Page in Objects, May be issued interactively (individually) or delayed (once to get all pages sent to Queue)

eTags A common “REST way” to detect whether any objects have changed since the last

Consumer Query, implying the Consumer can avoid re-querying all objects to synch up with the data held by the Service.

Data Query Selection Options

Query Retrieval Option

Provider Response if successful

XQuery Template Uses an XQuery Token “Name” to indicate a registered XQuery Template which will be

executed by the “named” Service Provider to return the predefined set of elements (including possible calculated aggregates) in the qualified set of objects satisfying the query.

Parameters to the Template are supplied as URL Query Parameters Ex: StudentSnapshot/?id=1234

Service Path Used to bridge “Associative” objects in a single Query, by specifying a Service Path rather

than a Service Name: Assuming an

Ex: schools/4567/students returns all Students in School 4567

Dynamic Query A “where” clause specified as a URL Query parameter can constrain the set of objects

returned by the Query

•

Typical Environment Provider

– Direct Single Service (ex: SIS with Student Data)

– Could be other Services available which are not used

•

Consumer Functionality

– Consumer is pre-provisioned.

– Register - then issue CRUD requests to preset Services – All Responses immediate and single object

– Default Zone & Context use only

– Full object Query only (Paged or not) • No XQuery, no Dynamic Query

– Issue ”create” to Alert Utility Service in case of errors

 Standard RESTful Client

•

Typical Environment Provider

– State Level Brokered (SLDS with full CEDS Data Model)

•

Consumer Functionality

– Register, examine Service Registry, dynamically provision – Gather, update, monitor data from multiple Zones

– Acquire multiple Queues, subscribe to & process Events – Utilize single & multiple object Requests

– Utilize XQuery scripts to support Composite and Report Object types

– Utilize Service Paths and Dynamic Query

– Provision as a Service Provider (Brokered Environs only)

Environment Providers

(concentric levels of increased functionality)

SIS

Application

Application

Environ

A Simple Consumer Session

•

Initialization

• Immediate Object Retrieval

1a. Consumer Registration Request

1b. Environments Response

REQ Connector URL

Environment

Provisioned Access Rights

Event Connector URL

Returned URLs may correspond to a testing, staging or production environment Session Token Queue Instance Service

• Initialization

•

Immediate Object Retrieval

• Object Creation & Event Reception

2. Immediate Query Request /

Response

Zone Provider Interface

Consumer Provisioning URL

Queue Manager URL REQ Connector URL

Environment

Provisioned Access Rights Authorization Token

•

URL determines destination

– Object Type or Service Path or XQuery Template is in URL segments

– Zone & Context are URL Matrix Parameters

– Request routed (Brokered Environment) directly to Service Provider

•

Multiple Object Requests

– Create, Query, Update, Delete “students”

•

Single Object Requests

– Create “students/student”

– Query, Update, Delete “students/12345”

•

Immediate: HTTPS Response from Service

•

Delayed: HTTPS Response is “Accepted”

– Actual Service Response asynchronously delivered to Queue

•

Multi-object Request has Multi-object Response

– Total Operation Status: “Success”,

– Per object substatus: Ordered, “Success” or “Failure” – All successful changes are reflected in issued Event

• Initialization

• Immediate Object Retrieval

•

Object Creation & Event Reception

3.a Change Object Request

Queue Manager URL REQ Connector URL

Environment

Provisioned Access Rights Authorization Token

3.b Provider Response & Change

Event

Zone Provider Interface

Consumer Provisioning URL

Queue Manager URL REQ Connector URL

Environment

Provisioned Access Rights Authorization Token

• Read This First (RTF)

– Explains and highlights the standard documentation

• Base Architecture

– Framework for Infrastructure

– All Request Types and Options explored

• Infrastructure Services

– Core Components comprising the Infrastructure – Environment Security, Connectors, Queues

• Utility Services

– Zone & Provider Registry & Alert Service

• Associated XML Schema

• A self-explanatory “hands on” learning tool for SIF v3.0. • Maps the SIF v3.0 specification to the REST platform. • Serves as a detailed reference implementation / test

harness with an associated human interface to help REST programmers understand and use the SIF v3.0

infrastructure.

• Provides quick Consumer access to a defined set of SIF v3.0 RESTful services (Environ, Connector, Provision, Student)

• Is a maturing developer tool and now features the ability to subscribe to Events and access them through a Queue. • Available at: http://rest3api.sifassociation.org/jsp/index.jsp

•

Converts Logical Data Model to multi-app

Solution

•

Independent of Payload

•

REST Technology (plus extensions)

•

Middleware leveraged but optional

•

Low Entry Point for Consumers (Register and Go)

•

Massively scalable at high end

– Environment spans multiple Zones and Contexts – Pub / Sub Events and Asynch Message Queues – Multi-object Responses and Events

– Enforcement of Site Data Security Policies

•

Documents & Sandbox in place:

1. Independence from the SIF Data Model Check

2. Increased Scalability (Multiple Objects/ Message) Check

3. A more flexible SIF Zone (Environment). Check

4. Support for “Lightweight” Client / Server connection type. (Direct Environment) Check and double check

5. Alignment of Infrastructure Service interfaces with existing middleware products Check

6. Define educational process “behavior” as well as data - Functional Services, (SRE is the POC) 1/2

7. Provide a REST Transport Story Check and double check

8. Support XQuery functionality Check

9. Extend Security Functionality Check

10. Standardize Zone Diagnostic and Preventative Maintenance “hooks” (Alerts & Queue statistics are a part of this). 1/2

SIF 3.0: Where do you want it to

take you?

Ron Kleinman, CTO

[email protected]

SIF 3.0 Infrastructure:

SIF REST Developer Sandbox

http://rest3api.sifassociation.org/jsp/index.jsp

Questions?