SIF 3: A NEW BEGINNING

(1)

(2)

The SIF Implementation Specification



Defines common data formats and rules of

interaction and architecture, and is made up of

two parts:



SIF Infrastructure Implementation Specification



defines the rules of interaction and messages

framework, and



SIF Data Model Implementation Specification



an XML-based data model that models entities in the

learning environment as SIF Data Objects to be shared

between applications.

(3)

Existing SIF 2.x infrastructure

Network Account H.R. & Finance Data Warehouse VLM Library Automation MIS Meals Post 16

Zone Integration

Server (ZIS)

SIF Agents

Applications

SIF Data Objects

(4)

Existing SIF 2.x infrastructure

Zone Integration

Server (ZIS)

SIF Agents

Applications

SIF Data Objects

(5)

Existing SIF 2.x infrastructure

Zone Integration

Server (ZIS)

SIF Agents

Applications

SIF Data Objects

(6)

Existing SIF 2.x infrastructure

Zone Integration

Server (ZIS)

SIF Agents

Applications

SIF Data Objects

(7)

Existing SIF 2.x infrastructure

Zone Integration

Server (ZIS)

SIF Agents

Applications

SIF Data Objects

(8)

Existing SIF 2.x infrastructure

Zone Integration

Server (ZIS)

SIF Agents

Applications

SIF Data Objects

(9)

(10)

Getting started has never been easier...

… as well as flexible and scalable!

(11)

New Terminology



New SIF 3 ‘environments’



Direct



Brokered

SIF 2

SIF 3

Agent

Adapter

Zone

Environment

ZIS

Middleware

(12)



SIF functionality now fits in with everything else

deployed in your IT data centre.

Your developers and

administrators will appreciate that the SIF 3.0 infrastructure is

REST based, and fully conformant with all accepted REST design

patterns.



A SIF 3 enabled application doesn’t have to be in

your data centre.

It can be located in the Cloud. As a

result, your SIF 3.0 solutions can span “hybrid” data centre /

cloud service deployments.



Middleware is no longer mandatory in deploying

SIF 3.0 solutions.

For example, a tablet based client

application can use the SIF 3.0 standard to directly access and

update the data maintained by your MIS or Data Warehouse,

independent of which particular SIF-enabled supplier products

you have deployed.

(13)



Data Security is baked into the SIF 3.0 standard

from the beginning

as it was in earlier releases. New

security profile extensions standardise how sites currently

utilising client certificates, shared secrets, and / or HMAC SHA

256 authentication technologies may leverage them when

constructing SIF solutions.



Data Privacy policies are enforceable

through a

combination of Data Profiles and administrative best practices.

For example a School or Academy could configure and dedicate

a separate External Environment in which all student data was

“de-identified”, and then allow the access of any Local Authority

or Academy Trust-level data analytics application to that

environment.

(14)



SIF remains an established, community built, free

and open data blueprint

that is supported by hundreds of

educational applications regardless of platform or vendor

provider. SIF 3.0 is “tighter” in terms of defining requirements

than its predecessors and as a result allows educational

institution architects to select “best of breed” SIF-enabled

components, assured in the knowledge that their choices will

interoperate with little or no integration effort, and the value of

their IT investments will be preserved.

(15)

SIF 3 Functionality



Completely Independent of Data Model



Leverages existing infrastructure

technologies- REST

SIF Implementation Infrastructure Specification 3.0.1

SIF Implementation Data

Model (UK) 2.0

OR

‘X’ Data Model

OR

‘Y’ Data Model

(16)

Why is the separation of infrastructure

& data model so important? (1)



Each locale can publish a release as often as they wish,

enabling responsiveness.



The SIF Association can release the global infrastructure

free from data driven pressures, increasing quality.



Network architects now find many of the best practices

they expect in the SIF specifications, promoting adoption.



Integrators can apply generic tools to each layer, allowing

for the better sharing of existing network resources.



Network programmers can use common REST tools to

(17)

Why is the separation of infrastructure

& data model so important? (2)



Everyone implementing a SIF published data model may

utilise data schema driven tools, greatly reducing effort.



Software architects are freed to design simpler components

that naturally fit together, increasing maintainability.



Infrastructure products can be more flexible when working

with modular data models, creating opportunities for reuse.



Everyone following a few conventions is now empowered to

use their own data model with the SIF infrastructure,

encouraging innovation.



Decision makers are free to chose the form of their data

payloads including: alternate formats, efficient compression,

and even end-to-end encryption, meeting more specialised

needs.

(18)

The SIF 3 infrastructure defines:



a secure REST-based transport protocol for defining both

direct and brokered connections between software

applications (which can act as REST servers or clients)



a set of services to enable interactions between REST

servers and clients, including creating environments, other

administrative task, synchronous exchanges of data, and

asynchronous message queues



an abstract, platform-independent definition of the API

between one software application and another or a

broker.

(19)

SIF 3 infrastructure definitions:



A SIF Environment is a distributed system that may consist of

middleware (a broker) and one or more software applications with a SIF

Adapter (a SIF-enabled application) sharing/reporting one or more SIF

data objects over a network. A SIF Implementation consists of one or

more SIF Environments deployed and configured to meet customer

data sharing and reporting needs.



The SIF Implementation Specification defines architecture requirements

and communication protocols for software components and the

interfaces between them; it makes no assumption of specific hardware

or software products needed to develop SIF-enabled applications

and/or middleware implementations, other than their ability to support

technologies leveraged as the foundation for SIF, most prominently XML

and HTTP(S) and including JSON and REST.

(20)

SIF 3 Environments (1)



Direct



Connects Consumers to its Services (no

middleware)



Support multiple Consumers



Support mobile Consumers



All Direct Consumers run in Brokered

(21)

SIF 3 Environments (2)



Brokered



Enterprise Service Bus + other middleware



All Direct Consumers run in Brokered

Environments



Centralised Security Policy & Workflow



Preventative Maintenance & Diagnostics

(22)

SIF 3 Transport: REST



Goal: HTTPS usage matches REST Design

Patterns



Service operations correspond to Get/Post/Put/Delete



Service Provider is a REST Resource



“Class” is URL, “Object” is URL/RefId (Students/1234)



Responses are immediate



Header arguments in:



HTTPS Header fields



URL Segments



URL Matrix and / or Query Parameters as appropriate



HTTPS payload is pure data



Interactive Paged Query with ETags

(23)

SIF 3 infrastructure

Management Information System

Application

Direct Environment

Adapter

Consumer (optional)

(24)

SIF 3 – Direct Environment



Implemented by a Central Data Service



Example: MIS, Data Store



Connects directly to Consumer Applications (no middleware)



No 3

rd

Party Services made available to Consumers



Supports multiple Consumers



Each Consumer is the “only” one in its Environment (promotes Data

Security and Privacy)



Consumers can see data change Events generated by others



Ideal for supporting mobile device based Consumer

Applications



Tablet Dashboards, Smart Phone Queries



Any ‘Direct’ Consumer Application can be seamlessly

(25)

SIF 3 infrastructure (2)

Application

Brokered Environment

Adapter

Message Broker Middleware

Consumer

(optional)

Management Information System

(26)

SIF 3 - Brokered Environment



Contains a Central Message Broker



Connects multiple Service Consumers and Providers



Enterprise Service Bus + other middleware components



Modular (Queues, Connectors, Registries) vs. ZIS



Provides a More Mature API including a Superset of

Direct Environment



All Direct Consumers can also run in Brokered Environments



Supports Administrative Interfaces



Centralised Security Policy & Workflow



Registries & Service Governance



Preventative Maintenance & Diagnostics



Provides a highly scalable solution

(27)

SIF 3 infrastructure (3)

MIS Data

Provider

Application

Hybrid Environment

Adapter

Message Broker Middleware

(Any of 20,000 RESTful

Mobile Client Applications)

Direct Environment

Student Portal

LMS Data

Provider

Student Contact

System

Brokered Environment

(28)

SIF 3 Solutions: Summary

Direct

Environment

Single source for all data

Brokered

Environment

Multiple Data Sources

Hybrid

Environment

 Connects Consumers to Providers with no

middleware - can be multiple connections.

 Low End: MIS accessed by students via mobile devices  High End: Data warehouse

seeded by multiple end-user MIS and accessed by data analytic and reporting applications

 Connects Consumers to multiple Providers leveraging existing IT middleware (ESB)  All Direct Consumers can run in

brokered environments

 External applications register to provide their data – highly scalable

 Standardised Consumer interface to MIS, data warehouse

 Centralised enforcement of site data security and privacy

policies

Direct

 School Portal utilises Direct Environment for 10000 students accessing via a mobile device.  The mobile device software sees

standard RESTful Service to display  Identical information whether School

Portal is local or in the cloud Brokered

 School Portal is also Consumer in a Brokered Environment accessing MIS, Assessment and VLE services

 School MIS Provider is also a

Consumer of a Authority/Trust-wide Brokered Environment where it seeds Learner Data into the Data warehouse.

(29)

(30)

Incorporating SIF 2.x and SIF 3.x

Library Automation VLM Operational Data Store H.R. & Finance Identity Management MIS Meals Assessment

(31)

Incorporating SIF 2.x and SIF 3.x

Library Automation VLM Operational Data Store Identity Management MIS Meals Assessment

(32)

Incorporating SIF 2.x and SIF 3.x

Library Automation VLM Operational Data Store H.R. & Finance Identity Management MIS Meals Assessment

(33)

Incorporating SIF 2.x and SIF 3.x

H.R. & Finance

Identity Management

Meals

Assessment

SIF 2 (SIF Classic)

Student Contact System Library Automation VLM Operational Data Store

(34)

Incorporating SIF 2.x and SIF 3.x

H.R. & Finance

Identity Management

Meals

Assessment

SIF 2 (SIF Classic)

SIF 3 (Direct and/or Brokered)

(35)

Incorporating SIF 2.x and SIF 3.x

H.R. & Finance

Identity Management

Meals

Assessment

SIF 2 (SIF Classic)

SIF 3 (Direct and/or Brokered)

SIF 2 (SIF Classic)

(36)

Choice

SIF Implementation Infrastructure Specification 3.0.1

SIF Implementation Data

Model (UK) 2.0

OR

‘X’ Data Model

OR

‘Y’ Data Model

(37)

Scalability (1)

Performance Enhancement Description Reference Multiple Consumer

Sessions for same application

Site Administrators can enable the same application instance, using multiple identifier values (ex: Johnson JHS and Miller Elementary) to register itself as multiple completely

independent and parallel Consumers.

The Environments Service “create Environment” section in Infrastructure Services.

Multiple FIFO Message Queues for same Consumer

Any Consumer can set up separate Message Queues and assign different Event types to them.

This allows the less important Events (ex: Attendance) to sit in Queue B until the messages in Queue A have all been retrieved and processed.

The Environment Scalability section in Infrastructure Services.

Multiple concurrent Consumer connections option to same Queue

A Consumer may use (where the Queue Service supports it) more than one multiple simultaneous concurrent connection by having multiple HTTP Requests open with a single Queue.

The Queue Service section in Infrastructure Services. “Get Next and Pop”

Queue Service Query Request

This replaces the SIF 2.x “double handshake” between Consumer and Polling Queue with a single handshake, reducing the number of message exchanges necessary to retrieve an incoming Event during periods of high message traffic by a factor of two.

The Queue Service section in Infrastructure Services.

“Long Polling” This option eliminates the message retrieval latency time previously associated with repeated polling for an arriving

The Queue Service section in Infrastructure Services.

(38)

Scalability (2)

Performance Enhancement Description Reference Multiple Objects

conveyed in single Request and single Event

The ability to “pack” a single Request to or Event with multiple affected objects from a specific Service Provider offers huge performance benefits for several important use cases (ex: a thousand small Attendance Objects created at the end of a Reporting Period result in the generation of 10 multi-object Events rather than 1000 single object Events).

The Service Operation section in the Base Architecture.

Immediate Service Responses

In another major change from SIF 2.x, a Consumer may now request that the Service Response to any given Request be returned synchronously (i.e. immediately) on the HTTP Response to its issued HTTP Request, rather than appear asynchronously at some later time, intermixed with arriving Events.

This will likely be the common case for most Requests, it conforms to the standard REST design pattern, it maximises the effectiveness of Direct Environments and it is expected to improve overall Request / Response processing throughput significantly.

The Basic Infrastructure

Framework section in the Base Architecture.

(39)

Scalability (3)

Performance Enhancement Description Reference eTag An eTag is equivalent to a “checksum” on all the objects of the

type being queried which are maintained by the Service Provider, and it is optionally returned in all Query Responses. If a valid eTag is included in a Query Request, and if there were no object data changes since the eTag was created, no objects are returned.

This offers an efficient way for Consumer applications

executing on mobile devices (which are unlikely to subscribe to Events) to determine whether they already have the latest data on one or more objects without comparing all the elements.

The Service Operation (Query) section in the Base

(40)

Use Cases & examples from

the ‘real-world’

(41)

(42)

Norfolk County Council (SIF 2)

Identity Management Schools People Net (HR) Norfolk Cloud Portal Capita Open Hive MIS PupilAsset (Assessment) eSafety (Google Apps) Capita Discover (Assessment) eSchools

(register) Different _Class Classroom _Monitor _HomeworkShow My Pearson _Fronter

eChalk

Capita Learning Gateway

Google Classroom

(43)

Norfolk County Council (SIF 3)

Identity Management Schools People Net (HR) Norfolk Cloud Portal Capita Open Hive MIS PupilAsset (Assessment) eSafety (Google Apps) Capita

Discover eSchools (register) Different _Class Classroom _Monitor _HomeworkShow My Pearson _Fronter

eChalk Capita Learning Gateway Google Classroom ‘Best of Breed’ /

mobile applications Data

Warehouse

Government reporting

(44)

(45)

Next Steps?

Now in terms of providing an Adapter which will integrate your application into a SIF

Environment (allowing it to seamlessly and securely exchange data with other

applications), basically your options, which determine the kind of tasks your

consultant will do and how long it will take, are:



Do it yourself:

SIF 3 is a standard of standards. Another way of looking at this is it has a solid core of

web technologies and patterns. So your favourite languages, libraries, and frameworks for your favourite environment should serve you very well. For the application requiring only simple interoperability, this may be the best option.



Leverage Open Source:

Say you want to kick it up a notch and leverage a lot of SIF 3’s functionality

or just build your own software without becoming an expert on SIF. There exist some excellent Open Source tools specifically for SIF 3. For the sophisticated application that needs to be completely under the owner’s control, this is a top option.

 AU examples: https://github.com/nsip/sif3-framework-java / https://github.com/nsip/Sif3Framework-dotNet



Find a Partner:

It is fair to say that SIF would not be where it is today without our infrastructure

vendors. Beyond providing middleware, they combine database expertise and SIF know-how to quickly produce quality products for others. When quality and timeline are your primary concerns, bringing in the