Ensuring quality experience for consumers Testing & Compliance

Ensuring quality experience for

consumers – Testing & Compliance

Mike Tometzki

2

Media & Device Testing Specialists

Certification – Interoperability – Conformance - Quality

Testing and Certification Services for over 40 Platforms Globally

Introduction to Digital TV Labs

UKAS accredited

Testing Laboratory

No.8159 to ISO/IEC 17025:2005

3

Services & Products

Consultancy

Services

Products

Specification Analysis

Device Specification Authoring Test Regime Design

Deployment Implementation Knowledge Transfer and Training

Digital TV Market Requirements Receiver Technical Reference Requirements

Test Regime Operation Test Suite Design Test Regime Auditing Device Test Services Play-out Validation Application Test

RF Front End Testing

HbbTV Testing

CI+ Testing

Conformance and Pre-Testing DLNA Testing

Dolby and HE-AAC Testing

Custom Test Suites

Automation Test Tools & Harnesses

HbbTV App Validator

Evora DVB Test Suite

Ligada iSuite for DASH Test Suite CI+ Test Tool

Ligada iSuite for HbbTV Test Suite

HEVC for DVB

4 Official Test lab for

Freeview Aust + NZ DTT

Supplier of TNT 2.0 Test Suite and DRM services

HbbTV Consultancy services

OIPF Test Suite and Test Harness

HbbTV Services

Custom HbbTV/OIPF Test Suite

Supplied Official Test Harness

Test Suite and flexible Test Automation System

Digital TV Labs & HbbTV

Supply of TDT Hibrida Test Suite and Test Tools

Supply of HbbTV 2.0 and DRM Test Suite

• Early Member of HbbTV Association and part of the Steering Board & Marketing Group



Testing



What test materials and procedures are available to

manufacturers in that country?



Any special tests for that country?



Certification



Is there a logo and associated trade mark licensing scheme?



What benefits does it bring?



Right to use platform logo/brand



Access to marketing campaign and brand awareness



Receiver listed on website



Access to client certificates



How is receiver testing carried out?



Self-certification versus certification



Use of audits

Deploying HbbTV – what is needed?



Define and create the HbbTV applications



Web application server infrastructure



Application Testing



Define and create the platform



IP-video delivery: DASH origin server, Content Delivery Network

(CDN), DRM solution (optional), DVB broadcast signalling



Migration plan (MHP/MHEG/HbbTV co-existence)



Platform Testing



Ensure availability of HbbTV receivers in the market



Platform specific receiver specification

Conformance regime

7

HbbTV Broadcaster Heaven

• HbbTV App works on

multiple Vendor

STB/TV’s

• Single version of the App

is deployed and without

need for updates

• New TV/STB platform

software releases do not

break App

• Testing the App requires

limited effort

• Users have a reliable,

good consumer

experience

• App is released on time

& within budget

8

HbbTV Broadcaster Hell

• HbbTV App works differently on some platforms

• HbbTV App does not work

on some platforms at all

• App need to be deployed as

multiple versions

• New releases of App break

on some platforms

• New TV/STB platform

software updates break the App

• Testing the App is a massive TV platform debug session • Users have an unreliable,

poor consumer experience • App is late to deployment

and over Budget Thailand – NBTC found that ~70% of STBs in the market

don't support HE-AAC properly and the audio goes mute on those services. Spec is basically too vague and no test

regime in place. Recall of boxes had to happen

Freeview Plus : various issues with Broadcaster Applications found - none conformance to

certain TV’s, HbbTV API’s in conformance on client device, Applications written in PC

environment and using extensions outside of HbbTV spec. Re-authoring of Apps was

9 Pay-TV Operator _OperatorHbbTV Old Model New Model

 Receivers not controlled by operator/content owner

 Cannot guarantee app operation on every device

 MHEG-5 and MHP were first horizontal interactive deployments

 HbbTV will be first mass-deployment Complex TV standard

 Testing and conformance vital to interoperability

 Differences between TV/STB platforms may mean different app version for each device or at best conditional code branches

• Some markets use

optional HbbTV behaviour. • Extra tests can be required

to get full coverage

• These tests shorten the roll-out as app-authors, receiver manufacturers, etc have something to work to.

• Testing platform with applications ok

• But leaves unknown areas

untested

• New applications can fail and large support costs for broadcaster, platform

manufacturer and app developer

• Proven solution:

• Use Official HbbTV Test Suite • + Digital TV Labs Test

Harness

Why Test and Certify?

HbbTV Platform

API 1 API 3 API 4

HbbTV Test Suite

HbbTV

App 1 HbbTV App 2

API 5 API 6 API 7 API Untested

Mandatory Optional

Extra Test

Suite



But - real apps may do things not (yet) covered by a Test Suite!



Manufacturers will Test against applications in recorded streams and also

those live on air

 Testing catch-up services can be problematic if they are geo-blocked

 Manufactures can only test against applications that are available (e.g. HbbTV

Live Apps)



The HbbTV Test Material Approval Procedure (TMAP) includes

some suggestions:



Interoperability Workshops

 Digital TV Labs have tools & services:

 Validator

 Syntax Check

 HbbTV Device Zoo

 Live Apps Hosting

 Testing by CE manufacturer

 PC HbbTV Simulation

 Selenium on Firefox

 HbbTV Selenium API Automation

 Device app automation testing

HbbTV Official Testing

12 Combined Effect of v1.3

Manufacturer

In-House Test Registered Test

Centre

Purchase Official Test Suite

Purchase Test Harness (optional)

Lab Testing

Test Report Test Report

HbbTV Logo TML Audit Programme Market Complaints Monitoring System

Conclusions



Interoperability is challenging: testing much more important than in a

simple digital broadcast world



Testing with just applications is important, but not sufficient

Implementations won’t be inter-operable without a test suite



Variety of different approaches to test and certification being

adopted in the market



The HbbTV Association’s official test suite is a great resource and is

available today



Extended coverage for operator specific tests still needed



Testing and certification helps migration to HbbTV and adoption of

better services, and helps avoid confusing consumers

THANK YOU!

www.digitaltv-labs.com

[email protected]

http://twitter.com/DigitalTVLabs

Any Questions?