Video streaming and playback

playback

This document is published by Sony Ericsson Mobile Communications AB, without any

warranty*. Improvements and changes to this text necessitated by typographical errors, inaccuracies of current information or improvements to programs and/or equipment, may be made by Sony Ericsson Mobile Communications AB at any time and without notice. Such changes will, however, be incorporated into new editions of this document. Printed versions are to be regarded as temporary reference copies only.

*All implied warranties, including without limitation the implied warranties of merchantability or fitness

These Developers Guidelines are published by: Sony Ericsson Mobile Communications AB, SE-221 88 Lund, Sweden

Phone: +46 46 19 40 00 Fax: +46 46 19 41 00 www.SonyEricsson.com/

© Sony Ericsson Mobile Communications AB, 2006. All rights reserved. You are hereby granted

Preface

Purpose of this document

This Developers Guideline contains descriptions of the streaming video standards supported in

Sony Ericsson mobile phones. The document also mentions some important issues to take into consider-ation when setting up a streaming media server and preparing media for streaming to a mobile phone.

Readers who will benefit from this document include:

• Business decision-makers

• Corporate buyers

• IT professionals

• Support engineers

3

January 2006

Sony Ericsson Developer World

On www.SonyEricsson.com/developer, developers will find documentation and tools such as phone White Papers, Developers Guidelines for different technologies, SDKs and relevant APIs. The website also contains discussion forums monitored by the Sony Ericsson Developer Support team, an extensive Knowledge Base, Tips & Tricks, example code and news.

Sony Ericsson also offers technical support services to professional developers. For more information about these professional services, visit the Sony Ericsson Developer World website.

Document conventions

Products

Sony Ericsson mobile phones are referred to in this document using generic names as follows:

Generic names

Series

Sony Ericsson mobile phones

Z1010 Z1010

K700 K700i, K700c

S700 S700i, S700c, S710a

K500 K500i, K506c, K508i, K508c, F500i

Z500 Z500a V800 V800, Vodafone 802SE K300 K300i, K300c, K300a J300 J300i, J300c, J300a K750 K750i, K750c, D750i W800 W800i, W800c Z800 Z800i K600 K600i, K608i V600 V600i Z520 Z520i, Z520c, Z520a W600 W600i W550 W550i, W550c

Abbreviations and acronyms

Trademarks and acknowledgements

The Bluetooth word mark and logos are owned by the Bluetooth SIG, Inc. and any use of such marks by Sony Ericsson is under license.

RealAudio and RealVideo are trademarks or registered trademarks of RealNetworks, Inc.

Other product and company names mentioned herein may be the trademarks of their respective owners.

W900 W900i

W810 W810i, W810c, W810a

3GPP 3rd Generation Partnership Project

AAC Advanced Audio Coding

AMR Adaptive Multirate

EDGE Enhanced Data rates for GSM Evolution

GPRS General Packet Radio System

HTTP Hypertext Transfer Protocol

MPEG Motion Picture Expert Group

NAT Network Address Translator

PSS Packet Switched Streaming

QCIF Quarter Common Intermediate Format

RTSP Real Time Streaming Protocol

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

5

January 2006

Document history

Change history

2004-06-30 Version R1A First edition

2004-08-16 Version R1B Minor revision

2004-10-15 Version R2A Second edition. Information about K500, Z500 and V800 series added.

2004-10-26 Version R2B Minor editorial changes

2005-04-28 Version R3A Information about K300, J300, K750, W800, and Z800 series added

2005-07-15 Version R4A Information about K600, V600, Z520, and W600 series added

2005-08-08 Version R4B Information about W550 series added

2005-09-26 Version R4C Real8 streaming support in K600 and V600 series removed.

2005-11-30 Version R5A Information about W900 series added

Contents

Overview ...7

Applications ...8

User scenarios ...8

Standards, architecture and protocol ...9

Streaming support ...9

Supported standards ...10

The MPEG-4 standard ...12

MPEG-4 video compression ...12

Audio compression ...13

File format ...13

Content recommendations ...14

7

January 2006

Overview

The video player in Sony Ericsson mobile phones plays video content that is locally stored or streamed. It can also play audio-only material, encapsulated in a file or stream.

What is streaming?

Streaming is a method for making audio, video and other multimedia available in near real-time, in differ-ent types of networks. Streaming to computers has been used for a number of years. With the introduction of faster mobile networks such as GPRS, EDGE and UMTS, streaming can also be utilized in mobile phones.

The name “streaming” refers to the technique it is based on. Previously it was necessary to download an entire file to a computer hard disk or a mobile phone memory, and then play it. Through streaming, the user can begin to watch or hear the content of a requested media file after only a short delay. The data in the file is split into small packets that are sent in a continuous flow, a stream, to the end-user's computer or mobile phone. The user can begin viewing the contents in the first packets, while the rest of them are still being transferred. The short delay at the start is due to time it takes for the client to buffer a small amount of data. The data buffer makes it possible for the client to playback the video or sound without interruption, even if the rate of received data varies slightly.

Applications

Applications for streaming services can be divided into “on-demand” and “live” information delivery appli-cations. Examples of the first category are music and video, news-demand applications, and on-demand instructional material. Receiving of radio and television broadcasting are examples of the second category.

User scenarios

• Streaming of music on demand

• Streaming of news (video, audio) on demand

• Streaming of movie trailers on demand

• Streaming and download of video on demand

9

January 2006

Standards, architecture and protocol

Sony Ericsson supports the architecture, protocols and codecs of PSS (Packet Switched Streaming) within the 3GPP system, as well as supporting all ongoing standardization activities within 3GPP. Sony Ericsson works constantly to follow standards and to ensure interoperability between business solutions, while at the same time endeavouring to meet additional market requirements within this area. The relevant 3GPP specification is TS 26.234, “Transparent end-to-end packet switch streaming service (PSS)”. The PSS includes media codecs for video, still images, bitmap graphics, text, audio, and speech.

Streaming support

The video player in Sony Ericsson mobile phones supports local playback and streaming of 3GPP media, mainly for Video On Demand (VOD) and live video applications.

Supported standards

Streaming and playback specifications for Sony Ericsson mobile phones are based on the 3GPP PSS standards.

11

January 2006

The 3GPP PSS standard

The video player application supports multimedia streaming as specified in the 3GPP Packet-Switched Streaming Services specification (TS 26.234). The following codecs are supported for streaming:

• H.263 Baseline, profile 3, level 10

• MPEG-4 Simple Visual Profile, level 0 (Not supported in K300 and J300 series)

• RealVideo® (ver. 8) (W900 series only)

• AMR narrowband

• AMR wideband (V800, Z800, K600, V600, and W900 series only)

• AAC-LC

Note: For the Z1010, V800, Z800, K750, W800, Z520, K600, V600, W550, W600, W900, and W810 video player some parameters of the specifications above can be exceeded. However, Sony Ericsson does not recommend higher video bit rates than 128 Kbps.

Note: AAC stereo is not supported in the K300 and J300 series

The following streaming session initiation options are supported in WAP applications:

• Plain rtsp:// URL entered in the Bookmarks or Enter address dialogs

• http:// URL pointing to a .SDP file

In addition, streaming sessions may be started from outside the video player via file association. For example, the user may click on a link to an SDP file from inside the Web browser, or on a local .sdp file from the File Manager. In either case, the video player application will be invoked to start the streaming session.

MIME and file type support

Note: Sony Ericsson mobile phones support version 4.2 of the SDP format. When SDP files are sent to the mobile phone via Bluetooth™ wireless technology or the infrared port, the files are saved in the Other folder, in the mobile phone’s File Manager.

The MPEG-4 standard

MPEG-4 was developed in 1998 by the Motion Pictures Expert Group, and has been incorporated into the 3GPP specifications for mobile multimedia. The earlier standards, MPEG-1 and MPEG-2, are widely used for multimedia CD-ROMS and digital broadcast television. MPEG-4 has functionality for support of both small mobile terminals and larger stationary devices within the standard.

MPEG-4 has the following advantages:

• Flexible range of bit rates supported, from 9.6 Kbps to 6Mbps (compared to 1.5 to 12 Mbps for MPEG-2)

• High error resiliency

• Variable frame rate, enabling optimization based on the transmission path and the overall load on the server.

MPEG-4 video compression

MIME type Extension Notes

video/mp4 .MP4 Not supported in K300 and J300 series

audio/mp4 .MP4 MPEG 4 Audio only, AAC or AMR

audio/amr-wb .3GP AMR wideband, V800, Z800, K600 V600, and W900

series only

video/3gpp .3GP

audio/3gpp .3GP All 3GPP treated as video

application/sdp .SDP

audio/x-pn-realaudio .ra RealAudio® ver. 8, K600, V600, and W900 series only

13

January 2006 rates and functionalities. Profiles are used to describe functionality packages. The Simple Visual profile provides efficient and error-resilient coding of video content, and is supported by the mobile phone’s video player.

MPEG-4 Visual is also broken down into levels, describing such things as frame size, bit rate and buffer capacity.

Level 3 is targeted at mobile devices and provides for frame sizes up to QCIF (176 x 144 pixels) at a max-imum bit rate of 64 Kbps at 15 frames per second.

Note: The K750, W800, Z520, K600, V600, W550, W600, W900, and W810 series support a maximum bit rate of 128 Kbps.

Audio compression

AAC (Advanced Audio Coding) is one of the audio compression formats defined within the MPEG-4 stand-ard. Compared to MP3 it has more advanced features and is more efficient.

The 3GPP standard uses AMR for audio coding, though this is actually outside the MPEG-4 standard. This is because AMR is highly optimized for the mobile environment, requiring as little as 4.75 Kbps bandwidth.

File format

The file format defined by MPEG-4 has extension MP4 or 3GP. It is applicable for both streaming and local storage/playback. MP4 uses a structured yet flexible method to describe and encapsulate multimedia material.

Content recommendations

The following content parameters are recommended as a baseline for testing. They should provide excel-lent quality for the screen size and processing power of the Sony Ericsson mobile phones.

Supported frame sizes are QCIF (176x144 pixels) and SQCIF (128x96 pixels).

Note:MPEG-4 video decoding is not supported in the K300 and J300 series.

Streaming:

• 32 Kbps, Video MPEG-4 or H.263, QCIF or SQCIF, 15 fps, Audio AMR 12.2 Kbps (for the V800, Z800, K600, V600, and W900 series also Audio AMR-WB over 12.2 Kbps)

• Audio only: AAC, 42 Kbps

(AAC stereo is not supported in K300 and J300 series)

Download / local playback:

• 64 Kbps, Video MPEG-4 or H.263, QCIF or SQCIF, 15 fps, Audio AAC 64 Kbps mono

15

January 2006

Network limitations

When streaming content is sent over a wireless network, the network’s capabilities might affect the practi-cal parameter limits of the content.

The K700, S700, K500, Z500, K300, J300, K750, W800, Z520, W550, W600, and W810 series are designed for GPRS class 8, 4:1 (4 download and 1 upload time-slots).

Normally the Channel Coding Scheme CS-2 is used for the GPRS network. This scheme specifies a data rate of 13.4 Kbps, which with 4 time-slots, gives the maximum data speed 53.6 Kbps.

If the CS-1 coding scheme is used, the maximum data speed with 4 time-slots is 4 x 9.05 = 36.2 Kbps.

In addition to GPRS, the Z500a, W600i, S710a phone models, and the W810 series support EDGE 4+2, Multislot Class 10. Modulation Coding Schemes MCS1- MCS9 are supported, with typical throughput rate of 50 – 120 Kbps

Server considerations

The server for streaming should be compliant with 3GPP Packet-Switched Streaming protocols. The con-tent should consist of 3GPP-compliant .3GP files generated by some compliant authoring tool.

Issues to take into consideration when setting up and testing a server for streaming media:

• Streaming standards supported by the mobile phone clients, see above

• Content recommendations, see above

• Proper configuration of Firewall/NAT

To minimize interoperability problems, it is strongly suggested to select a server that has been through PSS-AG testing.

17

January 2006

Configuration for firewall/NAT

Firewalls on both the client and server sides introduce complexities into end-to-end video streaming sys-tem deployments that are not present in a direct client-server link environment. Firewall components have evolved from simple packet filters to complex stateful inspection firewalls, and will continue to evolve in the future. It is the carrier's responsibility that any firewall/NAT present in the network be properly config-ured to allow UDP media streams to pass correctly between the server and the client.

The first required action is to be sure that any Firewall has opened the ports needed for communication between the mobile phone’s video player application and the server. On the player side, the following port range must be opened in the firewall:

• Mobile phone video player application: UDP ports 3456 to 3459

In most cases, the video player application will use one port-pair per media stream. So for a simple audio/ video presentation, it would use all four ports; 3456-3459, assuming these are not in use by another appli-cation.

On the server side, the port range to open on the firewall will depend on the particular server used and how it is configured.

Port 554 is used for RTSP by default.

A second required action is to be sure that any firewall or NAT is RTSP-aware and thus is transparent to the streaming session. A typical problem seen in carrier networks is that the introduction of non-RTSP-aware port remapping prevents the UDP packets from reaching the client. The problem is described in detail in the following IETF Internet draft:

http://www.ietf.org/internet-drafts/draft-ietf-mmusic-rtsp-nat-03.txt.

If the firewall or NAT is RTSP-aware, then it understands that the port information is sent in RTSP and may thus adjust the port remapping so that it is transparent to both the client and the server.