Web-based Mobile Applications

(1)

Web-based Mobile Applications

Mobile Communication and Mobile Computing

Prof. Dr. Alexander Schill

http://www.rn.inf.tu-dresden.de

(2)

Table of Contents

§

Motivation and Challenges

§

Main Principles

• Responsive Web design • Mobile First

• Design patterns

§

Further Features in HTML5

§

Server-side adaptation

(3)

Technologies for Web Applications

§

HTML5 and CSS 3

• advanced interface styling

• plugin-less multimedia and computing capabilities • layout transformation for multiple screen sizes

(4)

Challenges of Mobile Web Design

§

Dependency to network connection

§

Heterogeneous device support (desktop, tablets, smartphones)

• Slower network and CPUs

• Screen size can no longer be fixed • Native look and feel

• Touch/gesture input

• Access to native functions (e.g. location)

Ø

Flexible adaptation of information/media towards different

(5)

Mechanisms of Content Adaptation

§

adapt interface presentation (layout)

• use different layouts depending on device attributes • show/hide content and interface elements

(e.g. navigation)

§

provide optional and alternative content

• images in multiple resolutions

• audio/video in different formats • replace media with placeholders

§

compress files

(6)

Responsive Web Design

§

Main adaptation problem: interface scaling over different

device screens

• inofficial default screen width for PCs: 1024 px

• screen width range on mobile devices: 320 - 2048 px • classical approach: different versions of web site/app

(e.g. mobile.example.com)

§

Responsive Web Design

• principle described by Ethan Marcotte • single HTML page markup

(avoid separate mobile version)

• adapts to screen size through CSS • main concepts:

o Fluid grids

o Flexible images

o CSS Media Queries

(7)

Fluid Grids

§

page-layout based on grid system

• maximal layout width as starting point

• layout defined by columns with fixed width / margins

§

translate fixed grid sizes to proportional values

• % or em for block elements, fontsizes

• value for each element proportional to parent (context):

o target ÷ context = result

§

existing fixed/fluid CSS grid frameworks facilitate use

• 960 Grid System (960.gs)

• 1200px Grid System (1200px.com) • Blueprint (blueprintcss.org)

(8)

Fluid Grids

§

page-layout based on grid system

• maximal layout width as starting point

• layout defined by columns with fixed width / margins

§

translate fixed grid sizes to proportional values

• % or em (relative font size) for block elements and fontsizes • value for each element proportional to parent (context):

element size / parent size = relative size

960px 69px 900px 12px 566px 331px

Article

566 ÷ 900 = 62.8888889% .article { width: 62.8888889%; float: left; }

Sidebar

.sidebar { width: 36.7777778%; float: right; } 331 ÷ 900 = 36.7777778%

Banner

.banner {width: 93.75%;}900 ÷ 960 = 0.9375 #page {width: 90%;} (90% of browser window -‐> equals to 960px)

(9)

Flexible Images

§

large images don't automatically resize

• break layout if larger than parent element

§

simple solution:

• img {max-width: 100%;}

• force image to match width of its container

• resized proportionally in most browsers (keeps ratio) • also for other elements (video, object, embed)

(10)

Multi-resolution images

§ Problem: images don't scale across current screen resolutions (72 - 320 dpi)

• recent problem due to high res (retina) screen on mobile devices • many mobile devices need higher resolution images than PCs • network bandwidth needs to be considered

§ Solution: <picture> element

• currently developed by W3C, not implemented yet • similar to <video>

• multiple sources bound to media attributes (e.g. min-width)

• src-set attribute contains list of image files bound to device-pixel-ratios

• device-pixel-ratio: ratio between physical pixels and pixels on device, given in 1x, 1.5x, 2x

iPhone: 320 physical px and 320 browser px => 1x

retina iPhone: 640 physical px and 320 browser px => 2x

<source media="(min-‐width: 18em)" srcset="med.jpg 1x, med-‐highres.jpg 2x"> //small layout

<source media="(min-‐width: 45em)" srcset="large.jpg 1x, large-‐highres.jpg 2x"> // larger layout

<img src="small.jpg" alt="Description of image subject."> //fallback for older browsers

(11)

Multi-resolution images

§

<picture> not yet in browsers, but possible through

Javascript

• e.g. picturefill (http://scottjehl.github.com/picturefill/) • emulates <picture> element programmatically

• uses <div> and data-* attributes

(12)

CSS Media Queries

§

Conditionally apply styles based on browser attributes

• multiple CSS styles for the same elements

§

Mainly: style content based on the screen size

• define multiple page layouts with CSS on same HTML markup • define breakpoints for different layouts (e.g. screen width) • show/hide/resize/move elements depending on layout

§

List of example sites: http://mediaqueri.es

(13)

CSS Media Queries

§

limit scope of CSS properties by media type / feature:

• <link rel="stylesheet" media="only screen and (color)“ href="example.css" />

• @media screen and (max-‐width: 600px) { ... }

§

any combination of single media type and chained media features

(and operator)

• @media screen and (min-‐device-‐width: 480px) and (orientation: landscape)

• @media screen and (max-‐width: 1200px) and (min-‐resolution: 260dpi) and

(aspect-‐ratio: 1/1)

§

media types:

§

media features:

• resolution | min-‐resolution | max-‐resolution

• orientation | ...

stylesheet loading

(14)

Mobile Device Layout Patterns

§ Luke Wroblewski has compiled list of popular responsive layouts:

Mostly Fluid Column Drop Layout Shifter

Tiny Tweaks

(15)

Example - Mostly Fluid

(16)

Example - Mostly Fluid

§ Mobile first - base CSS (320px): § 2nd media query (768px):

§ 1st media query (480px): § 3rd media query (1000px+):

body {

margin: 10px 0; }

header, nav, article, footer { width: 100%; } nav ul li { width: 100%; } @media (min-‐width: 480px) { nav ul li { float: left; width: 25%; /* 4 Items */ }

article div.list div { float: left; width: 50%; } } @media (min-‐width: 768px) { body { margin: 20px 0; } header { float: left; } nav { float: right; width: auto; } } @media (min-‐width: 1000px) { article { float: right; width: 75%; } aside { float: left; width: 25%; } }

(17)

Mobile First

§

mobile web is the future market

• "Mobile phones overtook PCs as the most common Web access

devices worldwide” - Gartner

• optimize content for mobile also benefits desktop

o Focus on most important content, tune content for limited resources

• Mobile First approach promoted by Luke Wroblewski

§

Principle: start with mobile first

• small screen

o condensed navigation / content (reduce to most important parts) o responsive layout

• slow network

o reduce requests and file sizes

o image sprites, single CSS/JS files, minfied o ApplicationCache, localStorage

o CSS 3 / <canvas> / SVG instead of raster images

(18)

Mobile First - Innovation

§

touch input

• touch target sizes:

o recommended: 9mm/34px

o minimum: 7mm/26px

• visual size is 60-100% of target size • use multi-touch input:

• interface paradigms: drag to reveal, drag to refresh, ...

§

device features

• location

o adapt content to user location

• compass

o augmented reality based on user's location and direction

• orientation (portrait, landscape)

o optimize layout (navigation, content) to device orientation

• camera

(19)

Further Mobile Enablers in HTML5

§

based on XHTML 1.1

§

authored by W3C and WHATWG

§

new elements and attributes

§

new technologies, modularized under sections:

• promoted under HTML5, but most are separate standards

§

more capabilities for web applications

§

reduce necessity of proprietary plugins

Semantics Multimedia

Offline & Storage 3D, Graphics & Effects

Device Access Performance & Integration

(20)

Semantics

§

semantic elements:

• <article> • <footer> • <header> • <nav> • <section>

§

add structure to HMTL markup

§

microdata:

• itemscope, itemprop attributes

§

add extra semantic information to

HTML markup

§

using existing vocabularies

• schema.org

• Rich Snippets

(21)

Multimedia

§

<video> and <audio> tag:

§

play video / audio files without plugin

• control via browser agent or Javascript

• multiple file sources possible (codec alternatives) • attributes: autoplay, controls, muted, ...

• varying codec support across browsers • video: theora / h.264 / webM

• audio: ogg / wav / mp3 / webM / aac

<video id="myVideo"> //alternative sources

<source src="http://media.w3.org/2010/05/sintel/trailer.mp4" type='video/mp4; codecs="avc1, mp4a"'> <source src="http://media.w3.org/2010/05/sintel/trailer.ogv" type='video/ogg; codecs="theora, vorbis"'> Your user agent does not support the HTML5 Video element. //fallback text

</video>

<button type="button" onclick="vid_play_pause()">Play/Pause</button> //Javascript video API <script>

function vid_play_pause() {

var myVideo = document.getElementById("myVideo"); if (myVideo.paused) {myVideo.play();}

(22)

Device Access

§

access device sensors and data sources:

• Location, camera, microphone, screen orientation, motion / acceleration, contacts, calendar

§

still early development

• currently only few APIs:

o Javascript Geolocation API

o deviceorientation, devicemotion DOM events

§

mobile frameworks (e.g. Phonegap) also offer similar device

access

§

Example: Geolocation API

• provides highlevel interface to location information through Javascript

• adds navigator.geolocation element to DOM • User can allow/disallow location access!

(23)

// test if Geolocation is supported

if (typeof navigator.geolocation === 'undefined'){

alert("Geolocation not supported by your web browser."); } else {geo = navigator.geolocation;}

// get Position

if (geo) {geo.getCurrentPosition(successFunction, errorFunction); }

// read position object and display lat./long.

function successFunction(position) { var lat = position.coords.latitude; var long = position.coords.longitude; var acc = position.coords.accuracy; var speed = position.coords.speed;

var heading = position.coords.heading; alert('Your latitude is :'+lat+' and

longongitude is '+long); }

// display error message

function errorFunction(position) { alert('Error!');

Device Access

§ supported in most current browsers

(24)

Connectivity

§

WebSockets:

• bidirectional communication channel over TCP • WebSocket API + WebSocket protocol

• optimized, standardized version of Comet

• long-lived, open connection allows server push instead of polling • protocol:

o IETF RFC 6455

o open WebSocket through HTTP upgrade request

• API (Javascript):

o create new WebSocket object: new WebSocket(url, protocols) o send(data), close() methods

(25)

Performance & Integration

§

Web Workers:

• client background Javascript tasks

• long-running, multi-threaded, computationally expensive

• independent from user-interface tasks

• no access to DOM, uses message-passing

§

two types:

• dedicated: linked to creator script

• shared: accessible by all scripts from same origin/domain

§

XMLHTTPRequest 2:

• additional functionality

• tracking data transmission progress through progress event • binary data uploads

(26)

Offline & Storage

§

ApplicationCache:

• control caching of user agent • instruct with cache manifest file

• list files in CACHE, NETWORK (online only) and FALLBACK

sections

• Javascript API to force cache updates

§

Web Storage:

• store named key/value pairs locally • Javascript API

• value can be any data type supported by Javascript • size limitation per origin/domain (5-10MB)

§

two storage types:

• localStorage: no expiration date for stored data

(27)

Offline & Storage

§

IndexedDB:

• noSQL transactional local storage

• persist (complex) Javascript objects as key/value pair • indexing of value properties

• no hard size limit (user permission needed for 50MB+)

§

asynchronous Javascript API:

• transaction requests, retrieve data via callbacks • synchronous API for Web Workers planned

§

File API:

• access local files in Javascript

• user has to select files through <input> element or drag and drop

• read single File, directory (FileList) or binary data (Blob) • uses FileReader interface

(28)

CSS 3.0

§

more advanced interface styling:

• rounded borders

• shadows

• color alpha channel (RGBA) • multiple backgrounds

§

transitions and animations:

• timed transition between two CSS styles

• animations: complex transitions over multiple styles

§

web fonts:

• embed type font with @fontface

• Embedded Open Type (.eot), TrueType (.ttf), OpenType (.otf)

§

reduce necessity for

plugins

(Flash, ...) and

images

(e.g. for

(29)

Mobile Web Application Development

§

Responsive CSS Frameworks:

• Bootstrap (twitter.github.com/bootstrap) • jQuery UI (jqueryui.com) • Foundation (foundation.zurb.com) • many more...

§

Mobile Web Application Frameworks:

• Sencha Touch (www.sencha.com/products/touch) • jQuery Mobile (jquerymobile.com)

• M-Project (the-m-project.org) • LungoJS (www.lungojs.com)

• Joshfire (framework.joshfire.com)

§

Testing / Debugging

(30)

Server-side adaptation

§

Opera Mini

• requested web page rendered on proxy server

• rendered page compacted to binary Opera Binary Markup Language (OBML) and sent to client

• reduces download sizes by up to 90% • limitations:

o only onLoad-events (server-side Javascript) are fully supported o on client, only user-input events (onclick, onchange, ...) supported o limited Ajax support -> page only changes if user clicks something

Opera Mini (Java ME) Opera Mini Server Web Server1 Web Server2 Web Servern ... HTML OBML 1 2 3 4 5 6

(31)

Server-side adaptation

§

Amazon Silk

• browser on Kindle® devices

• browser subsystems (fetching, Javascript, rendering) also implemented in AWS (Amazon Web Services) cloud (EC2, Elastic Compute Cloud 2)

• dynamically split which subsystems executed in EC2 and on device:

• cloud advantages:

o computationally expensive tasks (rendering, scripts) done in cloud o faster fetch of referenced files (CSS, Javascript, images, ...)

o predictive caching, through all Silk users o optimized image compression

(32)

Cross-Platform Web-based mobile Apps

§

Apache Cordova (formerly PhoneGap)

• Cross-platform tool to create mobile apps based on HTML, CSS and Javascript

• Supported platforms: iOS, Android, Windows Phone, BlackBerry • Approach:

o web content wrapped in PhoneGap App

o Native code to create native browser UI element (UIWebView (iOS) or WebView (Android))

o and present locally stored web content

Ø Runnable as App

(33)

§ Web-technologies to create UI + logic

§ Cross-platform PhoneGap Javascript API to access device specific features Ø Wrapped to native code § Often used in combination

with Mobile Web Application Frameworks like jQuery

Mobile or Sencha Touch § Native UI elements not

supported

Phone Gap Approach

HTML, CSS, JavaScript

iOS PhoneGap Bib (JavaScript) iOS PhoneGap Bib

(Nativ)

PhoneGap Cross-Platform-API for JavaScript

(Nativer Wrapper Code) iOS-Plattform-API Android PhoneGap Bib (JavaScript) Android PhoneGap Bib (Nativ) (Nativer Wrapper Code) Android-Plattform- API

App project for iOS

App projekt for Android

(34)

PhoneGap Development

§

Start: Cross-platform web project

§

Compilation with platform-specific IDEs

Ø

Web service „PhoneGap build“ for creating Apps

without native IDEs -> build.phonegap.com

Xcode -‐ iOS Eclipse IDE -‐Android App content HTML, CSS, Javascript Co rd ov a.js ... co rd ov a.js An dro id co rd ov a.js iO S ... Android App iOS App ...

(35)

PhoneGap API

§ Cross-platform API provides common interface to access device specific features

(36)

Summary

§

Adaptive Web applications for mobile devices

§

Major principles:

• Responsive Web design

Ø Scalable layout and images

Ø Alternative layout and content based on Media Queries

• Mobile Device Layout Patterns

Ø Adapt layout for different devices/view sizes

• Mobile First

Ø optimize content for mobile -> also benefits desktop

§

HTML5 provides further enablers for mobile

• Device access, CSS3, Multimedia, Offline and Storage,… • Extended browser capabilities

§

Content adaptation in the infrastructure

Ø Reduce data volume for transfer

§

Cross-Platform Web-based mobile Apps

(37)

Some further readings

§ HTML5 Technologies Specifications: • HTML5: http://dev.w3.org/html5/spec/single-page.html • CSS: http://www.w3.org/Style/CSS/Overview.en.html • Microdata: http://www.w3.org/TR/html5/microdata.html • SVG 2: https://svgwg.org/svg2-draft/ • WebGL: https://www.khronos.org/registry/webgl/specs/1.0/ • Device Access: http://www.w3.org/2009/dap/

• WebSockets: http://www.w3.org/TR/websockets/ • Web Workers: http://www.w3.org/TR/workers/

• XMLHttpRequest 2: http://www.w3.org/TR/XMLHttpRequest/ • ApplicationCache: http://www.w3.org/TR/offline-webapps/ • Web Storage: http://www.w3.org/TR/webstorage/

• Indexed Database: http://www.w3.org/TR/IndexedDB/ • File API: http://www.w3.org/TR/FileAPI/

§ Crowd-Docs of Web Standards: www1.webplatform.org

§ Ethan Marcotte: Responsive Web Design, A Book Apart, 2011

§ Luke Wroblewski: Mobile First, A Book Apart, 2011