• No results found

Testing & Assuring Mobile End User Experience Before Production. Neotys

N/A
N/A
Protected

Academic year: 2021

Share "Testing & Assuring Mobile End User Experience Before Production. Neotys"

Copied!
31
0
0

Loading.... (view fulltext now)

Full text

(1)

Testing & Assuring Mobile End User Experience Before Production

Neotys

(2)

Agenda

Introduction

The challenges

Best practices

NeoLoad mobile capabilities

(3)

Mobile devices are used more and more…

At Home

In 2014, more people will connect to the Internet from mobile devices than from PCs

In 2015, 98 billion mobile applications will be downloaded

1.2 billion new smartphones enter the market in the next 5 years

In the Office

30% of companies have or develop applications for their

employees. 88% of employees use their

personal devices in their work.

(4)

Different type of mobile testing

Functional

• Is this application does what it is supposed to do?

Equipment

• How is the

application works on the device?

• Rendering

• CPU, Battery

Performance

• Is 1000 users can use this application simultaneously?

• What is the user experience when the server is loaded?

• What is the behavior of the application with different network conditions?

Functional testing Device testing Performance Testing

(5)

Agenda

Introduction

The challenges

Best practices

NeoLoad mobile capabilities

(6)

Mobile Performance Testing is Challenging for Enterprise Organizations

6

For 64%

Mobile Performance Testing is #1 Testing Priority

http://www.capgemini.com/thought-leadership/world-quality-report-2012-13

Don't have the right tools

65%

Don't have the devices they need

readily available

52%

Don't have the right processes or

methods

34%

But

(7)

Mobile Project are shorter with less budget

Back End services

• Common services for all devices

• Don’t need the mobile application to start testing

• Testing driven development

Mobile application

• Use the service calls to retrieve the data

• Last mile of the development life cycle

(8)

Validate the Performance of a Mobile Application is complex!

Back-End

EXECUTION PLATFORM

RIA/FRAMEWORKS

Database

Network

3G, 4G / LTE, Wifi

Video

HTTP Adaptive Streaming

Throughput Latency

Packet loss Server

Infrastructure

OS Client APP

Device

App

iPhone 4, 5, 6 Samsung Blackerry

(7,000 different device types are used to access Facebook)

Screen size CPU Memory

Native Mobile Mobile Browser Hybrid

(9)

Impact of the Network

(10)

Latency and bandwidth

 The Most important network constraints are :

• Latency

• Bandwidth

 From a certain level of bandwidth, increasing the

bandwidth will have a minor effect.

(11)

Packet size on a 3G Network

 What is the average packet size on mobile networks ?

Traffic recorded on a 3G Network by an European Operator during 1 week in October 2010

0 50000 100000 150000 200000 250000 300000 350000

100b 1kb 10 kb 100 kb 1 Mb 10 Mb More

Number of packet

(12)

Packet LosT influEnce

Packet loss consequence

• Packet Retransmission

• End user experience degraded because Packet loss will increase jitter

• Errors could occur if Packet loss are in pure data transmission

Packet loss is inevitable in Mobile Network

• Mobility/Fast switch to next cell/temporary loose the signal for congestion of the current cell

Mobile protocols try to reduce

the packet loss rate

(13)

Wrap-up…

(14)

Different kind of application

(15)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Mobile Context – Lots of Devices

Device Challenges

• Operating Systems - iOS

- Android - Windows

• Technologies used in the app - HTML 5

- Web Socket

- Streaming Video/audio - Google SPDY

• Screen Sizes

- Different content for different devices

- Design of what content is above/below the fold - When that content gets delivered

(16)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Mobile Context – High, Distributed Load

Lots of users

• Mobile applications often have large user bases

• High growth in popularity can create the need for highly scalable load generation infrastructure

Geographical distributed load

• Users are accessing apps from locations around the globe

• Distance and location can greatly affect performance

Mix of Web & Mobile Users

• Many mobile apps have a corresponding web app with a shared backend

• High load on the web app can affect performance on mobile

(17)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

CDN principles

(18)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

CDN advantages example

(19)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

How to test CDN-based architecture

2 kinds of CDN

• Caching CDN

• Streaming CDN

Tests must be realistic

• Spread the load among different geos and network

• Cache usage

• Streaming emulation

• CDN Backbone are hosted by Cloud providers: must choose appropriate Cloud provider(s)

• DNS resolution is mandatory in order to avoid stress a cdn node

Measure response times from different locations

• Cloud or multiple load generators are needed to stress several node of the CDN

(20)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Agenda

Introduction

The challenges

Best practices

NeoLoad mobile capabilities

(21)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Constructing Scenarios

Creating Real World Conditions

• Application under realistic load

• Real devices for capturing actual

• application performance on the device

• Real mobile CDN paths for testing the whole delivery chain

(22)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Include Network Constraint during your test

Select most represented location and carries

• Measure the latency /packet Loss from the major locations

• Store your measurements into a network Matrix

Involve the Management to get statistics

• Ratio of user connected on Wifi/4G/3G/Edge

• Ratio or number of user per locations

Include the Network constraint in your load policy

• Organize your load policy depending on the network constraints

• Combine the usage of Cloud Based Load generator and Wan emulation

(23)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Take profit of the Cloud Based Load Generator

Test the entire chain

•Firewall

•DNS

•ISP

•Network

Traffic generated from different geos

•Different networks

•Multiple Cloud providers

•Response time comparison

Save Time

•Deployment time is reduced

•Administration time is reduced

•No maintenance

•No Upgrades

Save Cost

•Pay only for what you use

•No machines acquisition

•No Maintenance

(24)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Analytics to Enable Decision-making

Actionable Insight

• Monitoring on all infrastructure layers (App, Web, DB)

• Stats on backend and frontend to pin point root cause of performance problems

Shift Left

• Test integration with CI systems to get feedback to development teams earlier

• Fix problems earlier when they’re less expensive

(25)

Mobile Performance Testing Product

What are you really measuring?

(26)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Real Device

Testing Tool Real Device Users

00:00:00 00:00:30 00:01:00 00:01:30

Load

NeoLoad

Application Server

App Error Rate

Virtual Users Load

Rendering Time

Complete Mobile EUE

(27)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Report proper metrics

Pie I have not yet eaten

Pie I have eaten

(28)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Agenda

Introduction

The challenges

Best practices

NeoLoad mobile capabilities

(29)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

NeoLoad’s Mobile Capabilities

Record from any device

• Browser-based application

• Native application

Simulate any device or browser

• Ensures the right content is delivered for that device or browser

• Simulates parallel connections for each specific device and browser

Emulate realistic network conditions

• Bandwidth

• Latency

• Packet Loss

Generate load from the Cloud

• Geographic realism

• Rapid scalability for large tests

Load Command

H+ 3G 4G WiFi

Monitor APP

Network Emulation

& Device Simulation

Cloud Load Generators On-premise Load Generators

Database Server

App Server Web Server

(30)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

NeoLoad + MobileCloud

Real Device Metrics Under Load

• Automatically trigger MobileCloud test with the start of a NeoLoad test

• See real device rendering times and vitals while the

backend of the application is under load

• Results are

correlated in real time for accurate and fast analysis

Load Command

H+ 3G 4G WiFi

Monitor APP

Cloud Load Generators On-premise Load Generators

Database Server

App Server Web Server

NeoLoad Controller

Real Devices &

Networks MobileCloud

Simulated Devices w/ Network

Emulation

Command Real Device Metrics

4G H+ 3G

(31)

www.neotys.com

© 2014 Neotys. All Rights Reserved.

Next Steps

NeoLoad PoC Assistance: Email sales@neotys.com

Visit Neotys Community at answers.neotys.com

Download NeoLoad Free Edition from www.neotys.com

Evaluation support at www.neotys.com/support/contact.html

Questions?

Visit Neotys booth

References

Related documents

This chapter takes Paige McCullers and Emily Fields, who are characters on the television show Pretty Little Liars, mark the queer identity in society and resist the

In these cases, directors and teachers are able to build partnerships with parents to meet the needs of young children with challenging behavior, with support from the mental

ASERO provides law enforcement and security training seminars to government and private organizations around the world, including: the Canadian Air Transport Security Authority;

>> Created and produced for Mr. Quintin Scally from Dublin. HolidayHomeAds.com is a Search Rentals, Sales, Time-Shares and Exchange Homes,serving people across

Guidelines on the Pharmacological Treatment of Persisting Pain in Children with Medical Illnesses (World Health Organization, 2012).. • Appropriate Route of Administration 

ARCHELON’s Mediterranean Workshop on the Rehabilitation of injured sea turtles provided the perfect opportunity to explore the benefits and the possibilities of creating a Network of

Debido a que la información se extrae de páginas web, y dado que las películas por género, país o década se puede acceder a diferentes páginas para mostrar

To make learning much effective and systematic,teachers should be clear about the long term goals and specific aims of teaching English (James, 2001). The aims of teaching