IERG 4080 Building Scalable Internet-based Services

(1)

IERG 4080

Building Scalable Internet-based Services

Department of Information Engineering, CUHK Term 1, 2015/16

Lecture 10

Load Testing

(2)

2

Software Performance Testing

(3)

How do you know that your system can perform well enough?

• Can it serve the users quickly?

• Can it serve the anticipated number of users?

• Can it cope with the anticipated amount of workload?

Performance Testing

Application Server Application

Server Database Server Load

Balancer Clients

(4)

Questions you should ask even before your application goes online • How many users will the application need to serve

at release, after 3 months, 6 months and 1 year?

• Where will the users come from (geographically, device used, etc.)? • How many users will use the application simultaneously?

• What are typical scenarios of using the application?

• …

4

Performance Testing

(5)

What is it?

• To test a system in order to understand its responsiveness, throughput, reliability and scalability under certain conditions

Why?

• Assess how ready is the system for production • Evaluate the system against some criteria

• Compare different systems

• Locate problems and identify components to be

(6)

Different categories (or types) of performance testing • Load test • Stress test • Capacity test • Regression test • Volume test • … 6

Performance Testing

(7)

• Subject the system to extremely high work load and see what problems will happen under those conditions

• Identify problems that might occur (e.g. synchronization issues, race conditions, memory leaks, …)

• Spike test is a subset – study the system’s behaviour and problems under sharp increase in work load in a short period of time

• Questions to answer:

• What is the max load a system can support before it breaks down? • How does the system break down?

(8)

8

Baselines and Benchmarking

IERG 4080 – 2015/16 Term 1 – C. M. Au Yeung Baseline

• A reference point against which any performance-improving changes to the application can be evaluated

• A baseline can be created for different layers in the application (e.g. load balancer, application server, database server, etc.)

Benchmarking

• The process of comparing your application’s performance against a baseline, or against a standard widely accepted in the community

(9)

(10)

• The process of putting

demand

on a system and measure its

response

• Understand the behaviour of the system under:

1. Normal load conditions

2. Anticipated peak load conditions

• Identify

bottlenecks

• In Internet-based services, the load is defined in terms of

the

number of concurrent users

or

HTTP connections

10

Load Testing

(11)

SLA – Service Level Agreement

• An agreement between your application and the users on the quality of the service provided

• Common metrics

› Uptime (e.g. 99.99%)

› ASA (Average speed to answer)

› TSF (Time service factor, e.g. 95% of requests within 1s) › TAT (Turn-around time)

(12)

12 https://slack.zendesk.com/hc/en-us/articles/204113126-Plus-Plan-Service-Level-Agreement-SLA-

(13)

Some

examples

of load testing:

• Editing a very large document in a word processor • Sending a huge file to a printer for printing

• Creating many concurrent users to browse a e-commerce Website • Creating many concurrent users to view and edit a public wiki

• …

(14)

What should we study in a test?

• The time required for a request to be processed

(i.e. response times)

• Errors

• Are requests finished?

14

Load Testing

(15)

Things to consider in a load test

• What are the common scenarios when users are using your

application

• Read-to-write ratio

• What will be the habits of the users

• Expected load in different situations (e.g. weekdays vs. weekends)

Load Testing

(16)

What should you

measure

in a load test?

• Response times (how long does the user have to wait?) • Error rates (how many requests are dropped?)

• How many requests can be answered in 1s?

• How many database queries can be performed in 1s? • How many files can be written/read in 1s?

16

Load Testing

(17)

Load Profiles

• The variation of work load across time and situations (ref. load profile in electrical engineering)

• Factors to be considered

› The possible activities of the users

› Common behaviours of the users (and their ratios)

› Time taken for a user to respond to changes in the application (think time) › Abandonment rate (how often does a user chooses to leave?)

(18)

Consider a e-commerce Website

What are the possible actions?

• Browsing • Searching

• Adding items to shopping cart

• Checking out and settling payments • Changing their profiles

• Writing comments on items

• …

18

Load Profile

IERG 4080 – 2015/16 Term 1 – C. M. Au Yeung

(19)

• In order to really understand whether your system can serve

your users efficiently and effectively, your tests should be

realistic

• In order words, you need to have a

good model of the

application’s usage

• Identifying application usage profiles is known as

workload

modelling

(20)

What are the methods for coming up with a good model of workload? • What are the objectives?

• What are the key scenarios?

• What are the common user behaviours?

• What are the navigation paths in key scenarios? • How much data would each user generate? • Which scenarios are more likely?

• What are your target load levels?

20

Modelling Application Usage

(21)

(22)

22

Tools for Load Testing

(23)

Available Tools

• Ab (Apache HTTP server benchmarking tool)

(https://httpd.apache.org/docs/2.2/programs/ab.html) • Apache JMeter (http://jmeter.apache.org/)

• httperf (https://github.com/httperf/httperf) • Tsung (http://tsung.erlang-projects.org/) • Funkload (http://funkload.nuxeo.org/) • Locust (https://github.com/locustio/locust) • Loads (http://loads.readthedocs.org/)

Load Testing

(24)

• https://httpd.apache.org/docs/2.2/programs/ab.html

• A command line tool for benchmarking HTTP servers

• Can be used to easily study how many requests per second your

Web server can handle

• Install by

24

Apache Bench (ab)

(25)

Examples

• Send 200 requests with maximum 50 concurrent users to google.com

• Send 500 requests with maximum 10 concurrent users to yahoo.com, output data into a .csv file

Apache Bench (ab)

$ ab -n 200 -c 50 http://www.google.com/

(26)

26

Apache Bench (ab)

Benchmarking www.yahoo.com (be patient)...done

Server Software: ATS

Server Hostname: www.yahoo.com Server Port: 80

Document Path: /

Document Length: 374 bytes Concurrency Level: 10

Time taken for tests: 0.681 seconds Complete requests: 100

Failed requests: 0 Non-2xx responses: 100

Total transferred: 76700 bytes HTML transferred: 37400 bytes

Requests per second: 146.91 [#/sec] (mean) Time per request: 68.069 [ms] (mean)

Time per request: 6.807 [ms] (mean, across all concurrent requests)

Transfer rate: 110.04 [Kbytes/sec] received

Connection Times (ms)

min mean[+/-sd] median max Connect: 26 32 3.2 32 38 Processing: 28 33 3.2 34 42 Waiting: 28 33 3.2 34 42 Total: 54 65 6.4 66 80

Percentage of the requests served within a certain time (ms) 50% 66 66% 68 75% 69 80% 70 90% 73 95% 76 98% 79 99% 80 100% 80 (longest request)

(27)

Apache Bench (ab)

(28)

• Main Site: http://locust.io/; Doc: http://docs.locust.io/en/latest/ • Open source: https://github.com/locustio/locust

• A user load testing tool written in Python

• You can define the user behaviour in Python code and load test your application

• A Web UI for monitoring the process in real-time

• Distributed – allow running load tests over multiple machines

28

Locust.io

(29)

Locust.io

Defining user behaviour

from locust import HttpLocust, TaskSet, task

class UserBehavior(TaskSet): def on_start(self):

self.login() def login(self):

self.client.post("/login", {"username":“user", "password":“1234"}) @task(2)

def index(self):

(30)

30

Locust.io

IERG 4080 – 2015/16 Term 1 – C. M. Au Yeung Defining user behaviour

from locust import HttpLocust, TaskSet, task

class UserBehavior(TaskSet): def on_start(self):

self.login() def login(self):

self.client.post("/login", {"username":“user", "password":“1234"}) @task(2) def index(self): self.client.get("/") @task(1) def profile(self): self.client.get("/profile")

Define one behaviour of a user (the actions that the user will take when using your application)

on_start will be invoked whenever a new user is created (in this case the user signs into the service)

(31)

Locust.io

Defining user behaviour

from locust import HttpLocust, TaskSet, task class UserBehavior(TaskSet):

def on_start(self): self.login()

def login(self):

self.client.post("/login", {"username":“user", "password":“1234"}) @task(2)

def index(self):

self.client.get("/")

Define a function in which the user will carry out one action

self.client is an HTTP client by which you can make requests to your application, it supports GET,

(32)

32

Locust.io

IERG 4080 – 2015/16 Term 1 – C. M. Au Yeung Defining user behaviour

from locust import HttpLocust, TaskSet, task class UserBehavior(TaskSet):

def on_start(self): self.login()

def login(self):

self.client.post("/login", {"username":“user", "password":“1234"})

@task(2) def index(self): self.client.get("/") @task(1) def profile(self): self.client.get("/profile")

@task(x) defines an action that the user will carry out while he is

online

x represents the ratio of how often the user will perform this action.

(33)

Locust.io

Define a user Run class WebsiteUser(HttpLocust): task_set = UserBehavior min_wait = 5000 max_wait = 9000

$ locust -f locust_file.py --host=http://www.yoursite.com

Defines a user

• task_set: the set of tasks the user will perform

• min_wait: minimum time in ms that the user will wait between tasks • max_wait: maximum time in ms that

(34)

34

Locust.io

• If you want to simulate a lot of work load, you will need to use the distributed mode (using multiple machines)

• First, put the test script in both the master and the slave(s) machine. • Then, start Locust in the master by

• Start Locust in the slave(s) by

$ locust -f locust_file.py --host=http://www.yoursite.com --master

$ locust -f locust_file.py --host=http://www.yoursite.com --slave --master-host=192.168.0.1

(35)

Locust.io

(36)

36

Locust.io

IERG 4080 – 2015/16 Term 1 – C. M. Au Yeung Running load test

Statistics for

response time Size of the content returned Number of requests per second Number of failed requests Total number of requests sent

(37)

• http://funkload.nuxeo.org/

• A functional and load testing module written in Python • Some features include:

› Can be used to emulate a Web browser

› Support cookies, https, HTTP authentication, file upload, etc. › Can be used for functional testing, load testing or stress testing › Generate reports in HTML format for review

• Installation:

(38)

Creating a simple test

38

Funkload

import unittest

from random import random

from funkload.FunkLoadTestCase import FunkLoadTestCase

class Simple(FunkLoadTestCase): def setUp(self):

self.server_url = self.conf_get('main', 'url') def test_simple(self):

server_url = self.server_url

res = self.get(server_url, description='Get url') self.assertEqual(res.code, 200)

self.assertEqual(res.body, "Hello World")

if __name__ in ('main', '__main__'):

(39)

Configuration file (a .conf file with the same name as the class in your test script)

Funkload

[main]

title = Demo

description = Simple demo url = http://localhost:5000

[test_simple]

description = Access our Demo app

[ftest]

log_to = console file log_path = simple-test.log result_path = simple-test.xml sleep_time_min = 0 [bench] cycles = 5:10:20 duration = 10 startup_delay = 0.01 sleep_time = 0.01 cycle_time = 1 log_to =

(40)

Running a benchmark

Data will be stored in an XML file. HTML-formatted report can be generated by

40

Funkload

$ fl-run-bench -c 1:10:20 test_Simple.py Simple.test_simple

$ fl-build-report --html simple-bench.xml

(41)

(42)

You have finished writing your code. How do you know that it is

efficient

? Is your implementation good enough? Can the application

perform 20%

faster

or use 20%

less memory

?

And how can you understand that if you want to?



Profilers

42

Profiling

(43)

A profile is ‘a set of statistics that describes how often and for how long various parts of the program executed’.

Profiling helps us to understand:

• How fast is the program running? • What are the bottlenecks?

• How much memory is the program using?

(44)

44

Timing Program Execution

$ time python test_profiling.py

real 0m2.458s user 0m2.108s sys 0m0.344s

The simplest way: use the Unix utility ‘time’ to measure the execution time of a program

• real – the actual elapsed time from start to end

• user – the amount of CPU time spent outside of kernel

(45)

cProfile

• A C extension which adds relatively small overhead profile

• A pure Python module adds significant overhead to the program being profiled. Can be extended to build a customised profiler

Other tools

(46)

46

The profile Module

import profile import random def swap(x, i, j): x[i], x[j] = x[j], x[i] return x def bubbleSort(x): for i in xrange(len(x)): for j in xrange(i): if (x[i] < x[j]): x = swap(x, i, j) return x x = [] for i in xrange(1000): x.append(random.randint(1,50000)) profile.runctx('bubbleSort(x)', globals(), {'x':x})

(47)

The profile Module

250239 function calls in 0.956 seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)

1 0.000 0.000 0.000 0.000 :0(len) 1 0.000 0.000 0.000 0.000 :0(setprofile) 1 0.000 0.000 0.956 0.956 <string>:1(<module>) 1 0.000 0.000 0.956 0.956 profile:0(bubbleSort(x)) 0 0.000 0.000 profile:0(profiler) 250234 0.436 0.000 0.436 0.000 test_profiling.py:4(swap) 1 0.520 0.520 0.956 0.956 test_profiling.py:8(bubbleSort)

(48)

48

The cProfile Module

Call cProfile on the command line to profile a program

Save output to a text file

Read and print out file content

$ python –m cProfile testing.py

$ python –m cProfile –o output.txt testing.py

$ python

>>> import pstats

>>> p = pstats.Stats(“output.txt”)

(49)

(50)

50

Load Test Your Application

In your project, after finished the application, you are required to load test your application and study its performance

• Come up with a simple but reasonable workload model

(e.g. a common scenario in which the user signs in and carry out several actions)

• Investigate the distribution of response times of major endpoints (e.g. how long does the user have to wait when submitting a new article or a new photo?)

(51)