TESTING AND OPTIMIZING WEB APPLICATION S PERFORMANCE AQA CASE STUDY

(1)

TESTING AND OPTIMIZING

(2)

Intro to Load Testing

2

(3)

What is Load Testing?

 Load testing generally refers to the practice of

modeling the expected usage of a software

program by simulating multiple users accessing the program's services concurrently.

 As such, this testing is most relevant for multi-user

systems, often one built using a client/server model, such as web applications.

(4)

Why Load Test Your Application?

 High cost of mission-critical application failure  Simulate real-life workload conditions

 Find boundaries of your system

 Locate potential problems before your

customers do

 Reduce development cycle and time-to-market

 Predict and reduce infrastructure costs

(5)

Load Testing Goals

Performance

 Ensure your application’s performance under basic

workload (baseline)

Stress

 Find break points and boundaries of your application

Reliability

 How stable is your application under extended period

of load

Scalability

 Determine how your application scales for the

(6)

Load Testing and Product Life Cycle

• Define performance requirements

Requirements

• Design your system based on performance requirements

Design

• Load test separate components

Development

• Load test a whole system

Testing

• Monitor your system’s performance

(7)

Load Testing Workflow

(8)

AQA Case Study

8

(9)

Background – AQA

 The Assessment and Qualifications Alliance (AQA)

is the largest of the three English exam boards.

 Awarding 49% of full course GCSEs and 42% of

A-levels nationally.

 Processing over 3.5 million exams each year.  Providing online services to teachers, exams

(10)

SUT – ERA (Enhanced Results Analysis)

 ERA is a free on-line tool that gives teachers in UK

an instant analysis of exam results.

 ERA’s key features:

 Grades reports by centre and subject

 Cross-centre and years results comparison

 Individual students grades analysis

 Question by question analysis

(11)

The Challenge

 Database size over 100

million records

 10 million active records

 70,000 users

22/08/2007 - Requests per 10 minutes

0 500 1000 1500 2000 2500 3000 00:0 0 01:0 0 02:0 0 03:0 0 04:0 0 05:0 0 06:0 0 07:0 0 08:0 0 09:0 0 10:0 0 11:0 0 12:0 0 13:0 0 14:0 0 15:0 0 16:0 0 17:0 0 18:0 0 19:0 0 20:0 0 21:0 0 22:0 0 23:0 0

14/08/2007 - Requests per 10 minutes

0 500 1000 1500 2000 2500 3000 00:0 0 01:0 0 02:0 0 03:0 0 04:0 0 05:0 0 06:0 0 07:0 0 08:0 0 09:0 0 10:0 0 11:0 0 12:0 0 13:0 0 14:0 0 15:0 0 16:0 0 17:0 0 18:0 0 19:0 0 20:0 0 21:0 0 22:0 0 23:0 0

 Workload of less than

10 pages/min at usual day

 Pick of 300 pages/min

(12)

12

Deployment Architecture

(13)

Test Architecture

Server Configuration

Oracle Web Cache  Microsoft Windows Server 32bit  Oracle Web Cache 10.2.2

Oracle AS  RedHat Linux ES4 U6

 Oracle Application Server 10.3.3  ChartFX 6.2

Oracle RDBMS  RedHat Linux AS4 U7

(14)

Identifying Test Scenarios

 Pick 5 most common

scenarios in your system

 Don’t add scenarios

which operate similar back-end flows

 Identify problematic

back-end flows ahead and make sure

scenarios cover them.

14

(15)

Identifying Potential Bottlenecks

 Search

 Usually sensitive to various input values.

 Simple or not implemented paging can cause to slow response time

 Search Results

 If paging is not implemented, result page can be very large

 Back to search results page from item details can execute search again

 External and 3rd Party Components

 3rd party components can cause to deadlocks or poor performance  OSSO server can perform slower than the rest of the system

 Export to Excel

(16)

Execution Plan

 Test the system with “light” input parameters first  Get through the “obvious” bottlenecks, constantly

increasing the workload

 Run nightly tests every night to ensure reliability

and stability of the system

 At level of 500 users, apply “heavy” input

parameters

 Test system for randomly distributed workload

(17)

10 Users – Conn. Pool Starvation

0% 20% 40% 60% 80% 100% Connection Pool Connections

Measurement Avg./Sum Max.

Page Time 1.6 sec 34 sec HTTP 200 3,348

HTTP 304 2,190 HTTP 302 410 HTTP 500 48

(18)

18

100 Users – Apache Shows Its Limit

Measurement Avg./Sum Max.

Load Size 100

Page Time 5.6 sec 67 sec Pages per Second 2.1 7.5

(19)

500 Users – Slow DB & OSSO

Measurement Avg/Sum Max.

Load Size 1000

Page Time 24 sec 2.5 min Pages per Second 7 19 HTTP 500 368

(20)

1000 Users – Search & Large Content

Sum

Max.

Load Size 1000

Page Time 1 min 7.5 min HTTP 500 27

(21)

Concluding 1000 Users 8 hrs Test

Measurement Average/ Sum

Max.

Load Size 1000

(22)

Load Issues Pyramid

10 – Conn. Starvation 100 – Apache Limit 500 – Slow DB Queries 1000 – Code, Search, Cache

(23)

Achieved Results

 Supported concurrent sessions/users:

 From 10 to 1000 (x100)

 Reliability

 From 90% failure to ~100% reliable system

 Response Time at 1000 VU

 From 1 min to less than a second

 Pages per Minute

(24)

Tips for Effective Load Testing

24

(25)

Optimizing Search Mechanism

 Use existing solutions.

 Use free text search only as the last option.  Limit user input to reasonable values.

 Implement paging on both search and result levels.  Improve “Back to search results” mechanism.

(26)

GZip Compression Effectiveness

 Pros:

 Reduce size of large text pages by 80%-90%  Improve response time by 50%

 Decrease throughput

 Cons:

 Increased CPU utilization by 35%

 Tip:

 Apply GZip selectively to text-based static resources and

large dynamic pages

 Do not apply GZip to every response!

(27)

Database vs Web Servers Costs

 Additional database setup and maintenance costs

are significantly higher than of web server.

 Make DB work easier by making web server work

harder.

 Memory costs less than CPU or new hardware

– rely more on memory, but keep in mind that everything has its limits.

(28)

Developers and Testers Tips

 Get to know your user.

 Evaluate his system usage behavior or try to

predict the common flow.

 Know the “costly” operations of your system. Try to

build the flow that omits recurrent usage of the operations.

 Do not be reluctant to cache data, still don’t

overuse the cache. Remember that shared data should be stored once for all users.

(29)

Project Manager Tips

 Make sure all involved parties are available at the

moment of test execution.

 Share test findings among all project members.  Log all changes to the test environment. You will

need to reapply them later to the production environment.

 Make sure the final results meet the performance

(30)

Team Synergy

 Synergy and cooperation between the following

teams is crucial for the success of the test:

 Database Team

 Application/Development Team

 System Team

 Performance Team or Specialist

(31)

Nachum (Nick) Dimer

Owner & Chief Consultant

IL: +972.54.4600515 UK: +44.783.8673770 PL: +48.519.717875 [email protected]

TESTING AND OPTIMIZING WEB APPLICATION S PERFORMANCE AQA CASE STUDY