SUNGARD SUMMIT 2007 | sungardsummit.com
A Community of Learning
Load Testing your Banner Systems
Presented by: Scott Hollen University of Richmond
March 22, 2007 Course ID 116
Course ID 116
Introduction
¾ What is load testing?
¾ Why test?
¾ “Load” vs. “Performance” vs. “Stress” testing
¾ Tools
Course ID 116
What is load testing?
From http://computing-dictionary.thefreedictionary.com
A test of a computer system and its applications by running under a full complement (full load) of
transactions or users. A load test can be real or simulated by testing software
Course ID 116
Why test?
¾ Maximize efficiency of your systems
¾ Identify potential bottlenecks
¾ Discover potential limitations
¾ Ensure that critical systems behave adequately under peak loads
¾ Provide measurements to anticipate all sorts of stuff like growth and response times
Course ID 116
It’s all so confusing!
Okay, we get load testing, but what about PERFORMANCE testing?
What about STRESS testing?
It may not seem all that important, but there is a difference between Performance, Load and Stress testing
Course ID 116
Performance Testing
¾ Profiling your applications and/or servers
¾ Goals are to find and eliminate bottlenecks
¾ Establish baselines and benchmarks, and acceptable levels of response for expected loads
¾ NOT to find bugs
¾ Performed on stable systems
¾ “White Box Testing” – inspected/monitored from the inside out (via monitoring tools); exercising programs internals
Course ID 116
Load Testing
¾ Comes after Performance testing – after tuning
¾ Exercise the system by increasing the load on the system – this exposes bugs
¾ Determines the highest load the system can handle and still work properly
¾ “Black Box Testing” – using client load tools against the application to measure its responsiveness
Course ID 116
Stress Testing
Simply put – this is where you get to try and break the system by overwhelming it
¾ Tests recoverability
¾ Allows for testers to see how the system behaves in/reacts to failure
Course ID 116
Sequence of Events
1) Performance test and tune the heck out of your stuff – eliminating any bottlenecks
2) When you can’t optimize anymore, move on to Load testing
http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html
Course ID 116
Tools
Front End Tools
¾ Prepares the data and scripts needed to drive the test’s actions
Back End Tools
Course ID 116
Examples
Open source
1) JMeter
2) The Grinder
3) Apache Flood
4) Httperf
Commercial products
1) LoadRunner (Mercury.com) 2) WAPT (loadtestingtool.com) 3) Silk Performer (by Seque.com)
Course ID 116
Now, for Banner
University of Richmond purchased LoadRunner in June, 2004
Allows for recording of scripts against Banner and for running them in many different configurations
• Also allows for analyzing of results following the tests
• Not cheap though!
First used extensively a year later when Registrar wanted to move from staggering out student registration to opening it up to the entire
Course ID 116
First step – Registrar’s office, DBAs, SAs and Banner team met.
Second step – Acceptable levels of performance was agreed upon
Course ID 116
Goals for Information Services
1) To provide an accurate estimate of the number of
Registering users our current system can acceptably support
2) To make useful recommendations regarding the use and expansion of our production environment
Course ID 116
Assumptions
1) We have a license for only 150 users in LoadRunner, so this is the size we decided to use in our simulation
2) Current environment based on current registration peak usage
3) Number of CPUs on database server is the critical hardware resource
4) Number registering vs. number who are available to register between 75% and 90%
Course ID 116
Our Test
Record with LoadRunner 1 student performing the following: 1) Login to Bannerweb
2) Select a registration term 3) Repeat the following 5 times
• Pauses – based on LoadRunners “wait time” and verified by Registrar and student as being realistic
• Looks up a class
Course ID 116
Repeated performance/load tests determined that our
production systems could handle 75 registration users per CPU
Conclusion
• With current hardware (4 CPUs), the maximum number of registering students we can handle is 300
Course ID 116
Was load testing valuable?
Yes!
We found that our current environment could not handle the Registrar’s request – this led to an upgrade of our CPUs Tests were repeated after upgrade and showed that we now
Course ID 116
Another example
Banner 7.2 to 7.3 Upgrade
Ran a series of performance tests against Banner 7.2 for baseline – system monitored by DBAs and SAs
¾ AGCGIFT – query only form
¾ APAIDEN – constituent data entry
¾ 150 user Registration test
• Modified our existing environment for Red Hat and mid-tier upgrades and re-tested, recording and saving
results
• Results – via LoadRunner and other tools – showed overall improvement
Course ID 116
• Ran in Banner 7.3 modifications and re-ran the tests • Improved again!
Course ID 116
So what does all of this show us besides an improvement in our system?
1) Establishment of baseline tests (both the test execution
scenarios) that can be executed over and over again provides consistency in comparisons
2) Well thought out test plans are crucial to overall results – knowing what part of your system the tests target allow for trusting the final numbers
Course ID 116
Overall Conclusions?
¾ Understanding the differences in your tests is critical to goal of the test
¾ Large systems such as Banner benefit greatly from well-engineered load testing (regardless of your chosen tool)
Course ID 116
Great resources
In addition to the two previous links:
http://www.wilsonmar.com/
http://www.awprofessional.com/articles/article.asp?p=2146 8&seqNum=6&rl=1
Course ID 116
Thank You!
Scott Hollen
Please complete the online class evaluation form Course ID 116
SunGard, the SunGard logo, Banner, Campus Pipeline, Luminis, PowerCAMPUS, Matrix, and Plus are trademarks or registered trademarks of SunGard Data Systems Inc. or its subsidiaries in the U.S. and other countries. Third-party names and marks referenced herein are trademarks or registered trademarks of their respective owners.
