E-Guide
Agile Testing in the Cloud
Software applications are closely intertwined with modern business models that demand for their development has never been higher. However, challenges exist, particularly for geographically dispersed teams. For test teams in some application development shops, cloud computing has broken down many of the limitations caused by testing on internal resources. This E-Guide will cover how to use the cloud for improved Agile testing and how this move to the cloud may help conquer internal limitations.
E-Guide
Agile Testing in the Cloud
Table of Contents
How to use the cloud for improved Agile testing
QA teams are moving testing to the cloud, conquering internal limitations Resources from IBM
How to use the cloud for improved Agile testing
By Francis Miers
Software applications are so closely intertwined with modern business models that demand for their development has never been higher. But challenges exist, particularly for
geographically dispersed teams. In this tip, we find out how the use of the cloud can provide many benefits for test management of a geographically dispersed team.
Demand for software continues
The constant search for competitive advantage has fostered an increase in global demand for software development. With that demand comes an escalating pressure to deliver software that’s effective straight out of the box, with minimal defects, and all in double quick time. Given the increasing complexity of software, it’s perhaps not surprising that less than a third of development projects started are brought to a successful conclusion.
While the co-location of teams may still be the preferred solution for effective software development, the emergence of technically competent, low cost workforces in traditionally non-technical countries, has added a new dynamic. With most obviously India, but
increasingly Eastern Europe and elsewhere, now being able to offer skilled IT resources, work has flowed in their direction.
Nearly 60% of Agile testing teams are geographically dispersed
A striking illustration of this trend is contained in a recent survey, which revealed that nearly 60% of all Agile testing teams are now geographically dispersed. Close-proximity working, even in Agile teams, is ever less the norm and increasingly the exception.
But while globalization offers the opportunity to access lower-cost resources and maintain momentum through 24/7 working, it also brings with it the potential for delay and
disruption. Most particularly, trying to utilize and manage geographically dispersed resources to the full poses a problem for standard and Agile testing methods, which have
traditionally assumed development and test teams to be located in a single room or campus.
The heart of the difficulty: communication
At the heart of this difficulty lies just one thing -- communication -- or rather lack of it. This alone has the disruptive ability to throw off schedules and inject costs and inefficiencies into testing projects.
For though our ‘village’ may well be global, it is far from a tight-knit community, with people spread around the world speaking in different first languages, underpinned by different cultures and operating in different time zones. Introduce these elements to the mix, and test management, particularly of Agile projects, immediately becomes a much more complex task, especially where Agile testing runs alongside development and so is 'requirements driven.'
Security and reliability of data transfer is complex
With dispersed teams, the security and reliability of data transfer becomes more complex. Companies looking to offshore their development and test management regimes, must extend their networks, typically Virtual Private Networks (VPN) to far away locations. Yet deploying a leased line or VPN to a developing country, where the infrastructure is still catching up, can be an expensive option. Add in the need to configure servers and set up databases, and this becomes a process not to be undertaken lightly.
Companies are turning to the cloud
These unfavorable ground conditions are leading more and more companies to turn to the cloud. For companies working with teams all over the world, using the cloud offers the opportunity to bring distant resources into the equation as and when needed, while doing away with the daunting requirement to invest in proprietary communications infrastructure. This method of working doesn't just deliver great time savings, but also removes the
What's more, the set-up and process of management testing is immediately simplified in the cloud, with no complicated training of personnel required, since team members can access a project simply and instantly via a familiar browser.
Though cloud-based systems may not yet be able to cope with all the complexities of
software development, when combined with cheap internet communication systems such as Skype, the cloud offers the perfect environment for test management. It even lets teams work within Scrum's iterative framework: enabling Sprint meetings; conducting meetings; the public posting of real-time, up-to-date burn down charts; and immediate collaborative thinking and comment for faster defect resolution.
The cloud is also well suited for the storage of test assets. Test scripts, recorded defects and results analysis can be all be stored in one place, readily accessible to all who need them, no matter what time zone they are working from. Projects become much easier and more effective, with performance improved as no work effort is lost through duplication or compromised through the introduction of errors and omissions during updates.
In an ever-more 'distributed world' where even relatively simple software projects can generate many hundreds of tests, the ability to communicate and access shared information is an enormous advantage.
By using the power of cloud-based test management, companies can do this simply and more cost-effectively than ever before. For that reason, those involved in overseeing projects, irrespective of scale or location, should consider it as a primary option.
About the Author: Francis Miers is a Director of Automation Consultants, an IT testing
consultancy with a specialist interest in cloud-based test management tools. The company has consulted on testing projects and IT transformations for clients that include BSkyB, Vodafone, HSBC and T-Mobile.
QA teams are moving testing to the cloud,
conquering internal limitations
For test teams in some application development shops, cloud computing has broken down many of the limitations caused by testing on internal resources. Early adopters say cloud testing has given QA teams freedom from internal hardware availability issues, a more accurate picture of potential runtime problems and an effective way to collaborate. Companies that move to cloud testing often arrive at a more streamlined and
communicative application lifecycle, said Theresa Lanowitz, analyst with Voke Inc. Where developers and testers communicate, she said, there is often a lot of back-and-forth in describing defects found only in test environments.
"Testers are now able to say, we've tested this software in virtual environment in the cloud, here's the defect and here's a link," said Lanowitz. "And the developer can click on that URL, see where the defect is and fix it."
Eliminating the "back-and-forth tension" between developers and testers speeds up the test cycle and lets testers focus on more strategic work, Lanowitz said.
Retailer uses cloud to speed up testing
Startup M-Dot Inc. is preparing to roll out its digital coupon and point-of-sale applications at 200 grocery stores in August, M-Dot CTO Mike Kavis already knows what 1 million
concurrent users would do to its in-house systems. "In the on-premise world you could never afford to do that," he said. "Even if you could, to set up and care for those servers could take months and months."
Since signing up to do load testing with cloud test provider SOASTA Inc., Kavis said he was able to put the architect on his four-person IT team on other tasks. It took M-Dot just a day to successfully simulate a million users. Kavis said once they flipped the switch, a few things broke around 700,000 users and a few more around 900,000. His team fixed the issues and load was no longer a 15-minute discussion with potential customers.
For Kavis, the ability to spin up a test lab at will and scale it to nearly any size has enabled a much more flexible style of application testing. He said his team can start testing as early as the prototyping phase and continue to test more frequently as it creates more services and optimizes the applications. The underlying goal is to root out bugs before they get into production. In the past, he said, load testing to scale was rarely even feasible.
"Where I came from, there wasn't a whole lot of money for hardware and testing, so
basically you tested within the limitations of your own infrastructure," Kavis said. "We would always find errors in production because we just didn't have the horsepower to really pound at it."
Virtual labs can liberate testers from IT
Cloud testing gives QA managers control over their test labs, said Ron Yun, QA director at mortgage software vendor Ellie Mae Inc.
Before setting up an on-demand, virtual test lab, QA teams at Ellie Mae had to rely on the company's IT department to schedule and configure internal machines. But IT's top priorities were usually production issues. Often times, test projects had to wait. ¬¬ "Our request, which is an internal request to them, comes at the bottom of their priority list," said Yun. "It could take days to weeks to satisfy a request."
Last summer the company signed up with virtual lab provider Skytap Inc. and, suddenly, the test lab was available on demand from any workstation and could scale up or down as needed. Now IT is virtually out of the picture and QA teams can carry out their functional, load and performance tests without having to fuss over physical machines, said Yun. Yun's testers can now collaborate with a second team in Beijing on a
keystroke-by-keystroke level of real time. Before, there could be as much as a day between the offshore team sending some data and screenshots and a problem really getting addressed, Yun said. He estimated Ellie Mae's testing operations are now five times faster.
Getting IT's attention and finding enough hardware is a problem facing testers at many companies, said Tom Lounibos, president and CEO of SOASTA. And that holds back production.
"The testing of applications over the last 10 years has been a very slow process," said Lounibos. "There were a lot of people involved and it could take four to eight weeks just getting the hardware and setting up the tests." In a cloud testing environment there are often intelligent agents that log user actions to generate test scripts, he continued, and testers can spin up their configurations of choice in minutes.
Most of SOASTA's customers have been companies with consumer-facing Web applications. Now that many applications handle sudden traffic from online marketing campaigns and are connected to social media websites like Facebook, unprecedented traffic spikes can cripple an application without warning.
Large-scale testing not always necessary
Not all applications are equal, however, and neither are their needs in the test lab. While M-Dot's application is designed for unpredictable load spikes, JetBlue's website has fairly predictable traffic, said Sagi Varghese, QA manager at JetBlue. He said people can get mixed up about the need to load test at a tremendous scale.
"If a system is designed to handle X number of concurrent connections then, theoretically, you can only run that many concurrent virtual users," said Varghese. "So if the system is built to handle only 500 concurrent connections, if you give it a million hits it will process the first 500 and put everything else into a queue."
JetBlue QA teams use HP LoadRunner on internal machines, because LoadRunner and is able to support a wide variety of protocols, Varghese said. He'll be evaluating LoadRunner in the Cloud when it comes out soon.
The biggest frustration facing QA teams at JetBlue is trying to simulate real-life scenarios on new systems, Varghese said. With no historical load data to draw on in a new system, he said, the company's licenses for 1,200 simulated users can only go so far.
"Where the cloud comes in is the need to scale the size of the simulations up and down," Varghese said. "That is very hard to do with fixed assets."
Resources from IBM
Leveraging the Cloud to Transform Software Delivery
The Future of Software Delivery is Cloud: Leveraging the Cloud to transform development and test
Planning for cloud: How enterprise architecture management can help drive your IT transition
About IBM
At IBM, we strive to lead in the creation, development and manufacture of the industry's most advanced information technologies, including computer systems, software, networking systems, storage devices and microelectronics. We translate these advanced technologies into value for our customers through our professional solutions and services businesses worldwide.
