Design of Web Services Testing Tool
Girish M Tere1, Bhatat T Jadhav21Department of Computer Science, Shivaji University, Kolhapur, Maharashtra – 434004 2Y.C. Institute of Science, Satara, Maharashtra – 415001
ABSTRACT-Software testing is one of the most important
techniques used to assure the quality of Web services. With the increase of the popularity of Web services, more
and more Web applications are developed with this
technique. This new way of software development brings about new issues for software testing, which has been widely recognized as a realistic means for ensuring the
quality of software systems. Web Services testing is
essential to achieve the goal of scalable, robust and successful Web Services especially in business environment where maybe exist hundreds of Web Services working together. In this paper, we give detailed explanation about the Web Services testing methodology and skill, which are very helpful to the testers. Compared with traditional programming testing, the Web Services testing have its own feature such as performance, authorization, and security. Based on the knowledge of the aspects of Web Services, we design and implement a testing tool to perform some tests automatically. This Web Services Testing tool automates the load testing of Web services before they are put into production. It can test Web Services that expose their functionality via SOAP interfaces. With an easy to use interface, the user can quickly generate test scripts from a WSDL document and verify the actual SOAP responses with the expected response.
KEY WORDS: SOAP; Web services; Web services testing
tools
1. INTRODUCTION
With the growing of using XML Web Services (Aoyama & Kawaguchi 2000), (Brereton et al. 1999), we find that the Web Services testing technique should be enhanced in the Web Service developing cycle. Although, Web Services are web application we can use tradition web testing methods (Nakamur, Hada & Neyama 2002). Testing is essential to achieve the goal of scalable, robust and successful Web Services, testers should concentrate in some key points of Web Services testing in order to design test cases specific to the task. In this paper, we give the basic concepts for the Web Services testing and implement an automatic test tool for some testing. Almost all software components expose their interface through Web Services for SOAP & XML messaging (Sahai et al. 2002). In the era of modern, web services-based SOA, distributed software components are readily integrated
and dependent. This distributed and flexible nature of software components makes web services testing challenging. A. Exploring Testing Issues
1. A key to testing Web services is ensuring their functional quality, because when you string together a set of services, you introduce many more opportunities for error or failure 2. Developers are typically poorly versed in security coding
scrutiny driven by performance issues
3. Few development organizations within enterprise IT shops understand need for vulnerability testing
4. Testing was the last step of the waterfall method. But even here the concept of a “freeze” is ending; testing is becoming a continuous activity.
B. Web Service Performance Testing Challenges
Since Web services are composed of loosely coupled distributed over networks, we must test the application: - end to end;
- service by service; - interface by interface.
Overall functionality of web services should be easy to test. All these loosely-coupled, platform-independent, highly scalable services are not free
– Major performance problems typically are a result of – Large services without adequate hardware support – Small services with significant overhead
– Layer on layer; abstraction on abstraction
– Services distributed on a network with its own latency Application needs to be performance tested in the following manner:
– End to end from the requester perspective
– At the unit level during development (by provider) – At service level (generally by requester and provider) – Interface validation (generally by requester and
provider)
– To ensure functionality under boundary load conditions
2. WEB SERVICES TESTING METHODOLOGY
A Web service is an interface that describes a collection of operations that are network accessible through standardized XML messaging. A Web service is described using a standard, formal XML notion, called its service description. It covers all the details necessary to interact with
the service, including message formats (that detail the operations), transport protocols and location. The interface hides the implementation details of the service, allowing it to be used independently of the hardware or software platform on which it is implemented and also independently of the programming language in which it is written. This allows and encourages Web Services-based applications to be loosely coupled, component-oriented, cross-technology implementations. Web Services fulfil a specific task or a set of tasks. They can be used alone or with other Web Services to carry out a complex aggregation or a business transaction.
Fig. 1 Web Services roles and operations
Web Services are modular, described and self-contained applications (Sahai et al. 2002). With the open standards, Web Services enable developers to build applications based on any platform with any component modular and any programming language. More and more companies are now exposing their information as Web Services and what’s more, it is likely that Web Services are used in mission critical roles, therefore performance matters. Consumers of web services will want assurances that Web Services will not fail to return a response in a certain time period. So the Web Services testing is more important to meet the consumers’ needs. Fig. 2 shows how XML messaging and network protocols forms the basis of Web Services architecture.
The basic requirements for a network node to play the role of requestor or provider in XML messaging-based distributed computing are the ability to build, parse a SOAP message, or both, and the ability to communicate over a network (receive, send messages, or both).
Typically, a SOAP server running in a Web application server performs these functions. Alternatively, a
programming language-specific runtime library can be used that encapsulates these functions within an API. Application integration with SOAP can be achieved by using four basic steps:
In Fig. 2, at (1) a service requestor’s application creates a SOAP message. This SOAP message is the request that invokes the Web service operation provided by the service provider. The XML document in the body of the message can be a SOAP RPC request or a document-centric message as indicated in the service description. The service requestor presents this message together with the network address of the service provider to the SOAP infrastructure (for example, a SOAP client runtime). The SOAP client runtime interacts with an underlying network protocol (for example HTTP) to send the SOAP message out over the network.
Request (service invocation) Fig. 2 XML messaging using SOAP
At (2) the network infrastructure delivers the message to the service provider’s SOAP runtime (for example a SOAP server). The SOAP server routes the request message to the service provider's Web service. The SOAP runtime is responsible for converting the XML message into programming language-specific objects if required by the application. This conversion is governed by the encoding schemes found within the message.
The Web service is responsible for processing the request message and formulating a response. The response is also a SOAP message. At (3) the response SOAP message is presented to the SOAP runtime with the service requestor as
Application WS SOAP Network Protocol SOAP Network Protocol 1 2 3 4 Clien Service Respons Application Service Provider Service Requeste r Service Registry Service Description, Service Description, WSDL Publish WSDL, UDDI Find WSDL, UDDI Bind
the destination. In the case of synchronous request/response over HTTP, the underlying request/response nature of the networking protocol is used to implement the request/response nature of the messaging. The SOAP runtime sends the SOAP message response to the service requestor over the network.
At (4) the response message is received by the networking infrastructure on the service requestor’s node. The message is routed through the SOAP infrastructure; potentially converting the XML message into objects in a target programming language. The response message is then presented to the application.
A. Unit Testing
Unit testing is much more like tradition program test. We could apply this technique into web services unit test. B. Functional Testing
Functional testing ensures that the functionality of Web Services is as expected. In functional testing, we should not only examine the basic input/output, bounds testing, error checking and so on, but also include the basic security/authorization examination, and test if the service should support all the communications protocols it applied which is also very important to Web Services.
Although Web Services have no user interface, but they provide web methods to invoke, which provide us a way to use automatic script to test them.
C. Performance Testing
Performance testing is often to determine the relevant product statistics. For example, the number of messages per second or the number of simultaneous users acceptable of a service. There are basically three ways to conduct performance testing: Ad hoc performance testing; Observational testing; Measured testing.
Compared with tradition program and web program, there exist variety factors to effect the performance of Web Services, which include: Differentiated Web Services solutions; System model; Workload model (Cardellini, Casalicchio & Colajanni 2001), Transaction model (Sahai et al. 2002), or even Security model.
There are some key parameters to determine Web Services performance; we can use tradition web testing parameters to describe the efficiency or ability of Web Services. These common performance measurements include: MCs(Megacycles), Memory footprint, BoW(bytes over wire, TTLB(time to last byte), user-perceived response time, and the TTFB(time to first byte). In research paper (Cardellini, Casalicchio & Colajanni 2001), SLA (service level agreement) will rule the relationship between users and services providers. It is very important for testers
D. Load/Stress Testing
The aim of Load/Stress testing is to find out the Web Services’ scalability in the growing of the number of the clients invoke them. Load/Stress testing can be applied with performance testing together. Through Load/Stress testing, the typically bugs would be found more easily than other testing methods, such as:
Memory leaks: Memory leak would be common in tradition programs. In Web Services, the programming language such as Java, C# is designed to automatic de-allocate memory when objects are no longer used. But it is still possible that the objects would not be de-allocated, such as in java’s programming using JDBC. Memory leaks not only appeared in programming language in Web Services, but also in some circumstances in database server. If too many connections appeared at the same time, the database server would not release the cursor in time in some circumstances. The memory leak is extremely difficult to detect. With few use of Web Services, memory leaks are very rarely found since the test does not generate enough usage of the product before it completes. Even through Load/Stress testing, few memory leaks still cannot be found. Concurrency and Synchronization: In Load/Stress testing, the testing program will generate many threads to act as virtual users to invoke Web Services, while Web Services can be invoked by others. In such complex circumstance, many different code paths and timing conditions will be performed. In general, the more code paths performed, the more error will be shown.
The load/Stress testing is ideal for automatic testing. The implementation of the testing tools will be introduced at section III.
E. Security Testing
Generally, there are two kinds of Web Services, the Web Services are used in Intranet and the Web Services are used in Internet (Zhu 2006), (Zhu & Zhang 2010). Both of them face the security risk since message could be stolen, lost, or modified. The information protection is the complex of means directed on information safety assuring. In practice it should include maintenance of integrity, availability, confidentiality of the information and resources that used for data input, saving, processing and transferring. The complex character of this problem emphasizes that for it solution should be realizing the combination of legislative, organizational and software-hardware measures. So, it is very difficult to test these automatically, but testers should carefully design the test cases according to the real environments and the protocols. The WS-Security 1.0 specification provides varieties ways for Web Services security, such as XML DIGSIG, XML Encryption.
The information protection is just one aspect of the security. In Web Services, the main challenge we faced is to consider the protection of resources such as data and applications so that this important information should be only
accessed by the appropriate entities. This calls for authorization
F. Authorization Testing
In real world, one Web Services would invoke another Web Services, In EAI, there exist thunders of Web Services that could be used together. Not everyone could access these services. So testing the authorization is very important to protect the invaluableness data. Authorization denotes granting authority, in practice, we often use access control list (ACL) or role-based access control (RBAC) to map from the entities to resources in order to assign rights for each resource. Authorization testing should test each entity (user categories) to Web Services to get an access table to determine if the authorization is assigned correctly. We designed a semiautomatic tool to perform the authorization test, which will be discussed later.
3. THE DESIGN OF THE WEB SERVICES TESTING
TOOL (WSTT)
A. Related Work
The nunit.org (NUnit 2011) developed an open-source unit-testing tool for all dot net languages. Some primary developing environments provide the functional testing tools, such as Visual studio .NET, IBM WebSphere Studio Application Developer, Weblogic workshop. But they’re basic functional tools; none of them provides full functional testing. Few companies develop third party Load/Stress tools, but some tools just use http post and get to simulate the user request, none of them provides security or authorization testing environment.
B. The Framework of WSTT
Since there exist good unit and functional test tools, we put our focus on the design of the Performance Testing, Load/Stress Testing and Authorization Testing. Fig. 3 shows the framework of WSTT. Detail information will be discussed later.
Fig. 3 The framework of WSTT C. Performance and Load/Stress Testing
Lots of Load/Stress automatic testing tools perform several tests at the same time, for example calling a number of
Web Services on the same server simultaneously (NUnit 2011). The tools first record user’s action, then change the action into scripts; the runtime engine executes the script concurrently.
Some Load/Stress tools’ capture programs catch all the http requests URL and then concurrently simulate number of users to open the URL, using http post get request to simulate the virtual user, whereas in production most Web Services will be invoked directly using SOAP. Here we use JavaScript as the script language to record the action that user invokes Web Services from browser. After record, the SOAP Composer transfers the http post request from the script to SOAP message and save as a configure file to let runtime engine to process. Because one Web Services can provide more than one web method, and one server can hold more than one Web Services, we consider Load/Stress testing includes two aspects:
− Repetition testing. Repetition testing is the basic testing for one functions or web method. That means running one Web Services over and over again. Functional test is to examine if the Web Services are working well, while repetition test is to justify if the Web Services could continue to work repeatedly.
− Concurrency testing. Repetition testing is to test one special web method; the aim of concurrency testing is to simulate the real world of the using of Web Services, it performs several operations simultaneously. With the concurrency testing, we can find some problems like thread safety, transaction problems and so on. We designed the runtime engine which can simulate up to 120 users concurrency, with each one can call one or more Web Services according to the configure file.
The performance testing can be a part of Load/Stress testing. The tradition way to analyze the performance is to use the site’s log data by using web mining technique (Sankar, Varun & Pabitra 2002) or use cookies to analyze user’s visiting (Cooley, Mobasher & Srivastava 1999). But these are not very suitable for Web Services. We currently use SOAP extension to catching SOAP message (Sahai et al. 2002). A SOAP message is composed by three major parts: a SOAP envelope, a SOAP header and SOAP body. We apply a new SOAP extension to Web Services to catch the SOAP message and log it to a text file for testing analysis. Analyzing these data should be very important. In the concurrency testing, Web Services might have transaction; some Web Services would be broken by another Web Services, or even cannot be executed. Currently, we do not provide such analyze tools to help testers to found such problems.
D. Authorization Testing
The authorization testing using WSTT currently cannot be performed automatically, because there exist many authorization methods. In different organizations, there may exist different authorization models such as LADP,
authorization services. We cannot get the users’ or roles’ information from these authorizations directly. In WSTT, we can simulate authorization in following ways:
1. Invoke and record all Web Services manually through the recorder. That can reuse the test case of the Load/Stress testing. Or we can get such information from UDDI. 2. Manually assign users’ (roles’) rights in authorization
server with same token or password. The users’ rights should be covering all the Web Services and there are no two users have exactly the same rights.
3. Export the users’ name form the authorization server and import them to WSTT
4. WSTT uses the each username, password to assemble a SOAP message, to invoke each Web Services and recorder the state.
5. Generate the report.
The SOAP message assemble must be modified by hand, because we cannot prognosticate what encryption method Web Services are used.
4. CONCLUSIONS
In this paper, we give some methods for the Web Services testing. Web Services testing includes unit testing, functional testing, performance testing, Load/Stress testing, security testing and authorization testing. We give detailed information about the key points of the Web Services testing and designed an automatic testing tool for some of these testing. Web Services are the foundations of modern distributed applications. The ease and flexibility of integrating disparate components has made rapid application development possible across platforms, languages, operating systems and devices. This flexibility makes it critical for developers and security professionals to thoroughly test and certify their Web Services before publishing them to consumers. The automatic testing tool we currently used has some features, such as SOAP-based log analysis, Repetition and Concurrency Load/Stress testing, and the authorization testing. In future, companies shall concentrate more on their core business and use services provided by experts in a specific area without re-inventing the wheel making SOA a reality. This will call for more stringent testing methodologies considering the scope involved in testing these services in an integrated manner. Industry will pay more attention to this technology as it is one of important technique to be used in cloud computing.
ACKNOWLEDGMENTS
Our thanks to teachers of Department of Computer Science, Shivaji University, Kolhapur for motivating us for this research work.
REFERENCES
Brereton, P., et al., (1999), “The Future of Software”, CACM, Vol. 42, No. 21, 1999, pp. 78-84.
Cardellini, V., Casalicchio, E. & Colajanni, M. (2001), “A Performance Study of Distributed Architectures for the Quality of Web Services”, Proceedings of the 34th Hawaii International Conference on System Sciences – 2001. Cohen, F., (2011), “Design of Web Services Testing Tool”, DOI=http://www.ibm.com/developerworks/webservices/library/ws-testsoap/, Accessed on 12 Jan 2011
Cooley R., Mobasher B. & Srivastava J, (1999), “Data Preparation for Mining” World Wide Web Browsing Patterns, Knowledge and Information Systems, 1999, 1 1(1) 5~32.
Felix, T., Castanet, P. & Berrada, R., (2009), “Testing Web Services Composition Using the TGSE Tool Services – I”, World Conference on Issue Date : 6-10 July 2009 Los Angeles, CA
M. Aoyama, E. Kawaguchi, Intelligent Software Services over the Internet, Information Modeling and Knowledge Bases, IX, IOS Press, Feb. 2000, pp. 128-135.
Mei, H. & Zhang, L., (2005), “A framework for testing Web services and its supporting tool”, Service-Oriented System Engineering, IEEE International Workshop, SOSE 2005, 20-21 Oct. 2005, pp 199
Muscogiuri, C., Jaeschke, G., Paradiso, A. & Hemmje M., (2002), “FAIRWIS: An Integrated System offering Trade Fair Web-based Information Services – A R&D Case Study”, Proceedings of the 35th Hawaii International Conference on System Sciences - 2002
Nakamur, Y., Hada, S. & Neyama, R., (2002), Towards the Integration of Web Services Security on Enterprise Environments, Proceedings of the 2002 Symposium on Applications and the Internet (SAINT 02w).
NUnit, DOI=http://nunit.org/index.php, Accessed on 20 Jan 2011
Sahai, A., Machiraju, V., Ouyang, J. & Wurster, K. (2002), “Message Tracking in SOAP-Based Web Services”, 0-7803-7382-0/02, IEEE 2002 Sahai, A., Machiraju, V. & Ouyang, J. (2001), “End –to-End Transaction Management for Composite Web based Services”, IEEE, 2001
Sankar K., Varun, T. & Pabitra, M., (2002), “Web Mining in Soft Computing Framework: Relevance, State of the Art and Future Directions”, IEEE Transactions on Neural Networks, 2002, 13(5) 1163~1177.
Sneed, H. & Huang, S., (2006), “WSDLTest - A Tool for Testing Web Services and Web Site Evolution”, 2006, WSE '06, Eighth IEEE International Symposium on Issue Date : 23-24 Sept. 2006 , Philadelphia, PA
Zhu, H., (2006), “A Framework for Service-Oriented Testing of Web Services”, Computer Software and Applications Conference, 2006, COMPSAC '06, 30th Annual International, Sept.2006, Volume: 2, page(s): 145 , Chicago, IL
Zhu, H. & Zhang, Y., (2010), IEEE Transactions on Collaborative Testing of Web Services, December 2010
