Performance Testing:
Factors that
Impact Performance
It is an accepted fact that no system is perfect from the viewpoint of performance. Problems pertaining to performance affect all types of systems, regardless as to whether they are client/server or Web application systems. It is imperative to understand the factors affecting performance of the system before embarking on the task of tackling them. The constraints affecting the performance may be many but can be broadly classified into (i) technical and (ii) project related factors. The former is very much pronounced in the case of Web-based systems, as the relevant technology has not attained the stage of maturity to address the issues on its own. On the other hand, the project managers struggle to develop an optimal system within the existing technology framework. This in turn, generates many project related factors sub optimizing the performance. Likewise, many factors affecting the performance of the system are discussed in this chapter. The subsequent chapters deal at length with the methods and strategies to tackle these factors.
Project Peculiarities
Management of projects broadly encompasses planning activities during the phases of development and implementation of a system. For instance, the planning at the systems development stage calls for selection of appropriate technology, skill sets, and software.
Defective planning at this stage would have a severe impact on the performance of the final product. The planning stage includes understanding various techniques available for development, skill sets required as well as the use of effective software tools. Therefore, the sooner these peculiarities are understood and addressed, the lesser the possibility of severe performance problems.
Rapid Application Development Approach
The fast changing technology coupled with ever increasing competition had a radical change in the perception of customers toward the product under development. They want to have a feel of the product as it evolves during the development. They would like to be involved in the development phases, and more importantly, they desire to launch the product in the market at an earliest point in time. This, in turn, leads to a pressure on the software development team to reduce the development cycle time for the product. Rapid Application Development (RAD) approach is now increasingly applied for rapid prototyping. Although RAD is based on the typical classic life cycle model of software engineering, its emphasis is more on prototyping and involves iterative design and development phases.
RAD approach follows traditional SDLC steps, but the phases are combined. Since the product has to be released in a tight time interval, thorough involvement of dedicated and well trained developers is required. Some of the issues that need to be addressed in RAD approach, particularly from the angles of performance, are (see Ames, 2005):
•
Shorter development time in development may lead to release of low quality product due to lack of concentration on performance.•
Chances of missing information due to the rapid approach may disqualify the performance objectives.•
Inconsistent internal designs like too much of cluttering of objects, sequence of screen navigation (for instance, navigating from one screen to another screen, out of which one screen may be having connections with a database and another screen may contain the presentation of data which is retrieved from the database[filling a combo box, etc.]) may be observed over time, after the product is deployed.•
High possibility of violating coding standards, and unoptimized code may con- sume too many resources.•
Module reuse for future projects may not be possible due to the project specific design.•
Module may not be flexible for graceful scaling.•
System may collapse for sudden surge in user base.In RAD, the importance is given to the functionality of the system, rather than to its performance, presumably due to the short development cycle time (see Software
lifecycles, 2004). There may be instances wherein the product may be released without
testing for its performance. In some cases, the projects may be subjected to the testing of performance, but a compromise may be made regarding the methodology to be adopted.
Technical Peculiarities
Technical peculiarities have a severe impact on the overall performance of the system. Choice of technology platform for the product plays an important role in its performance. To illustrate, developing the product on a new and untried technology with the belief that it would boost the performance may lead to other contraindications in terms of other quality attributes such as security, consistency, compatibility, and integrity. As an example, client side scripting is a new technology which can be used to reduce the number of interactions with the server. But the browser level scripting does not ensure total security. This may result in a possibility of exposing sensitive data to malicious access. Likewise, many challenges have to be faced by the development team on the technology front. These are described in the following sections.
Security Threat
Security threats to Web applications can make users worry about online transactions. Their privileged information may be leaked to the outside world. They may lose their credit card information to others and lose money.
While developing Web applications, the major focus is on the functionality rather than on security. If the Web users are able to see a security lapse in the system, then the product loses its market in spite of other interesting features because users no longer trust the Web site. Thus, a significant impact on performance of the site is directly related to the security of the applications. Threats such as SQL injection, cross browser scripting, sniffing can access the valuable, sensitive information of the user. Security and performance are usually at odds with each other. Current implementations of security on the Web have been adopted at the extreme end of the spectrum, where strong cryptographic protocols (see Paulson, 2004) are employed at the expense of performance. The SSL protocol is not only computationally intensive, but it makes Web caching impossible, thus misses out on potential performance gains.
Developer’s Negligence on Performance
It is a common practice among many developers not to optimize the code at the development stage. To illustrate, a developer may use more buffer space against a variable in the program than is actually required. This additional reserved space may later impact the performance of the system by denying space for others. Since both optimized
and unoptimized code exhibit the same functional behavior, an optimized code is preferred for better performance of the system.
Unoptimized code may utilize scarce system resources such as memory and processor unnecessarily. Such coding practice may lead to severe performance bottlenecks such as memory leaks, array bound errors, inefficient buffering, too many processing cycles, larger number of HTTP transactions, too many file transfers between memory and disk, and so on. These problems are difficult to trace once the code is packaged for deployment.
Complex User Interface
Web users appreciate an elegant, simple user interface. A complex interface would promote a user to abandon the site due to the difficulties in interacting with the site. More complexity also impacts the performance of the site. Only the necessary user identifica- tion information should be made mandatory, and all other fields should be optional. Users do not like lengthy, complex interfaces which are irritating to use.
Web Site Contents
In the early days of Internet evolution, Web contents were static information and online traffic was comparatively light. Now, Web contents are dynamic, warranting powerful Web servers and robust techniques to handle data transfer. Database driven Web sites such as e-commerce applications typically display dynamic contents. The visual appear- ance of the Web page depends on the code executed, which in turn is based on the data stored in the tables. These dynamic Web pages require more processing power that puts stress on the server.
Hyper Text Markup Language (HTML)/Text
Plain HTML/Text (see WHL2004) has negligible impact on the performance of the Web application. Storing them does not require too much space compared to other contents of the site. The organization of the text and the font may have some effect on the quality of the site. Displaying large text pages on Web sites is not good from the user point of
Table 2.0. HTML/Text type contents in a Web page
HTML/Text
JS JavaScript HTML Hyper text markup language XML eXtensible Markup Language VB Script Visual Basic Script
view. Overall, HTML/Text does not impact much on the server resources. Table 2.0 shows the HTML/Text type of contents available in a Web page.
Images
Images (see Image file formats, 2004) require more network bandwidth, storage, and CPU power than the text information. Using too many colors in images requires more storage and bandwidth. Inappropriate file formats of images can impair performance. Various performance aware techniques for the storage of images are available which require a proper evaluation for an optimized Web site. Table 2.1 shows the type of images that a Web page may contain.
Multimedia
Multimedia (see Lemay, 2004) components such as audio and video applications embedded in Web pages will have an impact on the performance of the Web site. Sound, bounded with text, takes significantly more time to download than the plain text pages. The type of encoding techniques used to store the audio and video files will have an impact on the quality and the size of the page. Proper selection of encoding techniques depending on the multimedia data can exhibit optimized Web pages. The performance of the multimedia embedded pages depends on the playback applications supported by the browsers. Table 2.2 gives sample multimedia files that a Web page may contain.
Executable Applications
The executable applications embedded in HTML pages, documents present in a Web site, also impact the overall performance of the site. Appropriate storage format and an
Table 2.1. Images that Web page may contain
Images
JPG/JPEG Joint Photographic Experts Group GIF Graphic Interchange Format PNG Portable Network Graphics BMP Bitmap
Table 2.2. Multimedia files that a Web page may contain
MPG Moving Pictures Expert Group (MPEG SWF Flash Files
appropriate display format are needed to ensure the quality and size of the pages. Some of the executable files associated with the Web page are shown in Table 2.3.
Client Environment
The environment of the client system has a significant bearing on the performance of the product. Though the contents are stored at the server end, users always closely interact with client systems to get the required information. Clients must be properly configured and connected. They must have required hardware resources with the necessary operating system. Since a designer of Web applications has no control over the client system and its resources, Web applications must work with heterogeneous clients. Thus the client’s environment may consist of different browsers, divergent platforms, and different types of security settings, multisize caching, different communication proto- cols, and different network topology, all having a significant influence on the perfor- mance of the product.
Different Browsers and Their Impact
Recent years have witnessed an in break in the number of browsers (see People use
different browsers, 2004). Unfortunately, a majority of them are not conforming to the
WWW standards. Even the population of available standard browsers in the Internet is sizable. The architecture underlying the browser and the security issues to be addressed vary from browser to browser. Due to these constraints, it is a difficult task to develop a Web product which is compatible with all standard browsers. The best example is that of the compatibility problems observed in 2000 by the Web users who have Internet Explorer (IE) and Netscape Navigator as their browsers. Users browsing with Netscape were safe from malicious access whereas users browsing with IE and Netscape installed on their machines were exposed to security threats. This happened due to the privileges granted by Microsoft’s IE browser to run a script placed on the Web user’s machine by Netscape Communicator. Due to the browser security issues, the number of online transactions drastically decreased on the Internet.
Sometimes problems arising due to usage of multiple browsers may have an impact on performance. The scripts running on one browser may not run on another browser. These browser problems can be eliminated by writing two different pages for the same
Table 2.3. Executable files that Web page may contain
PDF Portable Document Format XIP Executable in Place ZIP WinZip Compressed files
functionality. Browser redirect script could be used depending on the browser privileges. One more option is to use server scripting to understand the type of browser and display the page accordingly. Incomplete scripts on the browser may impact the performance. A browser is an important tool to access Web sites and resides on the client machine. As already said, not all the browsers available on the Internet are in conformance with WWW standards. A nonstandard browser may not be compatible for accessing all Web sites. A typical browser could be configured in many ways for optimal performance. Client machines can also be set up with different browsers to simulate the operating environment. A typical testing environment will have clients with multiple browsers. For instance, if the load has to be tested for 2,000 users, then simulation may be made for 1,000 users browsing through IE, 500 users with Netscape Navigator, and balance 500 with Java. Multiple versions could also be simulated. This is required as different versions of the browsers display the screen differently. Not just the browser, client machines can also be set with a mix of hardware and software components. It is difficult to predict the client environment for Web applications. Users of Web applications are unknown and can access from different geographical locations with different hardware and software setups. Therefore setting up an environment with a thorough mix of clients is advisable.
Different Platforms
Configuring clients across different platforms such as Windows 2000, Windows NT, Windows 98, and Linux are needed for a typical test environment. These operating systems contain the TCP/IP stack and the information required to communicate with the Internet. Each operating system implements the stack differently.
Protocol’s Impact on Performance
TCP/IP is the basic protocol needed for communicating with the Internet. It is a point to point protocol which allows the client to communicate across the Internet to different types of systems and computers. SMTP (Simple Mail Transfer Protocol) and POP (Post Office Protocol) can be used to send and receive mails in some clients. The rest can be set with IMAP (Internet Mail Access Protocol) for remote mail access activities. Variants of protocols (see Protocol, 2003) like TCP and HTTP affect performance. Main TCP variants are Reno, NewReno, and Tahoe, which have negligible impact on perfor- mance. Throughput of the TCP connection is reduced when maximum segment size of a TCP is increased. This is because the average value of the congestion window of the sender will decrease, which in turn increases the probability of a packet loss. Network traffic can be stopped because of the Initial Congestion Window (ICW) size of a TCP sender. ICW is the amount of the data the sender is allowed to send at the beginning of the connection.
HTTP parameters include parallel or simple connections, persistent connections, and pipelining. An HTTP persistent connection allows the already established connection to stay open beyond a single request response exchange. Persistent connections reduce the need for multiple TCP connections to be set up and separate down, avoiding the high
overhead for the typically short Web transaction or downloads. By sending multiple requests on the same persistent connection without waiting for other individual re- sponses from the server, performance bottleneck can be further reduced; this type of connection management by pipelining avoids the roundtrip delay of waiting for each of the responses before sending additional requests. For larger transfers, Selective ACKnowledgment (SACK) could be useful with HTTP 1.1. Web performances are improved and hence reduce page download time by this type of connection management (see Farkas & Huang, 2003).
Server Environment
Server and network setup of a Web-based system plays a major role in performance testing. Since all the applications are hosted in servers and the output of the applications are transferred to the clients via network, these form the heart of the functioning as well as performance of the system.
Servers are the repository of the services which are needed by the clients. These servers can have files, Structured Query Language (SQL) information, objects, data, information about work, and HTTP services. Different types of servers are shown in Table 2.4 designed for specific services. For instance all transaction services execute SQL requests at the server side.
Server environment plays a dominant role in performance. Setting up a proper server environment for PT is as important as in the production site.
Table 2.4. Types of servers and their services
Type of Server Services
File Server Provides file sharing services which is requested by client over a network. They are useful for sharing repositories like
documents, images and other large data objects.
Database Server Client passes SQL requests as message to the database server. The results of each SQL command are returned over the network. The code that processes the SQL request and the data reside on the same server.
Transaction Server Transaction server and the client invoke remote procedures that are residing on the server with SQL database engine. These remote procedures on the server execute a group of SQL statements called transactions. The network exchange consists of a single requests/reply message.
Object Server The client Server application is written as a set of communicating objects. Client and server communicate by means of objects using Object Request Broker(ORB) Groupware Server This addresses the management of semi structured information
such as text ,image mail, bulletin
Web Server The client and server talks using Remote Procedure Call (RPC) like HTTP. This Protocol defines a simple set of commands; parameters are passed as strings, with no provision for typed data.
Server Farm
The ever increasing traffic in the Web and the complexity of Web applications have warranted a need for multiple servers. A server farm is a group of networked servers that are housed in one location. A server farm streamlines internal processes by distributing the workload between the individual components of the farm and expedites computing processes by harnessing the power of multiple servers. The farms rely on load balancing software that accomplishes such tasks as tracking demand for processing power from different machines, prioritizing the tasks, and scheduling and rescheduling them depend- ing on priority and demand that users put on the network. When one server in the farm fails, another can step in as a backup.
As the performance of a Web site soars due to the increase in user base, the system