Specifying Cost - Figure 4.6 Performance Engineering process and SLAs.

Figure 4.6 Performance Engineering process and SLAs.

4.7.3 Specifying Cost

• Table of Contents

Performance by Design: Computer Capacity Planning by Example

By Daniel A. Menascé, Virgilio A.F. Almeida, Lawrence W. Dowdy

Publisher: Prentice Hall PTR Pub Date: January 05, 2004

ISBN: 0-13-090673-5 Pages: 552

Individual organizations and society as a whole could face major breakdowns if IT systems do not meet their Quality of Service (QoS) requirements on performance, availability, security, and maintainability. Corporations stand to lose valuable income, and public interests could be put at great risk. System designers and analysts usually do not take QoS requirements into account when designing and/or analyzing computer systems, due mainly to their lack of awareness about the issues that affect

performance and the lack of a framework to reason about performance. This book describes how to map real-life systems (e.g., databases, data centers, e-commerce applications) into analytic performance models. The authors elaborate upon these models, and use them to help the reader thoroughly analyze and better understand potential performance issues.

< Day Day Up >

choice of the number and capacity of servers of a datacenter affect both the performance and the cost of a system [3].

Without cost estimates, it is meaningless to discuss service level and performance objectives for a specific business. Managers need to calculate cost and estimate benefits to make good IT decisions. For example, to specify the availability goals for a credit card authorization system or a catalog sales center, one has to know the cost of a minute of downtime. Depending on the nature of the application, the average cost of one minute of downtime can be as high as $100,000 in the case of systems that support brokerage operations [19].

The Total Cost of Ownership (TCO) [20] model has been applied to evaluate the cost of a system. It means the total cost of owing a given system over some time horizon (e.g., five-year period). TCO intends to identify and measure elements of IT expenses beyond the initial cost of implementing a system. The most significant contributing items to TCO are:

Hardware costs, including acquisition expenses or leasing expenses of equipment, such as servers, storage boxes, and connectivity components.

Software costs, including personal productivity software, applications, database management, transaction monitors, intrusion detection, performance monitoring, and operating system software. Communication costs, including leased lines and communication services access.

Management costs, including network management, systems storage, maintenance, and outsourced services.

Support costs, including support services, support personnel training, end-user training, and help desk services.

Facilities costs, including leasing of physical space, air conditioning, power, and physical security. Downtime costs, including both the cost of lost productivity of employees and the cost of lost income from missed business [19].

Measuring the Return on Investment (ROI) is also critical to evaluate the cost and benefits of a system. To measure the success of the investment of information technology two factors are considered: the improvement in the quality of service (e.g., user satisfaction and system use) and the return on

investment (e.g., increased productivity and organizational impact). Thus, ROI methodologies are useful for analyzing the cost/benefits of IT projects.

Based on the requirement analysis of a system, one should develop a cost model to understand the cost/benefits of the system. When different system choices are evaluated, one needs to predict how much additional computing and communication resources will be needed for each choice and how much these elements cost now and in the future.

Example 4.6.

Consider the call center system described in the motivating example. Management is planning to replace the database server with a powerful cluster of servers. Two brands of clusters are being compared, system Y and system Z. What factors should be considered in the process of choosing between the two systems?

First, IT management is considering the TCO model, instead of the pure purchase cost of the system. Thus, the basic cost of both systems includes hardware and software costs, hardware and software maintenance, operational personnel, and facilities costs for a three-year period. System Y costs $300,000 and system Z costs $350,000.

Second, IT management is looking at the performance attributes of both systems. Standard benchmarks indicate that the throughput of systems Y and Z are 220 tps and 230 tps with 90% of the transactions responding in less than 0.5 sec, respectively.

< Day Day Up >

• Table of Contents

Performance by Design: Computer Capacity Planning by Example

By Daniel A. Menascé, Virgilio A.F. Almeida, Lawrence W. Dowdy

Publisher: Prentice Hall PTR Pub Date: January 05, 2004

ISBN: 0-13-090673-5 Pages: 552

< Day Day Up >

Third, IT management is evaluating other aspects of computing in their analysis. In addition to

performance, management also consider the dependability of the systems [19]. Information obtained on the dependability of the two systems suggests an expected 38 hours of unavailability for system Y and 21 hours of downtime for system Z, over a period of three years.

The call center charges $5 per call from customers. The company estimates that in the next three years, the average rate of telephone calls into the call center will be 1,000/hour. Therefore, the estimated average cost per hour of downtime due to revenue loss is $5,000. The total cost of a system is calculated as:

Using the above expression, the total cost for a three-year period for the two systems is:

Cost of system Y = 300,000 + 38 x 1000 x 5 = $490,000 Cost of system Z = 350,000 + 21 x 1000 x 5 = $455,000

The above methodology helps management justify selecting system Z. Although Z is more expensive initially, it is more dependable, which makes its total cost more attractive.

• Table of Contents

Performance by Design: Computer Capacity Planning by Example

By Daniel A. Menascé, Virgilio A.F. Almeida, Lawrence W. Dowdy

Publisher: Prentice Hall PTR Pub Date: January 05, 2004

ISBN: 0-13-090673-5 Pages: 552

< Day Day Up >

4.8 Concluding Remarks

The principle concepts of performance engineering are presented in this chapter. A motivating example is described involving the planning, development, and implementation of a call center application. Five examples of how performance considerations should be included in the analysis, design, operation, and evolution of a system are presented. The main issue in performance engineering is how to guarantee that a system will meet its performance goals. The steps required to carry out a performance engineering effort are described. A central issue in the methodology are the models needed to represent workload and the performance of a system. The basic steps of a practical performance engineering methodology include the following.

Specify the system performance goals. Understand the current environment.

Perform a workload characterization and generate a workload model. Develop a performance model.

Verify and validate the system model, which is composed of the workload and performance models. Forecast workload growth.

Use the system model to analyze the performance of different system architectures and various workload scenarios.

Select the best alternative (based on the performance model predictions) that presents the best cost-performance relation while satisfying the service levels specified.

< Day Day Up >

• Table of Contents

Performance by Design: Computer Capacity Planning by Example

By Daniel A. Menascé, Virgilio A.F. Almeida, Lawrence W. Dowdy

Publisher: Prentice Hall PTR Pub Date: January 05, 2004

ISBN: 0-13-090673-5 Pages: 552

< Day Day Up > < Day Day Up >

4.9 Exercises

Consider the motivating example of Section 4.3. Describe the performance requirements of the system. Be as precise as possible.

Assume that the design phase of the call center application has been completed. Analysts are now working on the development phase. Describe the workload model for the two stages and comment on the additional information that could be added to the model during the development phase.

Consider the call center motivating example and the data provided in Example 4.4. What is the maximum throughput that can be achieved by the database server?

In the architecture of the motivating example, the application server receives and executes the functions submitted by the representatives. Suppose that the application server has 150 active processes that service the received functions. The analyst monitors the system and verifies that an application process executes for an average of 550 msec between database server requests. The database server handles 120 requests/sec. What is the average database response time?

In Example 4.4, the workload model consists of two classes of transactions. Aggregate these two

classes into a single class. What is the bottleneck of the server? What is the maximum throughput achieved by the server, in terms of the aggregated transaction?

• Table of Contents

Performance by Design: Computer Capacity Planning by Example

By Daniel A. Menascé, Virgilio A.F. Almeida, Lawrence W. Dowdy

Publisher: Prentice Hall PTR Pub Date: January 05, 2004

ISBN: 0-13-090673-5 Pages: 552

< Day Day Up >

Bibliography

[1] V. A. F. Almeida, M. Crovella, A. Bestravos, and A. Oliveira "Characterizing reference locality in the WWW," Proc. IEEE/ACM International Conference on Parallel and Distributed System (PDIS) , December 1996, pp. 92–107.

[2] V. A. F. Almeida and D. A. Menascé, "Capacity planning: An essential tool for managing Web services," IEEE IT Professional, vol. 4, no. 4, July / August 2002.

[3] D. Ardagna and C. Francalanci, "A cost-oriented methodology for the design of Web-based IT architectures," Proc. 17th ACM Symposium on Applied Computing , Madrid, March 2002.

[4] M. Barbacci, T. Longstaff, M. Klein, and C. Weinstock, "Quality Attributes," Technical Report CMU/SEI- 95-TR-021, Software Engineering Institute, December 1995.

[5] R. Bodnarchuk and R. Bunt, "A synthetic workload model for a distributed file system," Proc. 1991

SIGMETRICS Conf. Measurement Comput. Syst. , ACM, May 1991.

[6] M. Calzarossa and G. Serazzi, "Workload characterization," Proc. IEEE, vol. 81, no. 8, August 1993. [7] R. Dumke, C. Rautenstrauch, A. Schmietendorf, and A. Scholz (eds.), Performance Engineering: State

of the Art and Current Trends, LNCS, vol. 2047, Springer, 2001.

[8] D. Ferrari, G. Serazzi, and A. Zeigner, Measurement and Tuning of Computer Systems , Prentice Hall, Upper Saddle River, New Jersey, 1983.

[9] D. Gifford and A. Spector, "The TWA reservation system," Comm. ACM, vol. 27, no. 7, July 1984. [10] C. Koch, "Put IT in Writing," CIO Magazine, November 15, 1998.

[11] M. Klein, "State of the Practice Report: Problems in the Practice of Performance Engineering," Technical Report CMU/SEI-95-TR-020, Software Engineering Institute, February 1996.

[12] D. Levine, P. Ramsey, and R. Smidt, Applied Statistics for Engineers and Scientists: Using Microsoft

Excel & MINITAB, Prentice Hall, Upper Saddle River, New Jersey, 2001.

[13] D. A. Menascé, V. A. F. Almeida, R. Riedi, F. Pelegrinelli, R. Fonseca, and W. Meira Jr., "In search of invariants for E-business workloads," Proc. 2000 ACM Conf. in E-commerce , Minneapolis, Minnesota, October 17-20, 2000.

[14] D. A. Menascé and V. A. F. Almeida, Capacity Planning for Web Services: Metrics, Models, and

Methods, Prentice Hall, Upper Saddle River, New Jersey, 2002.

[15] D. A. Menascé, B. Abrahão, D. Barbará, V. A. F. Almeida, and F. Ribeiro, "Characterizing E-Business Workloads Using Fractal Methods," J. Web Engineering, Rinton Press, vol. 1, no. 1, 2002, pp. 74–90. [16] D. A. Menascé, V. A. F. Almeida, and L. W. Dowdy, Capacity Planning and Performance Modeling:

From Mainframes to Client-Server Systems, Prentice Hall, Upper Saddle River, New Jersey, 1994.

[17] R. B. Miller, "Response time in man-computer conversational transactions," Proc. AFIPS Fall Joint

Comp. Conf., 33:267–277, December 1968.

[18] J. Nielsen, Usability Engineering, Morgan Kaufmann, San Francisco, California, 1994.

[19] D. Patterson, "A simple way to estimate the cost of downtime," Proc. 16th Systems Administration

Conference (LISA '02), USENIX, 2002.

< Day Day Up >

• Table of Contents

Performance by Design: Computer Capacity Planning by Example

By Daniel A. Menascé, Virgilio A.F. Almeida, Lawrence W. Dowdy

Publisher: Prentice Hall PTR Pub Date: January 05, 2004

ISBN: 0-13-090673-5 Pages: 552

< Day Day Up >

[21] B. Schneiderman, "Response time and display rate in human performance with computers,"

Computing Surveys, ACM, vol. 16, no. 3, September 1984.

[22] E. Shriver, Performance Modeling for Realistic Storage Devices, Ph.D. dissertation, Computer Science Dept., New York Univ., May 1997.

[23] P. Singleton, "Performance modeling - what, why, when and how," BT Technology Journal, vol. 20, no. 3, July 2002.

[24] C. U. Smith, Performance Engineering of Software Systems , Addison-Wesley Publishing Company, Inc., 1990.

[25] C. U. Smith and L. G. Williams, Performance S olutions: a Practical Guide to Creating Responsive,

Scalable Software, Addison-Wesley Publishing Company, Inc., 2002.

• Table of Contents

Performance by Design: Computer Capacity Planning by Example

By Daniel A. Menascé, Virgilio A.F. Almeida, Lawrence W. Dowdy

Publisher: Prentice Hall PTR Pub Date: January 05, 2004

ISBN: 0-13-090673-5 Pages: 552

< Day Day Up >

In document Performance by Design - Computer Capacity Planning by Example (Page 101-108)