• No results found

Application Performance Monitoring: Trade-Off between Overhead Reduction and Maintainability

N/A
N/A
Protected

Academic year: 2021

Share "Application Performance Monitoring: Trade-Off between Overhead Reduction and Maintainability"

Copied!
29
0
0

Loading.... (view fulltext now)

Full text

(1)

Application Performance Monitoring: Trade-Off

between Overhead Reduction and Maintainability

Jan Waller, Florian Fittkau, and Wilhelm Hasselbring

(2)

1. Introduction 2. Foundation

3. Performance Benchmark

4. Overhead Reduction and its Impact on Maintainability 5. Related Work

6. Future Work and Conclusions 7. References

(3)

Introduction

Introduction

I Application level monitoring introduces monitoring overhead

I Live trace processing approaches rely on high throughput

I How to achieve?

(4)

Introduction

Introduction

I Application level monitoring introduces monitoring overhead

I Live trace processing approaches rely on high throughput

I How to achieve?

→ Structured process for performance tunings utilizing benchmarks

(5)

Kieker Architecture

(6)

Causes of Monitoring Overhead

Performance Benchmark

Figure 2: UML sequence diagram for method monitoring with the Kieker framework [WH13]

(7)

Benchmark Engineering Phases

Performance Benchmark

(8)

Measured Timings

Performance Benchmark

Figure 4: Time series diagram of measured timings

(9)

Overhead Reduction Tunings

Overhead Reduction and its Impact on Maintainability

I Four performance tunings (PT1 to PT4)

I Used the benchmark for structured performance optimizations

I Goal: Low monitoring overhead and high throughput

I Every tuning is evaluated by the benchmark

(10)

Experimental Setup

Overhead Reduction and its Impact on Maintainability

I Modifying Kieker 1.8

I X6270 Blade Server with

I 2x Intel Xeon 2.53 GHz E5540 Quadcore processors, I 24 GiB RAM, and

I Solaris 10

(11)

Starting Point

Overhead Reduction and its Impact on Maintainability

No instr. Deactiv. Collecting Writing

Mean 1 176.5k 757.6k 63.2k 16.6k

95% CI ± 25.9k ± 5.5k ± 0.1k ± 0.02k

Q1 1 189.2k 756.6k 63.0k 16.2k

Median 1 191.2k 765.9k 63.6k 16.8k

Q3 1 194.6k 769.8k 63.9k 17.2k

(12)

Analysis

Overhead Reduction and its Impact on Maintainability

I High monitoring overhead in:

I Collection of data and

I actually writing the gathered data

I Expensive Reflection API calls

I Reuse of signature of operations

(13)

PT1: Caching & Cloning

Overhead Reduction and its Impact on Maintainability

No instr. Deactiv. Collecting Writing

Mean 1 176.5k 757.6k 63.2k 16.6k

Table 2: Throughput for basis (traces per second)

No instr. Deactiv. Collecting Writing

Mean 1 190.5k 746.3k 78.2k 31.6k

(14)

Discussion

Overhead Reduction and its Impact on Maintainability

I Will be used in Kieker since not impacting interfaces

(15)

Analysis

Overhead Reduction and its Impact on Maintainability

I From PT1: Queue is saturated and the monitoring thread waits for

a free space in the queue

I Target: Decrease the synchronization impact of writing data

I Optimize the communication between monitoring and writer thread

(16)

PT2: Inter-Thread Communication

Overhead Reduction and its Impact on Maintainability

No instr. Deactiv. Collecting Writing

Mean 1 190.5k 746.3k 78.2k 31.6k

Table 4: Throughput for PT1 (traces per second)

No instr. Deactiv. Collecting Writing

Mean 1 190.5k 757.6k 78.2k 56.0k

95% CI ± 3.6k ± 6.2k ± 0.1k ± 0.2k

Table 5: Throughput for PT2 (traces per second)

(17)

Discussion

Overhead Reduction and its Impact on Maintainability

I Will be used in Kieker since only impacting communication

(18)

Analysis

Overhead Reduction and its Impact on Maintainability

I From PT2: Monitoring thread is waiting for the writer thread to

finish

I Target: Decrease the writing time

I Reduce the conducted work of the writer thread

I Flat record model (ByteBuffers)

(19)

PT3: Flat Record Model

Overhead Reduction and its Impact on Maintainability

No instr. Deactiv. Collecting Writing

Mean 1 190.5k 757.6k 78.2k 56.0k

Table 6: Throughput for PT2 (traces per second)

No instr. Deactiv. Collecting Writing

Mean 1 176.5k 729.9k 115.7k 113.2k

(20)

Discussion

Overhead Reduction and its Impact on Maintainability

I Will not be used in Kieker since monitoring records now writing

bytes directly to buffers (less maintainable)

(21)

Analysis

Overhead Reduction and its Impact on Maintainability

I From PT3: About 80% spent time in collecting phase

I Target: Decrease the collecting time

I Remove interface definitions, configurability, and consistence

checks

(22)

PT4: Minimal Monitoring Code

Overhead Reduction and its Impact on Maintainability

No instr. Deactiv. Collecting Writing

Mean 1 176.5k 729.9k 115.7k 113.2k

Table 8: Throughput for PT3 (traces per second)

No instr. Deactiv. Collecting Writing

Mean 1 190.5k 763.3k 145.1k 141.2k

95% CI ± 2.0k ± 4.0k ± 0.2k ± 0.3k

Table 9: Throughput for PT4 (traces per second)

(23)

Results and Discussion

Overhead Reduction and its Impact on Maintainability

(24)

Threats to Validity

Overhead Reduction and its Impact on Maintainability

I At least one core was available for the monitoring

I Common threats of micro-benchmarks (relevance and systematic

errors)

I Different memory layouts of programs or JIT compilation paths

(25)

Summarized Tuning Results

(26)

Related Work

Related Work I Dapper I Magpie I X-Trace I SPASS-meter

(27)

Future Work

Future Work and Conclusions

I Reduce the impact of deactivated probes by, for instance, DiSL

I Generator handling the monitoring record byte serialization

I Multi-threaded versions of our monitoring benchmark

(28)

Conclusions

Future Work and Conclusions

I Proposed micro-benchmark for monitoring frameworks

I Tunings show an upper limit for the monitoring overhead

I Useful for live trace processing in the context of ExplorViz1

1

http://www.explorviz.net

(29)

Jan Waller and Wilhelm Hasselbring.

A benchmark engineering methodology to measure the overhead of application-level monitoring.

In Proceedings of the Syposium on Software Performance: Joint Kieker/Palladio Days (KPDays), pages 59–68, 2013.

References

Related documents

This study will use the most robust dataset available at the time (to my knowledge) to investigate gender differences in (1) prevalence of psychological distress and (2) the

Though livestock is still constituted the most crucial component of Borana livelihood, the survey result indicated that pastoralists are eager to reduces their livestock

Ten years ago when I was teaching my first class, the sophomore-level materials and energy balances course, I was fortunate enough to have dinner with Rich Felder and to talk about

Numerical values of equilibration temperatures and oxygen fugacities were obtained only for the Fe-Ti oxides of the Sorkka diabase and Mänty- harju gabbroanorthosite. Even with

These results, point to a more pronounced effect of the less effective peptide (CEM2) on the biophysical properties of the bacterial cell membrane model when high concentrations

Preamble Computing Introductory MATLAB Data Analysis in MATLAB Programming Tools Control Statements.

The Master’s Psychologists shall be familiar with the American Psychological Association’s Ethical Principles of Psychologist and Code of Conduct (1992), as well as the

This study has shown that building craftsmen's and project supervisors'/engineers' perceptions of the relative effects of labour-related factors on construction labour productivity