Performance
Monitoring for the
Java
™Virtual
Machine (JVM
™)
Brian Doherty, Ben Kim, David StoutamireJ ava P erformance E ngineering S un Micros ys tems , Inc.
Presentation Goal
L earn about non- intrusive performance
monitoring and the tools available for
monitoring production systems utilizing
the J ava
™software stack.
Speaker’s Qualifications
•
B rian, B en, and D avid are all members ofthe J ava™ P latform P erformance E ngineering group at S un Micros ys tems
•
W e represent expertis e in performancemeas urement, analysis, and tuning of industry standard benchmarks, cus tomer applications, the J ava™ clas s libraries , and the J ava
Performance Measurement—
A Product Lifecycle Process
D on’t stop monitoring your application
just because it’s been deployed!
O bserve your production applications
using minimally invasive tools that are
becoming better integrated across the
software stack.
Agenda
•
W hat is performance monitoring•
Monitoring J ava™ applications─ J ava™ s oftware stack
─ D is tributed applications
•
D emonstration─ T ools
─ A nalys is
Monitoring Goals
•
Maintain and manage system health– D etection of performance events
– D etection of application progres s or livenes s
• P erformance tuning
– D rilling down to find caus es
Performance Management Process
Monitor Start Analysis Design Code Deploy Profile No Yes Performance OK BenchmarkProfiling Vs. Monitoring
• P roduction vs. D evelopment
– P roduction s ys tems have more res trictive
meas urement needs
– P roduction loads vary over time
Profiling Vs. Monitoring
Profiling Monitoring
Intrusion High Low
D etail High Low
A ccuracy High Low
Complexities
Hardware Operating System Virtual Machine Application Server ApplicationComplexities
C lient T ier W eb T ier J 2E E ™ A pplication S erver T ier D atabase T ierT he D eployment A rchitecture
Monitoring Tools
•
Network management s ystems• O S monitoring tools
• J V M monitoring tools
What We Saw
•
S ymptoms─ E xces s ive memory utilization
─ R estart
•
T ools s een here─ R untime.freeMemory()
─ gcore
─ S erviceability A gent
Other Tools
•
P roduction─ - verbos e:gc
•
D evelopment─ - verbos e:gc
What We Saw
•
S ymptoms─ Incons is tent respons e times
•
T ools s een here─ V isualG C
Other Tools
•
P roduction─ - verbos e:gc
─ mps tat—identify s ingle threaded G C
─ ps tack—print O S thread dump
─ S IG INT —print J ava thread dump
•
D evelopment─ - X prof
What We Saw
•
S ymptoms─ P oor s ys tem utilization
─ L ow throughput
•
T ools used─ Y A X (perfmeter)
• C P U only ~50% utilized
─ S IG INT (thread dump)
Other Tools
•
P roduction─ B ytecode ins trumentation
─ prstat / top
─ ps tack
•
D evelopment─ - X prof
What We Covered
Hardware Operating System Virtual Machine Application Server ApplicationWhat We Covered
• UNIX stat tools
– vmstat, ios tat, mps tat, nets tat ks tat
• UNIX proc tools
– pstack, pmap, prs tat/top, trus s /s trace
• Hardware counters
– cpus tat, bus s tat • Misc. tools
– S E T oolkit
Monitoring the J ava
™S oftware S tack
Hardware
Operating System
Virtual Machine Application Server
What We Covered
• jvmstat tools
• C ommand line options
– - verbos e:[gc|class |jni]
– - X prof – - X X options • S erviceability A gent • J S R 174 interfaces Hardware Operating System Virtual Machine Application Server Application
What We Covered
• J MX interfaces
– S NMP , C IM/W B E M
• B ytecode instrumentation
– V arious commercial tools • J V MP I, J V MD I tools
– C ommercial and free
Hardware
Operating System
Virtual Machine Application Server
Application
Summary
•
P erformance measurement needs to be a product lifecycle process─ Not jus t a development proces s
•
Non- intrusive tools are critical formeas urement of production applications
•
T ools integrated across the s oftware s tack eas e the measurement and analys is proces sResources
• HotS pot performance documentation
http://java.sun.com/docs/performance • jvmstat http://www.sun.com/developers/coolstuff • S olaris™ performance documentation http://docs.sun.com
• J S R 174: Monitoring and Management
