7.9 APPENDIX I - Measurement Methods
7.9.4 Component Measurement
software resource monitoring using Windows based devices, please refer to Microsoft‟s document entitled 7
„Mobile Battery Life Solutions for Windows 7‟.
8
An example of the type of data documented from a utilization record is provided below and was taken from 9
setting up a process specific „processor time % use‟ within Microsoft Windows 7‟s „Performance Monitor‟ tool.
10
The average value should fully reflect the benchmark time period.
11
Application 1 12
Process Name : EXCEL.exe 13
Duration Mins : 1:40 14
Average % Processor Time : 60%
15
Application 2 16
Process Name : wmplayer.exe 17
Duration Mins : 1:40 18
Average % Processor Time : 8%
19
To calculate an individual application‟s energy consumption using the resource data, begin by calculating the 20
application set‟s energy consumption which includes all applications. This can be achieved by following the 21
steps presented in the „Power subtraction‟ part of Section 7.6.4 „Benchmarking‟. This calculation separates 22
the applications from the OS‟s energy consumption.
23
For the above example, to assess Application 1‟s energy consumption, initially calculate the total processor 24
utilization across all applications (e.g. 60 + 8 = 68%). Next calculate the required applications overall % of 25
the measured processor utilization (e.g. for Application 1, 60/68 = 88 %). The utilization factor for 26
application 1 is thus 88% which can be used in the following equation to calculate proportional energy 27
consumption for application 1.
28 29
30
For example if , Application 1 and 2‟s energy consumption would be 31
(0.019*0.88) and (0.019*0.12) respectively.
32
7.6.5 Utilization and OS data 33
Where a device specific energy consumption measurement cannot be made using the benchmarking 34
method, the energy consumption of an application (or a set of applications) may be allocated using 35
knowledge of the OS idle and maximum power consumption together with an application utilization factor.
36
Each of these variables can use either primary or secondary data and thus represents the most flexible 37
method, although conversely represents the highest uncertainty. This section describes methods to calculate 38
each of these variables.
39
page 26
Idle and Maximum Power 1
The average idle and maximum power consumption of a device can be obtained via direct measurement or 2
by using secondary data (e.g. 139 W and 238 W). Please see Section 7.5, „OS Measurement‟, for methods 3
to obtain direct or secondary data on the idle and maximum power consumption of a device.
4
Utilization 5
Utilization values can be defined in this case as the measure of a device‟s resource being consumed by an 6
application. These values can be obtained from a direct benchmarking measurement. This section‟s 7
definition of utilization should not be confused with that given in the „Multiple Applications Allocation - 8
Utilization‟ part of Section 7.6.4 „Benchmarking‟, which calculates utilization differently. Alternatively some 9
software developers publish average utilization factors for applications which can also be utilized.
10
To measure utilization of an application or application set, a utilization only benchmarking process should be 11
carried out. Performing a utilization benchmark does not require external measuring equipment but does 12
require commonly available software utilization measurements. To complete this refer to steps 1, 3, 4, 5, 13
5.2, 6, 7, 8 and 10 in the „Local Device Benchmarking‟ part of Section 7.6.4 „Benchmarking‟ (Other steps are 14
for direct power measurements only).
15
The following example uses only a CPU utilization measurement. However, where the device is more 16
complex or the application or task is known to use other hardware components more than the CPU, an 17
investigation into the power consumption of the components within the hardware must be undertaken (see 18
section 7.9.4 „Component Measurement‟ for methods to measure component power consumption). For 19
software resource monitoring using Windows based devices, please refer to Microsoft‟s document entitled 20
„Mobile Battery Life Solutions for Windows 7‟.
21
An example of the type of data documented from a utilization record is provided below and was taken from 22
setting up a process specific „processor time % use‟ within Microsoft Windows 7‟s „Performance Monitor‟ tool.
23
The average value should fully reflect the benchmark time period.
24
Example of the type of data documented from a utilisation record 25
Application 1 26
Process Name : EXCEL.exe 27
Duration Mins : 16:40 28
Average % Processor Time : 60%
29
Application 2 30
Process Name : wmplayer.exe 31
Duration Mins : 16:40 32
Average % Processor Time : 8%
33
For example, to assess Application 1‟s energy consumption, the utilization factor would be 60%. One 34
could also bind multiple applications into one application set which for the above example would 35
equate to a utilization factor of 68%. This factor can be used in the following equation to calculate 36
proportional energy consumption:
37
For example, if the OS power consumption for Idle and Maximum had been defined as 139 w and 238 38
w then for the duration of the utilization benchmark (16:40 Mins) , 39
, for Application 1 and 2, and respectively.
40 41
page 27
7.6.6 Transaction allocation 1
This section describes an allocation method that will allow a simplified per application process or transaction 2
energy consumption to be derived. This method‟s results should be described as the efficiency of a process 3
or transaction according to use of the software and devices, not a defined energy consumption.
4
This method can be utilized in the case of a remote device such as a server that cannot be directly accessed 5
and/or in the case that limited information is available. In this case it is acceptable to report in terms of a 6
combined OS and Application power consumption; however this must be clearly communicated and justified.
7
This is the case for servers as server OSs are commonly a very small portion of overall power consumption.
8
An application or set of applications can perform multiple processes or transactions concurrently that may 9
not be directly related to the task being analyzed. Minimizing the number of processes per application when 10
performing measurements of power is desirable and decreases result uncertainty, however this may imply 11
processes or transactions are independent of each other and do not share resource (i.e. create efficiency), 12
which may not be true. Some applications are specifically designed to be most efficient when highly loaded 13
with processes. Therefore in the case where the number of processes per application cannot or should not 14
be reduced, the transaction based allocation method can be applied to an application or application set‟s 15
energy consumption measurement (obtained using methods from either section 7.6.4 „Benchmarking‟ or 16
7.6.5, „Utilization and OS data‟). This method may result in uncertain results where each transaction is not 17
equal in terms of resource use.
18
Measure the energy consumption of the application or application set using methods from either 19
Section 7.6.4 „Benchmarking‟ or 7.6.5, „Utilization and OS data‟.
20
Assess the number of transactions that are occurring within the application or application set.
21
Where a benchmark is being carried out, the definition of number of transactions falls within the 22
time boundary of the benchmark (e.g. 100 transactions in 16 minutes). Where a benchmark has not 23
been carried out, define transactions within a time boundary. We use the term transactions in this 24
case to account for multiple processes that could be accounted as one transaction, thus making 25
accounting simpler. A transaction will be one or a set of application processes that is unique to a 26
request that can be defined and be accounted for within the application. For example a CRM 27
application serving one external user request could be defined as a transaction. This assessment 28
will be highly dependent upon the task and application being assessed. It is crucial that however 29
the transaction is defined all processes within the application are accounted for. If not included the 30
total application power consumption would not be accounted for.
31
When the total number of transactions have been defined (e.g. 100 transactions in 16 minutes) 32
those that are required for analysis should be calculated in % terms of the total. For example 80 33
transactions out of 100 equates to 80% of the total transactions.
34
To allocate energy consumption to the relevant transactions multiply the percentage of task 35
relevant transactions by the relevant application or application set energy consumption kWh value.
36
For example where the application energy consumption was 0.017 kWh and 80% of transactions 37
were task relevant, the task relevant energy consumption is 0.017*0.80 = 0.0136 kWh.
38
a. Specific power consumption for the relevant tasks can also be obtained. E.g. where the 39
transaction benchmark time was 16 minutes (0.0136*1000)/(16/60) = 51 watts.
40
Energy consumption per transaction can also be calculated. This can be achieved by dividing the 41
total energy consumption measured for the application or application set by the number of 42
transactions. For example for 100 transactions in 16 minutes where the application energy 43
consumption was 0.017 kWh would equate to 0.017/100 = 0.00017 kWh per transaction 44
a. Specific power consumption per transaction can also be obtained. E.g. where the 45
transaction benchmark time was 16 minutes (0.00017*1000)/(16/60) = 0.6 watts.
46
page 28 7.7 Virtual Machine Power Assessment
1
This section describes methods to assess the power consumption of a virtualized device. Four methods are 2
described to allow the allocation of a server device‟s power to the virtual machines running within.
3
Virtualization is commonly applied to a server system within a data center environment. Therefore this 4
section is aimed at server systems; however, the method can be applied to most types of virtualization 5
software. Various virtualization software systems exist and these guidelines aim to be applicable to all.
6
7.7.1 Background and Aims of Virtual Machine Power Consumption 7
A virtualized device can be described as one that can simultaneously run more than one OS with software 8
managed resources. Commonly a virtualized device will feature a Virtual Machine Manager (VMM) that 9
allows multiple OSs to be managed and executed, each OS is known as a Virtual Machine (VM). Each VM 10
can vary in its use type and resource use and can host a variety of OSs and applications.
11
Many VMs (1000‟s) can be running simultaneously on one server; therefore, a part of the power 12
consumption of the entire server should be attributed to each VM. Whilst the host device power 13
consumption can be empirically measured, the attribution of power consumption to a particular VM requires 14
calculation. This section describes methods that can be used for various virtualized device contexts.
15
CPU Memory Disk Network
VM 1
VMM Host
VM 2 VM 3 VM 4
16
Figure 7: A host device features a VMM with many VMs. The VMM manages resource use by 17
each VM 18
19
Several assessment methodologies are presented in this section ranging from simple to advanced levels. The 20
described methods take into account the power consumption of the physical host device that the VM is 21
executed on and attributes a portion of power according to the resource use of the VM.
22
7.7.2 Scope and Goal 23
Common goals for VM power assessment are:
24
Assess the power consumption of a task running inside a VM.
25
Assessment of VM power consumption for a defined set of tasks in its own right; for example for 26
management purposes.
27
This section considers VM host devices with one or multiple VMs. The impact of power saving technologies 28
such as VM migration between hosts is not considered within this chapter.
29
Any assessment of VM power is specific to the software executed inside the VM and the host system and 30
thus non-transferable to differently configured host systems.
31
7.7.3 Overview of VM assessment methods 32
Four main methods are described in the following sections that enable the energy of a virtual machine to be 33
calculated. Due to the complexity of virtualized devices, the following section is best followed using the 34
process flow diagram in Figure 8.
35
Method 1: Virtual Machine Manager Power Monitoring 36
page 29
The first method relies upon in built device power and resource monitoring systems to allocate VM energy 1
consumption.
2
Method 2: Virtual Machine Power Calculation 3
The second method uses estimation techniques from various device power and resource statistics. This 4
method is spilt between a simplistic (Method 2a) and more advanced (Method 2b) approach dependant on 5
data availability. To improve the estimation method dynamic power and resource measurements can be 6
utilized with method 2b and is required for method 3.
7
Method 3: Isolated Virtual Machine Calculation 8
Method 3 focuses upon direct device power and resource testing for a single VM in order to get high quality 9
and detailed results.
10
Methods 2 and 3 also involve a decision between dynamic and static use profiles which is explained below.
11
Method 4: Transaction Allocation 12
A further method to calculate a single transaction or process within a VM is included, and this has been 13
described as part of the Application Measurement (section 7.6.6 „Transaction allocation‟).
14
Dynamic or Static Use Profile
Start
2a. Estimation Method 1 2b. Estimation
Method 2
Static (Simple)
Static (Intermediate)
Measure Power Consumption
Dynamic (Advanced)
Focus on one or many VMs?
Measure Resource Dynamically Measure Resource
Dynamically
3.Isolation testing Method
Multiple Transactions per
application?
Yes
Transaction allocation (Section 7.6.6) Many
One
VMM Power Monitoring
No
1. VMM Power Monitoring Method Yes
15
Figure 8: Selection of a method for assessment of a Virtual Machine 16
page 30
1 2
7.7.4 Method 1: Virtual Machine Manager Power Monitoring 3
Increasingly VMMs provide live monitoring of power consumption values per VM based upon software 4
monitors. These systems can replace the need to carry out in depth assessments and can provide good 5
results. However, information regarding the uncertainty associated with these monitoring results is not 6
widely available. Therefore, it is recommended that information regarding correct configuration and areas of 7
uncertainties is requested from the vendor of the virtualization solution.
8
Where VMM monitoring is not available, or too uncertain use the following sections to obtain a per VM 9
power consumption.
10
Dynamic or Static Resource Monitoring 11
When using methods 2 or 3, a decision between static and dynamic should be made. The static use profile 12
can only be used with method 2a. Dynamic use profiles can be used with any other method.
13
Resource utilization can be measured dynamically within the virtualized device, or can be a static estimate of 14
average use. The power consumption of the host device depends on the utilization of each VM on the host 15
device resources. Therefore, information about the levels of resource utilization by each VM is required to 16
allocate power consumption to a VM. Dynamic measurement should be preferred due to the high variation 17
that is common in a virtualized device. A more accurate estimation of resource use will result in a less 18
uncertain assessment result.
19
A static use profile is an informed guess or average measured value of likely utilization of a device‟s 20
resources relative to its capabilities. This represents the simplest and quickest process and data 21
center operators can typically provide such estimates. Hardware vendors may also be able to 22
characterize expected resource utilization depending on utilization of the server hardware 23
components. Transaction based applications will utilize the machine depending on the number of 24
transaction requests. Therefore, static resource values should represent an average or typical 25
transaction or process set. A network service such as a DHCP server will use resources periodically 26
between phases of idle wait. Therefore, utilization values in this case should represent multiple 27
states, for example idle, average and maximum periods. It should be noted that this strongly 28
impacts uncertainty associated with the results.
29
A dynamic use profile can either be a direct monitoring of in-situ device activity or can be based on 30
a benchmarking activity of the likely VM behavior such as application load test cases or network 31
traffic traces. A dynamic profile is essential to performing empirical device power consumption 32
measurements.
33
7.7.5 Method 2: Virtual Machine Power Calculation 34
This section describes two sub-methods to calculate the power consumption of a VM as a portion of total 35
device power based on the size of the VM relative to other VMs on the system. The device power 36
consumption, device utilization and VM number and size are the main inputs to this section and methods to 37
calculate these are explained. Once an average VM power consumption is calculated, the time of the 38
scenario being analyzed is required to calculate an overall energy usage.
39
The results of this method have a higher uncertainty associated due to the estimation technique used. When 40
reporting assessment results the applied assessment methods need to be documented and a note regarding 41
the uncertainty of the results should be included.
42
Before any testing is carried out, the steps below (which describe how to prepare a virtualized device) 43
should be followed.
44
page 31
Virtualized Device Preparation 1
The following basic steps should be applied to prepare a virtualized device for direct assessment. Not all 2
steps are relevant for all methods.
3
A virtualized device benchmarking profile should be defined and implemented per device. See 4
section OS preparation for steps to complete this.
5
Where possible, implement a new install of the VMM and each VM.
6
Activate and connect relevant VMM and VM hardware services (e.g. connected to the network to 7
access the internet).
8
Where relevant, run the virtualized device and ensure that any VMM to VM and internal VM 9
exchanges and initial, one time only setups are completed (unless this is what is to be analyzed).
10
If detail into each VM is required an OS task and service listing should be recorded and listed before 11
and after each test period.
12
A VMM resource and service listing is also required for some methods, this is explained per method.
13
Setting Virtualized Device Benchmarking & Use Profiles 14
Where testing is to be undertaken, a device wide benchmark profile should be defined and documented.
15
This profile should document the details of the virtualized system, such as what VMM is running and what 16
VM‟s constitute a typical scenario for the assessment. The power consumption of each VM will vary with the 17
utilization of the host device‟s resources by processes and applications. This therefore means that when 18
performing an assessment, each VM should be running a „typical‟ operational profile of the OS and 19
applications. This use profile determines the resource utilization of the host system. The processes and their 20
resource utilization over time are referred to as a benchmark application task profile. An example benchmark 21
could be running an online web shop system with a specified number of concurrent users.
22
A benchmark can either be a real workload or a synthetic benchmark composed of smaller units of work.
23
Ideally, the typical application profile of a VM will be understood from historical use. An assessment of VM 24
power consumption based on a representative benchmark will reduce uncertainty associated to the results.
25
In some cases instead of developing a custom benchmark it is recommended to use an existing benchmark 26
that closely matches the anticipated VM application. For web applications popular benchmarks are provided 27
by organizations such as TPC1, SPEC2 and SPC3. 28
29
Method 2a: Single data point power consumption (Secondary Data) 30
This method involves the calculation of the average device power consumption and the size of the VM being 31
analyzed.
32
Average Power Consumption Calculation 33
Where no primary measurement data is available average server power consumption calculation must rely 34
on secondary data. Any reported power consumption depends on the precise server configuration in 35
hardware as well as software. Data is often reported in different ways which are required to be standardized 36
to the average power consumption using the following method.
37
1 http://www.tpc.org/
2 http://www.spec.org/
3 http://www.storageperformance.org/home/
page 32
When sourcing data one should closely match the device‟s hardware. Important hardware configuration 1
aspects which impact power consumption are: the number and type of CPUs, the amount and type of 2
installed memory, the number of make of hard disks and network controllers, the mainboard model, the 3
model and number of power supply units. Important software configuration options impacting power 4
consumption are the BIOS configuration and the VM host configuration. The uncertainty associated to host 5
power values depends on the similarity of the host system in regards of those configuration options.
6
Average power consumption can be estimated from single points although this results in increasing 7
uncertainty. A common value published by manufacturers is the Maximum Measured Electricity value (MME) 8
which refers to maximum observed power consumption by a server model and can increasingly often be 9
calculated with online tools for a particular server configuration. These tools commonly allow the
calculated with online tools for a particular server configuration. These tools commonly allow the