Measurement indicators are ‘containers’ for the definitions and values of measurement data defined and captured by agencies to support their business operations.
4.8.1 Quantitative versus qualitative
There are two types of data: qualitative data and quantitative data. The type of data that will result from measuring an attribute of an entity will depend on the sub-type attribute being measured, the entity, and the focus of the domain in which the measurement is being defined.
For example:
The cost (attribute) of a technology entity (sub-type) as an input (domain) will be a quantitative value, whereas the effectiveness (attribute) of an information entity (sub-type) as an input (domain) will be a qualitative indicator.
Qualitative data is a categorical measurement expressed by means of a natural language description; for example, temperature = ‘hot’. The categories applicable to a measurement indicator may have an order applied to them, which can be either natural or artificial.
When the ordering of the categories is artificial, they are referred to as nominal categories. Examples might be gender, race, religion or sport. When the categories have a natural order, they are called ordinal categories (or variables).
Categorical variables that are ordinal include those that judge size (small, medium, large etc.) and customer attitudes (strongly disagree, disagree, neutral, agree, strongly agree), but without additional information it is difficult to ascertain which of the values is the best or worst.
Quantitative data is a numerical measurement expressed not by means of a natural language description, but rather in terms of numbers, but not all numbers are continuous, measurable and quantitative. For example, a Medicare or Centrelink reference number is a number, but not something that one can add or subtract with any integrity.
Quantitative data is always associated with a scale measure (or ratio scale variable), such as:
currency: dollars and cents
temperature: Celsius scale
distance: metres, kilometres
weight: grams, kilograms, tonnes
time: seconds, hours, years etc.
4.8.2 Types of measurement indicator
The PRM framework is designed to accommodate three forms of measurement indicator:
1. First order indicators, also known as primary indicators, are basic quantitative measures that describe a single attribute (dimension) of an entity only in a single domain of the PRM.
2. Second order indicators, also known as secondary indicators, are complex quantitative measures that combine and compare multiple first order indicators from multiple PRM domains, or a single PRM domain, to describe an attribute of an entity in a single domain of the PRM.
3. Higher order indicators are qualitative (subjective) measures derived from the combination of multiple first and second order indicators. They are possibly represented as an index value or matrix-based
95
95
First order Second order Higher order
Definition Measurements are taken within a single
measurement domain
One-dimensional
Non-composite (simple) unit of measurement
Describes an entity without intent.
Multiple measures from two measurement domains
Two-dimensional
Two or more first order indicators
Composite unit of measurement
Describes an entity with intent.
Measurements taken from two or more measurement domains
Multidimensional representations
Agency indices that factor multiple first and second order indicators
Units of measurement are invented.
Type of data Quantitative Quantitative Qualitative
Domain
suitability Inputs, outputs Work, usage, inputs, outputs Inputs, outputs, work usage, outcomes
Example Price
Duration
Volume
Process efficiency
Program effectiveness
Agency efficacy
SWOT rankings
Risk assessments
Project health
Vehicle reliability
Process flexibility
Grades
Cycles/second
Kg/unit
0 and 1 (balanced scorecard)
High, low, medium (risk)
Hot, moderate, cool
96
4.8.3 Indicator framework
What a measurement indicator will look like (what attributes it has) is largely at the discretion of the agency, and is likely to be dependent upon the agency’s motivations for establishing the measurement indicator (for
example, for conformance or performance evaluations).
This section provides a framework for measurement indicator definition (Figure 4-34). Not all agency measurement indicators will define values for every attribute of the indicator framework. If the indicator is intended to operate as an output performance indicator, it will contain attributes such as ‘baseline value’, ‘target value’, ‘schedule’ and ‘current value’, whereas an output conformity indicator is likely only to include attributes such as ‘target value’ and ‘current value’, and will not specify a ‘schedule’ for measurement, as compliance for an output is measured only on delivery.
Figure 4-34: Measurement Indicator Framework
Measurement Indicator Rationale
Unit of measurement The unit of measurement attribute holds the unit of measurement of the measurement indicator. A unit of measurement can be a composite measurement unit, such as:
cycles/second
complaints/customer
customers served / hour
dollars/unit
output/dollar.
or a base measurement unit, such as:
percentage
dollars
hours.
Method of measurement This attribute describes the method of measurement to be employed in the gathering of data and gives an explanation of what the results of measurement mean.
Baseline This indicator attribute contains the measurement value at the commencement of a business initiative and will be used as the comparison value for conformance and performance determinations.
97
97
Measurement Indicator Rationale
compared to it and a performance determination can be made.
Current value Measurements that are intended to be taken in accordance with a schedule over time will involve the capturing of a measurement value for the current value multiple times.
Subsequent measurements will replace the pre-existing values of this attribute.
Schedule The schedule attribute entails a list of times and dates when the measurement is to be taken. The schedule attribute is most applicable in the measurement of outcomes, as outcomes are realised through the utilisation of outputs over time.
While outputs such as public infrastructure are capable of providing benefits indefinitely, it is important to define a target date by which target levels of benefits must be secured from an initiative.
Trend The trend attribute indicates the direction of movement in a measurement indicator value: increasing, decreasing or stable.
Forecast The forecast attribute indicates the predicted value of the metric at a prescribed time, or multiple times, in the future.