Nordic eHealth Indicators
Nordic collaboration for “developing, testing and assessing a common set of
indicators for monitoring eHealth in the Nordic countries and Greenland, Faroe
Islands and Åland for use by national and international policy makers and
scientific communities to support development of Nordic welfare”
.
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine
eHealth indicator methodology used in the Nordic work
1.
Defining the
context
(human and environmental) for
measurement
– identifying key stakeholders and
– defining the area(s) or system(s) that are relevant to be studied.
2.
Defining the
goals
– Top-down: analysis of (eHealth) strategies and interviews of
policy makers
– Bottom-up: review of existing indicator work, peer-reviewed
literature and stakeholder priorities
3.
I
ndicator selection and categorization
.
– Generating longlist of Grouping preliminary indicators
– rating on scientific soundness and feasibility
– Rating of importance: Policy makers, practitioners, researchers
4.
Defining the
data
– Selecting measures and sources, testing and collection of feedback
from different user groups.
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian Vimarlund 2
Policy analysis,
content
analysis of
existing
surveys
eHealth
evaluation
Literature,
existing
surveys
Source: Hyppönen H, Ammenwerth E, de Keizer N.Exploring a methodology for eHealth
indicator development. Stud Health Technol Inform. 2012;180:338-42
Modified from Reed Mark S., Fraser Evan D.G., Dougill Andrew J. An adaptive learning
process for developing and applying sustainability indicators with local communities.
ECOLOGICAL ECONOMICS 59 (2006) 406–418
Interviews
for
importance
rating
Framework for indicator categorization
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian Vimarlund 3
SYSTEM
AVAILABILITY
CONTEXT OF
IMPLEMENTAT
ION
GROUP 1: IT-related
Health care input
variables
GROUP 2: IT-related
Health care process
variables
GROUP 3: IT-related
Health care outcome
variables
What should we be using as measures within EHRs in
the Nordic countries: Policy analysis results
1.
Context to be focused in
a.
Common Key stakeholder groups
: Patients, Clinicians, HC Managers, Policy
makers
b.
Key areas or systems to be focused in:
1.
Common IT-services, IT-architecture; Information security tools and rules;
Standardization, regulatory framework
2.
The goals => indicators
•
Improve access to relevant health information through IT-services for Clinicians,
Patients/Citizens, secondary users
•
Improve interoperability of health information systems
•
Improve information security and privacy
•
Improve process support in the Health Care system
•
Improve usability of health information systems
•
Improve quality, effectiveness of health care
•
Empower and activate citizens
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine
Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian Vimarlund 4
GROUP 1:
IT-related
Health care
input
variables
GROUP 2:
IT-related
Health care
process
variables
GROUP 3:
IT-related
Health care
outcome
variables
Review of existing indicator work: Analysis of national
eHealth surveys in the Nordic countries and the OECD
model survey
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine
Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian Vimarlund 5
Norway
Denmark
Sweden
Finland
What is currently being measured in the Nordic surveys/ in the OECD
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas,
Sabine Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian Vimarlund 6
Indicator / measure
Fi
No
DK
SW
OECD
IT
costs
(investment/ maintenance budget)x
x
x
Access
to EHR Systems (
proportion of organisations/clinicians/ patients having)
x
x
x
x
Access
to key Information generated within own org/ outside own org
x
x
(x)
x
x
Access
to info sharing functionalities (HIE) - proxy for System integration/
interoperability/
x
x
x
x
x
Access
to PHR functionalities
x
x
(x)
x
x
Access
to telemedicine services
x
x
IT support quality
(UX = proportion of clinicians experiencing)x
x
IT
system quality
(e.g. reliability, down-time)
(UX)x
x
Most
challenging/ successful
aspects of information systems
(UX)
x
x
x
Information quality
(UX)x
x
Intensity
of use of EHR system/ information types / HIE/ PHR
x
x
x
x
x
Usability
(UX)x
x
x
Benefits
(general)
(UX)x
x
Productivity/ efficiency gains
(e.g. avoidance of duplicate tests)x
x
x
Professional development support
(UX)x
x
x
Support for collaboration/ care continuity
(UX)x
x
x
Patient safety
(several variables)x
x
Patient centredness
(UX)x
x
x
Health care input variables (GROUP 1) HC process variables (GROUP 2) Health care outcome variables (GROUP 3)
What is covered, what is missing in Group 1 - HC input variables?
6.11.2013
Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian
Vimarlund
7
Policy goals/
survey
indicators
Existing Measures
CON-TEXT
Improve information security
and privacy
1) Proportion of organisations with an information security plan (F)
2) Proportion of organisations conducting log monitoring regularly (S)
CON-TEXT
IT costs
1)
IT investment/ maintenance budget
2)
Proportion of IT costs in organisation’s cost budget
SYST
AVAILA
BILITY
Improve access to relevant
health information through
IT-services for Clinicians,
Patients/Citizens,
sec.users
1) Proportion of organisations/ clinicians/ patients with access to
systems
/ functionalities/ information
2) Product brands
SYST Q
Improve interoperability of
health information systems
1) Proportion of organisations/ clinicians with access to key information
generated outside own organisation, HIE functionalities
2) Proportion of organisations with defined codes and classifications
implemented in EHR systems (F)
3) Varying other System integration variables in different Nordic
countries
SYST Q
Standardization activities
1) Proportion of organisations that comply to specific HIE standards (F)
SYST Q
HELP Q
Technical quality of system
IT support quality
1) Proportion of clinicians experiencing the system technically stable
(little or no down-time) (F, S)
2) Proportion of clinicians getting sufficient help with IT systems (F, S)
INFO Q
Information quality
1) Proportion of clinicians experiencing information offered by system is
Policy goals/
survey
indicators
Existing Measures
Intensity of use
1)
Usage rate (proportion of actions performed electronically)
2) Proportion or organisations Using EHR data for defined secondary
purposes (N)
Improve usability of
health information
systems
1) Proportion of clinicians experiencing system Easy to use (F, S)
2) Prop. of clinicians experiencing syst. Efficient to use (F, S)
3) Prop. of clinicians satisfied with the system (F)
Improve process support 1) Prop. Of clinicians experiencing system helping avoid unnecessary
repetition of tasks (double documentation, duplicate examinations
etc.) (F)
2) Prop. Of clinicians experiencing system saving time (S)
3) Prop. Of organisations with access to DSS functionalities
4) Proportion of clinicians experiencing that IT facilitates collaboration
(B2B and B2C) (F, S, N)
6.11.2013
Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian
Vimarlund
8
What is covered, what is missing in group 2 – HC process variables?
The focus is on intensity of use, some variables in some countries monitor
usability.
Should we also monitor e.g. compliance to documenting guidelines,
experience in improvement of adherence to clinical guidelines or clinical
decisions?
Policy goal/
survey
indicators
Existing Measures
Improve quality,
effectiveness of health
care
1) Proportion of clinicians experiencing IT enhancing care quality (generic) (F, S)
2) Different variables measuring clinicians experience of impacts on
patient safety
(F, S)
Empower and activate
citizens
1) Proportion of clinicians experiencing IT enhancing
patient centredness/
empowerment
(D, F, S)
Professional
development support
1) Proportion of clinicians experiencing IT supporting assistance of own work
performance (S, N, F)
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine
Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian Vimarlund 9
What is covered, what is missing in Group 3 (Outcome variables)?
•
Focus on health outcomes (effectiveness) and savings is lacking at present
•
Current measures are mainly generic,
health benefits are functionality/condition-specific
, e.g.
•
Defined PHR functionalities e.g. Diabetes risk test+self care instructions +care path
•
-> empowerment of patients with increased risk
•
-> impact on service demand for patients with these conditions (may increase visits, since up
to 50% of diabetics are undiagnosed)
•
->Diabetes-related morbidity (e.g. cardiovascular disease, chronic renal failure, diabetic
retinopathy)
•
Comprehensive active medication list –> medication safety (e.g. adverse drug events; drug misuse)
•
=>
need to agree on definitions of key functionalities (including maturity level), define their
intended health impacts prior to monitoring health benefits
•
measuring health outcomes requires combining info from several sources e.g. surveys (or
logs) on availability and use rate of a specific functionality, register data on service use and
population health data.
What additional measures are in early adoption
or planning stages in the Nordic countries?
At present, we are in the process of:
1. defining the common functionalities and common
measures for eHealth-related input, process (and
outcome) variables, mapping these measures against
variables defined by the OECD (availability and use)
2. updating the existing eHealth surveys to allow
comparison between the Nordic countries
3. looking for alternative sources for the input, process
(and outcome) data
4. discussing eHealth outcome (benefit) variables
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine
Where are we regarding current, planned or anticipated steps for
evaluating effectiveness of these indicators in achieving outcomes
linked to stated national healthcare goals?
•
The current eHealth policies have been analyzed and indicators
mapped against the policy goals
•
Comprehensiveness of indicators under scrutiny
•
Challenges:
– Health outcome goals generic, measurements need to be
system-and condition specific
– systems that are targeted in the eHealth policies are changing
•
National ePrescription system => National patient data archive =>National PHR –
services…
•
=> We need to define some key functionalities, for which anticipated
health outcomes can be defined (may be disease-specific)
•
=> We need to specify varying sources of the needed data, and
explore availability of it (especially health data not necessarily easily
available for monitoring purposes)
•
=>We need to collaborate with HC quality indicator specialists
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine
Next steps and future work in indicator development and
evaluation within these contexts to meet national health goals
The Nordic group work plan for the next two years:
•
Deliverable I: A stakeholder-rated list of priority variables (groups 1-2 - (3)), with
common Nordic Variables identified. Indicators constructed from the Common Nordic
Variables to be included in the Nordic surveys in 2014.
•
Deliverable II: pilot data collection on comparable variables from national surveys:
comparison of results. The deliverable contains examples of results of the data
collection on comparable Nordic eHealth indicators. (Data will not be available from all
the Nordic countries)
•
Deliverable III: Results of the data collection from log/register data on comparable
Nordic eHealth indicators (for frequent monitoring)
•
Deliverable IV: final report: lessons learned and recommendations to achieve an efficient
and easy-to-use benchmarking information results. The deliverable condenses DI-III
results, with conclusions on the first and second mandate period and a suggestion on a
reporting system.
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine
6.11.2013 Hannele Hyppönen, Arild Faxvaag, Heidi Gilstad, Gudrun Audur Hardardottir, Lars Jerlvall, Maarit Kangas, Sabine Koch, Christian Nøhr, Thomas Pehrsson, Jarmo Reponen, Åke Walldius, Vivian Vimarlund 13
