Unobtrusive fall detection and sensor system interoperability for telemonitoring

(1)

fachhochschule stralsund

Unobtrusive fall detection and sensor system

interoperability for telemonitoring

Martin Staemmler

Medical Informatics, University of Applied Sciences, Stralsund

contact: [email protected]

(2)

fachhochschule stralsund Rostock Wismar Schwerin Neubrandenburg Greifswald

Stralsund

Saßnitz

Mecklenburg

-Vorpommern

Campus

(3)

-

Unobstrusive fall detection

- Motivation and background - Concept and design

- Validation with volunteers and in residential homes - Event processing

- Summary

-

Sensor system interoperability for telemonitoring

- Introduction and current status - CHA Guidelines for interoperability

- Prototype implementation for assessment

- Summary

- Telemonitoring status in Germany

Unobstrusive fall detection and sensor

(4)

fall statistics

- incidence 30% for those over 65 years*

- one fall per 6 months with persons in institutional care - 20% of falls lead to servere injuries and fractures

- admission to hospitals - requiring continuous care - less independent living

- costs for health and soical system - fall reasons

- leaving the bed at night (no light, not fully wake, …) - hinderances, carpets, wheelchair, rollators

- liability issues for institutional care - potential prevention

- clarifying the fall situation

- documenting the immedicate assistance after a fall

(5)

body worn sensor systems allowing to detect - acceleration

- position - orientation - localisation

- air pressure deviation

pressure sensitive matts or floors movement detectors - infrared - ultrasound - radar camera systems - 2D - 3D

but most approaches focus on the fall situation do not allow for prevention

(6)

advantages

- fall documentation

- support prevention by alerts (e.g. nurses) disadvantages

- instrusive with regard to privacy - extensive image processing

- costs MS-Kinect

- re-use from mass-market (gaming) - 2D colour images

- 3D sensor (640 x 480 pixel)

infrared daylight independent - 3D microphone

- low price (~100€) - API available

(7)

unobstrusiveness

- camera positioned on the floor - up to 50cm above the floor

targeted events - fall

- leaving the bed / feet in front - leaving the room

- activity in the room privacy

- 3D image only

- image processing locally

image analysis based on - width and hight of object - orientation

- compactness

- identified room regions

(8)

fachhochschule stralsund no probability += 10% yes probalility += 20%

two contours with an area difference up to maximal 30% ? contours in the area

in front of the bed ?

contour height >= 75% of distance bottom edge of the bed to floor ? orientation of the contour line vertical ?

distance between contour less than 110cm ? += 30% += 30% += 20% += 20% += 20% += 20% += 20% += 20% += 10% += 10% += 10% += 10%

Probability estimation

situation: feet in front of the bed

derived from 3D image

probablities: 100% feed in front of the bed 100% activity

(9)

fachhochschule stralsund Bad G a r d e r o b e S t u hl Ti sc h T i s c h Patientenb ett Tür -ber eich P a t i ent en b et t Bettbe reich B a d -b e r e i c h

validation with volunteers - simulated scenarios

- probability adjustment in decision tree prototype system

- easy installation

- low configuration requirements (region identification)

(10)

(11)

results:

Validation with volunteers

fall no fall sum

fall detected 47 80 127

fall not detected 2 219 221

sum 49 299 348

fall detection rate (47 of 49) 96%

true positve rate (47 of 127): 37%

false-positive-rate (80 of 127): 63%

fall classification acceptable

too many false positive alarms

next steps

validate in real-life situation

(12)

Residents

´

inclusion criteria:

- age _≥ 65 years

- resident institutional care in residential home

- no major limitations in movement capability of resident

- increased fall risk identified (e.g. due to fall assessment / previous falls)

- added value for resident and care support

- resident or person in charge fully informed, signed consent - 3 resident (66-87 years old, 1-5 falls within the last 12 months)

- 2 residents in residential home

- 1 person living in her own flat with frequent care support

Validation in residential home

study:

- maximum duration 2 months

(13)

Validation with residents in residential home

room for two residents (two beds) one aggreed to support validation)

private flat of one person room with one resident

(14)

residents:

- distrust/ refusal related to camera based system - 3D depth image inconceivable

- fall detection is expected to lead to immediate support after a fall care personal:

- positive attitude, expected support of their work

recommendation / experience:

- education targeted for different end user groups - present 3D image to increase acceptance

- no negative feedback during test, even more ideas for new use-cases - identify reason for a fall

- „peace of mind“ for residents

- monitor and get to know resident`s behaviour - in-time / immediate alerts

(15)

in theory

- direct network access, internet connectivity per room - fast recruitment of residents

- good health status of the resident throughout the validation (no drop out)

Results from validation (2)

in real-life

- up-to-date residential home, but with VDSL only

- residential home in start-up phase – lots of new residents - detoriation of residents´´´´ health status

- high data volumes per week (2 TB within 2 months)

- as expected from volunteer tests: high false-positive alarm rate

(16)

fachhochschule stralsund o v e m e n t in e c t o w e r s u p p ly n p lu g g e d a rr a n g in g rn ti tu re a l fu n c ti o n in e c t b s tr u c te d ie w ( ro ll a to r) a lf u n c ti o n ri v e r K in e c t

Results from validation (3)

(17)

- rollators and wheelchairs obstructing view

adapt image analysis, identify rollators/wheelchairs

Adresssing high false-positive rate

- insuffcient coverage of area in the room by only one Kinect

use wireless door sensor to detect e.g. going to the bathroom improve existing image processing relying on single images only

allow for rules taking previous events / states into account Measures taken

door sensor for bathroom

use of Complex Event Processing for evaluation - multiple persons in the room e.g. when providing care

obtain information from care personal alert system use typical schedule of care personal activity

(18)

Prototype – system architecture (1)

event type description

fall fall of a person

activity activity in the room, e.g. by walking

feet feet in front of the bed

Fath bath door opened

exit room door opened

resident

´

s room

Kinect image processing event generation USB 3D-images

event, status, central

processing alert alert personal fall detection

(19)

server room

storing events event analysis storing alerts / alarm configuration central processing event-DB -alert-DB

Prototype – system architecture (2)

processing alert alert personnal fall detection

(20)

residential home

Smartphone workplace central processing forwarding alert LAN/WLAN LAN/WLAN nurse

Prototype – system architecture (3)

processing alert Alert personnal fall detection

(21)

Complex event processing (CEP)

- Event Processing Network (EPN) using Event Processing Agents which are able to process high rate event stream

- abstraction of events

- sensor events (object detected, e.g. door closed)

- context events (activity in the room, e.g. feet in front of the bed)

- situational event (fall, leaving the room, prolonged stay in the bath) - NEsper-Framework* used for implementation

single events complex events a bs tr ac ti on event stream event stream

(22)

Architecture complex event processing

sensors services sensor data simple events complex events with context event message Image manager CEP interface messaging

(23)

(24)

Complex Event Processing configuration

select

a.Source

from pattern

[

every

(a=

ActivityEvent

->

timer:interval

(1 sec)

and

BathOpenedEvent

->

timer:interval

(1 sec)

and

BathClosedEvent

->

timer:interval

(5 sec)

and

not

ActivityEvent

)]

(25)

(26)

Improvement achieved by CEP

Szenario pure image based analysis complex event processing

activity in the room

some fall alarms no fall alarms

knees on the floor

frequent fall alarms some fall alarms sitting on

the floor

frequent fall alarms only if in front of the camera

rollator permanent fall alarms only if in front of the camera and sitting on the rollator

blanket out of the bed

frequent fall alarms no fall alarms standing

door area

frequent fall alarms no fall alarms standing

near door

(27)

Summary: Unobstrusive fall detection

results

- valuable information from real-life validation - new problems

- new use-cases

(probably more realistic as with usual living labs) - CEP reduces false-positive alerts

- further sensors needed to improve context

todo

- continued validation with residents

- extend to more than one Kinect, e.g. for rooms with two residents - improve system stability

(28)

-

Unobstrusive fall detection

- Motivation and background

- Concept and design

- Validation with volunteers and in residential homes

- Event processing

- Summary

-

Sensor system interoperability for telemonitoring

- Introduction and current status - CHA Guidelines for interoperability

- Prototype implementation for assessment

- Summary

- Telemonitoring status in Germany

Unobstrusive fall detection and sensor

(29)

Telehealth: Telemonitoring

scenarios: - patient health status monitoring - improving patient compliance - support for patient convalescence

- patient consultation („peace of mind“, emergencies) - medical device monitoring (ICD – cardiodifibrillators) participants: - telemedicine service centers, medical professionals

patient relatives information: - vital signs

- activity

- surveillance

implementation: - proprietory medical devices - company specific services

- lack in use of (available) standards or profiles - limited integration with other provider systems

- Personal Health Monitoring (PHM) - Ambient Assited Living (AAL)

(30)

vendor

Current status: Sensor systems

hinderance

- vendor specific and proprietory (i) sensor system, (ii) communikation (iii) data formats/representation (iv) vendor services

- vendor specific user interfaces for customers and providers product line 1 product line 2 product line n PH/AAL Provider eva- lua-tiom, feed-back, coa-ching adap- ta-tion for each pro-duct line at home

Customer view: sensor system limitation/vendor, no plug & play

PH / AAL Service view: high complexity, effort and costs

*

(31)

Continua Health Alliance (CHA)

- mission*: „Continua is dedicated to establishing a system of interoperable personal con-nected health solutions with the knowledge that extending those solutions into the home fosters independence, empowers individuals and provides the

opportunity for truly personalized health and wellness management.“

- approach: - reference architecture for PHM and AAL

- interface specification, plug & play, based on standards - from sensors … to electronic health records

Sensor-/Actor-system (Agent)

Basestation

(Manager) Monitoring server EHR

patient environment (home, mobile) telemonitoring service Electronic Health Record WAN Interface HRN Interface HRN Interface PAN Interface PAN Interface WAN Interface

Continua Health Alliance Guideline:

Assessment

(32)

Continua Health Alliance Guideline:

Assessment

patient environment (home, mobile) telemonitoring service Electronic Health Record WAN Interface HRN Interface HRN Interface PAN Interface PAN Interface WAN Interface ISO/IEEE 11073-20601 Bluetooth TCP/IP HL7 PHMR IHE XDR TCP/IP IHE PCD-01 HL7 ORU^R01

Open issues to be adressed:

- guideline usability

- interface functionality (plug & play, assignment, semantic, …) - Implementation support (tools, effort, …)

(33)

Basisstation (Manager)

sensor data plus context

process to establish plug&play /

connection

• Manager requires DIM (Domain

information Model) to establish a connection to the sensor

• if DIM is unknown with the

Manager it has to be provided by the Agent

• DIM is stored at the Manager • Sensor data are represented

according to the DIM

conecton request DIM unknown DIM accepted Glucometer : MDS Gewicht: Numeric Koerperfett: Numeric Glucometer : MDS Gewicht: Numeric Koerperfett: Numeric

PAN-Interface Concept

Sensor-/Actor-system (Agent)

(34)

PAN Interface

ISO/IEEE 11073 – PHD

- 10400 Common Device Specification

Telemonitoring Health & Fitness Independent Assisted Living

< < re fi n e > > sensor-/actorsystem (Agent) base station (Manager) Transportprofile Bluetooth - HDP USB - PHDC

transport and protocols:

- Bluetooth – Health Device Profile (HDP)

- USB – Personal Health Device Class (PHDC) application level (IEEE 11073-104xx): „Common Device Specification“

ISO/IEEE 11073 – PHD

- 20601 Optimized Exchange Protocol

Domain Information Model Service Model Communication Model

protocoll (IEEE 11073-20601)

„Optimized Exchange Protocol“ provides plug & play for multiple device types

10404 Pulse Oximeter 10407 Blood Pressure 10408 Thermometer 10417 Glucose Meter 10418 Weighing Scale 10441 Cardiovascular Fitness 10442 Strength and Fitness

10471 Independent Living Activity Hub

10419 Insulin Pump

(35)

PAN Interface Implementation

- 10408 Thermometer (µController) - Bluetooth communication

(36)

fachhochschule stralsund sensor data (Agent) base station (Manager) DIM (von Waage)

patient demographic data

HL7 V2.6 ORU^R01 (IHE PCD-01) m a p Monitoring-Server sensor data, patient demo-graphic data DIM (von Waage) 11073-20601 DIM DIM sensor data

WAN-Interface Concept

(37)

WAN-Interface

base station

(Manager) Monitoring-Server

IHE Patient Care Device - 01 (PCD-01)

HL7 V2.6 compliant to IHE PCD-01

unsolicited report (ORU^R01)

application level:

DIM compliant sensor data mapped to HL7 ORU^R01 message

according to IHE PCD-01

transportprofile

IHE IT Infrastructure TF Vol. 2 Appendix V

(Webservice WS-I Basic Profile)

transport level (TCP/IP based):

Web Service IHE ITI TF Vol.2 Appendix V (SOAP 1.2, WS-I BP* und WS-I BSP*)

semantic annotation

IEEE 11073-10101 and IEEE 11073-PHD

application domain:

Semantik according IEEE 11073-10101 and IEEE 11073-Personal Health Device

(38)

fachhochschule stralsund scale: MDS SystemType = MDC_DEVICE_SPEC_SCALE SystemId = FF:EE:DD:05:04:03:02:01 DevConfigId = 0x4000 (extended Cfg) …

body weight: Numeric

UnitCode= MDC_DIM_KILO_G SimpleNuObservedValue= 72

AbsoluteTimeStamp = 2010-11-03T17:31:0000 Type = MDC_MASS_BODY_ACTUAL

body fat: Numeric

UnitCode= MDC_DIM_PERCENT SimpleNuObservedValue= 12

AbsoluteTimeStamp = 2010-11-03T:1831:0000 Type = MDC_BODY_FAT

MSH|^~¥&|FHS^080019FFFF4F6AC0ÊUI-64||FHTMZ||20101103173021||ORU^R01ÔRU_R01|MSG00001|P|2.6|||NE|AL|||||IHE PCD ORU-R01 2010^HL7Ûniversal ID^HL7

PID|||0815^^^FHTMZ^PI||Doehring^Tom^^^^^L|||M|||Zur Schwedenschanze 2^^Stralsund^^1837^Germany^B

OBR|1|ABC12345^FHS 64|DEF12345^FHS Basisstation^080019FFFF4F6AC0ÊUI-64|528399^MDC_DEV_SPEC_PROFILE_SCALE^MDC|||20101103000001+0000|20101103235959+0000 OBX|1|NM|188736^MDC_MASS_BODY_ACTUAL^MDC|1.0.0.1|72|263875^MDC_DIM_KILO_G^MDC|||||R|||20101103173121+ 0000||||^^FFEEDD0504030201ÊUI-64 OBX|2|NM|188748^MDC_BODY_FAT^MDC|1.0.0.2|12|262688^MDC_DIM_PERCENT^MDC|||||R|||20101103173121+0000||||^^FF EEDD0504030201ÊUI-64 H L 7 2 .6 O R U ^ R 0 1 m it I H E -P C D -0 1

(39)

telemonitoring centre

Personal Health Monitoring Report (PHMR) m a p p in g base station (Manager) sensor data, patient demo-graphic data p ro c e d re s T re a tm e n t p ro to c o l c o n ta c t p e rs o n s e n s o r d a ta H L 7 O R U ^ R 0 1 report, patienten demographic data, sensor data a tt a c h m e n ts EHR / PHR

HRN Interface Concept

(40)

Monitoring-Server

HL7 Personal Healthcare Monitoring Report (PHMR)

HL7 Implementation Guide for CDA R2

EHR

HRN-Interface

application level:

mapping of sensor data (including DIM) to persistent CDA R2 Docu-ment (HL7 PHMR compliant)

transport profiles

IHE XDR

IHE Cross Enterprise Document Reliable Interchange

transport profile (TCP/IP) IHE XDR (HTTP/SOAP)

semantic annotation

SNOMED, UCUM, LOINC,

IEEE 11073-10101 bzw. IEEE 11073-PHD

application domain:

(41)

HRN-Interface implementation: Tool

**Integrationsserver (EAI*)**

business process:

mapping HL7-ORU message to CDA R2 document

mapping:

segment fields of the HL7-ORU message mapped to tags of the CDA R2 document, potentially with changing the represen-tation and adding e.g. title, effectiveTime, …

(42)

MSH|^~¥&|FHS^080019FFFF4F6AC0ÊUI-64||FHTMZ||20101103173021||ORU^R01ÔRU_R01|MSG00001|P|2.6|||NE|AL|||||IHE PCD ORU-R01 2010^HL7Ûniversal ID^HL7

PID|||0815^^^FHTMZ^PI||Doehring^Tom^^^^^L|||M|||Zur Schwedenschanze 2^^Stralsund^^1837^Germany^B

OBR|1|ABC12345^FHS 64|DEF12345^FHS Basisstation^080019FFFF4F6AC0ÊUI-64|528399^MDC_DEV_SPEC_PROFILE_SCALE^MDC|||20101103000001+0000|20101103235959+0000 OBX|1|NM|188736^MDC_MASS_BODY_ACTUAL^MDC|1.0.0.1|72|263875^MDC_DIM_KILO_G^MDC|||||R|||20101103173121+ 0000||||^^FFEEDD0504030201ÊUI-64 OBX|1|NM|188748^MDC_BODY_FAT^MDC|1.0.0.2|12|262688^MDC_DIM_PERCENT^MDC|||||R|||20101103173121+0000||||^^FF EEDD0504030201ÊUI-64 H L 7 2 .6 O R U ^ R 0 1 u s in g I H E -P C D -0 1 <component>

<templateId root="2.16.840.1.113883.10.20.1.31"/> <templateId root="2.16.840.1.113883.10.20.9.8"/> <id root="975c2f3b-2bd4-4e45-aed1-84af9ff51b10"/> <codecode="MDC_MASS_BODY_ACTUAL"

codeSystem="2.16.840.1.113883.6.24"

codeSystemName="MDC" displayName="Mass Body Actual"/> <statusCode code="completed"/>

<participant typeCode="DEV"> <participantRole> <id root="1.2.840.10004.1.1.1.0.0.1.0.0.1.2680" assigningAuthorityName="EUI-64" extension="FF-EE-DD-05-04-03-02-01"/> </participantRole> </participant> code system sensor data

(units according to UCUM)

sensor device

(43)

Assessment results

- encryption support on data transport level - prototyp implementation confirmed

- standard compliant communication sensor EHR - plug & play at the PAN Schnittstelle

- semantic annotation of sensor data based on the DIM - acceptable implementation effort

sensor-/actor-System (Agent)

base station

(Manager) monitoring server EHR

Patient Environment (home, mobile) Telemonitoring Centre

Electronic Health Record

measurement data, -context

caring, coaching of the patient

events, measures, report

(44)

patient environment (home, mobile) telemonitoring service Electronic Health Record WAN Interface HRN Interface HRN Interface PAN Interface PAN Interface WAN Interface ISO/IEEE 11073-20601 Bluetooth TCP/IP HL7 PHMR IHE XDR TCP/IP IHE PCD-01 HL7 ORU^R01

open issues: - bi-directional communication with sensor - pairing sensor system – user identifiction

at which level (Manager, TM Centre via device ID) - domain model limitations for event (e.g. fall)

- extending to further transfer protocols (e.g. ANT+) - mutual authentication at PAN-level

CHA well suited to establish cross-vendor interoperability CHA guidelines support customer and provider requirements

Continua Health Alliance Guideline:

Validation and assessment

(45)

Telemonitoring in Germany

medical device monitoring (e.g. ICDs)

- well established, costs covered by health insurance - reduction of visits to cardiological out-patient sevices patient health status monitoring

- multiple projects by health insurance companies to assess and provide evidence for monitoring,

coaching, adherance, and surveillance programmes - some service and infrastructure providers

- may be contracted by patients/citiziens at own costs - no reimbursement by health insurances

but

- on-going check (till April 2013) for medical service provision using telehealth approaches

status 2013: - sensors, communication, services available - limited interoperability

- lack of evidence for specific medical scenarios - missing reimbursement for wider use

(46)

thanks for your attention

? questions ?

Unobstrusive fall detection and

sensor systems interoperability

Acknowledgement:

Unobstrusive fall detection: Christian Marzahl, Peter Penndorf, Henriette Rau, Jacob Grieger

Unobtrusive fall detection and sensor system interoperability for telemonitoring