Virtual Patients in Clinical Medicine
Altion SIMO (1), Marc CAVAZZA (2) and Ryugo KIJIMA (1) (1) Virtual Systems Laboratory, University of Gifu
1-1 Yanagido, Gifu-shi, Gifu, 501-1193, Japan (2) School of Computing, University of Teesside,
Middlesbrough TS1 3BA, United Kingdom
[email protected], [email protected], [email protected]
Abstract. Virtual Patients are an essential component of medical virtual reality. Most work to date has been dedicated to virtual patients in surgery. However, virtual patients also have a strong potential in clinical medicine, where they can be used as visual interfaces to knowledge-based systems in various simulation and training applications. In this paper, we describe the implementation of a virtual patient in the field of cardiac emergencies and propose directions to extend the development of virtual patients towards the integration of multiple physiological sub-systems.
1. Introduction
Like in many other areas of Virtual Reality, the development of virtual humans for medical applications is becoming an important research topic. Virtual Patients can be part of realistic simulations which include visual, interactive and knowledge-based aspects. In the past, most work on virtual patients has been dedicated to surgical applications, where the emphasis was on anatomical knowledge and physical simulation. The only exception to date has been the work of Badler et al. [1] on the simulation of battlefield casualties, which used state-based representations to model the patient’s condition evolution.
Virtual patients in clinical medicine can be conceived of as visual interfaces to knowledge-based systems, simulating clinical situations from first principles. They offer the potential to embody the medical knowledge in a realistic context, supporting many forms of training and simulation.
In this paper, we describe the development of a 3D virtual patient. The system is implemented using a 3D game engine [2] (in our case, Unreal Tournament™ 2003) to support visualisation and interaction, which communicates with various simulation modules. Our application targets virtual patients for the ER (Figure 1). Most research in virtual patients has actually been carried in this context, largely due to the importance of proper training in the subject and the added value for training of dramatic visual simulations [1] [3] [4].
In the next sections, we describe the various techniques used for visualisation and simulation and introduce a strategy for the development of virtual patients in terms of integration of physiological models.
2. Generating Visual Signs from Physiological Simulations
The main idea behind virtual patients is that their appearance and behaviour should be determined by the pathophysiological models simulating the course of disease. The trainee will have to actively observe the patient to observe clinical signs, which can be used for diagnosis as well as assessing the gravity of the situation. Visually accessible signs can be decomposed into functional signs and inspection signs [5].
Figure 1. Patient Visualisation in the ER.
Functional signs are related to the patients’ behaviour and can be simulated using appropriate animations (e.g., for shortness of breath). Patient behaviour comprises functional signs such as the level of consciousness and spontaneous movements, pain and breathing patterns, etc. Mapping the results of the cardiac simulation to these signs involves simple inference from parameter values to the patient reaction. For instance, as cardiac output (CO) decreases so does cerebral perfusion: low CO values can thus be mapped to patient consciousness in the form of changing animations. Very high ventricular filling pressures result in elevated pulmonary capillary pressure, which can trigger cardiac edema, manifesting itself in the patient through altered breathing animation patterns.
Inspection signs are symptoms directly visible on the patients, such as sweat, edema, cyanosis, pallor, rashes, etc. These can be simulated using various forms of dynamic textures (supported by the Unreal™ visualisation engine) on the basis of the patient’s state as obtained by the pathophysiological simulation. Skin tones depend on peripheral perfusion. Hence, vasodilation (decreased Systemic Vascular Resistance, SVR) is accompanied by redness, while vasoconstriction results in intense pallor. The value of the SVR parameter is thus mapped to dynamic textures shifting between various skin appearances (Figure 2). Additional skin tones are implemented using commercial image processing packages in order they help the diagnosis of certain shock syndromes when they occur (rashes, purpura).
In addition, in the case of Emergency Medicine and critical care, monitoring devices (Figure 1) visualise in real-time vital physiological parameters, such as Heart Rate (HR) and Mean Arterial Pressure (MAP). The simulation will give direct access to these parameters by mapping the simulation’s variables to pre-set range values for HR and MAP. Other elements of visualisation include EKG curves and pressure curves (Figure 1).
Figure 2. Patient appearance in various situations: cyanosis (left), normal (centre), pallor (right).
3. Knowledge Representation and Physiological Modelling
The simulation of internal organs’ behaviour in a virtual patient should be based on physiological and pathophysiological models [6] [7] [8]. Physiological knowledge is most often represented through physiological laws. These tend to be described independently of one another (and “all other things standing equal”). One difficulty for knowledge representation is to integrate the various dependencies between the determinants of physiological functions into processes that would reflect integrated events, e.g. in the cardiac cycle. Some representations can assist in this integration process, such as, in the case of circulatory physiology, the Pressure-Volume curve representation for the cardiac cycle, as it identifies events related to ventricular filling, ejection, etc. Our virtual patient uses knowledge-based rather than numerical simulation for the organ systems. This has many advantages, which have been previously described [9] [10] [11], in particular the ability to provide explanations and to integrate different types of models, described at various levels of description. The Artificial Intelligence formalism we use is derived from work in Qualitative Physics and is known as Qualitative Process Theory (QPT) [12].
(defun Starling ()
"Method enforcing Starling's law " (let* ((d-EDV (physio_var-variation VTD)) (EDV (physio_var-value VTD)) (old-EDV (- EDV d-EDV)) (p (calcul_p)))
(unless (= d-EDV 0)
(cond ((and (>= EDV p) (>= old-EDV p)) nil) ((and (<= EDV p) (<= old-EDV p)) (let ((delta (- EDV old-EDV))) (set-DVS delta)))
((and (<= EDV p) (>= old-EDV p)) (let ((delta (- EDV p)))
(set-DVS delta)))
((and (>= EDV p) (<= old-EDV p)) (let ((delta (- p old-EDV))) (set-DVS delta)))))))
{Main Process Ejection [Process Frank-Starling (…)
I++(Contractility (I+(TeleDyastolic Volume, SV))] [Process Contractility (…) I+(Contractility, SV) ] [Process Afterload (…) I-(Afterload, SV)] [Process Cardiac-Pump (…)
I-(SV, TeleSystolic Volume)] }
Qualitative Process Theory represents processes by identifying the causal relations between qualitative variables, i.e. variables corresponding to discrete orders of magnitude. As such, it is well adapted to the representation of physiological laws. These are formalised using a set of
influence equations, which describe the evolution of one variable in response to variations of the others. For instance, the Frank-Starling relation, relating Stroke Volume (SV) to preload, can be represented by the following influence equation (where preload is approximated by the End-Diastolic Volume): I+(SV, EDV). A qualitative process encapsulates several influence equations as well as qualitative proportionalities, which relate variables whose reciprocal influence is not based on a physiological law (e.g. the relation between amount of fluid and its volume). A fragment of a qualitative process (in pseudo-code) for the cardiac ejection process is represented on Figure 3, together with the Common Lisp code representation of one of its methods.
Through the use of multiple influence equations, it is possible to model multiple dependencies, such as those occurring in Frank-Starling law (e.g. influence of inotropic levels). To represent integrated events in the cardiac cycle, various processes are encapsulated into macro-processes: for instance the ejection process controls various sub-processes such as the Frank-Starling process, the afterload-effect process, the inotropism-effect process, etc.
One specific difficulty is to represent physiological concepts such as cardiac reserve, or similar concepts such as reserve of preload: though at the stage the latter is not explicitly represented, it is taken into account by the dynamics of the influence equations governing the Frank-Starling representation (“breaking point” p in Figure 3).
4 Results for the “Core” Cardio-vascular Model
The cardio-vascular system model has been fully implemented [13] and is able to simulate various short-term evolutions of cardiac output under different pathological circumstances, which are created by primitive alteration of physiological parameters (e.g., blood volume, inotropism, intrathoracic pressure, etc.). It produces correct simulations (with an accuracy of qualitative value on the 10-value qualitative spectrum) for the following classical conditions: cardiogenic shock, cardiac tamponade, anaphylactic shock, septic shock, primitive tachycardia, hemorrhagic shock, etc.
The system operates by simulating the various physiological processes following an ordering based on the cardiac cycle. For instance, in the case of cardiogenic shock (Figure 4), the primitive variables decreased are inotropism and relaxation. The simulation of the ejection process will result in a decreased stroke volume (through influence equation I+(SV, inotropism) and in an increased residual telesystolic volume. The simulation of the arterial system will compute a new (decreased) value for the Mean Arterial Pressure and trigger compensatory mechanisms (baroreceptors, which affect various parameters including venous tone). The next process simulated, venous return, shows an increased venous return due to baroreceptor
response. Next, the ventricular filling process calculates an increased ventricular filling volume (End-Diastolic Volume) due to both increased venous return and residual volume. The
simulation of the ejection process in the next cycle would fail to improve ejection (stroke volume) despite the increase in prelaod, due to the low inotropic state (this is a consequence of the formulation of the Frank-Starling relation). As a result of this and the limited compliance of a failing heart, the End-Diastolic Pressure increases further. The next simulation cycle reaches a steady state (CO ↓↓, MAP ↓↓, EDP ), which causes the system to output the final results to
the virtual environments (showing values on monitors and generating a new patient’s appearance).
In addition, it is also able to simulate some basic physiological adaptation mechanisms, which do not correspond strictly speaking to pathological conditions. For instance, how the cardio-vascular system would adapt to an isolated increase of afterload: the stroke volume is decreased and the end-systolic volume increased. As a result, preload (EDV) increases during the next cardiac cycle, which restores stroke volume. In a similar fashion, it is possible to model the consequences of artificial ventilation, which is simulated using an increase in intrathoracic pressure, which would decrease ventricular venous return hence ventricular filling. A near-normal situation can be restored by mild fluid expansion (on a normal heart).
Figure 4. Patient in cardiogenic shock (left) and recovering after proper treatment (right).
5. Towards Full Virtual Patients: Development and integration strategy
The key idea in developing a complete virtual patient is to be able to integrate various physiological sub-systems that could support the modelling of a wide range of pathological situations. The main challenge is that these sub-systems differ greatly both in their nature and their level of description. From a knowledge representation perspective, classical physiological models appearing in the literature of in Medical textbooks [8] differ widely in their use of concepts, states and the nature of the causal links that relate them.
Our strategy for the development of a virtual patient consists in using a central physiological model around which to experiment with the integration of more sub-systems (Figure 5). This central model, originally developed for the simulation of shock states, describes circulatory physiology, with an emphasis on left ventricular physiology and short-term regulation of arterial pressure. Model integration takes place at the level of key variables in the central model, such as blood volume, inotropism, systemic vascular resistance, etc.
In order to devise a principled integration method, we have classified the sub-systems to be integrated into the following categories:
Connected pathophysiological models. These represent global evolution of the cardio-vascular system in certain pathologies in addition to the simulation produced by the qualitative model. One example is the peripheral model of shock states. This model is essentially a pathophysiological model whose nodes correspond to states and links between states represent a causal as well as temporal relation. The rationale for using here a model directly representing
pathological states, rather than deriving these from first principles is that it enables to work at a higher level of description (without having to model micro-circulation and tissues metabolism).
External pathophysiological models. These describe the evolution of a disease in an organ system which is not part of the circulatory system but can affect it under pathological circumstances. One example consist in modelling events associated with acute pancreatitis, and its associated shock state (which is interfaced at the level of blood volume and inotropism).
Connected sub-systems. This is for instance the case of the coronary circulation, which can be modelled through a causal network involving concepts such as oxygen consumption, cardiac wall tension, etc. It can be interfaced to the cardiac system at the level of inotropism, heart rate, etc. Other examples include Starling’s law of filtration (determining the occurrence of edemas, etc.).
Additional Organ Systems. These represent all remaining physiological systems in their interconnection with the circulatory system. Examples are the respiratory system and the renal system in its long-term control of arterial pressure. They can be interconnected at various levels with both the core models and the other sub-systems as described above.
Heart Rate Wall Tension MV02 Inotropism C. Flow Ischemia Tissue ischemia Venous pooling
Cardiac depression Decreased venous return Decreased cardiac output
… …
Tissue ischemia Venous pooling
Cardiac depression Decreased venous return Decreased cardiac output
… … Heart Rate Wall Tension MV02 Inotropism C. Flow Ischemia Tissue ischemia Venous pooling
Cardiac depression Decreased venous return Decreased cardiac output
… …
Tissue ischemia Venous pooling
Cardiac depression Decreased venous return Decreased cardiac output
… …
Figure 5. Integration of Multiple Physiological Sub-systems around the Main Cardio-vascular Model.
6. Conclusions
We have described early results from the development of a virtual patient in clinical medicine, mostly targeting emergency medicine and critical care. Our development strategy consists in integrating more physiological sub-systems around a core model of the cardio-vascular system, which has been implemented and evaluated for the correctness of its simulation results (comparing these with standard literature descriptions of the main shock syndromes). However, the difficulties in adding more systems vary greatly according to their nature. Local sub-systems (e.g. coronary circulation) are easily integrated into the central model, while complex
systems working on a different time scale (e.g., long-term regulation of arterial pressure through the renal system) require modifications to the simulation process enabling it to integrate various timescales. Our long-term goal is to be able to carry experiments with multiple integrated sub-systems, e.g. circulatory, respiratory and renal systems.
7. References
[1] Chi, D., Kokkevis, E., Ogunyemi, O., Bindiganavale, R., Hollick, M. Clarke, J., Webber, B., and Badler, N.. Simulated casualties and medics for emergency training, In K.S. Morgan, H.M. Hoffman, D. Stredney, and S.J. Weghorst (eds.), Medicine Meets Virtual Reality, IOS Press, Amsterdam, 1997, pp. 486-494.
[2] Lewis, M., Jacobson, J., Communications of the ACM, 45, 1, January 2002. Special issue on Games Engines in Scientific Research.
[3] Ponder, M., Herbelin, B., Molet, T., Schertenlieb, S., Ulicny, B., Papagiannakis, G., Magnenat-Thalmann, N. and Thalmann, D., Immersive VR Decision Training: Telling Interactive Stories Featuring Advanced Virtual Human Simulation Technologies, Ninth Eurographics Workshop on Virtual Environments (EGVE 2003).
[4] Halvorsrud, R., Hagen, S., Fagernes, S., Mjelstad, S., and Romundstad, L. Trauma Team Training in a Distributed Virtual Emergency Room. Medicine Meets Virtual Reality 2003, Newport Beach, CA, USA.
[5] Simo, A. and Cavazza, M. Visualising Pathophysiological Simulation with a Virtual Patient.
Seventh International Conference on Information Visualization (IV 2003), London, IEEE Computer Press, pp. 549-554
[6] Baan, J. Arntzenius, A.C., Yellin E.L. (Eds.) Cardiac Dynamics. Martinus Nijhoff, The Hague, 1980.
[7] Honig, C.R., Modern Cardiovascular Physiology. Boston: Little, Brown & co., 1988. [8] Guyton , A.C. and Hall, J.E., Textbook of Medical Physiology, WB Saunders, 2000.
[9] Bylander T., Smith J.W. and Svirbley J.R., Qualitative Representation of Behaviour in the Medical Domain. Computers and Biomedical Research, 21, pp. 367-380, 1988.
[10] Widman, L.E. Expert System Reasoning About Dynamic Systems by Semi-quantitative Simulation. Computer Methods and programs in Biomedicine, 29, pp. 95-113.
[11] Long W.J., Naimi, S., Criscitiello, M.G., Pauker, S.G., Kurzrok, S. and Szolovits, P. Reasoning about therapy from a physiological model, in Proceedings of MEDINFO'86
(Washington DC).
[12] Forbus, K.D. Qualitative Process Theory. Artificial Intelligence, 24, 1-3, pp. 85-168, 1984.
[13] Cavazza, M. and Simo, A. A Virtual Patient Based on Qualitative Simulation. Intelligent User Interfaces 2002, Miami, USA (ACM Press).