Complementarity of Reminder-based and
On-demand Decision Support According
to Clinical Case Complexity
J Bouauda, B Séroussia, H Falcoffb and A Venotc a
STIM,DPA/DSI/AP–HP & INSERM, U000729, Paris, France b
SFTG, Paris, France c
LIM&BIO, Bobigny, France Abstract
Clinical decision support systems are mostly conceived as reminder-based systems automatically providing alerts when physicians’ orders differ from recommended actions. We assume that this approach is appropriate for simple clinical cases when physicians think they know how to prescribe and don’t need to be helped. On the opposite, on-demand approaches allowing for flexibility in the interpretation of patient’s conditions and of the guidelines themselves are more appropriate for complex cases. ASTI is a guideline based decision support system that operates in two modes, the “critic mode” working as a reminder-based system, and the “guided mode” working on demand. We proposed a clinical case complexity score in the hypertension domain. A preliminary evaluation of 15 cases of hypertensive patients by 10 GPs tends to validate our assumption that reminder-based interaction is appropriate for simple cases and that physicians are willing to use on-demand systems as clinical situations become more complex.
Keywords:
Clincal decision support systems, Clinical practice guidelines, User interaction, Primary care, Hypertension
1. Introduction
Numerous studies have shown that the sole dissemination of textual clinical practice guidelines (CPGs) is inefficient to modify physicians’ behaviour. On the contrary, computer-based reminders, intervening at decision time, to automatically provide patient-specific recommendations from coded patient data, appear to be the most efficient means to influence healthcare professionals in the adoption of CPGs [1, 2]. However, the success of reminder-based clinical decision support systems (CDSSs) is not warranted [2]. Although general practitioners (GPs) have good intentions to adhere to best practices, their compliance with recommendations remains very low. GPs’ appraisals of the value of guidelines interact indeed with prior knowledge and beliefs, as well as individual and organisational working practices.
More importantly, relevance and accuracy of the advices automatically provided as well as the flexibility to respond to other factors influencing decision making in primary care are often brought into question with usual CDSSs. Classical automated approaches can hardly account for such flexibility. That is why other approaches, where guideline knowledge is
structured in a way a user could retrieve patient-specific recommendations more easily than within texts, have been proposed to provide physicians with guidance. Browsing and reading such structured guideline representations, the physician plays the role of a mediator of patient information which does not need to be coded and can be available among different supports. The OncoDoc system [3] has been developed according to these principles. It relies on a knowledge base, formally structured, through which a user navigates according to the informal description of a given patient to obtain patient-specific recommendations. In the same spirit, Prodigy III [4] offers entry points in complex management guidelines, known as scenarios, describing typical situations a phycisian could meet in his daily practice. To enter the guideline at the right place, the user has to select the relevant scenario corresponding to his patient. Arguments against such “on-demand” systems, which do not function as reminders, is that clinicians must be willing to use them.
We developed ASTI1 [5], a prototype guideline-based CDSS, applied to therapeutic
prescribing in primary care, according to this hypothesis. ASTI can be used according to two modalities, the “critic mode”, operating as a reminder-based CDSS, for the simple cases, and the “guided mode”, operating on demand, for the complex cases. ASTI has been first applied to the management of hypertension. An analysis of the patterns of usage of both modes, automatic control of the critic mode vs. search for guidance of the guided mode, has been carried out according to case complexity to evaluate the complementarity of the two modes in fulfilling GP’s needs for decision support.
2. The ASTI system
2.1 General description
The ASTI system must be considered as an add-on to an existing POE system it improves by adding extra functionalities. It has to be connected to an EMR. In this project, we used the éO2 electronic medical record, because it is the EMR used by the SFTG3, the GP’s association, partner of the project. éO is a problem-oriented medical record and stores free textual information as well as ICD-10 codes or any user-customized thesaurus. The “Banque Claude Bernard4 ” is the drug database associated with the éO EMR. Its contents is used for drug-centered checks of a physician order and provides alerts in case of drug contraindications or toxic drug interactions. ASTI’s value is to provide decision support at the strategic level by issuing guideline-based patient-specific recommendations.
2.2 Two modes of interaction
In the “critic mode”, ASTI operates downstream from the order-entry phase, automatically, as a reminder system. The GP enters her therapeutic prescription as usually. Working as a background process, the system analyses whether the physician’s prescription differs from guideline recommendations. When it is the case, the input prescription is criticized and alternative evidence-based recommendations are displayed that the GP may follow or not.
On the contrary, in the “guided mode”, ASTI operates upstream from the prescription process. The guided mode is thus triggered on GP’s demand, when he needs support to establish the drug prescription for a given patient. The guided mode is not automatically
1
ASTI stands for “Aide à la Stratégie Thérapeutique Informatisée” (Computer-based decision support to therapeutic strategy).
2
The éO EMR is a product of Silk-Informatique, Angers, France.
3
SFTG stands for “Société de Formation Thérapeutique du Généraliste” (Society for Therapeutic Education of General Practitioners).
4
processed. According to the principles first developed with OncoDoc [3], the knowledge base used in the guided mode is operationnalized by the GP while he interactively navigates through the decision tree. When completing the navigation, the user has selected a path from which appropriate therapeutic recommendations are derived.
2.3 Two knowledge bases
As proposed by Tu and Musen [6], two classes of guidelines have to be distinguished: on the one side, one-shot guidelines considered as “consultation guidelines” that specify actions and decisions whose consequences are not being tracked over time. On the other side, the management of chronic diseases relies on “management guidelines”, more complex, that model decisions and actions that lead to dependent changes in patient states over time. If consultation guidelines can be efficiently represented as if-then-else statements for decision making (Arden syntax, Prodigy phase I and II, etc.), the complexity of management guidelines is better handled by task-network models.
As criticized by GPs and even advocated as one of the reasons of physicians low compliance with recommendations, CPGs often only provide recommendations for the initial treatment, even when focused on the management of chronic diseases. This was the case with the guidelines on hypertension we implemented in ASTI’s knowledge bases. Concerning the critic mode, since only therapeutic decisions established for clinical situations covered by the CPG can be legitimately criticized, the knowledge base solely formalized the guideline content. Equivalent to “consultation guidelines”, guidelines have thus been formalized as if-then rules. However, when physicians are looking for management support, we think that all medical decisions have to be supported by a CDSS in order to be accepted, and therefore, actually used in practice by physicians. As a consequence, we designed the guided mode of ASTI so that it could provide therapeutic options for any cases, whether they are covered or not by the CPG. Therapeutic options relying on professional agreement have thus been added and are displayed as advices (with the corresponding grade of evidence, D) when evidence-based recommendations were missing, e.g. in the case of multiple-condition patients or for treatments after more than two lines of therapy. More complex and equivalent to complete “management guidelines”, the knowledge base of the guided mode, represented by a decision tree, relies on a rich model including clinical information and patient therapeutic history [7].
3. Preliminary evaluation of ASTI
We carried on an experimentation to check our working assumption based on the hypothesis that GPs would use the critic mode for simple cases and the guided mode for complex cases. This is based on the following formal characterization of clinical case complexity in the context of hypertension.
3.1 Quantification of clinical cases complexity
We formalized the complexity of a clinical case c by a complexity score built from 2 distinct parameters :
Clinical complexity , denoted CC(c), which measures the complexity of the clinical situation defined as the number of diseases or risk factors associated to hypertension,
e.g.CC(uncomplicated hypertension)=0, CC(hypertension+diabetes)=1, etc.
Therapeutic complexity , denoted TC(c), which measures the complexity of the therapeutic management defined as the number of treatments previously administered
in terms of therapeutic classes,
e.g.TC(never treated)=0, TC(ACEinhibitors)=1, TC(ACEinhibitors,ACEinhibitors+thiazides)=2, etc.
Each kind of complexity participates independently to the clinical case complexity and the complexity score CS of a clinical case c is simply defined by the sum:
CS(c)=CC(c)+TC(c)
Considering that cases presenting 2 elements of complexity begin to be moderately difficult to manage, three complexity classes were then defined according to complexity score thresholds as follows:
Low Medium High
CS<2 CS=2 CS>2
3.2 Protocol
The study was conducted within one day in a before-after framework. Recruited GPs had to be familiar with the éO EMR software. They should neither know the ASTI prototype nor be aware of complexity categorization principles. During the before phasis, each participant received the paper version of n clinical cases. For each one, she had to write down the drug prescription she decided to administer the patient without using ASTI. In the after phasis, participants had to use ASTI on the same n clinical cases according to the 3 modalities below:
Step 1: critic mode use. Participants had to use the critic mode alone. They entered the drug prescription they planned to administer in the order entry module of éO. Depending on the status of the entered prescription with respect to recommendations, alerts with suggestions may be or not produced. Each participant could modify or not his prescription as many times as he wanted until his final decision was made.
Step 2: guided mode use. Participants had to use the guided mode alone. Using the EMR to collect patient data, they navigated in the knowledge base (decision tree) to obtain ASTI recommendations. Then they had to make a decision for the clinical case, following or not the system’s recommendations.
Step 3: free use. Participants were free to use the ASTI system to help them make the best recommended therapeutic decision. They could use the mode that seemed the most appropriate to their needs, or both modes. In this step, they were asked to specify which mode they used, and in case they used both modes, which one they used first.
4. Results
Ten GPs were involved in the evaluation study. We used n=15 clinical cases of hypertension, extracted and anonymised from actual medical records, to represent at best the current practice of GPs in hypertension.
4.1 Case complexity distribution
Table 1 reports the complexity distribution among the 3 subsets : average low complexity for the case set used in step 1 (critic mode use), average high complexity for the case set used in step 2 (guided mode use) and average medium complexity for the case set used in step 3 (free use).
Table 1: Case complexity of case sets
4.2 Effect of case complexity on ASTI usage
We exclusively consider the results of the free use step and the effect of case complexity on system usage. These data are reported in table 2.
Table 2: System usage according to the level of cases complexity
Complexity # of decisions Critic mode alone Guided mode alone Both modes None Low 10 40 % 20 % 20 % 20 %
Medium 20 20 % 40 % 40 % 0 %
High 20 5 % 70 % 25 % 0 %
All 50 18 % 42 % 30 % 4 % Cumulatively, the critic mode is more used than the guided mode for the case of low complexity (60% vs 40%). It must be noticed that only for this case, 2 participants did not use ASTI at all. For medium complexity cases, the frequency of use of the critic mode does not change (60%), but, the frequency of guided mode use increases to 80%. For highly complex cases, critic mode use falls to 35% while guided mode use jumps to 95%.
Although on a limited set of cases, the tendency of participants to choose a given scenario of interaction with the system according to the case complexity seems plausible and consistent with our initial hypotheses. This is not invalidated when looking at the first mode used according to case complexity as described by table 3.
Table 3: First mode used according to case complexity
Complexity level Critic mode Guided mode None
Low 60 % 20 % 20 %
Medium 55 % 45 % 0 %
High 15 % 85 % 0 %
All 40 % 56 % 4 %
The critic mode is preferably used as the first choice for support for low and medium complexity cases with frequencies of 60% and 55%. This means GPs had sufficent knowledge and a good understanding of the cases so that they thought they were able to propose an acceptable prescription.
As for the guided mode, it is nearly never used (20%) for the low complexity case.
Class
n Low Medium High Average CS Mean complexity
Critic mode use 5 4 – 1 1.4 Low+
Guided mode use 5 – 2 3 3.2 High
Free use 5 1 2 2 2,4 Medium+
However, it becomes used as first support for medium (45%) and mostly for high complexity cases with 85% of the cases. In these later situations, the critic mode is not much used (15%). This suggests GPs might think they do not have sufficient knowledge to prescribe and are ready to actively search for guideline support.
5. Conclusion
In this paper, we presented the design of ASTI, a CDSS with two complementary modes of interaction offered to the physician willing to improve the choice of the pharmacological treatment adapted to a given hypertensive patient. Although obtained on small numbers and in a room, the results of the preliminary study of ASTI we presented, seem to validate the assumptions we made about the fact that reminder-based interaction is appropriate for simple cases and about the physicians’ willingness to use on-demand systems as clinical situations become more complex. A continuing project, ASTI 2, will evaluate this in a clinical trial carried out in actual general practice settings.
6. References
[1] Hunt DL, Haynes RB, Hanna SE, and Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes. JAMA 1998;280:1339–46.
[2] Shiffman RN, Liaw Y, Brandt CA, and Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. JAMIA 1999;6(2):104–14.
[3] Séroussi B, Bouaud J, and Antoine EC. OncoDoc, a successful experiment of computer-supported guideline development and implementation in the treatment of breast cancer. Artif Intell Med
2001;22(1):43–64.
[4] Johnson PD, Tu S, Booth N, Sugden B, and Purves IN. Using scenarios in chronic disease management guidelines for primary care. J Am Med Inform Assoc 2000;7(suppl):389–93.
[5] Séroussi B, Bouaud J, Dréau H, Falcoff H, Riou C, Joubert M, et al. ASTI, a guideline-based drug-ordering system for primary care. In: Patel VL, Rogers R, and Haux R, eds, Medinfo, 2001:528–32. [6] Tu SW and Musen MA. A flexible approach to guideline modeling. J Am Med Inform Assoc
1999;6(suppl):420–4.
[7] Séroussi B, Bouaud J, Chatellier G. Modeling patient-specific therapeutic strategy in the guideline-based management of a chronic disease. Stud Health Technol Inform. 2003;95:537-42
Address for correspondance
Jacques Bouaud
STIM/DSI, Hôpital Broussais, 96 rue Didot, 75014 PARIS, France [email protected]
