Distributed Decision Support Using a Web-based Interface:

157

Distributed

Decision

_{Support Using}

a

Web-based Interface:

Prevention

of

Sudden Cardiac Death

GILLIAN

D.

_SANDERS,

PhD,

C. GREG

HAGERTY,

MS,

FRANK A.

_{SONNENBERG, MD,}

MARK

A.

_HLATKY,

_MD,

DOUGLAS

K.

_{OWENS, MD,}

MSc

Although decision models can _provide a formal foundation for guideline development

and clinical decision _support, their widespread use is often limited by the lack of

plat-form-independent

software that

_{geographically}

_dispersed users can access and use

easily

without extensive

_training.

To address these limitations the authors developed

a World Wide Web-based interface for previously developed decision models. They

describe the use and functionality of the interface using a decision model that evaluates

the cost-effectiveness of strategies for preventing sudden cardiac death. The system

allows an _analyst to use a web browser to interact with the decision model and to

change the values of _input variables within _{pre-specified ranges,} to specify sensitivity

or threshold analyses, to evaluate the decision model, and to _{view the results}

gener-ated _dynamically. The web site also _provides

_linkages

to an _explanation of the _model, and evidence tables for _input variables. The _system demonstrates a _{method for}

pro-viding distributed decision support to remote users such as _{guideline developers,}

de-cision analysts, and potentially practicing physicians. The web interface _provides

plat-form-independent and almost universal access to a decision model. This _approach can

make distributed decision _support both _practical and economical, and has the _potential

to increase the usefulness of decision models

_{by enabling}

a broader audience to

in-corporate systematic analyses into both _policy and clinical decisions. _Key words:

de-cision-support techniques; practice guidelines; user-computer interface; decision

anal-ysis. (Med Decis Making 1999;19:157-166)

Decision models

_provide

an

_analytic

framework for

representing

the

_evidence,

_{outcomes, and}

prefer-ences in a clinical

decision.&dquo;’

_They

_represent

the available alternatives and events of

interest,

and

combine these elements in an

_objective

_and

pre-dictable way to

_produce

recommendations that are

consistent with

_underlying

data and

_assumptions.

Decision models enable

_analysts

and

_guideline

de-velopers

to

_{perform sensitivity analyses}

that

_identify

critical variables and

_highlight

the

_importance

(or lack of

_importance)

of

_uncertainty

about these

var-iables.

Thus,

decision models

_provide

a

_systematic

method for

_representing

what is known about a

clinical

_problem,

and an

_approach

for

_determining

whether what is not known is

_important.

Because of these

_advantages,

authors of

_practice

_guidelines

in-creasingly depend

on decision models to inform the

guideline

recommendations.3-18

Although

use of decision models has several

advantages,l’19-Zl

development

of clinical decision

Received April 17, 1998, from the Center for _Primary Care and

Outcomes _{Research, Department} of Medicine, Stanford

Univer-sity, Stanford, California (GDS, DKO); the UMDNJ/Robert Wood Johnson Medical School, New Brunswick, New _Jersey (CGH, FAS); the _Department of Health Research and _Policy, Stanford

University, Stanford, California (MAH); the VA Palo Alto Health Care _System, Palo Alto, California (DKO); and Stanford Medical

Informatics, Department of Medicine, Stanford _University (GDS,

DKO). Revision _accepted for _publication November 5, 1998.

Pre-sented at the 19th annual meeting of the _Society for Medical Decision _Making, Houston, Texas, October 26-29, 1997. Sup-ported in _{part by} the Cardiac Arrhythmia and Risk of Death

Pa-tient Outcomes Research Team _grant (HS 08362) to Stanford

Uni-versity from the _Agency for Health Care _Policy and Research, Rockville, Maryland. Dr. Owens is _{supported by} a Career

Devel-opment Award from the Health Services Research and

Devel-opment Service, Department of Veterans Affairs.

Address _{correspondence} and _reprint requests to Dr. Sanders: Stanford _University School of Medicine, HRP Redwood _Building,

Room T242, Stanford, CA 94305-5405; telephone: (650)723-5656;

FIGURE 1 Schematic _{representation} of the ICD

cost-effective-ness _{decision model The square node represents} a decision to

use one of an _implantable cardioverter defibrillator (ICD),

amio-darone, or a combination _strategy of amiodarone-to-ICD The circle represents a chance node. After a _regimen is chosen, the

patient enters a Markov tree (denoted by rectangles containing

circles and an arrow) An ICD patient enters the Markov tree

only if he or she survives ICD _{implantation.} The Markov trees represent the clinical events that can occur _during each

one-month _period as a _patient is followed until death. For example,

a _{patient receiving} the ICD _regimen is at risk each month for ventricular _tachycardia (VT), ventricular fibrillation (VF),

nonar-rhythmic cardiac death, and noncardiac death. If none of these

events occurs, the _patient remains well for the one-month pe-riod. A _patient who is well at the end of a _given month reenters

the Markov tree Subtrees are denoted by rounded _rectangles. Patients _receiving the amiodarone _regimen are at risk for V’T,

VF, nonarrhythmic cardiac _death, noncardiac death, and _toxicity from amiodarone. If _toxicity from amiodarone occurs, the pa-tient enters the _{amiodarone-toxicity} subtree.3’ _Reproduced with

permission from Owens DK et _al., Cost-effectiveness of implant-able cardioverter defibrillators relative to amiodarone _for pre-vention of sudden cardiac death, Annals of Internal _Medicine, 1997,126:1-12. Copyright American _College of _Physicians.

models also

_requires

extensive

_training

and exper-tise in decision

_analysis,

clinical

_medicine,

and

evi-dence

_synthesis.

As an

_example,

it took two _{years for}

a

multidisciplinary

team to

_develop

a decision model

that evaluated the cost-effectiveness of

_using

an

im-plantable

cardioverter defibrillator (ICD) to

_prevent

sudden cardiac death.22-z4

_Furthermore,

the

evi-dence used in a decision model may

_require

updat-ing

as new evidence is

_published.

For

_example,

sev-eral clinical trials that will estimate the effectiveness of ICDs were still in _progress when the model was

developed.&dquo;&dquo;

Because

_{paper-based publications}

are

brief,

sensitivity

or threshold

_analyses

that _{may be}

of interest to a

_particular

user _may not have been included in the

_{published analysis.}

Thus,

guideline

developers

or

_practicing

_physicians

_may

_question

whether a

_published

cost-effectiveness

_analysis

ap-plies

to their

_practices.

A

_potential

solution to these limitations is to make

the decision model available for use

_{by analysts,}

cli-nicians,

or

_{guideline developers.}

Until

_recently,

sub-stantial technical barriers

_(e.g.,

need for users to

have

_expertise

in

_{decision-analytic}

software,

the

di-versity

of

_computing

_platforms,

and the

_disperse

lo-cations of

_potential

users)

_prevented

such use of the

model. But now it is feasible to

_provide

an interface

to the model that enables remote users to

_perform

analyses.

The existence of such a distributed

deci-sion-support

system

would allow

_guideline

devel-opers or clinicians to use the decision model in the

development

of recommendations or

_guidelines,

to

update

the model as new evidence becomes

avail-able,

and to

_adapt

the model to reflect a

_particular

patient

population

or clinical

setting.1,27

In

_previous

work,

we have shown that such

_{site-specific}

guide-lines

_{potentially provide}

_greater

health

_benefit,

en-hance economic

_efficiency,

or

_both,

when

com-pared

with

_global

_guidelines.

1

We describe here our

_development

of an

inter-active web-based interface for a decision model that

enables remote use of decision models. We

dem-onstrate the use of this interface with the Cardiac

Arrhythmia

and Risk of Death Patient Outcomes

Re-search

_Project

(CARD PORT) decision model to

_analyze

the cost-effectiveness of

_strategies

to

_prevent

sud-den cardiac

death.22-24

Methods

We chose to illustrate the use of our

_system,

named PORTAL, with the CARD PORT ICD

cost-effec-tiveness decision model because new clinical data are

_becoming

available

rapidly,28,29

making

especially

important

the

_ability

to

_adapt

_existing

_{analyses easily.}

The ICD cost-effectiveness decision model

evalu-ates the cost-effectiveness of

_strategies

for

prevent-ing

sudden cardiac death (SCD)

_(figure

1). The two

strategies

are

_therapy

with a

_{third-generation}

ICD versus administration of

amiodarone,

the most

promising pharmocologic

alternative. The Markov

im-planting

an

_ICD,

_{administering}

_amiodarone,

or first

administering

amiodarone and then

_implanting

an

ICD after an

_arrhythmic

event occurs.

_Assuming

a

patient

survives the ICD

_operation,

he or she is at

risk for an

_arrhythmic

event and

_potential

death,

for

nonarrhythmic

cardiac

death,

or for noncardiac

death,

each month. Patients who receive the

amio-darone

_regimen

are also at risk for amiodarone

tox-icity.

The ICD cost-effectiveness decision model is

im-plemented

in the Decision Maker

_analytic

software. 30

To

_perform

remote

_analyses,

we created PORTAL, an interactive web-based interface for this decision

model

_(figure

2). PORTAL uses Windows

common-gateway

interface (Win-CGI)

_scripts

written in Visual

Basic (Microsoft Visual Basic 4.0). PORTAL has two

main modules: the web-interface control module

and the

_{analytic-software (e.g.,}

Decision Maker)

driver. The web-interface control module _manages

the

_organization

and content of the web

_pages,

as

well as interactions with the user. The web interface

allows the user to browse

_through

_{the many}

com-ponents

of the decision

model,

the model’s

under-lying

data,

and the

_analytic

results. When a user

en-ters data into the

_{analytic-results}

section of the web page, the control module interacts with the

analytic-software driver to

_{perform dynamic}

remote

analy-ses.

The

_{analytic-software}

driver uses

_{object linking}

and

_embedding

(OLE) commands to interact with

the

_underlying

decision model. The driver has nu-merous

_{capabilities, including}

_opening

the software

package, setting required

default values

_(e.g.,

indi-cating

whether or not the decision model is a

cost-effectiveness

_model),

_returning

the values of model

parameters

or software

_{settings, changing}

base-case

values,

analyzing

the tree with or without

cost-ef-fectiveness,

setting

up and

performing

sensitivity

and threshold

_analyses,

and

_saving

the

_analytic

re-sults.

Once the

_{analytic-software}

driver has

_performed

the needed

_analyses,

the web-interface control

mod-ule formats and

_displays

the results for the user. This module also creates

_dynamically

a

_graphic

dis-play

of the

_{sensitivity-analysis}

results. Much of the

parsing

and

_graphing

of the results is

_currently

done

through

interaction of the control module with

Ex-cel (Microsoft Excel 97)

_{spreadsheets.}

The web

in-terface controls these

_spreadsheets

as

_well,

_through

OLE commands. For

_example,

the

_{analytic-software}

driver for Decision Maker saves the results of

sen-sitivity

analyses

in a table format. This table is loaded

into Excel

and,

_using

OLE

_commands,

the control

module indicates the desired columns and rows to

create a

_graph

of the

_{sensitivity-analysis}

results. This

graph

is saved as a

graphics interchange

format

(GIF) file that the user _may view

_using

our web

in-FIGURE 2. Schematic _{representation} of poRTnr,’s architecture.

The PORTAL _system (the ellipse) has two main modules: the web-interface control module and the analytic software module. The web-interface control module interacts with the user _(A),

allow-ing the user to browse the _analysis, enter new _{input data,} and view custom-tailored _analyses. The web-interface control mod-ule interacts with the _{analytic-software} driver (B) to _perform the

needed remote _analyses. The _{analytic-software} driver uses _object

linking and _embedding commands to interact with the

under-lying decision model (C). _Although in this paper we describe the Decision Maker software module and the Cardiac Arrhythmia

decision model, other _{analytic-software} modules and decision models can interact with the PORTAL _system.

terface.

_Currently

we have

_developed

the Decision

Maker

_{analytic-software}

driver;

we have _the

capa-bility

to

_develop

other drivers for different decision-model formats.

System Description

and

_Example

Scenario

A user is able to

_perform

several actions

_using

PORTAL. He or she can browse the model

descrip-tion,

assumptions,

data,

evidence

_tables,

and

base-case results. As

_part

of the interactive

interface,

the

user can make

_changes

to the base-case

_input

var-iables and then view the

_changed

health and

eco-nomic results. PORTAL also can calculate the

mar-ginal

cost-effectiveness of the

_{competing strategies.}

Finally,

the user can

_perform

and view

_sensitivity

and threshold

_analyses

on the numerous model

var-iables.

As an

_example,

_{suppose that} a

_guideline

devel-oper at a remote site identifies a

_patient

_population

that he or she believes to be at risk for SCD. The

developer

wants information

_regarding

the use of an

ICD for this

_{particular population.}

He or she locates the

_published

cost-effectiveness

_analysis

_comparing

the use of an ICD and administration of

amioda-rone,24 noticing

that the values of some variables are

different from those for the

patient

population

in

question.

He or she wants to see how

_changing

these

values affect the results of the decision

_analysis.

First,

at the

_{guideline developer’s}

institution,

the

cost of ICD

_implantation

is much lower

($20,000)

than that listed in the

_published

_analysis

_input

table

($44,600). Second,

the

_probability

of an ICD-related

FIGURE 3 Home _page of the Sudden Cardiac Death (SCD) Decision _Modeling Group The menu on the left side allows the user to

navigate through a _description of the cost-effectiveness decision model for the _implantable cardioverter defibrillator (ICD) vs amio-darone and of other related projects The user is able to browse the modeling definitions, assumptions, schematics of the model, input

data, evidence tables, main results, and _{sensitivity analyses} These noninteractme _capabilities are detailed elsewhere 3°

The home _page also _provides links for the user to _perform interactive _analyses

FIGURE 4 _Input variable best-estimate values and sources The input table lists the variable name (with links to the _{corresponding} evidence tables where _{appropriate),} the base-case values, the _range used for _{sensitivity analyses,} the level of evidence based on a _rating

system developed by the U S Preventive Services Task Force, and the sources used to determine the chosen values The user is then able to enter a new value for a _given variable This screen shows the user _changing the _{perioperative mortality} from 1 8% to 0 5%

FIGURE 5 Parameters for _{sensitivity analysis.} The sensitivity-analysis section of the web interface allows a user to choose an _input

variable, and then to set the desired minimum and maximum values for the sensitivity analysis as well as the incremental step This

user uses the _pull-down menu to choose the decision-model variable of Frequency of battery replacement as the variable on which to

perform the _{sensitivity analysis. Notice that the base-case value is listed in square} brackets-[4 years]-after the variable name After

choosing the variable on which to _perform the _{sensitivity analysis,} the user sets the _{remaining parameters} (minimum value, maximum

value, and incremental _step) This screen shows the user _choosing a minimum _frequency of _{battery replacement of every three years,}

a maximum value of every eight years, and an incremental _{step of every} one _year

but a recent

_study

has determined that

_{perioperative}

mortality

is 0.5% at the

_{developer’s}

institution.

Fi-nally,

the

_{published analysis}

uses four years for the

frequency

of ICD

_battery

_replacement.

The

_guideline

developer

has read a recent

_report

from the ICD

manufacturers that claims that the ICD

_battery

life

is now

_{reaching eight}

_years. Because the

_developer

is still uncertain about this

value,

he or she wants

to

_perform

a

_{sensitivity analysis}

on the

_frequency

of

battery replacement

while

_taking

into account the lower ICD cost and lower

_{perioperative}

death rate.

Our

_{guideline developer}

_goes to the SCD decision

modeling

group home page

(figure

3). On the left side is a menu that is consistent

_throughout

the

website and that enables the user to

_navigate

through

the

_modeling

_{assumptions, input}

_data,

evi-dence,

and results. It also

_provides

links to the in-teractive

_analyses.

CHANGING BASE-CASE VALUES

The

_{guideline developer}

clicks on the

_Input

Data

link under the

_{Perform Analyses heading}

in the menu. An

_{input-variable}

table similar to the one

found in the

_published

cost-effectiveness

_analysis

appears. In addition to the variable name, base-case

value,

sensitivity-analysis

range, level of

evidence,

and source, there is a field for the user to enter a new value for any of the variables. For

_example,

the

probability

of ICD

_{perioperative}

death is listed as

1.8%. The

_{guideline developer changes}

this value to

0.5% to reflect the institution’s lower

_{perioperative}

mortality (figure

4).

_Similarly,

he or she

changes

the initial cost of the ICD device from $44,600 to

$20,000.

The

_developer

does not

_change

the

_frequency

of

bat-tery

replacement

because he or she would rather see the effect of this variable in a

_sensitivity

_analysis

over a _range of values.

SENSITIVITY ANALYSES

The

_{guideline developer}

uses the

pull-down

menu

to choose the

_frequency

of

_{battery replacement}

as

the variable to be tested in

_{sensitivity analyses.}

He

or she chooses a minimum

_frequency

_{of every three}

years and a maximum

_frequency

of _every

_eight

years, with an incremental

_step

of one _year

_(fig.

5). The

_{guideline developer}

then submits the

analy-ses. He or she checks the boxes

_{corresponding}

to

Main

_Results,

_Marginal

Cost

_{Effectiveness,}

and

Sen-sitivity

Analyses

to

_identify

those

_analyses

he or she would like

_performed

(not shown). The PORTAL

sys-tem interacts with the Decision Maker OLE interface and the

_underlying

decision model to return the

re-quested

results.

ANALYTIC RESULTS

PORTAL returns a web _page with the

performed

analyses (figure

6). The

_top

table lists the main

re-sults. Each row

corresponds

to one of the three

strategies

(ICD

_only,

amiodarone

_only,

and an

amio-darone-to-ICD

_strategy).

The columns indicate the

expected

costs and

_{quality-adjusted}

_{life years} asso-ciated with these different

_strategies.

The second

ta-ble shows the

_marginal

cost-effectiveness

_expressed

in cost _per

_{quality-adjusted}

_{life years saved when the}

FIGURE 6 Results of the interactive _analysis The _system returns the results in a tabular form. For _example, the Mam Results table shown here lists the three _strategies (ICD only, amiodarone _only, and amiodarone-to-ICD) The _remaining columns list the costs and the _{quality-adjusted} life _expectancies associated with the different strategies The second table lists the _Marginal Cost _{Effectiveness}

FIGURE 7 (bottom _{of facing page)} Results of interactive _{sensitivity analysis} The web interface returns a table of the _{sensitivity-analysis} results The columns (from left to _right) mdicate the chosen variable and values _{possible strategies, corresponding} costs, effectiveness,,

average cost-effectiveness marginal costs, marginal effectiveness, and _marginal cost-effectiveness This screen shows the

sensitivity-analysis results for _varying the _frequency of _{battery replacement from every three years} to _{every eight years} At the bottom of the table is a link that allows the user to mew a _graph of these _{sensitivity-analysis} results

I

FIGURE 8 _Graph of the _{performed sensitivity analysis} Our _system creates _dynamically a _graph of the _{performed sensitivity analysis}

On the X-axis is the variable chosen for the _analysis (here, BattRepl for the _frequency of _{battery replacement),} on the Y-axis is the

marginal cost-effectiveness of ICD versus amiodarone, _expressed in _{dollars per quahty-adjusted life year saved}

PORTAL’S web interface has allowed the user to

ex-plore

the

_published

decision model and to

_adapt

that model to

_represent

_patients

in his or her insti-tution. For

_example,

the

_marginal

cost-effectiveness of ICD versus amiodarone in the

_previously

pub-lished

_analyses

was $74,662 per

QALY

saved. In the scenario

described,

using

the new

data,

the

mar-ginal

cost-effectiveness has been reduced to $35,341

per

QALY

saved. PORTAL’S interactive abilities have

allowed the user to tailor the

_underlying

decision model to

_represent

more

_accurately

the

_patient

pop-ulation in

_question.

In this

_example,

the customi-zation of the decision model has

_produced

a

mar-ginal

cost-effectiveness that is lower than the

published expected

result and could

_potentially

change

the recommended treatment in _this popu-lation.

The third table shows the results of the

_sensitivity

analysis

on the

_frequency

of

_battery

_replacement.

For the

_specified

_{sensitivity-analysis}

_range of three

to

_eight

_{years, this} table lists the

_{corresponding}

cost,

effectiveness,

average

cost-effectiveness,

marginal

cost,

marginal

effectiveness,

and

_marginal

cost-ef-fectiveness of each

strategy

(figure

7). The user can

also view an

_accompanying

graph

of these

sensitiv-ity-analysis

results that is created

_dynamically

for the

requested

results

_(figure

8).

SYSTEM EXTENSIONS

To aid in the

_development

of

_{site-specific}

guide-lines,

Sanders and

_colleagues

have

_developed

ALCHE-MisT, a

_{computer-based}

tool that extends the PORTAL

system

by

using

decision models to

_generate

anno-tated clinical

_algorithms

_{automatically.31}

The ALCHE-MisT

_system

obtains information from the decision

model and the decision

_analyst

to

_{automatically}

cre-ate an annotated

_algorithm

of the

_optimal

test or

treatment

_strategy,

as determined

_by

the decision model. As with the PORTAL

_system,

a remote user can tailor the

_input

values of the decision model. ALCHE-MIST then

_{automatically}

creates an

_updated

algo-rithm that reflects the

_optimal

_strategy

based on

these new

_inputs.

The

_system

_dynamically

creates a web _page that

_displays

the

_updated

_algorithm

to the

remote user.

_Thus,

the ALCHEMIST

_system

_provides

a

method for

_creating

evidence-based,

_{site-specific}

clinical

_algorithms

that reflect the characteristics of

a user’s

_{practice setting}

or

_patient

_population.

This

system

is an

_important

extension of PORTAL because

it

_provides

a method for

_{automatically analyzing}

a

decision model and

_displaying

the results of the

analysis

in an intuitive

_compact

format-a clinical

algorithm. Preliminary

evaluation of the ALCHEMIST

system

demonstrated that users

_strongly

and

signif-icantly preferred guidelines developed

within this

framework to

_guidelines

_{published by nationally}

recognized

organizations. 31-33

Discussion

As the

_example

scenario

demonstrates,

our

web-based interface to a decision model can

_provide

dis-tributed

_dynamic

decision

_support

to remote users

such as decision

_analysts,

_guideline

_developers,

_peer

reviewers,

and clinicians. PORTAL’S interface allows

users to browse and

_interrogate

a

_developed

deci-sion model and to

_adapt

the

_input

variables and

analyses

to

_represent

their

_patient

_populations,

and

therefore we _{believe may}

_provide

_important

health

benefits. Prior to the

_development

of a

technology

such as _PORTAL,

_provision

of decision

_support

was

difficult because use of models

required

extensive

experience

with the available

software,

the models could not be used

_easily

across different

_computing

platforms,

and it was not

_possible

to

_analyze

models from a remote site. In

_addition,

each user had to

own

_{decision-analytic}

_software,

obtain the decision

model,

and

_{perform analyses locally.}

PORTAL enables

any user who has a web browser to

_analyze

a de-cision model

_{remotely. Although}

much of the

infor-mation needed to create the

_original

interface to a

decision model can be obtained

_directly

from the

decision

_model,

the PORTAL

_system

_requires

some

additional information from the decision

_analyst.

For

_example,

the PORTAL

_system

_requires

the

_analyst

to

_complete

a list of

_modeling

_assumptions,

the valid

sensitivity

analysis

ranges, necessary

definitions,

and references used in the

_analysis.34

This information

can be obtained

_directly

over the web

_using

a

de-cision-model annotation

editor.31

_Although

complet-ing

this information is additional work for the de-cision

_analyst,

it is information that should be

readily

available. In

_addition,

the

_{implementation}

of the annotation editor on the web allows decision

analysts

access to the editor from different

institu-tions,

and allows

_{decision-analysis}

teams to share

decision-modeling

tasks _among members located at

geographically

disparate

institutions who are

_using

different

_computing

_platforms.31

Similar interactive sites are

_{being developed}

else-where. Kattan and

_{colleagues developed}

a

_system

that allows a user to load a

_developed

decision

model,

to

_specify

which variables should be

inter-active,

and to

_publish

this interactive decision model

on his or her

website.35

_Similarly,

users of the DATA

decision-analytic

software

_by

_TreeAge

can create a

&dquo;Custom Interface&dquo; to a

_developed

decision model.

TreeAge

is beta

_testing

a web-based version of its

system.36

Currently,

PORTAL treats

_utility

variables the

same as _any other

_input

variable. An extension of

our work would

_incorporate

research on

computer-based

_utility

assessments to

_help

the user to

deter-mine his or her

_patient’s

utilities.34 3’-40

Web interfaces to decision models such as that

described in this article will

_provide

a means for

distributed decision

_support34

and

_potentially

for

guideline development.

A

_centrally

located

guide-line-resource

_group

could

_develop

a decision

model,

and could disseminate that model

_using

an

interactive web interface to

_{potential guideline}

de-velopers.

These users could then make modifica-tions to the evidence

_underlying

the decision model and

_subsequently

could

_modify

the

_{corresponding}

guideline.41

The

_availability

of distributed decision

_support

with a web-based interface to a decision model

raises several

_questions.

Should interactive decision

models be peer reviewed? How _{should such peer}

review occur?

_Then,

after access to an interactive

decision model is

_provided

on the

web,

who is

re-sponsible

for

_keeping

the model _up to date with

the latest clinical evidence? Previous researchers

have studied methods of

_critiquing

decision

anal-yses.42-46

Although

most of these studies concentrate

on

_paper-based

decision

_analyses,

their criteria

apply

to web-based decision

_analyses.23

We

_suggest

that a

_{comprehensive}

checklist of desirable

ele-ments of _{any decision}

_analysis

be

_compiled

and

used to evaluate web-based decision models before

they

are used. For

_example,

a checklist of desirable

elements would include peer review of the

analysis,

discussion of the

_quality

of the

evidence,

clear

state-ment of the

_underlying

_assumptions,

_listing

of the

data sources, and well-defined and relevant

strate-gies.

Conclusions

Our

_approach,

demonstrated here for the Cardiac

Arrhythmia

PORT ICD cost-effectiveness decision

model,

is

_{generalizable}

to _{any decision model} in _any

domain.&dquo;

It

_provides

distributed decision

_support

to remote users such as

_{guideline developers,}

decision

analysts,

and

_practicing

_physicians.

PORTAL’S web

in-terface

_{provides platform-independent}

and almost

universal access to _{the decision model. This}

ap-proach

can make distributed decision

_support

and

decision-model

_sharing

both

_practical

and

econom-ical. The

_approach

will increase the usefulness of

decision

_models,

and enable a broader audience to

incorporate

systematic analyses

into both

_policy

and clinical decisions.

The authors thank _{Kathryn McDonald,} Robert F. Nease Jr., and

Lyn Duprd for comments on the _manuscript and editorial

References

1. Owens DK, Nease RF. A normative _analytic framework for

development of _practice

_guidelines

for _specific clinical pop-ulations Med Decis _Making. 1997;17:409-26

2. Carter AO, Battista RN, Hodge MJ, et al. _Proceedings of the

1994 Canadian clinical practice guidelines network

work-shop. Can Med Assoc J. 1995;153.

3. _Aggleton P,

_O’Reilly

K, Slutkin G, Davies P. _Risking

every-thing? Risk behavior, behavior _change, and AIDS Science.

1994;265:341-5.

4. _Schapira MM, Matchar DB, Young MJ. The effectiveness of ovarian cancer _screening. A decision _analysis model. Ann Intern Med. 1993;118:838-43.

5. _Singer DE, Samet JH, Coley CM, Nathan DM. _Screening for diabetes mellitus. In: _Eddy DM (ed). Common _Screening

Tests. _{Philadelphia,} PA: American _College of _Physicians, 1991:

154-78.

6. Sox HC Jr., Garber AM, Littenberg B. The _resting

electrocar-diogram as a _screening test: a clinical analysis. In: _Eddy DM

(ed). Common _Screening Tests. _{Philadelphia,} PA: American

College of _Physicians, 1991:47-80.

7. Sox HC Jr., Littenberg B, Garber AM. The role of exercise

testing in _{screening for coronary artery} disease. In: _Eddy DM

(ed). Common _Screening Tests. _{Philadelphia,} PA: American

College of _Physicians, 1991:81-112.

8. Carlson KJ, Skates SJ, Singer DE. _Screening for ovarian can-cer. Ann Intern Med. 1994;121:124-32.

9. American _College of _Physicians. Guidelines for _counseling

postmenopausal women about _preventive hormone _therapy.

Ann Intern Med. 1992;117:1038-41.

10. American _College of _{Physicians. Screening} for ovarian can-cer : Recommendations and rationale. Ann Intern Med. 1994;

121:141-42.

11. _Eddy DM. _Screening for breast cancer. In: _Eddy DM (ed).

Common _Screening Tests. _{Philadelphia,} PA: American

Col-lege of _Physicians, 1991:229-54.

12. _Eddy DM. _Screening for _lung cancer. In: _Eddy DM (ed).

Com-mon _Screening Tests. Philadelphia, PA: American _College of

Physicians, 1991:312-25.

13. Eddy DM. _Screening for cervical cancer. In: Eddy DM (ed).

Common _Screening Tests. Philadelphia, PA: American

Col-lege of _Physicians, 1991:255-85.

14. _Eddy DM. _Screening for colorectal cancer. In: _Eddy DM (ed).

Common _Screening Tests. _{Philadelphia,} PA: American

Col-lege of _Physicians, 1991:286-311.

15. Fahs MC, Mandelblatt J, Schechter C, Muller C. Cost effec-tiveness of cervical cancer _screening for the elderly. Ann In-tern Med. 1992;117:520-7.

16. Grady D, Rubin SM, Petitti DB, et al. Hormone therapy to prevent disease and _prolong life in postmenopausal women. Ann Intern Med. 1992;117:1016-37.

17. Melton JL III, Eddy DM, Johnston CC. _Screening for

osteo-porosis. In: _Eddy DM (ed). Common _Screening Tests.

Phila-delphia, PA: American _College of _Physicians, 1991:202-28.

18. _Littenberg B, Garber AM, Sox HC Jr. _Screening for hyperten-sion. In: _Eddy DM (ed). Common _Screening Tests.

Philadel-phia, PA: American _College of _Physicians, 1991:22-46.

19. Nease RF, Owens DK. A method for _estimating the

cost-effectiveness of _{incorporating patient preferences} into prac-tice _guidelines. Med Decis Making. 1994;14:382-92.

20. Owens DK, Nease RF. Development of _{outcome-based} prac-tice _guidelines: a method for _{structuring problems} and

syn-thesizing evidence. Joint Commission J Quality

Improve-ment. 1993;19:248-63.

21. _Eddy DM. Guideline for _policy statements: the explicit

ap-proach. JAMA. 1990;263:2239-43.

22. Sanders GD, Harris RA, Hlatky MA, Owens DK. Prevention of

sudden cardiac death. a _{probabilistic} model for decision

support. Proceedings of the Nineteenth Annual _Symposium

on _{Computer Applications} in Medical Care, New Orleans, LA,

1995. J Am Med Informat Assoc 1995; 2, symposium suppl:

258-62.

23 Sanders GD, Dembitzer AD, Heidenreich PA, McDonald KM,

Owens DK Presentation and _explanation of medical decision models _using the World Wide Web. _Proceedings of the 1996

AMIA Annual Fall _{Symposium, Washington,} DC, 1996. J Am

Med Informat Assoc. 1996; 3, symposium suppl:60-4.

24 Owens DK, Sanders GD, Harris RA, et al. Cost effectiveness of _implantable cardioverter defibrillators relative to amio-darone for _prevention of sudden cardiac death. Ann Intern Med. 1997;126:1-12.

25. _Connolly SJ, Gent M, Roberts RS, et al. Canadian _Implantable Defibrillator _Study (CIDS): _{study design} and _{organization.} CIDS _{Co-Investigators.} Am J Cardiol. 1993;72:103F-108F.

26. Siebels J, Kuck KH. _Implantable cardioverter defibrillator

compared with _{antiarrhythmic drug} treatment in cardiac ar-rest survivors (the Cardiac Arrest _{Study Hamburg).} Am Heart

J. 1994;127:1139-44.

27. Sanders GD, Hagerty CG, Sonnenberg FA, Hlatky MA, Owens

DK. Distributed _dynamic decision _{support using} a web-based interface for _prevention of sudden cardiac death

(abstr). Med Decis Making. 1997;17:524.

28. The _{Antiarrhythmics} versus _Implantable Defibrillators (AVID)

Investigators. A comparison of _{antiarrhythmic-drug therapy} with _implantable defibrillators in _patients resuscitated from near-fatal ventricular _arrhythmias. N _Engl J Med. 1997;337:

1576-83.

29. _Bigger JT _Prophylactic use of _implanted cardiac

defibrilla-tors in _patients at _high risk for ventricular _arrhythmias after

coronary-artery bypass graft surgery. N _Engl J Med. 1997;337:

1569-75.

30. _Sonnenberg FA, Pauker SG. Decision maker: an advanced

personal computer tool for clinical decision _analysis.

Pro-ceedings of the Eleventh Annual _{Symposium Computer}

Ap-plications in Medical Care. _Washington, DC: IEEE Computer Society, 1987.

31. Sanders GD. Automated creation of _{clinical-practice} guide-lines from decision models. PhD Dissertation. Stanford Med-ical Informatics. SMI Report No. SMI-98-0712, Department of

Computer Science Report No. STAN-CS-TR-98-1609.

Stan-ford, CA: Stanford _University, 1998.

32. Sanders GD, Nease RF, Owens DK. _Development and _pilot evaluation of automated _{computer-based} creation of

site-specific clinical-practice guidelines from decision models

(abstr). Med Decis Making. 1998;18:462.

33. Sanders GD, Nease RF, Owens DK. _Design and

implemen-tation of a _{computer-based system} to annotate decision mod-els for use in _{guideline development} (abstr). Med Decis

Mak-ing. 1998;18:469.

34. Scott GC, Cher DJ, Lenert LA. _{SecondOpinion:} interactive web-based access to a decision model. _Proceedings of the AMIA Annual Fall _Symposium, Nashville, TN, 1997. J Am Med Informat Assoc. 1997; 4, symposium supplement:769-73.

35. Kattan M. Decision _{Analysis/Utility} Assessment over the

In-ternet [Web document]. 1998: _{http://utility.urol.bcm.tmc.edu.} 36. TreeAge Software Inc. Product Info: Client-Server Edition

[Web document]. 1998: _{http://www.treeage.com/activex.htm.} 37. Lenert LA, Michelson D, Flowers C, Bergen MR. IMPACT: an

object-oriented graphical environment for construction of multimedia _{patient interviewing} software. _Proceedings of the Nineteenth Annual _Symposium on _{Computer Applications} in Medical Care, New Orleans, LA, 1995 J Am Med Informat Assoc. 1995; 2, symposium suppl:319-23.

38 Nease RF, Hynes LH, Littenberg B. Automated _utility assess-ment of _global health. Qual Life Res 1996;5:175-82.

39. Sanders GD, Owens DK, Padian N, Cardinalli AB, Sullivan AN,

Nease RF. A _{computer-based} interview to _identify HIV risk behaviors and to assess _{patient preferences} for HIV-related health states. _Proceedings of the Annual _Symposium on

Computer Applications in Medical Care. _{Philadelphia,} PA:

Hanley & Belfus, Inc.; 1994:20-4.

40. Sumner W, Nease R, Littenberg B. U-titer: a _utility assess-ment tool. _Proceedings of the Fifteenth Annual _Symposium on _{Computer Applications} in Medical Care. New York:

McGraw-Hill, 1991:701-5.

41. Owens DK. The use of medical informatics to _implement and

develop clinical-practice guidelines. West J Med. 1998;168: 166-75.

42. _Sonnenberg FA, Beck JR. Markov models in medical decision

making: a _{practical guide.} Med Decis _Making.

1993;13:322-38.

43. _Hayward RS, Wilson MC, Tunis _SR, Bass _{EB, Guyatt} G. User’s

guides to the medical literature. VIII. How to use the clinical

practice guidelines. A. Are the recommendations valid? Evi-dence-Based Medicine Working Group. JAMA. 1995;274:

570-4.

44. Richardson WS, Detsky AS. Users’ _guides to the medical

lit-erature. VII. How to use a clinical decision _analysis. B. What

are the results and will they help me in _{caring for my} pa-tients? Evidence-based Medicine _{Working Group.} JAMA.

1995;273:1610-3.

45. Richardson WS, Detsky AS. Users’ _guides to the medical

lit-erature. VII. How to use a clinical decision analysis. A. Are the results of the _study valid? Evidence-based Medicine

Working Group. JAMA. 1995;271:1292-5.

46. Wilson MC, Hayward RS, Tunis _SR, Bass EB, Guyatt G. User’s

guides to the medical literature. VIII. How to use clinical

practice guidelines. B. What are the recommendations and will _{they help you in caring for your patients?} Evidence-Based Medicine Working Group. JAMA. 1995;275:1630-2.

47. _Cheng CHF, Sanders GD, McDonald KM, Heidenreich PA,

Hlatky MA, Owens DK. _Design of a modular, extensible de-cision _{support system} for _{arrhythmia therapy. Proceedings} of the 1998 AMIA Fall _Symposium. Orlando, FL, 1998. J Am

Med Informat Assoc. 1998; 5, symposium