Transcript Document

A Graphical Framework for
Specification of Clinical
Guidelines at Multiple
Representation Levels
Erez Shalom, B.Sc. and
Yuval Shahar, M.D., Ph.D.
Medical Informatics Research Center
Department of Information Systems Engineering
Ben Gurion University, Beer Sheva, Israel
The need for Automation of Clinical
Guidelines
 Following Clinical guidelines (GLs) has been
shown to improve the quality of medical care
 Automatic support provides:
Visual specification
Search and retrieval
Application of a GL
Retrospective quality assurance
 However: Most GLs are text based and
inaccessible at the point of care
The Required Infrastructure



A machine-comprehensible GL
representation language
Support for a gradual structuring of the
GL (from text to an executable code)
Runtime GL application and QA tools
The Structuring Process
The Guideline as a text document
The Guideline as a tree of plans
Plan 1
D P
In parallel
Plan 1.1
D P
Plan 1.2
D P
Involves 2 main types of knowledge:
Procedural knowledge – e.g. plan for administer the two medications in parallel
Declarative knowledge - e.g. the eligibility criteria “Is drug available?”
Several Unresolved Issues




Unclear division of the responsibility between
the knowledge engineer (KE) and the Expert
physician (EP)
The transformation into a formal representation
is often done in one step
Lack of a detailed methodology for the overall
process
Usually the structuring methodology is focused
on only one GL-specification language/ontology
The hybrid representation model
Gradually structuring the GL using increasingly formal representation levels
Expert Physician Knowledge Engineer
Expert
Physician
Collaboration
Knowledge
Engineer
DeGeL KB
Semantic
markup
Adding control
structure
Formalizing to
executable code
//Check HGB
If(HGB > 12)
{...
}
Free text
Guidelines
Semi- Structured
Level
Semi- Formal
Level
//Check HGB
If(HGB > 12)
{...
}
Formal
Level
 Implemented as part of DeGeL [Shahar et al, JBI 2004]
Used with the URUZ GL markup tool
URUZ (I):
Specification of declarative knowledge
Expert physician
Selects “filter
condition”
knowledge role
URUZ (cont’d):
Specification of procedural knowledge
Expert physician decomposing the GL
into tree of plans
The URUZ evaluation: Insights




Structuring guidelines by expert physicians, in
principle is feasible
Creating a ontology-specific consensus is an
essential preliminary step!
A comprehensive methodology was developed for
the overall process
An complete evaluation model and tool was
developed and used
The URUZ Evaluation: Conclusions
 Interface is cumbersome and not user friendly
(average Scalable Usability Scale: 47.5 [over 50 is
user friendly])
 The initial procedural structuring should be
independent of any specification language
 There is a need for customized graphical widgets
especially for the procedural knowledge
 Additional representation levels should be supported
 We now know what users need:
GESHER - A Graphical Framework for Specification of Clinical
Guidelines at Multiple Representation Levels
The GESHER Architecture
DeGeL KB
//Check HGB
If(HGB > 12)
{...
}
Procedural
Guideline
Specification
Knowledge
Base
Temporal
Absraction
KB
Medical Standard
Vocabularies
Server
LOINC
ICD-9
CPT
Declarative
Concepts
Standard
Terms
GESHER Client Application
Expert Physician
Collaboration
Knowledge Engineer
The GESHER’s Main Features
 User friendly graphical client application
 Support specification at multiple
representation levels
 Support to multiple specification languages
(GL ontologies)
 Access centralized resources such as
DeGeL [Shahar et al, JBI 2004] and a
knowledge base
GESHER: Semi-Structured Level
The Hybrid
Ontology Tree
showing KRs
at all
representation
levels
GESHER(I) : Semi-Formal Representation Level
Plan
Structuring
white Board
The basic set
of clinical -plan
types
The guideline’s
hierarchical
plan-body
generated
GESHER(II) :Semi-Formal Widgets
The Expression Builder –
Acquisition of Semi-Formal Expressions
E x p e r t
physician uses
pre-defined
condition types
for specifying
semi-formal
expressions
Ongoing Evaluation of URUZ and
GESHER
 Three GLs in different domains were used :
Pelvic Inflammatory Disease (PID)
Chronic Obstructive Pulmonary Disease (COPD)
 Hypothyroidism
 For each GL:
 Collaborative consensus was formed with Expert physician and
knowledge engineer
 Three markups performed by
experts physicians
 One of the markups which was build with knowledge engineer and
Expert physician was used as Gold Standard
 7 different questionnaires were filled by EPs and KEs participated in the
evaluation
Several Conclusions


For making an ontology-specific consensus,
understanding the Asbru semantics, as well as their
own knowledge, helped EPs more than knowing the
different DEGEL tools
EPs understand better intuitive KRs such as Actors,
level of evidence or clinical context, rather than
Asbru’s more semantically oriented KRs such as
eligibility criteria, or control structures such as
switch-case .
Summary
 The need for gradual GL specification
 Making an ontology-specific consensus as first step
Use a well defined methodology for the overall
process
 Current tool : URUZ
 Improved graphical framework: GESHER
 Both tools are now ongoing a formal evaluation
 Future work:
Finish evaluations
Develop formal widgets
Creation and use of graphical templates for GL
specification
Acknowledgments






All Medical informatics research center members
Our colleagues at Stanford and VA hospital: Drs. Mary
Goldstein, Susana Martins, Lawrence Basso, Herbert
Kaizer, Laszlo Tudor Vaszar
Our colleagues at Soroka’s university medical center :
Prof. Eitan Lunenfeld, Dr. Avi Yarkoni, Dr. Guy Bar,
Prof. Yair Liel and Dr. Tal Merom
NLM award No LM-06806
Contact info : [email protected]
Visit our web site : http://medinfo.ise.bgu.ac.il/medlab/
Questions?
Consensus Formation:
The Basis for The Structuring Process
EPs create a Clinical consensus :
Give Doxycycline 100 mg orally or IV every 12 hours Plus
Metronidazole 500 mg IV every 8 hours
EPs and KE adds procedural knowledge :
Order :parallel
Doxycycline
Metronidazole
EP+KE add
declarative
knowledge :
Ontology- specific
consensus
Filter condition for drug :
Is patient not sensitive to Doxycycline and
the drug available?
Methodology of the structuring
process

Before Markup:
 Make
Consensus
 Read consensus and GL sources
 Have some training with the tool
Perform Structuring according to
consensus and GL sources
 Evaluate the markup

completeness
num
category
grade
1.2.1
identical to GS
1
This is best markup
* (1.3.1 and 1.3.3 are checked)
1.2.2
exist in GS and missing in
markup
-1
We assume that missing content is wrong
* (1.3.1 and 1.3.3 cannot be checked)
1.2.3
exist in markup and missing in
GS
0
Generally don't care
* ( should check for correctness)
1.2.4
exist in markup partially
compared to GS
0
Most of the content complete
* ( should check for correctness)
-1
correctness
Why this grade
content not complete at all
* ( should check for correctness)
1.3.1
correct clinically
1
Not worsing patient outcome
1.3.2
incorrect clinically
0
Not worsing patient outcome
-1
worse patient outcome
1.3.3
correct according to Asbru
semantics
1
Not worsing patient outcome
1.3.4
incorrect according to Asbru
semantics
0
Not worsing patient outcome
-1
worse patient outcome
Question : In what way did each of the following subjects
help you to make an ontological consensus ? (-3 / 3 scale)
Subject
EP1
EP2
EP3
Average
Your expertise
2
3
3
2.67
Asbru KR’s – Declarative Part
3
2
2
2.33
Reading the guideline sources before
making consensus
2
3
2
2.33
Knowing the multiple representation
level model
3
3
0
2.00
Asbru KR’s – Procedural part
2
2
2
2.00
Having more than one source
3
0
3
2.00
Ontology
1
3
0
1.33
DeGeL
0
2
0
0.67
URUZ -main interface
1
1
0
0.67
IndexiGuide
0
2
0
0.67
URUZ –plan body wizard
1
0
0
0.33
Vaidurya
0
1
0
0.33
Vocabulary server
0
0
1
0.33
Spock
0
0
0
0.00
Question: How easy was it for you to understand each of
the following knowledge Roles? (-3/ 3 scale)
Knowledge Role
EP1
EP2
EP3
EP4
Average
Actors
3
3
3
3
3
A process still to be defined
3
3
3
3
3
Level of evidence
3
3
3
2
2.75
Strength of recommendation
3
3
3
2
2.75
Clinical context
3
3
3
2
2.75
Abort Condition
3
1
3
3
2.5
Complete Condition
3
1
3
3
2.5
Simple Action
3
2
3
2
2.5
Filter Condition
1
3
3
2
2.25
Reactivate condition
-1
3
3
1
1.5
Subplans – sequential order
-2
3
3
2
1.5
If-then -else
-3
2
3
3
1.25
Repeating plan
0
1
2
2
1.25
Subplans – any order
-3
3
3
2
1.25
Suspend condition
-1
1
2
2
1
Subplans – parallel order
-3
3
3
1
1
Guideline knowledge
-2
2
1
2
0.75
Intentions Overall - outcome
-2
-2
3
3
0.5
Question: How easy was it for you to understand each of
the following knowledge Roles? (-3/3) Cont..
Knowledge Role
EP1
EP2
EP3
EP3
Average
Plan activation
-3
0
3
2
0.5
Subplans –unordered
-3
2
-1
3
0.25
Intentions Overall - process
-2
-2
2
2
0
Intentions intermediate - outcome
-2
-2
2
2
0
Setup Condition
-1
-1
1
1
0
Intentions intermediate - process
-2
-2
-2
0
0.5-
Switch case
-3
-1
-1
3
0.5-
Consensus example – procedural part
Diagnosis (Squential
- any order)
PID(Squential)
History
Diagnosis
Patient and sex partner Treatment
and evaluation (parrallel)
Patient Treatment and evaluation
Sex partner Treatment and
evaluation(parrallel)
Treatment -
empiric treatment for
C. trachomatis and
N. gonorrhoeae
Evaluation
FollowUp (parallel)
patient
Sex partner
Physical
exmination
Laboratory
test
See 1.1
1.1 Patient Treatment and evaluation
Patient Treatment and evaluation
is PID
severe
?
NO
Outpatient Treatment and
evaluation (parallel)
Per OS Treatment
Per OS Treatment
Evaluation
YES
Hospitalization
See 1.6
See 1.7
See 1.2
Consensus example – declerative part
Top level - PID
Level of evidence
?
Strength of recommendation
?
Actors
Doctors - Gynecologist clinics
Clinical context
ER, ward , OR,
Intentions
Overall - outcome
To Cure PID Or
Filter Condition
(suspected pid) and (sexually active female)
Set Up Condition
?
Abort Condition
Elimination of diagnosis of PID Or patient died
Complete Condition
No further treatment (IV or Per Os or surgery) or follow up is needed
substantial clinical improvement
Hospitalization
Level of evidence
Strength of recommendation
Actors
Doctors - ward's Gynecologist
Clinical context
ward or operation room
Intention intermediate process
To treat severe PID
Filter Condition
Severe PID Or failure of oral treatment Or tubo-ovarian abscess Or Surgical emergencies or (mild pid and
(pregnant or HIV positive))
Set Up Condition
Abort Condition
Complete Condition
discharge from hospital
Digital Electronic Guideline Library )DeGeL)
Physician
Clinical
Guideline
Server
Guidelines
DeGeL Digital electronic
Guideline Library
GLVaidurya
Search and
Retrieve tools
retrieval
Knowledge
engineer
URUZ/GESHER
GL Specification,
Representation
and classification
tools
GL Application
and Quality
Assessment tools
Vocabulary
server
Server
IDAN Mediator to time
oriented clinical data
Sample GL Modeling Methods
Method
Knowledge acquisition tool
EON and SAGE, Prodigy,
GLIF
A Protégé-based interface
PROforma
Arrezo
GEM
GEM-Cutter
GLARE
"CG_AM" graphical interface
GUIDE
NEWGUIDE
Asbru
Asbru-View, GMT, Stepper,
URUZ, GESHER