Transcript Introduction to Knowledge Based Systems (KBS)

Clinical Decision Support Lecture 1b
Brief History and State of the Art of Clinical
Decision Support
www.cs.man.ac.uk/ai/modules/cds
1
The Hype of the Time
• Guidelines
• Evidence Based Medicine
• Clinical Errors (reducing)
– Improving prescribing practice
– Reducing adverse drug reactions
• Protocols
• Knowledge Management
• ...
www.cs.man.ac.uk/ai/modules/cds
2
Clinical Judgement and Clinical Errors
• To Err is Human
http://www.nap.edu/books/0309068371/html/
• Supporting a Humanly Impossible Task
• Johnson Articles - see resources
www.cs.man.ac.uk/ai/modules/cds
3
Computer Aided Decision Support Works
(sometimes)
• Evidence of effectiveness growing
– 25 years since Clem McDonald’s
Protocol-based computer reminders, the quality of care and
the non-perfectability of man
• Use still limited
• Meta studies and reviews a decade old
• Elson R E and Connelly D P (1995). Computerized patient records in primary care: Their
role in mediating guideline-driven physician behaviour change. Archives of Family
Medicine 4: 698-705.
• Grimshaw J and Russell I (1993). Effect of clinical guidelines on medical practice: a
systematic review of rigorous evaluations. Lancet 342: 1317-1322.
• Johnston M, Langton K, Haynes R and Mathieu A (1994). Effects of computer-based
clinical decision support systems on clinical performance and patient outcome. A critical
appraisal of research. Archives of Internal Medicine 120: 135-142.
www.cs.man.ac.uk/ai/modules/cds
4
Examples of Protocols –
See handouts
www.cs.man.ac.uk/ai/modules/cds
5
Types of Decision Support: Information Tasks
• Informative
– Guidelines e.g. eBNF, BMJ Clinical Evidence,...
– Literature search - DxPlain
• Information structuring
– intelligent records (EPRs)
• PEN&PAD, Medcin vocabulary, ...
• Triggers and warnings
– MLMs, McDonald’s original work, HELP, ...
• Critiquing - Perry Miller
• Advising
www.cs.man.ac.uk/ai/modules/cds
6
Types of Decision Support: Clinical Tasks
• Management Protocols
(often effective, Johnston et. al1994)
– Prescribing
– Protocol based care
• Oncocin, T-Helper, etc.
– Referral
• Diagnostics
(rarely effective, Johnston et. al1994)
• Mycin
• Internist I
• Knowledge Couplers
www.cs.man.ac.uk/ai/modules/cds
7
Reasons for success and failure(1)
• Understanding of problem
– Meeting real and recognised needs
• Forsythe D E (1992). Using ethnography to build a working system:
rethinking basic design assumptions. Sixteenth Annual Symposium on
Computer Applications in Medical Care (SCAMC-92), Baltimore,
MD, Baltimore, MD: 505-509.
• Meeting them effectively
– “The user is always right…
but the user is usually wrong”
– The technology is still crude at best
• Implementing it successfully
www.cs.man.ac.uk/ai/modules/cds
8
Reasons for success and failure(2)
• Most projects fail at implementation!
• The technology only works if people want it and use it
– Requires emphasis on participation, ownership, training, respect for
practicalities
• ‘Implementation’ begins with design
• Evaluation begins with design
– Formative evaluation essential
www.cs.man.ac.uk/ai/modules/cds
9
Potted History (1)
• Bayesian stream
– 1968 Ledley and Lusted: Diagnosis using ‘Idiot Bayes’
discriminant
• Followed by Pauker Decision Support using utility theory
– 1970-1985 - de Dombal: ‘Idiot Bayes’ abdominal pain and other
surgical diagnostic problems
• Meanwhile RCP Computer Workshop refined discriminants and then
stimulated Spiegelhalter to come up with practical algorithms for
belief nets in early 1990s
– 1980s Society for Medical Decision Making formed and statistical
work largely separated from rule based work
www.cs.man.ac.uk/ai/modules/cds
10
Bayes Methods
• Probablistic
– If you think the patient has one of a set of diseases:
• e.g. for Acute Abdominal Pain: Appendicitis, Obstruction, Perforating
Ulcer, Pancreatitis, Gallbladder Inflammation, Tubal preganancy (if
female), or ‘Other’
– If you know
• a) That a patient has an indicant
• a) The prior probability of each disease
• b) The probability that a patient with each disease has the indicant
– You can calculate
• c) the posterior probability that the patient has each disease given that
they have the indicant.
– And you can do so for all of a set of indicants.
www.cs.man.ac.uk/ai/modules/cds
11
Bayes: simple example
• Prior probabilities: appendicitis 70% obstruction 10% other 20%
• Probability of indicant given diagnosis: (“vomiting without nausea”):
– 1% of patients with Appendicitis have indicant
– 25% of patients with obstruction have indicant
– 5% of patients with ‘other’ have indicant
Easy to collect
• Posterior probabilities of appendicitis given indicant
– appendicitis70*1/correction  17% of pts with indicant have appe’is
– obstruction 1*25/correction  60% of pts with indicant have obstr
– other
 2*5/correction  23% of pts with indicant have other
www.cs.man.ac.uk/ai/modules/cds
12
Potted History (2)
• Rule based stream
– 1972 - Shortliffe Mycin: First rule based system
– 1970s US AIM Workshop produced “Big 4”
• Mycin/Oncocin/Puff - Backwards chaining ‘shells’
• Interist I - NEJM CPCs from a large network
– Became QMR as a general reference
• Casnet - Multilayer causal reasoning (glaucoma)
• Abel - Complex causal networks (acid-base metabolism)
– 1990s Protocol based reasoning
• Protégé/Eon successors to Mycin/Oncocin at Stanford
– Musen MA. Domain ontologies in software engineering use of Protégé with the EON
architecture. SMI Technical Report 97-0657. Methods of Information in Medicine
37:540-550, 1998.
• ProForma at ICRF
• ASBRU
• PRODIGY III
www.cs.man.ac.uk/ai/modules/cds
13
Rules: Example from MYCIN
• IF the site of culture is blood
AND the method of culture is sterile
AND the aerobicity of the organism is anaerobic
THEN there is good (.65) evidence that the Diagnosis of
the Organism is Enterobacter
www.cs.man.ac.uk/ai/modules/cds
14
Potted History (3)
• Reminders
– 1970 - Homer Warner, HELP, LDS
• 1980s - Arden Syntax
• 1990s - MLMs - standardised Arden
– 1970s - Clem McDonald - ‘…reminders and the nonperfectability
of man”
• Regenstrief laboratory systems
– Many variations
• PRODIGY II
• Systematic Review: Johnston M, Langton K, Haynes R and Mathieu A (1994).
www.cs.man.ac.uk/ai/modules/cds
15
Examples
• CONDITION: the serum potassium is over 4.8 & the
patient is on any digitalis derivative
• ACTION: issue alarm
www.cs.man.ac.uk/ai/modules/cds
16
Potted History (4)
• Offshoots and Idiosyncratics
– Critiquing - Perry Miller
• Also Johan van der Lei
– Quick Medical Reference - Chip Masari
– Intelligent Records - Alan Rector and Anthony Nowlan
– Knowledge Couplers - Larry Weed
www.cs.man.ac.uk/ai/modules/cds
17
Potted History (5)
• Knowledge Management and the Web
– 1980s Grateful Med (PubMed) and DxPlain
• Quick access to Medline abstracts and related
– 1990s “The Web with everything”
• Rise of Evidence Based Medicine
– Cochrane, NICE, NELH, Health on the Web (HoN),…
• Indexing and ‘meta data’ & the Semantic Web
– How do you find it
• Portals and certification
– How do you know if it is any good
• Information for Public and Patients
– Its an open world out there
• Type “Diabetes Support” at Google 776,000 hits, AllTheWeb 295,000
Yahoo 26, Netscape 2000
• Classic Information Retrieval and Librarianship
– Digital Libraries
• Different fields with little contact
www.cs.man.ac.uk/ai/modules/cds
18
Potted History 7:
Cuing and Intelligent Medical Records
• PEN&PAD, MedCin, …
• Almost took off
• Where we came from…
www.cs.man.ac.uk/ai/modules/cds
19
Where we come from
Clinical
Terminology
GALEN
Clinical
Terminology
Data
Entry
Clinical
Record
Decision
Support
Data
Entry
Electronic
Health
Records
Decision
Support &
Aggregated
Data
www.cs.man.ac.uk/ai/modules/cds
Best
Practice
Best
Practice
20
Potted History 6:
Re-use, Terminologies and Ontologies
• Transferring the logic is easy
• Transferring the access rules in curly brackets is hard
– And it takes your most skilled people
• Subtle dependencies and system indiosyncracies
• The need for a common vocabulary
www.cs.man.ac.uk/ai/modules/cds
21
Why isn’t decision support in routine use?
• Hypothesis one: “Pearls before swine”
– Doctors are ‘resistant’
• Hypothesis two: “The Emperor’s new clothes”
– Systems are not clinically worthwhile
•
•
•
•
•
Not clinically useful
Too time consuming - too hard to learn
Too expensive
Too inaccessible
Too sparse
– How many diabetic patients does a GP see per week?
• Easier ways to get help
– The technology is still primitive
• Developers misunderstand medicine
– They think it is rational!
www.cs.man.ac.uk/ai/modules/cds
22
Why isn’t decision support in routine use?
• Hypothesis 3: “The invisible computer”
– When it works, no one notices
• ECG interpretation
• Alerts and reminders
• NHS Direct
– Simple but effective?
– Junior doctors’ PDAs
• Convergence of communication and computing
• Upmarket PDAs have 10-100 times the power of the machine that
first ran Mycin!
– Why Web technology and XML are critical to this course
• divorce content and presentation
www.cs.man.ac.uk/ai/modules/cds
23
A modern View
The Tripod
• Electronic Patient Records
• Decision Support
• Terminology and Ontologies
Plus
•
Knowledge Management/Information Retrieval
www.cs.man.ac.uk/ai/modules/cds
24
Patient Specific Records
Information Model
(Patient Data Model)
Inference Model
(Guideline Model)
Dynamic Guideline
Knowledge
Concept Model
(Ontology)
Static Domain
Knowledge (2a)
A Protocol
www.cs.man.ac.uk/ai/modules/cds
26
Who Should Be Evaluated for UTI?
Under the assumptions of the analysis, all febrile children between the ages of 2 months and 24 months with no
obvious cause of infection should be evaluated for UTI, with the exception of circumcised males older than 12
months.
Minimal Test Characteristics of Diagnosis of UTI
To be as cost-effective as a culture of a urine specimen obtained by transurethral catheter or suprapubic tap, a
test must have a sensitivity of at least 92% and a specificity of at least 99%. With the possible exception of a
complete UA performed within 1 hour of urine collection by an on-site laboratory technician, no other test meets
these criteria.
Performing a dipstick UA and obtaining a urine specimen by catheterization or tap for culture from patients with a
positive LE or nitrite test result is nearly as effective and slightly less costly than culturing specimens from all
febrile children.
Treatment of UTI
The data suggest that short-term treatment of UTI should not be for <7 days. The data do not support treatment
for >14 days if an appropriate clinical response is observed. There are no data comparing intravenous with oral
administration of medications.
Evaluation of the Urinary Tract
Available data support the imaging evaluation of the urinary tracts of all 2- to 24-month-olds with their first
documented UTI. Imaging should include VCUG and renal ultrasonography. The method for documenting the UTI
must yield a positive predictive value of at least 49% to justify the evaluation. Culture of a urine specimen obtained
by bag does not meet this criterion unless the previous probability of a UTI is >22%.
FOOTNOTES
The recommendations in this statement do not indicate an exclusive course of treatment or serve as a standard of
medical care. Variations, taking into account individual circumstances, may be appropriate.
www.cs.man.ac.uk/ai/modules/cds
27
The Technologies
• Semi Structured text and XML – GEM/GEMCUTTER
– Systematising what is needed
– XML is now the generic syntax for everything
• Find out about it from any handy book or web tutorial
• Knowledge representation, Ontologies, terminologies, –
Protégé, OilEd and OWL
• Rule based systems - Tallis/ProForma
• Bayesian inference MicroHugin
www.cs.man.ac.uk/ai/modules/cds
28
Exercises/Lab
• Follow links or your own knowledge to find a range of
“guidelines” on the Web.
– Compare what is on offer
•
•
•
•
•
Who are they for? What are they for?
Who has supplied them?
How might you use them as a clinician treating patients?
How might you author one?
How might you provide computer support to follow one?
• Pick a disease
– Look it up in Google with “protocol”, “guideline”, “systematic review”
• Answer the questions above.
• Find some protocols in your institution, if you work in the NHS
– answer the questions above.
www.cs.man.ac.uk/ai/modules/cds
29
Apology
…We hope this is fixed…
• Illogical order due to technical difficulties
– Due to the wrong kind of .dll in the MS automatic upgrade, we will
discuss & demonstrate GEM later when the software is has been
upgraded to fit the new ‘standard’. Microsoft is unlikely to
apologise for the inconvenience this will cause to your module, but
we apologise and will do our best to cope.
www.cs.man.ac.uk/ai/modules/cds
30