Transcript Document
„Application of Probabilistic ILP II“, FP6-508861 Probabilistic Logic al and Relational Learning Probability Logic James Cussens University of York UK Kristian Kersting University of Freiburg Germany Learning ICML-Tutorial, Banff, Canada, 2004 www.aprill.org Special thanks to the APrIL II consortium • „Application of Probabilistic ILP“ Heikki Mannila • 5 institutes • www.aprill.org Stephen Muggleton, Mike Sternberg Subcontractor: James Cussens Luc De Raedt Subcontractor: Manfred Jaeger François Fages Paolo Frasconi ICML-Tutorial, Banff, Canada, 2004 • 3 years EU project ... special thanks ... Alexandru Cocura, Uwe Dick, Pedro Domingos, Peter Flach, Thomas Gaertner, Lise Getoor, Martin Guetlein, Bernd Gutmann, Tapani Raiko, Reimund Renner, Richard Schmidt, Ingo Thon, ... ICML-Tutorial, Banff, Canada, 2004 ... for discussions, materials, and collaborations to • Introductory survey • Identification of important probabilistic, relational/logical and learning concepts ICML-Tutorial, Banff, Canada, 2004 Tutorial´s Aims Objectives The integration of probabilistic reasoning with first order / relational logic representations and machine learning. Probabilitiy Logic Learning ICML-Tutorial, Banff, Canada, 2004 One of the key open questions of AI concerns Probabilistic Logic Learning: Why do we need PLL? Diagnosis Web Mining Prediction Classification Decision-making Computational Biology Description Economic Let‘s look at an example Text Classification Computer troubleshooting PLMs ICML-Tutorial, Banff, Canada, 2004 Robotics Medicine Web Mining / Linked Bibliographic Data / Recommendation Systems / … [illustration inspired by Lise Getoor] book author book author publisher publisher book Real World ICML-Tutorial, Banff, Canada, 2004 book Web Mining / Linked Bibliographic Data / Recommendation Systems / … books B2 authors B1 A2 B3 series author-of publisher-of A1 P2 B4 P1 Fantasy Science Fiction Real World ICML-Tutorial, Banff, Canada, 2004 publishers Structured Domains Not flat but structured representations: Multi-relational, heterogeneous and semi-structured Dealing with noisy data, missing data and hidden variables Uncertainty Machine Learning Knowledge Acquisition Bottleneck, Data cheap Let‘s look at some more examples ICML-Tutorial, Banff, Canada, 2004 Real World Applications Why do we need PLL? Blood Type / Genetics/ Breeding Father Mother AA AA AA 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 AA Prior for founders 0.4 0.2 1 0.8 0.4 0.2 0 AA aa Aa 1 Aa aa 1 Aa aa aa aa 0.8 1 0.8 0.6 0.8 0.6 0.4 0.6 0.4 0.2 0.4 0.2 0 Aa 0 0.6 Aa 0.6 Aa AA AA Offspring 0 Aa Aa 1 Aa aa 0.2 aa 0 AA Aa aa 0 CEPH Genotype DB,http://www.cephb.fr/ ICML-Tutorial, Banff, Canada, 2004 • 2 Alleles: A and a • Probability of Genotypes AA, Aa, aa ? Others Social Networks Protein Secondary Structure Scene interpretation ? Phylogenetic Trees Metabolic Pathways ICML-Tutorial, Banff, Canada, 2004 Data Cleaning Why do we need PLL ? Real World Applications Probabilistic Logics Uncertainty - no learning: to expensive to handcraft models + soft reasoning, expressivity Structured Domains - attribute-value representations: some learning problems cannot (elegantly) be described using attribute value representations + soft reasoning, learning Machine Learning Inductive Logic Programming (ILP) Multi-Relational Data Mining (MRDM) - crisp reasoning: some learning problems cannot (elegantly) be described without explicit handling of uncertainty + expressivity, learning ICML-Tutorial, Banff, Canada, 2004 Statistical Learning (SL) • • • • • • Rich Probabilistic Models Comprehensibility Generalization (similar situations/individuals) Knowledge sharing Parameter Reduction / Compression Learning – Reuse of experience (training one RV might improve prediction at other RV) – More robust – Speed-up ICML-Tutorial, Banff, Canada, 2004 Why do we need PLL? • When it is impossible to elegantly represent your problem in attribute value form – variable number of ‘objects’ in examples – relations among objects are important • Background knowledge can be defined intensionally : – define ‘benzene rings’ as view predicates ICML-Tutorial, Banff, Canada, 2004 When to apply PLL ? Overview – Logic Programming, Bayesian Networks, Hidden Markov Models, Stochastic Grammars 3. Frameworks of PLL – – – Independent Choice Logic,Stochastic Logic Programs, PRISM, Bayesian Logic Programs, Probabilistic Logic Programs,Probabilistic Relational Models Logical Hidden Markov Models 4. Applications ICML-Tutorial, Banff, Canada, 2004 1. Introduction to PLL 2. Foundations of PLL