Transcript Sequencing of Events and Their Durations Based on Event

Temporal information extraction:
Reasoning with events based on their
descriptions
Leon Derczynski
University of Sheffield
Introduction

Background

Anchoring events

Reasoning about events

Representing temporal data

Evaluating annotations
Background

Why bother?

Temporal information affects everything
described by language.
The world is in a state that changes with time.
Not all assertions made in written text are true
together.



Temporal information shows which sets of data
can concurrently be true.
Tense and temporal models

Zeno Vendler (1957)
“Verbs and Times”

Hans Reichenbach (1947)
“The tenses of verbs”

James Allen (1983)
“Maintaining knowledge about temporal
intervals”
Vendler

Vendler verb classification:

Verb instances fall into one of four groups:





Stative: a persistent state (“John sits”)
Activity: lasts for a finite period
(“Bob ran for an hour”)
Accomplishment: takes a finite period, and
culminates (“Kate climbed the hill in five minutes”)
Achievement: Instantaneous finishing events (“Lucy
reached the top of Everest”)
Tests are provided to see which group a verb
sense fits in.
Reichenbach

Reichenbach model of verb tenses:



Speech time: when the words were uttered.
Event time: when the described event occurred.
Reference time: like a viewpoint.
“The cat will break the door” – ST=RT, ET in the future
“The cat will have broken the door” –
ST = present, ET in the future, RT looks back onto ET


Allows simplistic description of any phrase.
Tracking reference time is sometimes very helpful:
“When John comes home, I will have gone”
In this case, when describes a reference time for the
whole sentence.
Allen

Interval logic:




All events are described as intervals, with start
and end points.
Interval relation types are defined (before,
includes, starts…).
A table for inferring about interval relations is
given.
E.g.: A before B, A includes D:


Before stipulates that A’s endpoint is before
B’s start.
We can infer D before B.
Anchoring events

Introduction to event anchoring

Dealing with named weekdays

TEA – an implemented anchoring system

Problems
Anchoring events




Fixing information from text to a timeline.
Calendrical time is a common reference, given a
calendar.
Expressions describing a time are sometimes
referred to as TIMEXs.
Once identified, a TIMEX may be normalised to a
fully specified date or interval.


Named entity recognition, finite state grammars and
machine learning have all been used to identify these
expressions.
Appropriate granularity should be chosen.
Weekday references


The English week has seven day names.
A single day name is often deemed sufficient reference for
a human:



“I’ll see you next Tuesday”
“Monday, and the markets are buzzing”
To anchor a weekday, given ST and a day name, we need to
choose direction from ST, and optionally distance.




Baldwin: an inclusive 7-day sliding window, centred on today.
Mani & Wilson: find controlling verb’s tense and use this to
determine direction.
Tense estimation: check PoS of sentence tokens for VBD; if
found, assume backwards.
Dependency-based: use Stanford parser to find controlling
verb.
Mazur & Dale - “What’s the Date?” (2008)
Generic vs. specific


Some expressions, that look like TIMEXs, should not be
normalised.
“Today” can mean:


the 24-hour period containing ST and bounded by midnights.
Modern times, or a change of frame of reference:




“In Victorian times, ladies wore long dresses. Today, modern
fashions do not dictate a single length.”
This second idea is not restricted to the period from 00:00 to
23:59 GMT on Thursday 7th May 2009!
As 90% of uses in some texts are specific 1, some systems
choose to accept a 10% error rate.
Features based on local words can help distinguish generic
from specific, but below this baseline accuracy. 2
1: Han, Gates & Levin – “From Language to Time: A Temporal Expression Anchorer” (2006)
2: Mani & Wilson – “Robust Temporal Processing of News” (2000)
TEA





Temporal Expression Anchorer: Han, Gates & Levin at CMU.
Calendar used as time ontology, dealing with various levels
of granularity.
Processes TCNL (Time Calculus for Natural Language).
Identifies temporal expressions in input, and associates
TIMEXs with their textually nearest verb.
Absolute and relative expressions are evaluated using
TCNL:



“Friday last week” is split, into “Friday” and “last week”
{fri} + {now - |1week|} = {fri,{now - |1week|}} = {now |1fri|}
Constraint satisfaction based on a calendar model narrows
the possible set of absolute dates.
Determining event durations



Given some normalised expressions, knowing event
durations can greatly increase our reasoning ability.
Data can be taken from human annotators.
Determining a typical event duration is difficult:





“The dog ran up the hill”
“Linda had finished her cleaning”
This results in low inter-annotator agreement.
A simplified approach would allocate durations into two
classes: shorter or longer than a day.
Possible to classify events this simply with 76% accuracy,
using hypernym and local word PoS features.
Pan, Mulkar & Hobbs – “Learning Event Durations from Event Descriptions” (2006)
Reasoning about events

Introduction

Temporal closure

Minimal notations and temporal inference

Help from linguistic models
Reasoning about events

Annotations often only describe a subset of a document’s
temporal information, perhaps as a number of labelled
events and times.

An annotation may also include some links between pieces
of temporal information.
It is possible to infer data about relations between points,
given a set of rules or logic, and some existing relations.
It is also possible to add detail and boundaries to an
annotation based on linguistic features of the source text.



This ability to reason about events saves human annotators
work, and allows us to maximise the available descriptions
from their efforts.
Temporal closure

A temporal closure can be thought of as a graph:



Times and events are node; relations are edges.
Every time and event is connected to every other.
E.g.





t1 is Tuesday 5th May 2009
e1 is hearing this talk
We can say: t1 before e1, thus giving a type to this
relation.
A temporal closure includes relations between
every node in the graph.
This can lead to very large amounts of data for
only a moderate-sized document.
Minimal annotations

It is rare for every relation (graph edge) to be
annotated. We can infer some relations:




(t1 before e1) ^ (e1 before e2) => (t1 before e2)
Inference can be used to complete a closure
without specifying every relation’s type.
When this applies, and no more relations can be
removed, we have a minimal annotation.
For example:



Three nodes: e4, e5, e6
Closure has 3 possible relations
A minimal graph may just say:



(e4 after e5)
(e5 simultaneous e6)
To infer the closure, we simply need to add:

(e4 after e6), or (e6 before e4)
Relation inference

Allen’s interval logic describes 13 relationships, and
provides a transitivity table for inferring a relation given
two related ones.



Some inconsistent labellings are possible.
Backtracking over the initial graph should detect these cases.
A set of ten inference rules can be used:

Allen’s 13 relations are reduced to just 3, including some
reversal of parameters.




Only before, simultaneous and includes are used
e9 after e10 => e10 before e9
These rules can be iteratively added to an agenda and used to
reason with a database of approved relations.
For small graphs (< 2000 edges) we can assign types to
around 10% of relations, given a human annotation.
Applying Reichenbach

Reference time can provide a boundary on an event.

“John had eaten all the pies”



“John had eaten all the pies when Annika arrived”


ET – RT = ET2 – ST
Because we know that RT is after ET and equal to ET2, we can
specify three temporal relations:




Event 2 = arriving
Reference time is the same across the sentence.


Event 1 = eating
ET – RT – ST
e1 before e2
e1 before ST
e2 before ST
Having a model for tenses allows us to confidently add
relations to a temporal graph of a discourse.
Representing temporal data

Introduction

TIMEX and TimeML

TCNL

T-BOX
Representing temporal data


Once we can identify temporal information, we
need to store this information.
Temporal information is rich, and favours a
format that can capture it well.





Aspect, polarity, tense, part of speech
Event class, event frequency
Hints about reference, speech and event time
Notation languages are available both for storing
and working with this data.
These languages are new (under a decade old),
and possibly not yet mature.
TIMEX



Standard for describing a time-specific
expression.
Evolved through the MUC conferences and TERN,
through TIMEX, TIMEX2 and TIMEX3.
TIMEX3 is currently used as the means of
describing absolute times in TimeML.
<TIMEX3
tid="t43" type="DATE"
value="1989-10-30"
temporalFunction="false"
functionInDocument="CREATION_TIME">
10/30/89
</TIMEX3>
TimeML
SGML-based language for temporal
annotation.
 Allows identification of events and times.
 Thorough provision of links between
events and times:





TLINK: temporal, possibly including a SIGNAL
tag to a linking word
SLINK: subordinate
ALINK: aspectual
ISO standard.
TimeML - TimeBank


Corpus of 181 newswire texts.
Temporal information annotated in TimeML:






6383 TLINKs,
7940 EVENTs,
3004kB in size.
Tiny compared to some other types of corpus.
Involved a large human annotator effort and a
few different versions.
Biggest temporally annotated corpus.
TCNL



Developed at CMU with L. Levin.
Useful for reasoning between events.
Captures intensional meanings of expressions.


“Yesterday” becomes {now-|1day|} instead of
something like 20090506
A set of operators are used to reason between
operands:


+/- for forward/reverse shifting
@ for in;


{|2sun| @ {may}} is “the second Sunday in May”
& for distribution;

{15hour} & [{wed}:{fri}]} is “3pm from Wednesday to
Friday”
T-BOX


Reading solid SGML is inconvenient for humans;
a visual representation of events may be
preferable.
Presenting events on a timeline may lead to
unintentional over-specification.





Suggests a distance.
Many intervals are left with one end open
Plotting parts of a sentence in temporal order will
destroy word order, making it hard to read
Annotating documents can be done more easily
when events are grouped locally and visually
connected.
T-BOX1 from Brandeis specifies a set of rules for
rendering events and their relations.
1: Verhagen – “Drawing TimeML relations with T-BOX” (2007)
T-BOX

Relations only exist between nodes that are directly
connected or contained.
This suggests:
- X contains Y
- Y is before Z


Drawing a temporal closure could provide a very cluttered
and messy graph.
A set of guidelines are provided for reducing graphs to
something more visually appealing.



Equivalence classes for some events.
Break cycles in graphs.
Remove derivable relations.
Evaluating annotations

Typical annotation evaluation.

Graph-based evaluation.
Evaluating annotations
Annotations can be compared in different
ways.
 When evaluating automated TIMEX or
relation identification against a gold
standard, we can measure precision and
recall.



TimeBank is often used as a gold standard for
training and evaluation or systems working in
TimeML.
Evaluating TIMEX normalisation needs a
different measure, as there are varying
degrees of correctness available.
Graph based evaluation


Based on the use of minimal temporal graphs.
Graphs between events (intervals) are converted
into graphs between points:




Smaller set of relations, needing only = and <
Simpler algebra
Simultaneous points are grouped into nodes.
Graphs over the same set of points can then be
compared, based on the number of node splits
and merges needed to reach one from the other.
Summary
Background and models useful for
temporal information extraction.
 Technical approaches to temporal IE.
 How to reason about events.
 Temporal closure & minimal annotations.
 Notations for temporal information.
 Evaluating temporal graphs & annotations.

Questions