Evaluation Exercises on Semantic Evaluation - ACL SigLex event
#13 TempEval 2
Description Evaluating Events, Time Expressions, and Temporal Relations
Newspaper texts, narratives and other texts describe events
occurring in time, explicitly and implicitly specifying the temporal
location and order of these events. Text comprehension requires the
capability to identify the events described in a text and to locate
them in time.
We provide three tasks that are relevant to understanding the
temporal structure of a text: (i) identification of events, (ii)
identification of time expressions and (iii) identification of
temporal relations. The temporal relations task is further structured
into four sub tasks, requiring systems to recognize which of a fixed
set of temporal relations holds between (a) events and time
expressions within the same sentence (b) events and the document
creation time (c) main events in consecutive sentences, and (d) two
events where one syntactically dominates the other.
Data sets will be provided for five languages: English, Italian,
Spanish, Chinese and Korean. The data sets do not comprise a parallel
corpus and sizes may range from 25K to 150K tokens. The annotation
scheme used is based on TimeML. TimeML (http://www.timeml.org) has been
developed over the last decade as a general multilingual markup
language for temporal information in texts and is currently vetted as
an ISO standard.
Participants can choose any combination of the three main tasks and
the five languages.
Tempeval-2 is a follow-up on Tempeval-1, which was an initial
evaluation exercise based on three limited temporal relation
tasks. See http://www.timeml.org/tempeval-2/
for more information.