BCCWJ-TimeBank: Temporal and Event Information Annotation on Japanese Text

Temporal information extraction can be divided into the following tasks: temporal expression extraction, time normalisation, and temporal ordering relation resolution. The first task is a subtask of a named entity and numeral expression extraction. The second task is often performed by rewriting systems. The third task consists of event anchoring. This paper proposes a Japanese temporal ordering annotation scheme that is used to annotate expressions by referring to 'the 'Balanced Corpus of Contemporary Written Japanese' (BCCWJ). We extracted verbal and adjective event expressions as in a subset of BCCWJ and annotated a temporal ordering relation on the pairs of these event expressions and time expressions obtained from a previous study. The recognition of temporal ordering by language recipients tends to disagree with the normalisation of time expressions. Nevertheless, we should not strive for unique gold annotation data in such a situation. Rather, we should evaluate the degree of inter-annotator discrepancies among subjects in an experiment. This study analysed inter-annotator discrepancies across three annotators performing temporal ordering annotation. The results show that the annotators exhibit little agreement for time segment boundaries, whereas a high level of agreement is exhibited for the annotation of temporal relative ordering tendencies.