- Original Article
- Open Access
Semantic relation identification for consecutive predicative constituents in Chinese
© The Author(s). 2017
- Received: 6 June 2016
- Accepted: 25 August 2017
- Published: 17 October 2017
In this paper, we propose a general methodology for designing semantic role/relation system. Based on this methodology, we establish a succinct semantic relation system for consecutive predicative constituents for Chinese, which includes serial verb construction, discourse construction, and other constructions describing serial events. This semantic relation system has 13 middle-level classes and 24 fine-grained sub-classes in contrast to conventional complex classification schemes and meets the uniqueness and completeness criteria of semantic relation identification. We conduct experiments on our system by training four annotators in 1 h to label 200 sentences extracted from Sinica Treebank and HIT-CDTB. With the help of our predesigned feature-based decision tree and a connective markers checklist, the annotators attain a 73% consistency with the reference standard annotation and substantial agreement by Cohen’s kappa coefficient for middle-level labeling. By analyzing the labeling error types, we slightly revise our classification scheme and propose six methods to improve the classification and labeling system, hoping to achieve even better agreement in the future.
- Semantic relation identification
- Semantic roles
- Feature-based semantic relation system
- Serial verb construction
- Discourse construction
- Discourse relation recognition
Essential to natural language understanding are the processes of part-of-speech tagging, parsing, and semantic relation identification. In this paper, our objective is to clarify the relations between consecutive predicative constituents (abbreviated CPCs), which include serial verb construction (abbreviated SVC), discourse construction, and other constructions describing serial events in Chinese text and to find a good and workable semantic relation system for semantic role/relation labeling tasks. As a consequence, a methodology of semantic role/relation design methodology was also established.
The adults hurried to go uphill (V1) to hunt the tiger (V2).
It tired the students out (V2) to go uphill (V1) early in the morning.
如遇下雪(V1), 一般車輛避免(V2)上山, 以免發生(V3)危險
condition-result between V1 and V2; event-avoidance between V2 and V3
If it snows (V1), it is better to avoid (V2) driving uphill lest danger occurs (V3).
Most studies discuss different constructions separately. However, when studying semantic relations between CPCs in different constructions, it is not necessary to regard them as distinct phenomena. Zhou and Xue (2012) described four characteristics which blur the boundary between discourse construction and serial verb construction. They are as follows: (i) semi-colon is not always used to separate the sentences; (ii) in most of the cases, no explicit discourse connectives are used to denote the discourse relations; (iii) no inflectional clues to differentiate free adjuncts and main clauses; and (iv) both subject and object can be dropped in Chinese. For instance in (2), there are no essential reasons we need to separate (a) and (b) into different categories of discourse construction and serial verb construction.
Her husband died in a car accident.
她丈夫車禍, 所以過世了。 Discourse Construction
Her husband met with a car accident, so died.
In this paper, we are targeting to identify semantic relations between CPCs in Chinese. However, our proposed design methodology is applicable to develop all semantic relation systems not limited to consecutive predicative units. To focus our studies, we exclude predicate-argument relations in our discussion and we do not account for the problem of delimiting related and unrelated two predicative constituents.
To sum up, CPCs are beyond any syntactic restriction, indicating the constructions including two events which describe the same subject or an identical topic such as discourse construction, serial verb construction, modifier–event construction (e.g., 傷重過世 shāngzhòng guòshì ‘seriously injured and died’), and causal event–resultative event construction (e.g., 過世留下遺產 guòshì liúxià yíchǎn ‘died and left a legacy’). In this paper, we aim to distinguish the relation between both inter- or intra-sentential CPCs; not only discourse construction and SVC are included but also sentences modified by a prepositional phrase or a complemental phrase are all taken into account. By following our design methodology, we integrate different surface forms of Chinese constructions from Sinica Treebank (Chen and Huang 2004) and HIT-CDTB (Zhang et al. 2014). It results in a hierarchical relation system of 24 fine-grained semantic relations, practically achieving the completeness and uniqueness criteria of relation identification.
The rest of the paper is organized as follows. In Section 2, we provide necessary background information and address the importance of design methodologies for semantic relation system. After introducing the distinction between semantic role labeling and semantic relation identification, we review the relevant literatures in Section 2.2. In Section 3, we motivate the need for a new relation system and propose a feature-based design methodology in Section 3.1. Following that, in Section 3.2, we describe the design of our semantic relation system; in Section 4, a guideline for semantic relation identification is addressed, and an experiment to verify the completeness and distinctness of subordinate relations is described and discussed in Section 5. We conclude the paper in Section 6.
2.1 Semantic roles and semantic relations
Semantic role is the role of a dependent daughter with respect to its head constituent. On the other hand, semantic relation means relations between any two related constituents. Therefore, semantic relations have a broader coverage than head-dependent relations, since other than head-dependent relations, they also include coordinate relations and discourse relations.
Conventional text annotation, such as treebanks, might annotate syntactic dependent structures and semantic roles of constituents, such as Sinica Treebank (Chen et al. 2003). How is the semantic role of a dependent daughter determined? Usually, it is a result of considering the parameters of head constituent, dependent daughter, role marker, and phrasal/sentential pattern. By those factors, we decide a best role to describe this dependent daughter. The premise of above role assignment scheme is that there is only one semantic role for each dependent daughter and there is a limited set of predetermined semantic roles which you may choose from. Such a simple schema annotates only the semantic role of dependent daughter of head-dependent relations without considering coordinate relations and relations across sentences, such as discourse relations. Furthermore, a naïve role labeling system may cause the following problems.
張三打破花瓶 (The vase is the patient of 打 dǎ ‘hit’ or theme of 破 pò ‘broken’.)
Zhangsan breaks the vase.
花瓶打破了 (The vase is the theme of 破 pò ‘broken’.)
The vase was broken.
The second problem is none-exclusiveness of semantic roles/relations. Semantic roles/relations may categorize into different feature dimensions. The relations between two constituents may be described by different categories of relations. For example in (3), the role of 花瓶 huāpíng ‘vase’ may be described by either dynamic relation of patient or static relation of theme. Similarly temporal relations, cause-result relations are falling into two different feature dimensions and the two relations may co-exist in CPCs. Conventionally, semantic roles may each other share the same characteristics and have idiosyncrasies. Each semantic role can be characterized by a few semantic features. The best role describing a dependent daughter is the role which matches most semantic features of the target constituent in its contextual environment (Dowty 1991). It causes competition among several possible candidate roles. Then, to determine the best role, in addition to feature matching, should each feature assign different weights in different contextual environments?
張三被屠殺。(Zhangsan is the patient of “slaughter”; it focused on the dynamic feature dimension.)
Zhangsan was slaughtered.
張三遭屠。 (Zhangsan is the theme of “being slaughtered”; it focused on the static feature dimension.)
Zhangsan was slaughtered.
張三把牛肉軟化了 (Role marker bǎ delimits “beef” as a patient, i.e. instead of the main verb, bǎ becomes the head.)
Zhangsan softened the beef.
牛肉軟化了 (“Beef” is the theme of “soften”.)
The beef is softened.
他買票(V1)上車(V2)。 (Do temporal relation, condition-consequence, cause-result, or elaboration all exist between V1 and V2?)
He bought a ticket (V1) and got on the bus (V2).
A better relational system for human annotators and also for future automation should meet the criteria of uniqueness and completeness; Uniqueness means two CPCs may assign a semantic relation (mostly are formed by a pair of semantic roles) which best describes their semantic relation. Completeness means two CPCs may be assigned some semantic role(s) to describe their semantic relations. That is, semantic relations between two CPCs are best described by one of the relation labels in the system without ambiguity.
Therefore, in designing a semantic relation system for CPCs, we encounter the following problems: How many relations are needed to meet the completeness criterion, that is, are all semantic relations between CPCs covered by the proposed classification system? How to meet the uniqueness criterion, that is, achieve a unique and consistent best labeling for CPCs? In the meanwhile, should we allow multiple relations while different interpretations occur? In the following subsections, by studying the related work and summarizing previous systems and offering our viewpoints, we attempt to answer the above questions and come up with a workable design methodology and a practical semantic relation system.
2.2 Related works
In the late 1980s, Mann and Thompson (1988) proposed a new interpretation of rhetorical structure theory (abbreviated RST) to describe natural texts, characterizing their structure primarily in terms of relations that hold between parts of the text. By examining real data, they provided a list which enumerates those which have proven the most useful 23 relations, including circumstance, solutionhood, elaboration, background, enablement, and motivation. Continuing the RST framework, Carlson and Marcu (2001; RST-DT) in their tagging reference manual claimed 16 classes covering a total of 78 relations, including attribution, background, cause, comparison, and condition.
Hovy and Maier (1992) summarized a survey of the conclusions of approximately 30 researchers who proposed more than 400 intersegment relations in different classification systems. Hovy then suggested using just as many relations as are required for determining the major aspects of English discourse structure, that is, approximately 70 relations, organized into a hierarchy of increasing specificity. The top-level classifications are divided into three parts: Ideational relation is defined between adjacent segments of material as those relations that express some experience of the world about us and within our imagination, for example, circumstance, cause/result, and general condition. Interpersonal relation is defined as holding between adjacent segments of textual material by which the author attempts to affect the address’s beliefs, attitudes, desire, and so on, by means of language, for example, interpretation and enablement. Textual relation is defined as holding between adjacent segments of text which exists solely due to the juxtaposition imposed by the nature of the presentation medium, for example conjunction and pre-sequence.
Miltsakaki et al. (2008) proposed four classes covering a total of 38 fine-grained relations which are annotated to the Penn Discourse Treebank (abbreviated PDTB), the largest-scale annotated corpus at the discourse level. Their relation structure is therefore most prevalent. The top-level classifications are divided into four parts: temporal is used when the situations described in the arguments are related temporally; contingency is used when the situations described in the arguments are causally influenced; comparison applies when a discourse relation is established between arguments in order to highlight prominent differences between the two situations; and expansion groups all the relations which expand the discourse and move forward its narrative or exposition.
Based on the lexically grounded approach of PDTB, Zhou and Xue (2012, 2015) presented a Chinese discourse annotation scheme and focused on the key characteristics of Chinese text which differs from English that we have mentioned in previous section. They claimed promising results on identifying a discourse relation; classifying the semantic type of explicit, implicit, or altLex; and determining the argument span. The agreements of the former are both over 95%, and the latter is over 80%.
Also based on PDTB classification, Huang et al. (2014) adopted discourse connectives to reveal explicit discourse relation in Chinese. They found there are 808 Chinese connectives which are eight times more than English connectives. Moreover, the Chinese discourse connectives have a variety of parts of speech that further deepen the difficulties of Chinese discourse relation labeling. By using semi-supervised learning method, the labeling result shows an F-score of 73.22%.
Zhou et al. (2014) followed the annotation scheme of PDTB and presented the first open discourse treebank for Chinese (abbreviated DTBC). They modified the PDTB sense hierarchy; three type level senses (i.e., CONTINGENCY. Inference, CONTINGENCY. Purpose and EXPANSION. Background) and two subtype level senses (i.e., EXPANSION. Conjunction. parallel and EXPANSION. Conjunction. progressive) were added to meet the needs of Chinese textual characteristics. They reported an over 90% inter-annotator agreement on discourse connective identification and an over 85% on sense annotation.
Li et al. (2014), based on the annotation of discourse connective in the Chinese discourse treebank (abbreviated CDTB), propose a three-level classification with a total of 17 fine-grained relations. The top-level classifications are divided into causality, transition, coordination, and explanation, and the third level merely lists the corresponding connective markers belonging to the middle-level types. The CDTB annotation is done by five long-term training annotators. The result showed a 94% inter-annotator agreement on explicit or implicit identification, an 82.3% agreement on explicit connective identification, and a 74.6% on implicit connective insertion.
Regarding practical relation identification problems, Wang et al. (2010) interpreted the implicit semantic relation between N-N compounds by adopting a dynamic approach using paraphrasing verbs. Not giving a set of relation candidates, they recognized each relation of N-N compounds by collocate verbs, for example, through Chinese word sketch engine finding people tell/seek a 愛情故事 àiqíng gùshì ‘love story’; and 民間故事 mínjiān gùshì ‘folk tales’ are stories that come from/spread in villages, then further recognize the fine-grained differences of meaning between the related compounds. On the contrary, Hong and Huang (2015) revealed the semantic relation between V-V compounds using an ontology-based conceptual classification, where three types of eventive relations, i.e., coordinate, modificational, and resultative, are predicted automatically. Xu and Huang (2014) discriminated sentences into general events, speech act, and modality types which is namely a task of event type classification.
As given above, many classification systems for discourse relations had been established and experiments on different aspects of discourse relations were carried out. The proposed systems were proven to be sound. Unfortunately, hardly any previous mentioned systems meet the uniqueness criterion, since it is almost impossible to achieve mutually exclusivity for all relations. We agree that multi-relation interpretations between CPCs do exist but to achieve a unique annotation scheme which is also in line with the procedure of human understanding, i.e., always select the best interpretation among many possibilities. Therefore, our semantic relation system adopts a feature-based decision-making methodology described below to achieve the best labeling.
3.1 Methodology for designing a semantic role system
Our proposed design methodology tries to provide a methodology for designing a semantic relation system to avoid the problems caused by conventional semantic role labeling system mentioned in the Section 2.1 and to achieve the uniqueness and completeness criteria of relation identification.
To deal with the first problem of multiple roles, we suggest labeling semantic relation between any two related constituents instead of semantic roles, i.e., each constituent may have many relations each with respect to a different related constituent.
For the problem of ambiguous role assignment caused by none-exclusiveness of semantic roles/relations, we propose a multi-level refinement and feature-based approach. A relation system should be designed in a hierarchical way from top level to bottom level. Each level of relations is differentiated by a salient feature from top to bottom to form a binary branching. For instance, the most salient feature to differentiate CPCs relations is +realis. Two types of relations are divided. One is relations regarding realis events (facts) and another is relations regarding irrealis events (opinions). We then refine each type of relations into different levels of fine-grained relations according to different dimension of relation types and semantic features from different aspects. For instance, the realis relations can be refined according to the feature of +intension into relations with purpose and relations without purpose. The resulting hierarchical relation system is also formed a decision tree which is utilized to find the unique best relation among ambiguous multiple relations to achieve the uniqueness criterion. As for the criterion of completeness, we propose a corpus-based approach; each preliminary designed feature-based semantic relation system should be tested and verified by a large set of real data from corpora. New features and new relations can be added, and the preliminary system will be refined accordingly until all real data can be satisfied.
3.2 Our design
- a.causality[-intentional] vs. purpose[+intentional]
搶劫(V1)被逮(V2) cause-result vs.
robbed and be caught
gather evidence to expose one’s secret
- b.purpose[+intentional] vs. background[-intentional]
出國(V1)留學(V2) means-purpose vs.
搭機(V1)前往(V2) manner- head event
go by airplane
- c.purpose[+intentional] vs. sequence[-intentional]
整裝(V1)出門(V2) means-purpose vs.
dressed to go out
go home after school
Since the type of purpose implies an intention behind an action, it is an effective feature to distinguish it from the other types as shown in (8). Comparatively, events labeled with purpose contain intention and motivation which lead the actor to pursue his act. On the contrary, the actors are less aware of what they can achieve by engaging in an action related with causality, background, or sequence relation types.
- (9)and [+temporal] vs. listing [-temporal]
忽然宣佈(V1)關閉機場, 也下令(V2)禁止所有集會活動 and vs.
Suddenly the closure of the airport was announced, and all gatherings were banned too.
暫停包括香港飛(V1)臺北, 以及臺北飛(V2)香港航班 listing
Paused flights include Hong Kong to Taipei and Taipei to Hong Kong.
Belonging to the synchronous type, and is used to connect events that occur at the same time, while listing is simply used to link items being described that are not involved within the timeline. And is also easily confused with apposition, and apposition sometimes confused with result; (10) and (11) demonstrate the difference between these subtypes.
- (10)and [−synonym] vs. apposition [+synonym]
一群人又跳(V1)又叫(V2) and vs.
A group of people dancing and shouting.
把自己好好梳妝(V1)打扮(V2)一番 head event-apposition
dress and decorate oneself properly.
- (11)apposition [+synonym] vs. result [+result state]
被騙(V1)上當(V2) head event-apposition vs.
be cheated and fooled
哭(V1)不停(V2) head event-result
Apposition is established only when V1 and V2 are synonymous; result is assigned when the second event denotes the result state of the first action. Along with these features, each subordinate type and subtype in Fig. 1 has been defined specifically; see Appendix 1; connective words that help reveal explicit relations are also attached to each correspondent subtype; see Appendix 2, Table 4.
To meet the completeness criterion, we have tried to identify as many semantic relations as possible that CPCs may have and we define the relation of elaboration and addition rather flexible and inclusive, which makes the completeness criterion easier to be achieved. The completeness criterion is verified by labeling over 50 paragraphs in HIT-CIR Chinese Discourse Relation Corpus (HIT-SCIR 哈工大社会计算与信息检索研究中心 2013) and 3000 sentences in Sinica Treebank (Chen and Huang 2004), to ensure sufficient coverage of the relations shown in Fig. 1 for Mandarin Chinese.
4.1 Three factors that affect the deduction of relation recognition
He finished the meal and left.
坐(V1)著看(V2)書 manner-head event
sit and read.
美(V1)得沒話說(V2) head event-result
[It is] too beautiful to describe.
He is poor but still happy.
[He] went to the capital to sit a civil service examination.
get sick and stay in the hospital.
In order to take a break [he] goes outside.
go outside and take a break.
Using common-sense knowledge, relations are often deduced from the knowledge of entailment, implication, or presupposition between CPCs, as shown in the following examples respectively.
消失(V1)不見(V2)了 head event-apposition V1 entails V2
vanish and disappear
喝酒(V1)醉倒(V2) cause-result V2 implies V1
drink wine and become drunk
水加熱(V1)到一百度沸騰(V2) condition-result V2 presupposes V1
Heat water up to 100 degrees and it will boil.
Spain was defeated and surrendered.
Spain surrendered because it was defeated.
Table was tidied up and cleaned.
open the door to go out
go out to open the door
In addition, combinations of specific verbs may be idiomatic and thus seldom reversed, as shown in (21).
Wishing you happy travels.
More examples are provided to illustrate that context and world knowledge are often necessary for a correct understanding of CPCs. As in (22), 想 xiǎng ‘think’ naturally precedes 回答 huídá ‘reply’ to reflect the process of human mental behavior, and in (23), the sequence of 救 jiù ‘rescue’ and 上岸 shàngàn ‘ashore’ demonstrate the factual process of rescue. In (24), we either infer from the context that one animal is sick so the speaker does not want it, or our common sense tells us that illness makes something undesirable; in either case, a cause-result relation between V1 and V2 is determined.
He thinks a while and answers me.
The fisherman rescues him and gets him on shore.
This one has long been sick; change to another one.
Since our primary concern is about whether the classification system allows users to find a comfortable linking relation between CPCs, and whether different users recognize the same linking relation without difficulty, the immediate goal is to achieve a fairly consistent manual tagging result, which is also an important foundation for the future development of automatic semantic relation identification.
4.2 How to determine a prior relation
他回家(V1)看(V2)準決賽了 TimeBefore-TimeAfter ? means-purpose ?
He has gone home to watch the semifinal.
The decision tree is generated by a complete semantic relation framework given in Fig. 1 and the features mentioned in Section 3.2 as discriminative decision node. In the tree structure, relation types also play roles as discriminative decision node. The most salient relation should be selected at first, and the closer to the bottom, the more inclusive and vague sense the relation linked. The decision tree demonstrates their priorities while determining the major relation. For example, in the sentence of 他在台灣出生(V1)成長(V2) tā zài táiwān chūshēng chéngzhǎng ‘he was born and raised in Taiwan’, we first use realis to determine it is a fact description; since there is no intention within V1 and V2, we look for causality in-between. Lacking causality, we then examine if V1 modifies V2 or if V2 is a result state of V1; disproving them both, we continue to look for a temporal relation between V1 and V2. With a positive answer, we then judge whether they are synchronous; since they are not, the TimeBefore-TimeAfter relation is determined.
- a.除非 chúfēi ‘unless’
Unless you agree, then I won’t go.
Unless I have textbooks, otherwise I am not going to school.
- b.除/除了 chú/chúle ‘In addition to; except for’
In addition to flowers, he even knelt down.
Except for the typhoon day, we open all year round.
- c.不管/不論/無論 bùguǎn/búlùn/wúlùn ‘no matter; whether… or…’
Whoever runs for election will win.
Whether on foot, by bike, or by bus
- d.也 yě ‘and; or; not only…but also’
He smokes and drinks.
You can learn Chinese or Japanese.
He not only sings but also dances.
We note that word-pair relation markers are more accurate than single-word markers, since their associated relations are usually definite and unique. For example, the three markers 除了chúle ‘in addition to’, 似乎 sìhū ‘seem’, 也 yě ‘and’ in (27) can establish five possible relations: except-event, and, or, theme-contrast, and restrictive-addition. However, 除了…也 chúle…yě ‘not only… but also’ which expresses the relation of restrictive-addition is the prior relation, since it is the most definite expression of the semantic relation in (27).
除了表彰(V1)他在中文文學創作的偉大成就之外, 似乎 也有意(V2)借此彰顯高行健 遭到中共當局迫害 restrictive-addition
chúle__biǎozhāng__tā__zài__zhōngwén__wénxué__chuàngzuò__de__wěidà__ chéngjiù__zhīwài,__sìhū__yě__yǒuyì__jiècǐ__zhāngxiǎn__gāoxíngjiàn__ zāodào__zhōnggòng__dāngjú__pòhài
It not only honors his great achievements in Chinese literature, but also seems to be meant to highlight the Chinese authorities’ persecution of Gao Xingjian.
liùshísìsuì__fútiánkāngfū__de__zhèngcè__ shǒuwàn__shòudào__le__ hǎopíng__rúhé__zài__jǐnpò__de__zhèngjú__dāngzhōng__déyǐ__fāhuī__ cái__shì__shòudào__zhǔmù__de
64-year-old Yasuo Fukuda’s policy strategies have been well received, but what receives the greatest attention is how he handles this pressing political situation.
64歲福田康夫的政策手腕受到(V1)了好評, 他可能被提名(V2) condition-result
64-year-old Yasuo Fukuda’s policy strategies have been well received; he may be nominated.
Finally, we note that in semantic relation identification for CPCs, we adopt a two-way linkage to avoid interference between head assignments. In the linking construction of Mandarin, Li and Thompson (1981: 631) identify essentially two kinds of sentence linking: forward linking and backward linking. As with linking elements then, connectives are also divided into two kinds. For example, in (29a), 假如 jiǎrú ‘if’ is a forward-linking element whose function is to signal the dependence of clause 1 on clause 2 to complete its message. However, constituents led by certain connectives always play the same role; thus as 下雨 xiàyǔ ‘rain’ must be a condition when introduced by 假如 jiǎrú ‘if’, it is unnecessary to decide which clause in the text is the main clause or the head verb. In fact, due to the influence of English syntax, we increasingly see sentences like (29b).
假如下雨(clause 1), 我們就在屋裡吃飯(clause 2) condition-result
If it rains (clause 1), we will eat in the house (clause 2).
我們就在屋裡吃飯(clause 1), 假如下雨(clause 2) result-condition
We will eat in the house (clause 1), if it rains (clause 2).
We thus propose two-way linkage as opposed to one-way linkage, because it prevents interference between head assignments and focuses on the relation structure of CPCs. For example, in (30a), we need not determine which verb is primary but instead clarify the verbs’ relation and the roles they play.
He works fast.
他做事(V1)快(V2), 說話也快 topic-comment (semantic focus on topic)
He works fast and talks fast too.
他做事(V1)快(V2), 說話慢 topic-comment (semantic focus on the comment of topic)
He works fast and talks slow.
He was sick and stayed in the hospital.
他住院(V1)了, 因為生病(V2) result-cause
He stayed in the hospital because of sickness.
When a relevant sentence occurs, as it shown in (30b) and (30c), a switch of semantic focus does not change our understanding of the original CPCs. Even when the VP positions change, the relation of the construction does not change, as illustrated in (31), where an unspecified temporal relation holds between the events, allowing for the inversion of the constituents without significant changes in meaning.
To evaluate the uniqueness and completeness criteria and to understand how well defined our classification system is, we conducted an annotation experiment involving two Chinese undergraduate students without a linguistic background and two second-year graduate students from the linguistic department. They had never participated in any semantic labeling task and were unfamiliar with our classification system. Before labeling, our classification system was explained using 100 examples with answers during a 1-h training course. Annotators were allowed to keep their 100 training examples (not part of the testing data) as references during the experiment. Each annotator was to label both middle-level types and fine-grained subclasses for 100 discourse relations extracted from HIT-CDTB and 100 serial verb relations extracted from Sinica Treebank, and at the same time reveal whether each tag of their selection was based on the decision tree or the connective markers. Since the annotators are tested for the agreement of relation identification, they are told which verb-pair is the analyzed target. The analyzed verb-pair must be on the same parsing level, i.e., the nested structure is not taken into account since verbs occur in nested structure are not regarded as CPCs. At the meanwhile, a reference standard annotation has been set by two proficient annotators with full understanding of our classification and prior labeling procedure. Part of our experimental data is listed in the Appendix 3, Table 5.
Cohen’s kappa coefficient for middle-level labeling
Annotator 5 (reference standard)
Labeling basis distribution
Distribution of labeling basis
In summary, we propose six ways to improve our classification and labeling system. The first is that according to the kappa coefficient, the more training provided to our annotators, the more agreement we can expect. The second is to extract more connective markers and linguistic cues from real-world text and add these into the checklist for reference. The third is that since addition and elaboration are broader relation types, we should consider relaxing our standard for their near node on the decision tree given more experimental evidence showing this need. The fourth is to consider the evaluation relation for both realis and irrealis subtrees. The fifth is to train annotators to test the relation type by fabricating connective markers. The sixth is to have annotators follow the decision tree step-by-step, that is, not allow annotators to neglect higher-level decisions and jump to latter choices.
The factors that influence the deduction of relation recognition are sense of events, markers, event ordering, and knowledge-based reasoning. The decision tree assistance in identifying a proper relation is based on event ordering and knowledge reasoning. In practice, however, labeling still depends heavily on human judgment. In order to clarify how well defined our classification system is, we conducted an experiment which shows an average of 73% accuracy and an approximate 70% agreement by Cohen’s kappa coefficient in middle-level labeling which indicates substantial agreement. We analyzed the error type for each relation and summarized six ways to improve our classification and labeling system, which is predicted to enhance the agreement by Cohen’s kappa up to 0.81–1.00, that is, almost perfect agreement in the future.
As a final remark, the proposed classification system is designed by following our proposed methodology. It results in a succinct semantic relation identification system for Chinese CPCs incorporated with a hierarchical decision tree to meet the uniqueness and completeness criteria of semantic relation recognition and which is in contrast to the conventional complex classification schemes discussed in Section 2.
We thank the anonymous reviewers for their very helpful suggestions and criticisms.
All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Aikhenvald, Alexandra Y. 2006. Serial verb constructions in typological perspective. In Serial verb constructions: A cross-linguistic typology, ed. Alexandra Y. Aikhenvald and Robert M. W. Dixon, 1–68. Oxford: Oxford University Press.Google Scholar
- Carlson, Lynn, and Daniel Marcu. 2001. Discourse tagging reference manual. Available at http://www.isi.edu/~marcu/discourse/tagging-ref-manual.pdf. Accessed 30 October 2015.
- Chen, Keh-jiann. 2011. E-HowNet. Available at http://ehownet.iis.sinica.edu.tw/index.php. Accessed 2 October 2015.
- Chen, Keh-jiann, and Chu-Ren Huang. 2004. Sinica Treebank Version 3.0. Available at http://turing.iis.sinica.edu.tw/treeview. Accessed 10 October 2015.
- Chen, Keh-jiann, Shu-Ling Huang, Yueh-Yin Shih, and Yi-Jun Chen. 2005. Extended-HowNet—A representational framework for concepts. In OntoLex 2005 - Ontologies and Lexical Resources IJCNLP-05 Workshop, 1–6. Stroudsburg: ACLGoogle Scholar
- Chen, Keh-jiann, Chi-Ching Luo, Ming-Chung Chang, Feng-Yi Chen, Chao-Jan Chen, Chu-Ren Huang, and Zhao-Ming Gao. 2003 Sinica Treebank: Design criteria, representational issues and implementation. In Treebanks: Building and using parsed corpora, ed. Anne Abeillé, 231–248. Dordrecht: Springer.Google Scholar
- Dowty, David R. 1991. Thematic proto-roles and argument selection. Language 67(3): 547–619.View ArticleGoogle Scholar
- HIT-SCIR 哈工大社会计算与信息检索研究中心. 2013. Chinese Discourse Treebank (CDTB) 中文篇章关系语料. Available at http://ir.hit.edu.cn/hit-cdtb/. Accessed 10 October 2015.
- Hong, Jia-Fei, and Chu-Ren Huang. 2015. Ontology-based event relation prediction: A SUMO based study of Mandarin VV compounds. Journal of Chinese Linguistics 43(1): 170–195.View ArticleGoogle Scholar
- Hovy, Eduard H., and Elisabeth Maier. 1992. Parsimonious or profligate: How many and which discourse structure relations?. Unpublished. Available at http://www.isi.edu/natural-language/people/hovy/papers/93discproc.pdf. Accessed 9 Sept 2015.
- Huang, Hen-Hsen, Tai-Wei Chang, Huan-Yuan Chen, and Hsin-Hsi Chen. 2014. Interpretation of Chinese discourse connectives for explicit discourse relation recognition. In Proceedings of the 25th International Conference on Computational Linguistics, 632–643. Stroudsburg: ACL.Google Scholar
- Huang, Hen-Hsen, and Hsin-Hsi Chen. 2011. Chinese discourse relation recognition. In Proceedings of 5th International Joint Conference on Natural Language Processing, 1442–1446. Stroudsburg: ACL.Google Scholar
- Li, Charles N., and Sandra A. Thompson. 1981. Mandarin Chinese: A functional reference grammar. Berkeley and Los Angeles: University of California Press.Google Scholar
- Li, Yancui, Wenhe Feng, Jing Sun, Fang Kong, and Guodong Zhou. 2014. Building Chinese discourse corpus with connective-driven dependency tree structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2105–2114. Stroudsburg: ACL.Google Scholar
- Lin, Jingxia, Chu-Ren Huang, Huarui Zhang, and Hongzhi Xu. 2012. The headedness of Mandarin Chinese serial verb constructions: A corpus-based study. In Proceedings of the 26th Pacific Asia Conference on Language, Information and Computation, 428–435. Stroudsburg: ACLGoogle Scholar
- Mann, William C., Sandra A. Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse 8(3): 243–281.View ArticleGoogle Scholar
- Miltsakaki, Eleni, Livio Robaldo, Alan Lee, and Aravind K. Joshi.2008 Sense annotation in the Penn Discourse Treebank. In Computational linguistics and intelligent text processing, ed. Alexander Gelbukh, 275–286. Berlin, Heidelberg: Springer-Verlag.Google Scholar
- Prasad, Rashmi, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K. Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of the 6th Language Resources and Evaluation Conference (LREC 2008), 2961–2968, Paris: ELRA.Google Scholar
- Tao, Liang. 2009. Serial verb construction in Mandarin Chinese: The interface of syntax and semantics. In Proceedings of the 21st North American Conference on Chinese Linguistics (NACCL-21), (vol. 2), ed. Yun Xiao, 209–228. Smithfield: Bryant University.Google Scholar
- Wang, Meng, Chu-Ren Huang, Shiwen Yu, and Bin Li. 2010. Chinese noun compound interpretation using verb paraphrases. Journal of Chinese Information Processing 24(6): 2–9.Google Scholar
- Wolf, Florian, and Edward Gibson. 2005. Representing discourse coherence: A corpus-based study. Computational Linguistics 31(2): 249–287.View ArticleGoogle Scholar
- Xu, Hongzhi, and Chu-Ren Huang. 2014. Annotate and identify modalities, speech acts and finer-grained event types in Chinese text. In Proceedings of the Workshop on Lexical and Grammatical Resources for Language Processing (LG-LP 2014), 157–166. Stroudsburg: ACL.Google Scholar
- Zhang, Muyu, Bing Qin, and Ting Liu. 2014. Chinese discourse relation semantic taxonomy and annotation. Journal of Chinese Information Processing 28(2): 28–36.Google Scholar
- Zhou, Lanjun, Binyang Li, Zhongyu Wei, and Kam-Fai Wong. 2014. The CUHK Discourse TreeBank for Chinese: Annotating explicit discourse connectives for the Chinese TreeBank. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014), 942–949. Paris: ELRA.Google Scholar
- Zhou, Yuping, and Nianwen Xue. 2012. PDTB-style discourse annotation of Chinese text. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, 69–77. Stroudsburg: ACL.Google Scholar
- Zhou, Yuping, and Nianwen Xue. 2015. The Chinese Discourse TreeBank: A Chinese corpus annotated with discourse relations. Language Resources and Evaluation 49(2): 397–431.View ArticleGoogle Scholar