- Research
- Open Access
Encoding emotion in Chinese: a database of Chinese emotion words with information of emotion type, intensity, and valence
- Jingxia Lin1 and
- Yao Yao2Email author
- Received: 23 September 2015
- Accepted: 23 June 2016
- Published: 19 October 2016
Abstract
Despite the increasing interest in emotion and sentiment analysis in Chinese text, the field lacks reliable, normative ratings of the emotional content and valence of Chinese emotion words. This paper reports the first large-scale survey of average language users’ judgment of perceived emotion type (e.g., anger, happiness), emotional intensity, and valence (e.g., positive, negative) of Chinese emotion words. The results of the survey reveal significant differences from previously proposed Chinese emotion lexicons, which mostly relied on a few researchers’ judgment or automatic annotation. Furthermore, the current study also explores the issue of lexical variation across different Chinese varieties with a comparison of emotion word perception by Chinese speakers from three different areas (Mainland China, Hong Kong, and Singapore). The emotion lexicons constructed in the current study will serve as an important reference for future research on emotion and language, including (but not limited to) topics related to sentiment detection and analysis, perception of affective language, and cross-regional lexical and semantic variation in Chinese.
Keywords
- Chinese emotion words
- Emotion type
- Emotion intensity
- Valence
- Language variation
1 Background
A fundamental element of humanity, emotion plays a vital role in human societies of all times, geographical locations, and cultures. Often defined as “a mental and physiological state associated with feelings, thoughts and behaviour” (Lee 2010:1; see also Frijda 1988, 1994; Harkins and Wierzbicka 2001; Plutchik 1962; etc.), emotion has invigorated at least two distinct lines of scientific inquiry1. The first line concerns the cognition and expression of emotion by humans, which has intrigued psychologists, cognitive scientists, and linguists for years (e.g., Desmet 2002; Frijda 1986, 1988, 1994, 2004; Kövecses 2000; Plutchik 1962, 1980, 1994; Russell 1980; Turner 1996, 2000, 2007; Wierzbicka 1992); the second line of research focuses on the automatic identification and classification of human emotion by machines (e.g., Ahmad 2011; Chuang and Wu 2002; Ortony et al. 1988; Pang and Lee 2008; Ravi and Ravi 2015; Strapparava and Mihalcea 2008; Subasic and Huettner 2001; Tang et al. 2009), which flourished in more recent years as a result of rising interest in artificial intelligence technologies.
Despite the possible difference in research interest, most emotion studies along both lines assume—and heavily rely on—the existence of certain emotion taxonomies. The two features that are most often discussed for categorizing emotions are emotion type and emotion intensity (e.g., see the emotion annotation scheme proposed in Wiebe et al. 2005). It is widely believed that there are some basic emotion types (e.g., happiness, anger, and sadness), and each emotion belongs to one or more basic types2; on the other hand, emotions may also differ in their intensity, for example, scared is a stronger emotion than afraid although they both belong to the category of fear.
In the studies that are specific to emotion and language, where the center of concern is the encoding and decoding of emotion in language, one of the most important tasks is to map the emotion taxonomy to linguistic expressions of emotion (i.e., emotion words and phrases), which would result in a representational model of the emotion lexicon3. In addition to emotion type and intensity, the third feature that is often coded in an emotion lexicon is valence, which generally speaking, refers to the overall positivity or negativity of the word. Valence judgment (or sentiment classification) of emotion words and emotion-laden words has been carried out in both lines of emotion studies, by psycholinguists and computer scientists, respectively.
But how do we obtain an emotion lexicon annotated with emotion type, intensity, and valence? How can we be sure if upset should be considered as anger or sadness in emotion type, high or low in emotion intensity, positive, negative, or neutral in valence? Obviously, questions of the latter type may solicit different opinions if we ask a group of English speakers because the way people perceive emotion words and the emotional content in these words may be profoundly individual4. Thus, models that only rely on a few people’s judgment, even if they are experts, are subject to criticism regarding the representativeness and generalizability of their results. An alternative approach is to conduct judgment experiments with large groups of participants so as to obtain a normative perception of the emotion lexicon that better represents the comprehension of the general population.
So far, there have been a few experimental-based emotion word models published for English (e.g., Bradley and Lang 1999; Nabi 2002; Strauss and Allen 2008), but similar studies are still lacking for other languages including Chinese5. The past decade or so has witnessed a fast growing body of literature on emotion expressions in Chinese, but most existing studies solely relied on a few individuals’ (usually researchers) judgment (Chang et al. 2000; Lee 2010; Xu and Tao 许小颖, 陶建华 2003) or automatic annotation (e.g., Xu et al. 2008). To further complicate the issue, Chinese is widely used in a number of countries and regions (Mainland China, Hong Kong, Singapore, Taiwan, etc.) and consequently has evolved into several varieties over the years of parallel development. Cross-regional variations have been found in almost every linguistic aspect—especially in pronunciation, lexicon, and grammar (see Huang et al. 2014; Li 李宇明 2010; Lin et al. 2014; Tsou and You 邹嘉彦, 游汝杰 2010)—which renders untenable the assumption of a homogeneous perception of the Chinese emotion lexicon. However, to our best knowledge, regional differences have not been examined comprehensively in previous research of Chinese emotion words.
Thus, the goal of the current study is twofold: (1) to collect judgment of Chinese emotion words from a sizable group of laymen and (2) to compare the judgment of Chinese speakers from different areas. Specifically, we focus on the judgment of emotion type, emotion intensity, and valence by native Chinese speakers from Mainland China, Hong Kong, and Singapore. To preview the results, the current study revealed both similarity and significant differences compared with previous studies, which can be attributed to either our participants’ background (i.e., laymen) or the fact that the current results were based on a sizable group of participants’ judgment; we also found important cross-regional differences in the participants’ perception of Chinese emotion words. The current results will serve as an important reference for future research on emotion and the Chinese language.
In the rest of this paper, we will first review previous studies on emotion and language in more detail; we will then introduce the experimental methods of the current study, followed by the results and discussion.
1.1 Emotion type, emotion intensity, and valence of emotion words
As stated above, previous literature generally agreed on the existence of some basic emotion types, but the exact identity of the basic categories is still under debate (see Ekman 1992; John 1988; Kövecses 2000; Oatley and Johnson-Laird 1987; Plutchik 1980, 1994; Turner 1996, 2000). The number of basic types proposed in previous research ranged from four (e.g., anger, anxiety, happiness, and sadness as in John (1988)) to five (e.g., happiness, sadness, anxiety, anger, and disgust in Oatley and Johnson-Laird (1987) or anger, fear, happiness, sadness, and surprise in Turner (1996)), six (e.g., anger, disgust, fear, happiness, sadness, and surprise in Ekman (1992)), eight (e.g., joy, sadness, trust, disgust, fear, anger, surprise, and anticipation in Plutchik (1994) or anger, anxiety, disgust, fear, neutral, happiness, sadness, and surprise in Strauss and Allen (2008)), and even 24 (e.g., Xu and Tao 许小颖, 陶建华 2003; see the discussion below). Regardless of how many basic types were proposed, most previous studies acknowledged that not all emotion words can be identified with a single emotion type—instead, there are complex emotions that must be understood as a blend of two or more basic emotion types. For example, in Turner’s (1996) model, worry belonged to both fear and sadness and guilt was a mixture of sadness, fear, and anger. Needless to say, when we examine across models, the classification of complex emotions will vary with the number and identity of basic types assumed in the model. A model that assumes more basic types will have fewer complex emotions compared to a model with fewer basic types. Given a specific model, it is also possible for an emotion word to not belong to any basic type, in which case one can say that the emotion word fails to be represented in the model. Thus, one way to evaluate the goodness of a model is to count the number of emotion words that are accounted for (or not accounted for) by the model.
Compared to emotion type, the representation of emotion intensity is less contentious, as intensity is always measured on a one-dimensional scale. Previous studies used either broad intensity bands (e.g., low, medium, and high in Plutchik (1980) and Turner (2000); see also Lee (2010)) or numerical scales with finer categorization (e.g., Strauss and Allen (2008) used a 7-point scale).
Coding valence presents another type of challenge. All previous studies agreed on using a one-dimensional scale to represent valence—either a numerical scale (e.g., Bradley and Lang 1999 used a 9-point scale; Khoo et al. (2015) used a 7-point scale) or a scale with ordered categories (e.g., positive, negative, and neutral in Baccianella et al. (2010)). Nonetheless, the interpretation of a valence judgment is still unclear. In most studies, the valence scale is interpreted as a continuum from “positive” to “negative,” via “neutral” in the middle. However, in the seminal work that produced the ANEW (Affective Norms for English Words) database, Bradley and Lang (1999) asked participants to rate a word on a scale of “pleasantness.” Their methodology was replicated in a number of subsequent studies that built similar databases for other languages, e.g., Spanish (Redondo et al. 2007) and French (Monnier and Syssau 2014). Regardless of how the valence scale is labeled, the interpretation of a valence judgment can be ambiguous, as the perceived valence of a word may be either a judgment of the emotional experience encoded by the word (i.e., from the perspective of the experiencer) or an attitude toward the encoded emotion (i.e., from the perspective of a reporter). In the current study, by including both valence and the word’s emotional content (emotion type and intensity) in the rating task, we hope to unveil the distinction between the two perspectives.
1.2 Construction of emotion word models
It is not uncommon for emotion word models to be based on a few researchers’ judgment. Consider models with emotion type and intensity information first. For example, Turner’s (2000) model was based on only one researcher’s judgment, and John’s (1988) model had three independent judges. More recently, Nabi (2002) suggested that researchers’ conceptualization of emotion words could be different from that of average language users. Along this line, Strauss and Allen (2008) carried out the first large-scale experiment with laymen participants for emotion type categorization and intensity rating. The study recruited 200 participants to rate a list of 463 words, with each word judged by 50 participants on average. The main results of the study were a list of highly representative emotion words of each emotion type, which elicited a consensus of judgment from the participants, and a list of complex emotion words, which received mixed judgment. For example, angry, mad, and rage are all representative of anger, and cheerful, enjoy, and joy are representative of happiness; by contrast, examples of complex emotion words include helpless (sadness + fear + anxiety) and desire (happiness + anxiety). Strauss and Allen’s study also revealed several significant differences from previous studies that relied on experts’ judgment. In addition to categorization differences in individual words (e.g., the word doom was categorized as fear in Strauss and Allen (2008), but as sadness in John (1988)), Strauss and Allen’s model also reflected richer emotional content of the test words—as many previously proposed single-type emotion words turned out to be complex—probably due to the large number of participants in Strauss and Allen’s experiment.
While the benefits of experimental methods are obvious, the risk of using unsupervised laymen’s judgment must not be overlooked. Nabi (2002) warned of the use of free recall tasks in emotion research due to the difference between the lay understanding of emotion words and theoretical definitions of emotions assumed in scientific research; more recently, Bai’s (2015) study of Chinese expressions also evinced that completely unsupervised experiments may produce incongruous results (see the discussion of Bai (2015) below). Thus, the key to the success of emotion word judgment experiments is to provide sufficient control of the test materials and to ensure the reliability of the responses.
Similar to emotion type and intensity, the coding of valence information may be achieved based on the judgment of a large number of laymen participants (e.g., the series of ANEW databases) or only a few expert annotators (e.g., the WKWSCI Sentiment Lexicon by Khoo et al. (2015)) or completely automatically using corpus analysis and lexical information (see Tang et al. (2009) for a more detailed review). While the first method is mainly adopted by psycholinguistic studies, the latter two are more often used in computer science studies. As far as we know, there has not been a comparison of valence annotations obtained with different methods, which is probably a result of these methods being used by separate populations of researchers.
1.3 Models of Chinese emotion words
Most existing models of Chinese emotion words regarding emotion type and intensity only relied on expert judgment, among which we notice two slightly different approaches. While some scholars generated emotion word models from the Chinese lexicon per se, others based their models on the translation equivalents of the English emotion lexicon. Along the first line, Xu and Tao 许小颖, 陶建华 (2003) proposed seven basic emotion types (好 hao “love,” 恶 wu “disgust,” 喜 xi (乐 le) “happiness,” 怒 nu “anger,” 哀 ai “sadness,” 惧 ju “fear,” and 欲 yu “desire”) following the tradition in Chinese classical philosophy. The same authors also proposed 24 finer emotion categories including, e.g., 羞 xiu “shame,” 烦 fan “annoyed,” 傲 ao “pride,” 信 xin “trust,” and 疑 yi “suspicion” based on 372 emotion-related words, but the relationship between the seven basic types and the 24 finer categories was not explained6. In another study, Chang et al. (2000) categorized 33 frequently used emotion verbs—each of which had more than 40 occurrences in the Sinica Corpus (Chen et al. (1996))—into seven basic categories (happiness, depression, sadness, regret, anger, fear, and worry). In both studies, the authors did not explain the criteria of emotion word classification, so we had to speculate that the classification was based on the authors’ judgment.
Along the second line, Lee (2010) published an emotion word model generated by mapping Chinese emotion words to the model of English emotion words in Turner (2000), with some additional emotion words from Plutchik (1980). As a result, Lee’s model had an identical structure as Turner’s model, with five basic emotion types (anger, fear, happiness, sadness, and surprise) and three intensity levels (high, moderate, and low). The classification of each Chinese emotion word in Lee’s model followed the classification of the word’s English equivalent in the English model. Being the first to represent both emotion type and emotion intensity in Chinese emotion words, Lee’s model has been applied in a number of subsequent emotion-related studies in Chinese (Lee 2010; Lee et al. 2010; Lee et al. 2013; Lee et al. 2014).
It should be noted that Lee’s model relies on an assumption that the classification of emotion words (by emotion type and intensity) can be transferred across languages via translation equivalents. This assumption necessarily requires that (1) for each Chinese emotion word, there exists a precise translation equivalent in English, and (2) each pair of Chinese and English translation equivalent emotion words are perceived (by native speakers of Chinese and English, respectively) with the same emotion type and equivalent emotion intensity. However, previous studies have found that the encoding of emotional concepts may very well vary across languages. For example, Pavlenko (2008b) pointed out that the concept of fun is not encoded in the Russian lexicon. A more relevant example was noted by Bai (2015) about the difference between the English word shameless and its (near) equivalent in Chinese, i.e., 无耻 wuchi “shameless”: while shameless may be used in a joking way in English (i.e., more similar to bold), 无耻 wuchi “shameless” is almost always insulting. In other words, even if translation equivalents do exist, the emotional content encoded in the words might not be exactly the same, which would further lead to differences in the perception of emotion type and emotion intensity across languages.
To be sure, there are a small number of Chinese emotion word studies that used laymen’s judgment, but the scope of research in these studies was limited to certain emotion types. For example, both Li et al. (2004) and Bai (2015) focused on shame expressions in Chinese. Li et al.’s study started with a list of 83 words that were related to 羞 xiu “shame/shyness,” 耻 chi “disgrace,” and 辱 ru “humiliation/shame” in the dictionary; the list was then expanded to 113 words and phrases by 10 native speakers; finally, the complete list of shame expressions were submitted to a judgment experiment for emotion sub-type with a separate group of 52 native speakers. Bai (2015), on the other hand, fully relied on laymen’s judgment for both generalization (via a free listing task) and categorization (via a similarity sorting task) of shame expressions in Chinese.
One unexpected result of Bai’s study is that in the free listing task, a few emotion words typical of anger (e.g., 愤怒 fennu “angry,” 生气 shengqi “angry”), sadness (e.g., 伤心 shangxin “sad”), and disgust (e.g., 讨厌 taoyan “hate,” 厌恶 yanwu “disgust”) were constantly proposed by lay participants as prototypical shame words. Granted that shame is often associated with anger, sadness, and disgust (in fact, the status of shame as a unique emotion type is still arguable), most researchers would not consider these words as core shame words in Chinese. In our view, these incongruous results are very likely attributable to the unsupervised nature of the free listing task; in addition, the lack of contrast with other emotions—as the experiment focused on shame expressions only—may also cause lay participants to confuse words of other related emotion categories as the core vocabulary of the emotion type of interest. Due to these concerns, the current experiment used a word list based on previous expert reports together with a categorization task that had a full range of emotion type options, in order to provide a more controlled experimental setting (see below for details of experimental methods).
Last but not the least, it is worth noting that Bai (2015) was one of the first to examine the variation of emotion word perception by Chinese speakers from different regions (Mainland China, Singapore) and language backgrounds (monolingual, bi(multi)lingual). Bai’s results showed significant differences in the perception of shame expression across participant groups, although it is unclear whether the observed variation patterns may be generalized to the perception of other emotions.
As for valence information, the two major Chinese sentiment databases are Affective Lexicon Ontology (Xu et al. 2008) and Hownet (Dong and Dong 2006), both of which are constructed by automatic or semi-automatic techniques. To our knowledge, there has not been a valence model for Chinese words based on large-scale manual annotation.
To summarize, there have been a few attempts in previous literature to construct databases of Chinese emotion words with annotated information related to emotion, but a comprehensive, cross-regional model of Chinese emotion words as perceived by lay speakers is yet to be proposed. The current study set out to fill this gap.
2 Methods
Design of the current study—especially regarding the coding of emotion type and intensity—mainly follows Strauss and Allen’s (2008) survey of English emotion words. In this section, we report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study.
2.1 Experimental materials
We used the list of Chinese emotion words from Xu and Tao 许小颖, 陶建华 (2003, cf. Lee 2010), which consisted of 372 words ranging from one to four syllables in length, with the majority being disyllabic compounds7. Since the complete list was too long to be tested in one experimental session, we randomly divided the words into four word lists (lists 1–4), with 93 words per list, and each participant only completed one word list.
Repeated items
Real words repeated within a list | Real words repeated across lists | Non-word repeated across lists |
|---|---|---|
List 1: 厌倦 yanjuan “bored,” 悲痛 beitong “grief” List 2: 担忧 danyou “worried,” 丧气 sangqi “discouraged” List 3: 愉悦 yuyue “pleasure,” 痛恨 tonghen “hate” List 4: 愤慨 fenkai “resentful,” 愁闷 choumen “depressed” | Lists 1, 2: 开心 kaixin “happy” Lists 1, 3: 焦躁 jiaozao “anxious” Lists 1, 4: 愤怒 fennu “angry” Lists 2, 3: 沉痛 chentong “grief” Lists 2, 4: 震惊 zhenjing “shocked” Lists 3, 4: 畏惧 weiju “fear” | Lists 1, 2, 3, 4: 几几 jiji |
2.2 Participants and procedure
The participants were 256 (192F, 64M) native Chinese speakers recruited from two university campuses, one in Hong Kong and one in Singapore. More than half of the participants (N = 149) were in the age range of 21–25 years, followed by the age range of 16–20 years (N = 81), and the rest of the participants (N = 26) were between 26–55 years. In terms of language background, 92 were born and raised in Mainland China and self-identified as Mainland Mandarin speakers, 87 Hong Kong and Hong Kong Chinese speakers, and the remaining 77 Singapore and Singaporean Chinese speakers. All the participants got cash compensation or partial course credit for participating in the study.
The judgment task was administered in the form of an online survey on the Google Form platform. Each participant was randomly assigned to work on one word list. The participant’s task was to judge the emotion type, intensity, and valence of each test item on the list. For emotion type, we followed the emotion taxonomy in Strauss and Allen (2008, see also Lee 2010; Xu and Tao 许小颖, 陶建华 2003), assuming seven basic emotion types (怒 nu anger, 焦虑 jiaolü anxiety, 厌恶 yanwu disgust, 害怕 haipa fear, 喜 xi happiness, 哀 ai sadness, 惊讶 jingya surprise) and the existence of complex emotions as blends of basic emotions. The participant was asked to choose—if possible—the most appropriate emotion type for the test word; if none of the seven basic emotion types was appropriate, the participant could choose one of three additional options, acknowledging that the test word belonged to a different emotion type other than the seven basic types (其他情感类型 qita qinggan leixing other emotions) or that the word was not an emotion word (中立/无情感色彩 zhongli/wu qinggan secai neutral/emotionless, or 不理解词义 bu lijie ciyi unfamiliar with the word). If the word was recognized as a true emotion word (i.e., one of the seven basic types or other emotions), the participant also needed to rate the emotion intensity of the word on a 7-point scale (1 = “basically no emotion”; 7 = “very strong emotion”)8. The participants were also asked to judge the valence of each word; following previous studies, we used three general terms to label valence, i.e., 褒义 baoyi positive, 贬义 bianyi negative, and 中立 zhongli neutral.
Number of participants for each list
Word list | Number of total participants | Number of participants by region | ||
|---|---|---|---|---|
Mainland China (MC) | Hong Kong (HK) | Singapore (SG) | ||
List 1 | 61 | 24 | 22 | 15 |
List 2 | 64 | 23 | 22 | 19 |
List 3 | 68 | 23 | 22 | 23 |
List 4 | 63 | 22 | 21 | 20 |
It should be noted that the emotion type categorization task only allowed single-category responses, i.e., the participants were forced to classify an emotion word with only one emotion type. As a result, words encoding complex emotions will be revealed in the pooled responses from a group of participants, but not in a single participant’s responses. Another noteworthy point about the categorization task is the availability of additional options apart from the pre-assumed basic emotion types. For one thing, the fact that other emotions was an available option made it possible for the results to shed light on the taxonomy of basic emotions: if a large group of words was categorized as other emotions, this would suggest the existence of additional basic emotion types. In addition, the word list we used included not only typical emotion words (e.g., 悲痛 beitong “grief,” 开心 kaixin “happy”) but also words whose emotion-word status may be arguable (e.g., 理解 lijie “understand”) and words that may only be recognized in certain dialectal regions (e.g., 背悔 beihui “regretful,” 来劲 laijin “in high spirits”). Instead of forcing the participants to identify these words as emotion words, we provided the options neutral/emotionless and unfamiliar with the word. Thus, results of this study can also distinguish the most typical emotion words from those that are less recognized or less commonly used.
3 Results
3.1 Reliability of the results
Summary of rating consistency of intra-list repeated words
List | Word | Emotion type | Emotion intensity | Valence | ||
|---|---|---|---|---|---|---|
Number (and percentage) of consistent raters | Mean rating at the first and second occurrences (correlation coefficient) | t value | p value | Number (and percentage) of consistent raters | ||
List 1 | 厌倦 yanjuan “bored” | 49 out of 61 (80.3 %) | 4.95, 5.36 (0.62) | −1.89 | 0.06 | 54 out of 61 (88.5 %) |
悲痛 beitong “grief” | 58 out of 61 (95.1 %) | 6.26, 6.34 (0.62) | −0.64 | 0.5 | 54 out of 61 (88.5 %) | |
List 2 | 担忧 danyou “worried” | 41 out of 64 (64.1 %) | 4.90, 4.92 (0.42) | −0.066 | 0.95 | 45 out of 64 (70.3 %) |
丧气 sangqi “discouraged” | 45 out of 64 (70.3 %) | 4.94, 5.10 (0.68) | −0.77 | 0.44 | 57 out of 64 (89.1 %) | |
List 3 | 愉悦 yuyue “pleasure” | 65 out of 68 (95.6 %) | 5.12, 5.32 (0.69) | −1.04 | 0.32 | 64 out of 68 (94.1 %) |
痛恨 tonghen “hate” | 40 out of 68 (58.8 %) | 6.38, 6.36 (0.51) | 0.14 | 0.89 | 62 out of 68 (91.2 %) | |
List 4 | 愤慨 fenkai “resentful” | 53 out of 63 (84.1 %) | 5.97, 5.77 (0.63) | 1.04 | 0.30 | 51 out of 63 (81.0 %) |
愁闷 choumen “depressed” | 46 out of 63 (73.0 %) | 5.03, 4.98 (0.67) | 0.23 | 0.82 | 54 out of 63 (85.7 %) | |
We also compared the emotion intensity ratings for the first and second occurrences of the intra-list repeated items. The results showed that all the intra-list repeated items elicited highly similar ratings of emotion intensity between the two occurrences (p > .05 in all t tests, correlation coefficient >0.5 for seven out of eight words; see Table 3 for detail). Taken together, these results indicate that the participants’ judgment of emotion type and intensity in the current experiment was stable and consistent overall.
Within-rater consistency in valence ratings was between 70.3 and 94.1 % (mean = 86.1 %, s.d. = 7.4 %). Similar to the emotion category results, lexical items with lower within-rater consistency tend to have greater degree of mixed ratings across rater. For example, 愤慨 fenkai “resentful” was rated as negative (70.5 %) + neutral (21.3 %) + positive (8.2 %) at the first occurrence and negative (64.5 %) + neutral (19.3 %) + positive (16.1 %) at the second occurrence.
Rating reliability (categorization and intensity rating) of real words across lists
Cross-list repeated real words | Emotion type | Emotion intensity | ||
|---|---|---|---|---|
Most frequently voted emotion type and valence (the corresponding percentage of voters) by list | Mean intensity rating by list | t value | p value | |
开心 kaixin “happy” | List 1: happiness (100 %), positive (96.7 %) | List 1: 5.39 | 0.97 | 0.34 |
List 2: happiness (98.4 %), positive (95.3 %) | List 2: 5.17 | |||
焦躁 jiaozao “anxious” | List 1: anxiety (96.7 %), negative (91.8 %) | List 1: 5.74 | 0.51 | 0.61 |
List 3: anxiety (85.3 %), negative (88.2 %) | List 3: 5.66 | |||
愤怒 fennu “angry” | List 2: anger (100 %), negative (82.8 %) | List 2: 6.20 | 1.46 | 0.15 |
List 4: anger (95.2 %), negative (77.8 %) | List 4: 5.97 | |||
沉痛 chentong “grief” | List 2: sadness (85.9 %), negative (84.3 %) | List 2: 6.11 | 1.35 | 0.18 |
List 3: sadness (83.8 %), negative (77.9 %) | List 3: 5.85 | |||
震惊 zhenjing “shocked” | List 3: surprise (100 %), neutral (67.6 %) | List 3: 5.96 | −0.40 | 0.69 |
List 4: surprise (98.4 %), neutral (63.5 %) | List 4: 6.03 | |||
畏惧 weiju “fear” | List 1: fear (93.4 %), negative (82.0 %) | List 1: 5.69 | 0.37 | 0.71 |
List 4: fear (92.1 %), negative (73.0 %) | List 4: 5.62 | |||
Finally, the non-word item, 几几 jiji, which appeared in all four lists, was recognized as unfamiliar by most of the participants (84.8 %), followed by neutral/emotionless (14.0 %). It should be noted that in Strauss and Allen’s study of English emotion words, the non-word item (Ytzok) was used as a gatekeeper, as participants who failed to recognize the non-word item as unfamiliar were excluded from the analysis. In the current study, we did not adopt this practice because not recognizing几几 jiji as a non-word item does not necessarily reflect a lack of competence of the participant. Although 几几 jiji is not a compound in standard Chinese vocabulary, the component 几 ji is a real morpheme with several possible meanings, e.g., pronominal “how many” and nominal “bench,” making it possible to assign meanings to几几 jiji. Furthermore, it is not unlikely for Chinese language users to associate 几几 jiji with real compounds such as 磨磨叽叽 momojiji “grumble” and 叽叽喳喳 jijizhazha “twitter”—both of which contain morphemes that are homophonic with 几几 jiji—in order to make sense of 几几 jiji. Thus, 几几 jiji is more word-like in Chinese than Ytzok is in English, and a failure to recognize 几几 jiji as a non-word should not be taken as evidence of unreliable judgment.
3.2 Emotion type, intensity, and valence judgment of Chinese emotion words
This section reports the general rating results for real-word items9. When analyzing the results of emotion type, intensity, and valence judgment, we first separated the survey results by participants’ Chinese variety (Mainland China, Hong Kong, and Singapore), resulting in three subsets of data, which we refer to as “MC,” “HK,” and “SG” datasets, respectively. For each test word in each dataset, we compiled the list of emotion categories the word was classified with and the percentage of participants that voted for each category. The emotion categories with the highest and second highest percentages of voters were considered as the primary and secondary emotion categories of the word.
Following Strauss and Allen’s study, words whose primary emotion type was voted by no less than 70 % of the survey participants were defined as representative emotion words, whereas words whose primary emotion type was voted by less than 70 % of the survey participants were defined as blended emotion words. As the names suggest, words in the former category are considered to be highly representative of certain basic emotion type, and those in the latter category express a mixture of emotions of multiple basic types (i.e., complex emotions). Some examples of universally agreed representative emotion words are 欣喜 xinxi “happy” and 伤感 shanggan “sad,” categorized as happiness (100 %) and sadness (>90 %), respectively, in all three datasets; by contrast, words such as 不安 bu’an “restless” (primary: anxiety, 55–68 %; secondary: fear, 22–32 %) and 忌恨 jihen “hate” (primary: disgust, 61–65 %; secondary: anger, 26–32 %) received consistent, mixed classification results across regions and are thus typical examples of blended emotion words in Chinese.
As for intensity judgment, we calculated the mean and standard deviation of intensity ratings of each test word in each dataset pooled from all the valid ratings (if a participant classified a test word as unfamiliar, his or her rating of emotion intensity, if any, was excluded; if a participant classified a test word as neutral, his or her rating of emotion intensity was recognized as 1). To give some examples, we found that among representative anger words in the MC dataset, 暴怒 baonu “rage” was perceived as an extremely intensive anger emotion (mean = 6.96, s.d. = 0.20), followed by other anger words such as 激愤 jifen “wrathful” (mean = 6.54, s.d. = 1.00), 愤怒 fennu “angry” (mean = 6.43, s.d. = 0.71), 愤懑 fenmen “resentful” (mean = 6.35, s.d. = 0.87), 愤慨 fenkai “resentful” (mean = 6.18, s.d. = 0.72), 忿怒 fennu “angry” (mean = 6.04, s.d. = 0.98), and 生气 shengqi “angry” (mean = 6.00, s.d. = 0.87) on the intensity continuum. We also noticed a few trends regarding cross-regional differences in the perception of emotion intensity and the relationship between emotion type and emotion intensity ratings (see more discussion below).
Based on the raw valence judgment, we calculated the percentage of participants who voted for positive, negative, and neutral for each test word10. In general, valence judgment shows higher cross-rater agreement than emotion type ratings—probably a direct result of the smaller set of valence categories. Furthermore, valence judgment also showed some general correlations with emotion type; however, such correlations were not strong enough to fully predict valence based on emotion type, suggesting that the valence judgment was not simply based on the sentiment of the emotion described in the emotion word (see more discussion below).
The full list of emotion words including primary and secondary emotion types, emotion intensity, and valence is provided in the Additional file 1. The complete database of emotion type, intensity, and valence ratings is available upon request.
4 Discussion
4.1 Major findings
Highly representative emotion words
Type | Words voted by all participants into the type |
|---|---|
anger | 愤怒 fennu “angry” (MC, HK, SG), 愤懑 “resentful” (MC), 生气 shengqi “angry” (MC, HK, SG), 暴怒 baonu “rage” (MC, HK, SG) |
anxiety | 心急 xinji “impatient” (MC), 心焦 xinjiao “anxious” (MC), 焦虑 jiaolü “anxious” (MC, HK), 焦急 jiaoji “worried” (MC, SG), 焦躁 jiaozao “anxious” (MC, SG) |
disgust | 厌烦 yanfan “bored” (MC), 反感 fangan “dislike” (SG), 厌恶 yanwu “disgust” (MC, HK), 腻烦 nifan “bored” (MC, SG), 讨厌 taoyan “hate” (MC, HK) |
fear | 怕 pa “fear” (SG), 恐惧 kongju “fear” (MC), 畏惧 weiju “fear” (MC) |
happiness | 高兴 gaoxing “pleased” (MC, HK, SG), 欢欣 huanxin “joyful” (MC, HK, SG), 欢快 huankuai “happy” (MC, HK, SG),欢乐 huanle “happy” (MC, HK, SG), 欢喜 huanxi “joyful” (MC, HK, SG), 开心 kaixin “happy” (MC, HK, SG), 乐 le “happy” (MC, HK, SG), 乐意 leyi “willing” (MC), 喜悦 xiyue “joyful” (MC, HK, SG), 快乐 kuaile “happy” (HK, SG), 欢 huan “happy” (MC, HK),欢愉 huanyu “delighted” (MC), 欢悦 huanyue “pleased” (MC), 欢娱 huanyu “pleasure” (MC, SG), 欣喜 xinxi “happy” (MC, HK, SG), 快活 kuaihuo “happy” (MC), 满意 manyi “satisfied” (MC), 愉悦 yuyue “pleasure” (HK), 兴奋 xingfen “excited” (MC), 欢畅 huanchang “joyful” (MC), 幸福 xingfu “happy” (MC), 喜爱 xiai “favorite” (SG), 欣慰 xinwei “pleased” (MC, HK), 热爱 re’ai “love” (MC), 窝心 woxin “pleased” (HK), 自豪 zihao “proud” (MC), 钟爱 zhongai “favorite” (MC) |
sadness | 哀愁 aichou “sad” (MC, HK, SG), 哀伤 aishang “sad” (HK, SG), 悲伤 beishang “sad” (MC, HK, SG), 伤心 shangxin “sad” (MC, HK), 感伤 ganshang “sentimental” (HK, SG), 悲痛 beitong “grief” (SG), 悲哀 beiai “sad” (MC, HK), 哀痛 aitong “sad” (MC, HK), 悲恸 beitong “grief” (MC), 悲愁 beichou “sad” (HK), 伤感 shanggan “sentimental” (SG), 悲凄 qibei “sad” (HK), 悲凉 beiliang “desolate” (HK), 沮丧 jusang “depressed” (SG) |
surprise | 惊讶 jingya “surprised” (HK, SG), 惊奇 jingqi “surprised” (HK, SG), 诧异 chayi “surprised” (MC), 惊诧 jingcha “astonished” (MC), 惊愕 jinge “stunned” (HK), 愕然 eran “astounded” (MC), 惊疑 jingyi “surprised” (MC), 吃惊 chijing “surprised” (MC, SG), 震惊 zhenjing “shocked” (MC, HK, SG) |
Number of representative emotion words in all regions
Region | ||||
|---|---|---|---|---|
Emotion type | MC | HK | SG | Shared by all regions |
anger | 8 | 11 | 7 | 7 |
anxiety | 23 | 14 | 13 | 9 |
disgust | 19 | 13 | 10 | 8 |
fear | 8 | 8 | 6 | 5 |
happiness | 57 | 49 | 33 | 32 |
sadness | 33 | 36 | 26 | 21 |
surprise | 13 | 9 | 8 | 7 |
neutral | 2 | 5 | 4 | 1 |
Total | 163 | 145 | 107 | 90 |
Mean intensity of representative and blended words across regions. representative words do not include words whose primary category is other emotions or unfamiliar
MC | HK | SG | |
|---|---|---|---|
representative | 5.25 (s.d. = 0.79) | 5.09 (s.d. = 0.91) | 5.17 (s.d. = 0.84) |
blended | 4.46 (s.d. = 1.07) | 4.23 (s.d. = 1.03) | 4.26 (s.d. = 1.01) |
Words classified as other emotions by the majority (≥70 %) of the raters and the corresponding percentage of raters in each language variety
MC | HK | SG |
|---|---|---|
害羞haixiu “shy” 86 % 不好意思 buhaoyisi “embarrassed” 83 % 倚重 yizhong “rely heavily upon” 82 % 怀疑 huaiyi “suspect” 82 % 羞惭 xiucan “ashamed” 78 % 敬慕 jingmu “admire” 78 % 珍惜 zhenxi “cherish” 77 % 同情 tongqing “empathy” 77 % 挂念 guanian “reminisce” 74 % 羞涩 xiuse “shy 74 % 敬重 jingzhong “revere” 73 % 信服 xinfu “convinced” 73 % | 无奈 wunai “helpless” 86 % 羡慕 xianmu “admire” 77 % 自傲 zi’ao “proud” 76 % 同情 tongqing “empathy” 71 % | 体贴 titie “considerate” 79 % 挂念 guanian “reminisce” 79 % 怜惜 lianxi “compassionate” 79 % 自傲 zi’ao “proud” 79 % 羞愧 xiukui “ashamed” 78 % 害羞 haixiu “shy” 75 % 溺爱 ni’ai “spoil” 75 % 同情 tongqing “empathy” 75 % 崇拜 chongbai “adore” 74 % 羞惭 xiucan “ashamed” 74 % 痴迷 chimi “obsessed” 73 % 情愿 qingyuan “willingly” 73 % 关切 guanqie “care” 70 % |
Following Strauss and Allen (2008), we do not treat shame as a basic emotion in this study, although some contend that shame is one of the core, self-conscious emotions that is particularly important in the Chinese culture (Li et al. 2004; see also Bai 2015). As mentioned earlier, previous studies have identified some prototypical shame expressions in Chinese, ranging from 40 in Bai (2015) to 113 in Li et al. (2004). A small subset of these words was also present in the current study (mostly in the MC and SG datasets): 6 from Bai’s list generated by Chinese monolinguals, 11 from Bai’s list by Chinese-English bilinguals, and 14 from Li et al.’s list. According to our results, a few of the overlapped words—不好意思 buhaoyisi “embarrassed,” 害羞 haixiu “shy,” 羞惭 xiucan “ashamed,” 羞愧 xiukui “ashamed”—was majorly categorized as other emotions in at least one regional dataset. Other overlapped, so-called shame words in these two studies, such as 生气 shengqi “angry,” 讨厌 taoyan “hate,” and 伤心 shangxin “sad,” were mostly categorized under anger, disgust, and sadness as their primary and secondary categories in our study. Thus, our results do not provide sufficient support for claiming shame as one of the basic emotion types lexicalized in Chinese; on the contrary, our results suggest that some of the shame expressions previously proposed by lay participants may indeed be an artifact of the eliciting task (see our discussion on Bai (2015) in Models of Chinese emotion words).
As for emotion intensity, our results showed that the test words covered a wide range of emotion intensity in all three datasets (MC: 1.17–6.96; HK: 1.29–6.45; SG: 1.58–6.47). We also found that MC participants overall rated the test words with higher emotion intensity than both HK and SG participants (p < .001 in both paired t tests; see Table 7 for by-region summaries), suggesting possible cross-regional differences in the perception of emotion intensity, although it is unclear at this stage what may have caused such differences.
Furthermore, representative emotion words (excluding other emotions and unfamiliar) received on average higher intensity ratings compared to blended emotion words, and the trend was significant in all three regions (p < .001 in all t tests). We propose two possible accounts for the observed pattern: first, the ambiguity in emotion type may interfere with the perception of emotion intensity, therefore participants may find it difficult to evaluate emotion intensity for words belonging to multiple emotion categories; second, for some participants, the perception of emotion intensity may be conflated with emotion prototypicality, and thus, blended emotion words were perceived as less intensive than representative emotion words. One reviewer suggested a possible effect of word frequency on perceived intensity, as high-frequency words may be perceived as less intensive than low-frequency words. However, this hypothesis was not supported in a post hoc analysis of the MC dataset, with frequency counts gathered from the BCC Chinese corpus (the assorted subcorpus)11. A linear regression model showed that after controlling for emotion type (both primary and secondary), (log) word frequency had no significant effect on perceived emotion intensity (t = −1.18, p > .1). Whether or not the frequency effect may exist in the other two datasets—as well as the origin of the effect (e.g., familiarity, register, or prototypicality) if it does exist—still awaits further investigation.
Mean percentage of raters who selected positive, negative, and neutral valence categories for representative emotion words by primary emotion type and rater region. other and unknown words are not included
The strong correlation with emotion type suggests that our participants’ valence ratings were to a large extent based on the perceived “pleasantness” of the emotional experience described by the emotion words. Nevertheless, emotion type cannot fully account for the valence ratings. For example, a simple analysis of representative happiness and sadness words in the MC dataset revealed that while these words were on average voted by 92.1% and 87.9% of the participants as happiness and sadness, respectively, only 87.1% and 78.4 % of the participants voted these words as positive and negative, respectively. That is to say, at least 5–10 % of the participants did not vote for the valence category predicted by the emotion type they chose; in most cases, these participants rated the valence as neutral, but there are also some cases where participants voted for the opposite of the predicted valence category. Some extreme examples include 痴迷 chimi “obsessed” (happiness 70.8 %; positive 16.7 %, negative 16.7 %, neutral 66.7 %), 得意 deyi “proud” (happiness 95.7 %, positive 34.8 %, negative 30.4 %, neutral 34.8 %), 偏爱 pianai “favor” (happiness 73.9 %, positive 34.8 %, negative 17.4 %, neutral 47.8 %). Although all three words were predominately categorized as happiness by MC participants, no more than one third of the participants recognized these words as positive; instead, there was a sizable proportion of the participants who recognized these words as negative. In our opinion, these results showed that the participants may have a neutral or negative attitude toward the emotional experience described by the word; in other words, the valence ratings reflected not only the polarity of the emotional experience (from the experiencer’s perspective) but also the evaluation of such emotional experience (from a reporter’s perspective). In the current study, since participants were simply instructed to rate the goodness of word meaning, it is possible that some participants took the experiencer’s perspective while others put themselves in an assessor position, thus resulting in mixed valence ratings when the two perspectives entail different—or even opposite—evaluations. Our results suggest the importance of examining the distinction between these two perspectives, which as far as we know, has not received much attention in either psycholinguistic research on word valence or computational research on sentiment analysis.
Relatedly, our data in general suggest a certain degree of individual differences in all three emotional measures. An important source of such variation is raters’ experience of emotion words used in specific contexts (for example, as discussed above, the emotional connotation of a word may change with perspective). However, the overall high cross-rater consistency in our dataset indicates that language users may extrapolate core meanings of emotion words based on relevant contexts and that such meanings can be queried in a context-free rating experiment.
4.2 Comparison with previous studies
Words classified with neutral/emotionless as the primary emotion type and the corresponding percentage of raters in each dataset
MC | HK | SG |
|---|---|---|
了解 liaojie “understand” 87 % 留神 liushen “careful” 74 % | 关注 guanzhu “pay close attention to” 86 % 了解 liaojie “understand” 86 % 留神 liushen “careful” 86 % 沉静 chenjing “calm” 82 % 自爱 zi’ai “self-esteem” 71 % | 理解 lijie “understand” 80 % 了解 liaojie “understand” 84 % 想 xiang “think” 79 % 拥护 yonghu “support” 73 % |
We also conducted a more detailed comparison with Lee’s (2010) emotion word model, which, like the current study, represented both emotion type and emotion intensity. Before we introduce the comparison results, it should be noted that the structure of Lee’s model is slightly different from the current one: while Lee also distinguished emotion words belonging to one basic emotion type (“basic” emotions in Lee’s terms) and those encoding a blend of two or (rarely) three basic emotion types (“first order” and “second order” emotions in Lee’s terms), Lee’s model assumed only five basic emotion types (anger, fear, happiness, sadness, surprise) and used a broader classification system of emotion intensity (high, moderate, low) than the 7-point scale in the current study.
Our comparison focused on the representation of emotion words shared between the two studies (N current = 372, N Lee = 236, N current∩Lee = 226), which yielded a few important differences. To start with, only about half of the shared emotion words were listed under the same primary emotion category in Lee’s and the current datasets (116 in MC, 122 in HK, 109 in SG), although the discrepancy might be largely due to the different emotion classification systems used in the two studies. Secondly, among the shared words considered as basic emotion words on Lee’s list (N = 141), less than 75 % of were also classified as representative emotion words in the current study (105 in MC, 90 in HK, 72 in SG). For example, 37 words in the shared lexicon were considered as basic happiness words in Lee’s model. While the majority of these words were indeed categorized as representative of happiness in our study, a good number of them (7 in MC, 7 in HK, 14 in SG) fell into the blended category, some of which were categorized as happiness by less than half of the participants in at least two regions (e.g., 放松 fangsong “relaxed”: <32 % in MC, 29 % in HK, <10 % in SG, whose primary category is neutral or other emotions in all three datasets).
Summary of current intensity ratings of the shared words by intensity categorization in Lee (2010)
Intensity level in Lee (2010) | Mean and range of intensity ratings in the current study | ||
|---|---|---|---|
MC | HK | SG | |
low (21words) | 4.20 (2.14–5.35) | 4.23 (2.85–5.19) | 4.19 (2.35–5.64) |
moderate (73 words) | 4.98 (2.55–6.26) | 4.90 (2.33– 5.95) | 4.86 (2.00– 6.00) |
high (47 words) | 6.00 (4.61–6.96) | 5.73 (4.57–6.46) | 5.65 (4.723– 6.47) |
Summary of current valence ratings of the shared words by valence categorization in Xu et al. (2008)
Valence category in Xu et al. (2008) | Mean and range of percentage of valence ratings for the same category (positive, negative, or neutral) in the current study | ||
|---|---|---|---|
MC | HK | SG | |
positive (76 words) | 82.3 % (17.4–100 %) | 78.3 % (4.5–100 %) | 74.6 % (5.9–100 %) |
negative (64 words) | 78.2 % (0–100 %) | 80.7 % (0–100 %) | 72.5 % (0–100 %) |
neutral (185 words) | 34.2 % (0–100 %) | 26.3 % (0–86.4 %) | 30.4 % (0–86.7 %) |
Taken together, results from the current study revealed both similarities and discrepancies from previous models of Chinese emotion words. Given our discussion above, the discrepancies may be attributable to three sources: (1) differences in model structure, (2) researchers’ vs. laymen’s judgment and the number of raters, and (3) cross-linguistic differences in the emotional content of translation equivalents between English and Chinese. It is beyond the scope of the current paper as to which model would yield more accurate predictions or better performance in psychological/computational studies of Chinese emotion language; in fact, we suspect that the assessment results will be highly dependent on the nature of the task. The current study provided a dataset of Chinese emotion word categorization that is significantly different from previous works and provides a wider geographical coverage, which may serve as an additional reference for future studies of Chinese emotion language.
5 Conclusion
In this study, we conducted a large-scale survey of common Chinese speakers’ judgment of Chinese emotion words in carefully controlled experimental settings. Results of this study generated the first database of normative ratings of emotion type, emotion intensity, and valence for a list of Chinese emotion words, based on the judgment of Chinese speakers from Mainland China, Hong Kong, and Singapore. A number of cross-regional differences in emotion word perception have been revealed in the database, although the cause of the observed variation is beyond the scope of the current study.
We believe that the current results will produce important implications for further research of Chinese emotion language. Specifically, the current emotion word database will benefit both psycholinguistic research on emotion word perception and processing and computational research on identifying emotion types and sentiment at both lexical and text levels. Since the methodology we used follows the behavioral experimental tradition in psycholinguistic research, the results can be easily adapted or used for stimuli selection and construction in other psycholinguistic experiments. On the other hand, our results, which relied on multi-rater annotation in controlled settings, provide more fine-grained details of word meanings than typical databases used in computational research. For example, instead of categorizing a word as simply belonging to one category (of emotion type, intensity, or valence), our results make it possible to distinguish words that are core members of a certain category and words that are on the borderline between two (or more) categories. Such heavily annotated resources may be used as the gold standard or training data in (semi)-automatic sentiment detection and classification tasks. Furthermore, the database will also be a valuable resource for linguistic studies, especially those related to lexical semantics and the syntax-semantics interface. For instance, previous studies propose that Chinese BA and BEI constructions denote the change of state of the patient involved, and their use is conditioned by a number of factors including verb lexical semantics, object definiteness (e.g., Liu 1997, 2007; Zhang 张伯江 2001). In addition, these constructions may also express speakers’ emotions (Jing-Schmidt 2005; Jing-Schmidt and Jing 2011). Thus, other things being equal, emotion words with higher intensity should be more likely to occur in BA/BEI constructions than words with lower intensity, and such a prediction can be tested given the intensity information in the current database. Last but not the least, the current database will also shed light on the research of cross-regional variation in Mandarin Chinese, with regard to both the patterns of variation in emotion word perception and the mechanisms of such variations.
The precise definition of emotion is still under constant debate; the issue is further complicated by related concepts and terms such as affect, mood, attitude, sentiment, etc. (see Frijda (1986, 1988, 1994, 2004) for more detailed discussion of the distinction among these concepts). In this study, we use the term “emotion” in a narrow sense and focus on words that express the most widely-recognized, core human emotions (e.g., happiness, anger, and sadness).
This paper uses small caps and italic for English category names and English emotion words, respectively.
Emotion words can be divided into two broad categories depending on their usage: expressive emotion words, which express the user’s emotion directly (e.g., taboo and swear words like shit and interjections like wow), and descriptive emotion words, which describe certain emotion states or processes (e.g., happy, sad, rage) (see Clore et al. 1987; Kövecses 2000; Pavlenko 2008a; Potts 2007; etc.). This paper focuses on descriptive emotion words, which directly encode emotional content in the semantic meaning.
Strictly speaking, variation in judgment may exist both in the understanding of the emotion word (e.g., what emotion is referred to by the word upset) and in the categorization of the encoded emotion (e.g., what emotion type and intensity level the emotion upset belongs to). In this paper, we do not make the distinction between an emotion word and the emotion it refers to (i.e., the word upset vs. the emotion upset), but see Nabi (2002) and Russell (1991) for a discussion on possible differences between the two.
In this paper, we use the term “Chinese” to refer to the Standard Mandarin Chinese, which has a standard orthography in either simplified Chinese characters (used in Mainland China, Singapore) or traditional Chinese characters (used in Taiwan, Hong Kong).
Xu and Tao 许小颖, 陶建华 (2003) manually extracted emotion words from the adjective and verb categories in Modern Chinese Grammar Information Dictionary (Yu et al. 俞士汶等 2003). The authors distinguish the list of emotion words from words that express attitudes (e.g., 粗暴 cubao “gruff,” 温和 wenhe “gentle,” and 冷漠 lengmo “indifferent”) and personality (e.g., 忠诚 zhongcheng “loyal,” 善良 shanliang “kind,” 英勇 yingyong “brave”).
Xu and Tao’s 许小颖, 陶建华 (2003) list has 374 words in total. Our study excludes two words that are repeatedly listed (喜欢 xihuan “like” and 惦念 diannian “reminisce”), which results in 372 emotion words.
A reviewer raised a question regarding the rationale of choosing a 7-point scale (e.g., as opposed to a 5-point scale). Previous studies have found that the number of points on a rating scale had no significant effect on rating results (see Krosnick and Presser (2010: 272) for a survey of related studies). Therefore, this study adopted a 7-point scale following Strauss and Allen’s (2008) experimental design.
Given the high intra- and inter-participant rating consistency, for repeated test words, we report only the rating results of one occurrence of each item, which is either the first occurrence in the list (for items repeated within a list) or the occurrence in a numerically earlier list (for items repeated across lists; eg, items appearing in Lists 1 and 2 will be reported based on the rating results in List 1).
If a participant classified a test word as unfamiliar, his or her rating of valence (if any) was excluded from analysis.
BCC (http://bcc.blcu.edu.cn/) was released in September 2014. The assorted subcorpus of BCC has about one billion words. The frequency information of the emotion words in the subcorpus was collected on April 8, 2016.
Declarations
Acknowledgements
This research was supported by Nanyang Technological University under Start Up Grant [M4081117.100] and The Hong Kong Polytechnic University, Department of Chinese and Bilingual Studies under the Departmental General Research Funds [4-ZZEL].
Authors’ contributions
LJ carried out literature research. LJ conducted the experiment with Singaporean participants; YY conducted the experiment with Mandarin and Hong Kong participants. Both authors participated in data analysis. Both authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- Ahmad, Khurshid (ed.). 2011. Affective computing and sentiment analysis: Metaphors, emotions and terminology. Heidleberg: Springer.Google Scholar
- Baccianella, Stefano, Andrea Esuli, and Fabrizio Sebastiani. 2010. SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010), ed. Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner, and Daniel Tapias, 2200-2204. European Language Resources Association (ELRA).Google Scholar
- Bai, Lin. 2015. Emotion categorization and early bilinguals: A study of Chinese-English bilinguals’ understanding of the shame category. Ph.D. dissertation. Singapore: Nanyang Technological University.Google Scholar
- Bradley, Margaret M, and Peter J Lang. 1999. Affective norms for English words (ANEW): Simuli, instruction manual and affective ratings (Technical Report C-1). Gainesville: University of Florida, NIMH Center for the Study of Emotion and Attention.Google Scholar
- Chang, Li-li, Keh-jiann Chen, and Chu-Ren Huang. 2000. Alternation across semantic field: A study of Mandarin verbs of emotion. In Chinese Verbal Semantics, ed. Yung-O Biq, special issue, Computational Linguistics and Chinese Language Processing 5(1): 61–80.Google Scholar
- Chen, Keh-Jiann, Chu-Ren Huang, Li-Ping Chang, and Hui-Li Hsu. 1996. Sinica Corpus: Design methodology for balanced corpora. In Proceeding of the 11th Pacific Asia Conference on Language, Information and Computation, ed. Byung-Soo Park and Jong-Bok Kim, 167–176. Seoul: Kyung Hee University.Google Scholar
- Chuang, Ze-Jing, and Chung-Hsien Wu. 2002. Emotion recognition from textual input using an emotional semantic network. In Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP 2002), 177-180. CO: Denver: ISCA Archive. http://www.isca-speech.org/archive/archive_papers/icslp_2002/i02_2033.pdf.
- Clore, Gerald L, Andrew Orton, and Mark A Foss. 1987. The psychological foundations of the affective lexicon. Journal of Personality and Social Psychology 53(4): 751–766.View ArticleGoogle Scholar
- Desmet, Pieter. 2002. Designing emotion. Ph.D. dissertation. Delft: Delft University of Technology.Google Scholar
- Dong, Zhendong, and Qiang Dong. 2006. Hownet and the computation of meaning. River Edge: World Scientific Publishing.View ArticleGoogle Scholar
- Ekman, Paul. 1992. An argument for basic emotions. Cognition and Emotion 6(3/4): 169–200.View ArticleGoogle Scholar
- Frijda, Nico H. 1986. The emotions. Cambridge: Cambridge University Press.Google Scholar
- Frijda, Nico H. 1988. The laws of emotions. American Psychologist 43(5): 349–358.View ArticleGoogle Scholar
- Frijda, Nico H. 1994. Varieties of affect: Emotions and episodes, moods, and sentiments. In The nature of emotion: Fundamental questions, ed. Paul Ekman and Richard J Davidson, 59–67. New York: Oxford University Press.Google Scholar
- Frijda, Nico H. 2004. Emotions and action. In Feelings and emotions, ed. Anthony SR Manstead, Nico H Frijda, and Fischer Agneta, 158–173. Cambridge: Cambridge University Press.Google Scholar
- Harkins, Jean, and Anna Wierzbicka (eds.). 2001. Emotions in crosslinguistic perspective. Berlin: Mouton de Gruyter.Google Scholar
- Huang, Chu-Ren, Jingxia Lin, Menghan Jiang, and Hongzhi Xu. 2014. Corpus-based study and identification of Mandarin Chinese light verb variations. In Proceedings of the 25th International Conference on Computational Linguistics (COLING 2014), Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, 1–10.Google Scholar
- Jing-Schmidt, Zhuo. 2005. Dramatized discourse: The Mandarin Chinese ba-construction. Amsterdam: John Benjamins.View ArticleGoogle Scholar
- Jing-Schmidt, Zhuo, and Ting Jing. 2011. Embodied semantics and pragmatics: Empathy, sympathy and two passive constructions in Chinese media discourse. Journal of Pragmatics 43(11): 2826–2844.View ArticleGoogle Scholar
- John, Carolyn H. 1988. Emotionality ratings and free-association norms of 240 emotional and non-emotional words. Cognition and Emotion 2(1): 49–70.View ArticleGoogle Scholar
- Khoo, Christopher SG, Jin-Cheon Na, and Sathik Basha Johnkhan. 2015. Comparison of lexical resources for sentiment analysis. Presentation at Singapore Symposium on Sentiment Analysis, 2016.Google Scholar
- Kövecses, Zoltán. 2000. Metaphor and emotion: Language, culture, and body in human feeling. Cambridge: Cambridge University Press.Google Scholar
- Krosnick, Jon A, and Stanley Presser. 2010. Question and questionnaire design. In Handbook of survey research, 2nd ed, ed. Peter V Marsden and James D Wright, 263–313. Bingley: Emerald Group Publishing.Google Scholar
- Lee, Sophia YM. 2010. A linguistic approach to emotion detection and classification. Ph.D. dissertation. Hong Kong: The Hong Kong Polytechnic University.Google Scholar
- Lee, Sophia YM, Ying Chen, and Chu-Ren Huang. 2010. A text-driven rule-based system for emotion cause detection. In Proceedings of Human Language Technologies: The 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, 45–53.Google Scholar
- Lee, Sophia YM, Ying Chen, Chu-Ren Huang, and Shoushan Li. 2013. Detecting emotion causes with a linguistic rule-based approach. Computational Intelligence 29: 390–416.Google Scholar
- Lee, Sophia YM, Shoushan Li, and Chu-Ren Huang. 2014. Annotating events in an emotion corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), ed. Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, 3511–3516. European Language Resources Association (ELRA).Google Scholar
- Li, Jin, Lianqin Wang, and Kurt Fischer. 2004. The organization of Chinese shame concepts. Cognition and Emotion 18(6): 767–797.View ArticleGoogle Scholar
- Li, Yuming 李宇明. 2010. Global Chinese dictionary 全球华语辞典. Beijing: Commercial Press.Google Scholar
- Lin, Jingxia, Hongzhi Xu, Menghan Jiang, and Chu-Ren Huang. 2014. Annotation and classification of light verbs and light verb variations in Mandarin Chinese. In Proceedings of the 25th International Conference on Computational Linguistics (COLING 2014), Workshop on Lexical and Grammatical Resources for Language Processing, 75–82.View ArticleGoogle Scholar
- Liu, Feng-Hsi. 1997. An aspectual analysis of ba. Journal of East Asian Linguistics 6(1): 51–99.View ArticleGoogle Scholar
- Liu, Feng-Hsi. 2007. Word order variation and ba sentences in Chinese. Studies in Language 31(3): 649–682.View ArticleGoogle Scholar
- Monnier, Catherine, and Arielle Syssau. 2014. Affective norms for French words (FAN). Behavior Research Methods 46: 1128–1137.View ArticleGoogle Scholar
- Nabi, Robin L. 2002. The theoretical versus the lay meaning of disgust: Implications for emotion research. Cognition and Emotion 16(5): 695–703.View ArticleGoogle Scholar
- Oatley, Keith, and Phillip N Johnson-Laird. 1987. Towards a cognitive theory of emotions. Cognition and Emotion 1(1): 29–50.View ArticleGoogle Scholar
- Ortony, Andrew, Gerald L Clore, and Allan Collins. 1988. The cognitive structure of emotions. New York: Cambridge University Press.View ArticleGoogle Scholar
- Pang, Bo, and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2(1-2): 1–135.View ArticleGoogle Scholar
- Pavlenko, Aneta. 2008a. Emotion and emotion-laden words in the bilingual lexicon. Bilingualism: Language and Cognition 11(2): 147–164.Google Scholar
- Pavlenko, Aneta. 2008b. Structural and conceptual equivalence in the acquisition and use of emotion words in a second language. The Mental Lexicon 3(1): 91–120.Google Scholar
- Pennebaker, James W, Tracy J Mayne, and Martha E Francis. 1997. Linguistic predictors of adaptive bereavement. Journal of Personality and Social Psychology 72(4): 863–871.View ArticleGoogle Scholar
- Plutchik, Robert. 1962. The emotions: Fact, theories and a new model. New York: Random House.Google Scholar
- Plutchik, Robert. 1980. Emotions: A psychoevolutionary synthesis. New York: Harper and Row.Google Scholar
- Plutchik, Robert. 1994. The psychology and biology of emotion. New York: Harper-Collins.Google Scholar
- Potts, Christopher. 2007. Into the conventional-implicature dimension. Philosophy Compass 2(4): 665–679.View ArticleGoogle Scholar
- Ravi, Kumar, and Vadlamani Ravi. 2015. A survey on opinion mining and sentiment analysis: Tasks, approaches and applications. Knowledge-Based Systems 89: 14–46.View ArticleGoogle Scholar
- Redondo, Jaime, Isabel Fraga, Isabel Padrón, and Montserrat Comesaña. 2007. The Spanish adaptation of ANEW (Affective Norms for EnglishWords). Behavior Research Methods 39(3): 600–605.View ArticleGoogle Scholar
- Russell, James A. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39(6): 1161–1178.View ArticleGoogle Scholar
- Russell, James A. 1991. Natural language concepts of emotion. In Perspectives in personality, vol. 3, ed. Daniel J Ozere, Joseph M Healy Jr., and Abigail J Stewart, 119–137. London: Jessica Kingsley.Google Scholar
- Stone, Phillip J, Dexter C Dunphy, Marshall S Smith, and Daniel M Ogilvie (eds.). 1966. The general inquirer: A computer approach to content analysis. Cambridge: MIT Press.Google Scholar
- Strapparava, Carlo, and Rada Mihalcea. 2008. Learning to identify emotions in text. In Proceedings of the 23rd Annual ACM Symposium on Applied Computing (ACM-SAC 2008), 1556–1560. New York: ACM.View ArticleGoogle Scholar
- Strauss, Gregory P, and Daniel N Allen. 2008. Emotional intensity and categorisation ratings for emotional and nonemotional words. Cognition and Emotion 22(1): 114–133.View ArticleGoogle Scholar
- Subasic, Pero, and Alison Huettner. 2001. Affect analysis of text using fuzzy semantic typing. IEEE Transactions on Fuzzy Systems 2(4): 483–496.View ArticleGoogle Scholar
- Tang, Huifeng, Songbo Tan, and Cheng Xueqi. 2009. A survey on sentiment detection of reviews. Expert Systems with Applications 36(7): 10760–10773.View ArticleGoogle Scholar
- Tsou, Benjamin, and Rujie You 邹嘉彦, 游汝杰. 2010. An international dictionary of Chinese neologisms 全球华语新词语词典. Beijing: Commercial Press.Google Scholar
- Turner, Jonathan H. 1996. The evolution of emotions in humans: A Darwinian–Durkheimian analysis. Journal for the Theory of Social Behaviour 26(1): 1–33.View ArticleGoogle Scholar
- Turner, Jonathan H. 2000. On the origins of human emotions: A sociological inquiry into the evolution of human affect. Stanford: Stanford University Press.Google Scholar
- Turner, Jonathan H. 2007. Human emotions: A sociological theory. NewYork: Routledge.Google Scholar
- Wiebe, Janyce, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation 39(2): 165–210.View ArticleGoogle Scholar
- Wierzbicka, Anna. 1992. Defining emotion concepts. Cognitive Science 16(4): 539–581.View ArticleGoogle Scholar
- Xu, Xiaoyin, and Jianhua Tao. 许小颖, 陶建华 2003. The study of affective word categorization in Chinese 汉语情感系统中情感划分的研究. In Proceedings of the 1st Chinese Conference on Affective Computing and Intelligent Interaction 第一届中国情感计算及智能交互学术会议论文集.Google Scholar
- Xu, Linhong, Hongfei Lin, Yu Pan, Hui Ren, and Jianmei Chen. 2008. Constructing the affective lexicon ontology. Journal of the China Society for Scientific and Technical Information 27(2): 180–185.Google Scholar
- Yu, Shiwen, Xuefeng Zhu, Hui Wang, and Yunyun Zhang 俞士汶, 朱学锋, 王惠, 张芸芸. 2003. The grammatical knowledge-base of contemporary Chinese: A completle specification, 2nd edition 现代汉语语法信息词典详解第二版. Beijing: Tsinghua University Press.Google Scholar
- Zhang, Bojiang 张伯江. 2001. Symmetries and asymmetries between bei-construction and ba-construction 被字句和把字句的对称与不对称. Chinese Language 中国语文 285(6): 519-524.Google Scholar
