Title: A multimodal construction of the Chinese dream: A case study of the multimodal metaphors and metonymies in Mengwa. Download the recorded talk.

Abstract: Metaphors and metonymies are not merely linguistic phenomena, but a way of thinking, so they exist not only in language but also in other modes. What’s more, multimodal metaphors and metonymies can be blended to create emergent structures and construct new concepts. This talk aims to analyze the metaphorical conceptualization of the Chinese Dream in Mengwa, a series of TV public service advertisement. With the use of gestures, images, discourses, sound, composition, colour, foreground, background, etc., multimodal metaphors and metonymies at the different levels of Image schemas, Domain, Frame, Mental spaces are constructed one after another, such as VIRTUE IS UP, DILIGENCE IS FOUNDATION, GOODNESS IS SOUL, HONESTY IS BODY, BEAUTY STANDS FOR GOOD FUTURE, VIRTUES MAKE A FAMILY HAPPY, VIRTUES TAKE A FAMILY TO A NICE FUTURE, etc.. And these multimodal metaphors and metonymies conceptualize the six virtues of goodness, diligence, thriftiness, honesty, filiality, and harmony as the foundation of an ordinary but happy Chinese family. In addition, China is made up of millions of thousands of ordinary families, therefore, CHINA IS A BIG FAMILY. Consequently, the Chinese Dream becomes every family member’s dream but rich with further specifications of any of the individuals, i.e., THE CHINESE DREAM IS MY DREAM.

Title: Gesturing toward the future: Casting & Gesture in Forced Entertainment’s “Complete Works: Table Top Shakespeare.” Download the recorded talk.

Abstract: As most American college kids will tell you, Shakespeare is confusing. There are books in English that promise to “translate” Shakespeare, as if it’s a foreign language. Yet productions have used casting, particularly celebrity casting, to offload some of the storytelling onto the bodies of the actors. Because we know so much about a particular actor in real life, we can use that to make sense of this new fictional world. Casting is also used to reimagine the Shakespearean classic, using a body we might not expect to tell the old story in a new way. You might think you know King Lear, for example, but what might we see differently about our world today if Lear is played by a woman? When a casting choice is meant to challenge or change your category (of leader or lover or human, say), I call it counter casting. The example of counter casting I want to talk about today is Forced Entertainment’s “Complete Works: Table Top Shakespeare” wherein actors tell the story of the play sitting at a bare table casting everyday household objects as the characters. I will examine the subtle gestures that they use to direct our attention, evoke our emotion, and signal conceptual ideas in the play such that a cheese grater is terrifying and the death of an iron is heart-breaking. This Shakespeare—no set, no lights, no actors--is not confusing.

Title: Old Norse sagas about Vikings, Saracens, and Buddha: misunderstandings or cultural adaptations? Download the recorded talk.

Abstract: The aim of this lecture is to discuss the nature of intercultural communication, based on literary and manuscript evidence from medieval Norway and Iceland. The North was, and still is, on the edge of inhabitable land and may seem peripheral to religious and political centers in the Middle Ages, such as Jerusalem or Rome. However, as known, the Vikings were renowned seafarers, Scandinavians travelled to Rome and Jerusalem on pilgrimages, and Norse kings had political alliances with rulers in not only Western Europe, but also kingdoms such as Tunisia and Armenia. Such intercultural contacts are recorded in Old Norse literature, which comprises of highly original indigenous prose and poetry, as well as translations from Latin, French, and German. The writers and translators of these texts often wrote under the commission of Norwegian kings or queens, or their powerful aristocrats. The scribes belonged to various religious orders of the Catholic Church, thus participating themselves in highly international European networks. Through a study of three types of stories – Icelandic family sagas about Vikings; translations of French chivalric literature and crusade stories, and religious texts, one of which is a Christian rendering of the life of Buddha – we will see that the various narratives about distant exotic worlds retell of intercultural communication and cultural diversity, but that the narratives also share some common traits and mode of representation. It is therefore relevant to ask: Were the common characteristics in the representation of ‘others’ due to the Christian scribes misunderstanding their multicultural past and environment? Or did they just adapt the stories about the ‘others’ to mirror and create their own self-image? These stories are fascinating in their own right, but they may also provide analogies for thinking about intercultural communication in general. By regarding literature as cultural translatio, or as a blend of old memories and histories, and new intentions and aspirations, we see how and why every literary text was written down, translated, or copied to fit its new cultural context. In translations, necessary changes are made in order for a text to be intelligible to its new target audience. In medieval manuscript culture, this was done even on a material level, as even the form and design of the manuscripts were adapted to the readership and their needs. Old Norse literature offers intriguing examples of unique world literature, but it also testifies to common principles in intercultural communication.

Title: Memories of wars: A multimodal study of war heroes and heroines on the website of British Imperial War Museum Download the recorded talk.

Abstract: War museum websites have been used widely as a primary site for the memories of wars, in which the narrating of various heroic deeds and life of heroes and heroines involved in the wars covers the major part of the multi-modal historic military discourse. For the this study, the official website of the British Imperial War Museum (IWM) was adopted as the major source of data, and three questions were raised as follows: What heroes/heroines are posed at the website of Imperial War Museum?; What image is constructed of heroes/heroines on the website of Imperial War Museum?; What multimodal discursive strategies are employed for the constructing of the heroic images on the website of Imperial War Museum? Based on 55 narrative stories, the study made a multimodal analysis, and it is found that the multimodal dimension of text and pictures/photos/paintings are more frequently used for the constructing of the heroic image of heroes, and a number of memorial effects were created and framed for the heroes/heroines, and the enemy and the war itself. The text focuses upon the heroic deeds and life while the pictures/photos/paintings depict the hardship, the dreadfulness and strong belief and optimism of the heroes/heroines. More specific functions of each modal dimension were also analysed and the three questions raised were also answered. It is found that multimodal dimensions are very effective means for the depicting images of the heroes/heroines and even memories of wars.

Title: The cross-cultural patterns of poetic performance in oral traditions: multimodality, communication, and cognition. Download the recorded talk.

Abstract: How do we learn to organize a language in chunks and to use those chunks creatively? Theories of chunking are based on abstract rules or on the storage of large numbers of exemplars. They view linguistic knowledge as a linear combination of discrete ‘chunks,’ such as phonemes or morphemes. The Parry-Lord theory of oral composition-in-performance argued that oral singers produce complex poems out of rehearsed improvisation through the mastery of a system of formulas, chunks that integrate phrasal, metrical, and semantic structures. Recently, computational linguistic models (Baayen et al.) based on discriminative learning propose that linguistic knowledge consists of statistical expectations within the complex dynamic system of cues and outcomes underlying language. Instead of discrete units, these computational models use a ‘wide’ learning algorithm with thousands of input units representing summaries of changes in acoustic frequency bands, and with proxies for distinctions in a lexical meaning vector space as output units. In this talk, I will reconsider formulaicity and creativity in oral poetic performance through these non-compositional models.

Title: Epistemic modulations in English and Chinese. Download the recorded talk.

Abstract: In communicating a particular message, speaker and listener seek to converge on a shared meaning. In Information Theory, Claude Shannon describes this as a process of encoding and decoding a message using a shared code, where each new character transmitted reduces uncertainty, or information entropy, by a measurable amount by guiding the listener towards a particular location in a multidimensional combinatorial space. Communication is judged to be successful when a message is transmitted unchanged from speaker to listener. Multimodal human communication encodes meaning in far more complex ways both verbally and non-verbally. Much of the intended meaning is never transmitted, but is assumed or must be inferred. Each word or gesture provides a cue that guides the listener in a multidimensional state space towards the intended meaning in an infinitely large multidimensional space of possible meanings. Do non-verbal cues provide a universal bedrock of a shared human code? In a cross-cultural communicative situation, are there non-verbal cues that allow speakers without a shared language to reduce the uncertainty of the intended message? In this talk we examine shared and divergent patterns of non-verbal epistemic qualifications in English and Chinese. These qualifications aim to communicate the degree of novelty, surprise, acceptance, and certainty of a particular message and help listeners orient towards a particular intended epistemic stance.

Title: The Case of Disappearing Ironies: A Multimodal Story. Download the recorded talk.

Abstract: In recent years, linguists studying irony have increasingly come to appreciate the importance of multimodal data, and to take irony seriously as a cross-modal phenomenon (see, for example, Colston and Athanasiadou’s recent volume The Diversity of Irony). In this talk I will bring this approach to the phenomenon of what I call “irony attrition”: the perhaps surprisingly common case when people start out by doing or saying something ironically, but become increasingly earnest in that usage, over time. We’ll consider ironies and erstwhile ironies from a wide variety of settings, and bring them into conversation with the trajectories of other kinds of perspective-embedding expressions, in both language acquisition and language change. Irony attrition turns out not to be a hazard of irony as irony per se, though its effects in that area are striking and important. Instead it is part of a more general tendency for intermediate-level, embedded discourse frames to get compressed or go missing in both memory and quotation. We’ll trace this process as a consequence of three things: (a) the complex viewpoint arrangement underlying the ironic interpretive stance, (b) the limitations of human memory for source information, and (c) routinization.

Title: Mistaking Our Futures. View the talk on YouTube or download it.

Abstract: Forms of expression—words, phrases, gestures, material anchors in the ground of communication exploited by the speaker—do not mean or carry meaning. They are instead just forms, tools we exploit to prompt others to construct meaning. Often these forms are connected in our minds to conceptual frames, and if those frames are common knowledge in a culture, commonly known by native speakers, we expect that the frame we activate will be the one activated by other participants. The slightest form can prompt us to call up great ranges of meaning from that frame. A frame is a mental bundle of meaning that we expect all native speakers to know, or, if they are children, to learn quickly. Knowing these common frames is indispensable to understanding basic native patterns in the language. We have a mental bundle for grave-digging, a mental bundle for marriage, and so on. We have a mental frame for buying and selling, and a more limited mental package for the special case of buying and selling securities, particularly stocks and bonds and other financial instruments. In that mental package, that little frame for buying and selling securities, there are roles for the buyer, the seller, what is sold, and the broker who arranges the transaction. When someone says, “I have to call my stockbroker,” everyone can activate the appropriate package, the appropriate bundle of related elements. When someone says that sentence, we imagine, unless we are told or have reason to believe otherwise, that the telephone call is about buying and selling securities. Nobody needs to explain that the call is about buying and selling, because the word “stockbroker” calls up that frame. To understand “I must call my stockbroker,” we activate information from that frame to build a small mental array containing the speaker, the phone call, and the broker. One of the greatest sources of misunderstanding in intercultural communication is the assumption by participants that they are calling up the same frame when they use certain expressions, as opposed to calling up frames that merely have some intercultural overlap. The resulting misunderstandings can be particularly insidious because the fact of the misunderstanding often goes unnoticed. This talk will look at attempts to provide instruments for reducing such frame-based misunderstanding, such as the DARPA-funded app “Negotiation Across Cultures using TACT: A Tool to Accelerate Cultural Talks.” We will focus especially on frame-based misunderstandings in intercultural communication about our possible futures.

Title: Visual metaphors in news cartoons on COVID-19 in China. Download the recorded talk.

Abstract: Recent years has witnessed a rapid growth of visual metaphor studies in a variety of genres. News cartoons, the carrier of visual metaphors, are applied to express ideas and comments on social concerns. The most unusual and impressive social concern this year is the war of fighting against COVID-19, a pandemic affecting the whole world. Everyone unites to fight the virus. Anti-pandemic heroes are fighting on the frontline, which we ordinary people stay at home and follow the related news every day. Many pictures are used to in news to communicate and transmit message. This research aims to identify the visual metaphors in the news cartoons on COVID-19 and explain the meaning construction of these metaphors to reveal how the pandemic is conceptualised and people’s attitudes towards it. All the visual metaphors demonstrate that the Chinese people are determined to fight against the pandemic, showing our confidence and insistence towards it.

Title: The transmission of mechanical technologies between the Classical Mediterranean and ancient China. Download the recorded talk.

Abstract: The Hellenistic, Roman and Byzantine empires of the Mediterranean on the one hand, and China on the other, developed a remarkable array of mechanical and geared technologies between the late first millennium BC and the end of the first millennium AD – water-lifting devices, water-powered machinery, mechanical clocks, and automata. Joseph Needham and Wang Ling in 1965 argued that the horizontal water-driven wheel reached the West from China, and that the vertical water-wheel was simultaneously adopted, perhaps independently invented. But in the last half-century new research on the Classical world has argued for a different chronology for the invention and uptake of the water-mill and water-lifting wheels in the Mediterranean. The implications of this for the question of the relationship with Chinese technology have not yet been explored. This paper assesses the evidence for the chronology of certain mechanical technologies – cogs and gearing, mechanical automata, water clocks, gimbals, hodometer carts, and water-powered machinery – in the East and the West, to see what light that might shed on ancient connections between China and the Mediterranean world. I identify a repeating pattern of the emergence of particular technologies in the Mediterranean, and then their appearance in China within 70–200 years of their appearance in the West. This happens in too many instances to be explained as independent invention, and diffusion along early incarnations of the Silk Roads appears to be the explanation. I examine whether in particular instances the device itself may have been carried by merchants or embassies, or whether it was just the idea that travelled – and sometimes became garbled in transmission. The new picture that emerges suggests that Han China was less isolated, and rather more receptive to foreign technological ideas, than it was often portrayed as being in the scholarship of the 20th century.

Title: Towards a Theory of Interdisciplinarity: Multimodal Blending in Teaching and Learning in Area Studies. Download the recorded talk.

Abstract: The world’s most pressing problems require solutions that cross disciplinary boundaries. Yet in an academy dominated by disciplinary thinking, interdisciplinarity is very challenging for researchers, teachers, and students. This is definitely the case in area studies which integrates numerous social sciences and humanities subjects. Problem-based learning (PBL) is a promising teaching methodology for integrating multiple disciplines. However, the literature on problem-based learning does not adequately articulate the underlying cognitive processes required for interdisciplinary knowledge construction, particularly outside the clinical sciences. I propose blending and conceptual integration as a theoretical foundation for interdisciplinary teaching and learning, particularly in area studies. I illustrate this theoretical approach with examples drawn from two masters courses - on Russian language and on the culture of Russia and Eastern Europe – constituting integral parts of a degree programme. I argue that understanding interdisciplinarity from a cognitive perspective allows both students and teachers to be more self-conscious about the practice of interdisciplinary studies, thereby enhancing the learning and teaching process. Furthermore, I argue that using multimodal data (e.g., videos published by media, recordings of political speeches and interviews, films) in teaching and learning in area studies provides the most needed conceptual bridges between various disciplinary components of these interdisciplinary programmes.

Title: Virtual communication between machines with the human as their object: A new stage of multimodal communication after oral, written, printed, electronic and machine-human communication. Download the recorded talk.

Abstract: Luhmann and Baecker have described the development of communication from orality over script, over print and finally to digital communication. In all these stages, technology played only an assisting role. This paper argues, that there is a fundamentally new stage of “virtual communication”, in which artificial intelligence has taken over and humans have become the object of manipulation (as customers, voters etc.). Algorithms do not only listen to oral or read written human communication (with bots or with other humans), but analyze multimodal communication (including likes, behaviour, surfing habits, mobility profile, values, dreams, aims, beliefs etc.), compare them with Big Data and base decisions of manipulation on correlations and on a personality profile. These algorithms target not just the explicit communication, but the emotions and thoughts of humans and also predict future behaviour, therefore allowing simulations of reality. Mightier algorithms also take over decision-making roles in societies, where they replace human court decisions, fine tune just-in-time and on-demand production, censor chatrooms etc. Sets of algorithms help to manage smart cities and a whole society. Although the human is still part of the communication, especially as the analyzed object and the target of the manipulation, the human is often unaware of the virtual communication and a passive receiver of the machine’s decisions, while the main actors in the virtual communication are machines. Research describes these forms of virtual communication, finds evidence in social management systems, credit systems (gamification of human life) or customized (fake) news filter bubbles and in customized consumption offers (Amazon, Facebook, Google, Netflix) and analyzes benefits including the security enhancement through such virtual communication. Research also has the duty to warn of abuse or harmful developments and to raise ethical questions. Especially the exosceletal ethics, imposed by gamifications like credit systems, need to be valued against intrinsic ethics.

About Us

Conference Organization Committee

Directors Prof. ZENG Yanyu, Dean of the College of Foreign Studies. Mark Turner, Director of the Center for Cognitive Science. Committee members: Prof. LIU Bai, Dr. CHEN Zhongping, Dr. ZENG Jiansong, Dr. QI Xingang, Dr. QIN Yong.

Hunan Normal University

Located in Changsha, a city of great historical and cultural interest, Hunan Normal University (HUNNU) is an institution of higher education listed in the national “211 Project” and the “Double Top-Class Project” constructed jointly by the Ministry of Education and Hunan Province. Founded in 1938 as National Normal College (NNC), it is one of the oldest normal universities in China. In the wave of university reforms in 1953, Hunan Normal College (HNC) was founded on the basis of NNC. In 1984, HNC was renamed HUNNU. Gloriously, it was admitted in 1996 into the “211 Project”—one of the “100 key universities to be promoted in the 21st century” by Chinese Ministry of Education. Since 2000, it has renewed itself by merging with Hunan Teachers’ College, Hunan College of Politics and Law and Hunan Medical College in succession.

HUNNU consists of 24 colleges and runs altogether 92 undergraduate disciplines, which fall into such 11 main categories as philosophy, economics, law, education, literature, history, science, technology, agriculture, medicine, management and art. It boasts such 6 National Key Disciplines as Ethics, English Language and Literature, Modern Chinese History, Developmental Biology, Theoretical Physics, Basic Mathematics, and 9 Key Disciplines sponsored by the 211 Project, and 22 provincial-level key disciplines rated in the 12th Five-Year Plan.

HUNNU has set up partnerships with 171 universities and institutions in 41 countries and regions to push forward personnel exchange and cooperation in teaching and scientific research. It has co-established Confucius Institutes at Kazan Federal University in Russia, Wonkwang University in South Korea and Southern Utah University in the U.S.

Over the 80 years, HUNNU has been developing steadily despite the warfare of WWII. The faculty, whichever generation they were, stuck to the motto “Be humane, benevolent, excellent and diligent”, and worked hard jointly for the prosperity today. In recent years, propelled by the “211 Project” and the “Double Top-Class Project”, HUNNU has achieved much in discipline development, student education, faculty construction, teaching research and social service in the satisfaction of more than Hunan’s needs in educational, economic and social development.

While going forward, HUNNU takes holistic education as the fundamental mission, and strives to be a key comprehensive university which, with great advantages in teacher training, is top-class in China and well known abroad.

College of Foreign Studies, 410081 36 Lushan Rd., Yuelu District, Changsha, China

Foreign Studies College of Hunan Normal University dates back to Dept. of Foreign Studies of National Normal College founded in 1938. The first dean was QIAN Zhongshu (1910-1998), a famous scholar of Western and Chinese culture. After him, LUO Kailan (1906-1988), LIU Zhongde (1914-2008) and other eminent scholars worked here in succession. Now it holds the first-level doctoral program of Foreign Language and Literature and a research station for post-doctors. Under the leadership of Prof. JIANG Hongxin, its discipline of English Language and Literature was evaluated as a national key discipline. In Sept. 2017, its discipline of Foreign Languages and Literatures was admitted into the national “World First-Class Discipline Construction Project”, being one of the 6 admitted disciplines of its type in China.

It consists of Dept. of English, Dept. of Translation Studies, Dept. of Russian, Dept. of Japanese, Dept. of Korean, Dept. of French and Dept. of Public English, and boasts such institutes as Hunan Center for International Cultural Communication, Hunan Center for Sino-Russian Cultural Exchanges, Center of American Studies, Center of Northeast Asian Studies, Center for Studies of British and Irish Literature, Center of Modern Foreign Language Teaching, Center of Cognitive Linguistic Studies, Center for Studies of British and American Poetry. It publishes Journal of Foreign Languages and Cultures and a Chinese journal of the same name, and supports 3 Confucius Institutes abroad.

It has a faculty of 26 full professors, 44 assistant professors and dozens of lecturers, of whom 51 have got doctoral degrees, 2 are members of the Discipline Assessment Group under the State Council, 2 are state-level teaching masters, and 2 are awardees of the New Century Talent Program of Chinese Ministry of Education. It is a partner of over 30 universities in America, Britain, Japan, Russia and South Korea. Now it has over 40 doctoral candidates, over 600 graduate students, and over 1,200 full-time undergraduates. Adhering to the motto “international perspective, global sense, honesty, integrity and versatility”, Foreign Studies College aims to cultivate more versatile and innovative talents who are both physically and mentally healthy, both virtuous and learned and are adaptable to societal changes.

Red Hen Lab™

The International Distributed Little Red Hen Lab™ is a global big data science laboratory and cooperative for research into multimodal communication. Red Hen deploys the contributions of researchers from complementary fields, from AI and statistics to linguistics and political communication, to create rich datasets of parsed and intelligible multimodal communication and to develop tools to process these data and any other data susceptible to such analysis. Red Hen’s social organization and computational tools are designed for reliable and cumulative progress in a dynamic and extremely challenging field: the systematic understanding of the full complexity of human multimodal communication. The study of how human beings make meaning and interpret forms depends upon such collaboration.