Friday, 9 June 2017, 3-5:30pm, Parallel Sessions
Panel 1, Room B01 Blending, Metaphor, & Metonymy in Multimodal Communication. Chair: Inés Olza.
3pm: Pleshakova, Anna (University of Oxford): "Viewpoint
Construction in Russian Media: 'A Right to Voice One's Opinion'."
3:30pm: Wozny, Jacek (University of Wroclaw): "Multimodal blending in meaning construction of Polish "chodzi o" and Czech "jde o" ('it is about')"
4pm: Mendes de Oliveira, Milene (University of Potsdam): "Blended classic joint attention in user-created Youtube videos"
4:30pm: Colautti, Jesse (University of British Columbia): "The Lord of the White House: Blending fictive narratives with reality in multimodal memes and gifs"
5pm: Mierzwinska-Hajnos, Agnieszka (Maria Curie-Skłodowska University): "When Music does not Need Words: Multimodality in Walt Disney’s Fantasia. A Conceptual Blending Analysis."
Panel 2, Room E26 Discourse & interaction 1. Chair: Paolo Caponetto
3pm: Menke, Peter (Universität Paderborn): "'Es war einmal' - A multimodal corpus of a collaborative storytelling card game"
3:30pm: Pfaender, Stefan (University of Freiburg): "Synchronization in Embodied Interaction: Collaborative Utterances"
4pm: Augustyn, Rafał (Maria Curie-Skłodowska University): "Verbo-audio-visual ensembles in science communication"
Panel 3, Room 104 Multimodality & Creative Expression. Chair: Anna Bonifazi
3pm: Colín Rodea, Marisela & Mihaela Mihaeliescu (Universidad Nacional Autónoma de México): "Modes and submodes in Mihai Eminescu’s literary work"
3:30pm: Bolek, Elwira (Maria Curie-Skłodowska University): "Play on signs and meanings in Polish artistic theatre posters"
4pm: Hsu, Hui-Chieh; Kurt Feyaerts & Geert Brône (KU Leuven): "The hand, the eye, and the bow: Embodied multimodal depicting in cello master classes"
4:30pm: Page, Jeremy (Macquarie University): "'The Michelangelo of Flow’: Revised Methods for Visualising Rhythmic Synchronicity in Rap"
5pm: Pearlman, Karen (Macquarie University): "Cutting Rhythms: Editing and Cognition Beyond Continuity"
Panel 4, Room 106 Concepts & Form. Chair Javier Valenzuela
3pm: Hedblom, Maria M. & Oliver Kutz (Free University of Bozen-Bolzano): "Multi-modal Image Schemas"
3:30pm: Rosiński, Maciej (University of Warsaw): "A multimodal analysis of the emergence of geometrical concepts"
4pm: Jelec, Anna (Adam Mickiewicz University): "Grounding conceptualisation in bodily mimesis"
4:30pm: Berindeanu, Florian (Case Western Reserve University): "Deleuze beyond Language: Communication Between Semiotic and Pragmatics"
Panel 5, Room 107 Body, Face, Gesture, Space 1. Chair: Alan Cienki
3pm: Hubbard, Amy L. (German Sport University Cologne): "Walk this way: How movement analysis is changing treatment options for one neuroclinical population"
3:30pm: Tong, Yao (Vrije Universiteit Amsterdam): "Words accompanied by representational gestures, their grammatical categories and semantic fields: A corpus-linguistic approach"
4pm: Stave, Matthew & Eric Pederson (University of Oregon): "When does a nod mean ”Yes”? The timing of back-channel head gestures during narration"
4:30pm: Rominger, Christian (University of Graz); Kurt Feyaerts (University of Leuven); Helmut K. Lackner (University of Graz); Geert Brône (University of Leuven); Bert Oben (University of Leuven) & Ilona Papousek (University of Graz): "In your face? Exploring multimodal resonance patterns involving facial responses to verbal and gestural stance-taking expressions"
5pm: Iriskhanova, Olga (Moscow State Linguistic University & RAS Moscow) & Olga Prokofyeva (Moscow State Linguistic University): "Focusing in visual perception, speech and gestures: А multimodal analysis of oral descriptive discourse in Russian"
Panel 6, Room E25 Media, Word, Image. Chair: Line Engh
3pm: Hougaard, Anders (University of Southern Denmark): "Hyperembodiment in Multimodal Instant Picture Messaging"
3:30pm: Lou, Adrian (University of British Columbia): "Internet Memes as Multimodal Similes: A Cognitive Linguistic Analysis"
4pm: Bolognesi, Marianna (University of Amsterdam): "Flickr Distributional Tagspace: Modeling the multimodal semantic information encoded in Flickr tags."
4:30pm Grogan, Kimberly L. (Case Western Reserve University): "Multimodality and Conceptual Blending: Political Graffiti Murals as a Form of Fictive Communication"
5pm: Arruda-Doná, Beatriz & Aline Pereira de Souza (State University of São Paulo): "The seduction of the image: an attractive way of constructing texts"
Panel 7, Room 215 Multimodality, Gesture, and Space. Chair: Leonid Berov
3pm: Avelar, Maíra (State University of Southwest Bahia) & Lilian Ferrari (Federal University of Rio de Janeiro): "Experiential Integration and Deixis: the discourse role of gestures"
3:30pm: Rekittke, Linn-Marlen (RWTH Aachen University): "Multimodal expressions of discontent: The role of interactive gestures in negotiating agreement during travel planning"
4pm: Hoetjes, Marieke (Radboud University Nijmegen): "The effect of gesture on fluency"
4:30pm: Brunner, Marie-Louise & Stefan Diemer (Saarland University & Trier University of Applied Sciences): "Developing a taxonomy of gestures in multimodal communication via Skype"
Saturday, 10 June 2017, 12:30-1:30pm, Poster Session. Location: foyer outside the main lecture hall
Stampoulidis, Georgios (Lund University): "Approaching Athenian Graffiti as a Multimodal Genre with GIS Application"
Pleshakova, Anna & Xavier Laurent (University of Oxford): "Developing a Teaching Module on Research Methods for Multimodal Communication and Discourse Analyses"
Keily, Holly (University at Buffalo): "The impact of memory limitation on gesture production"
Alekseenko, Natalia (Moscow State Linguistic University): “Fictive Interaction from Multimodal Perspective”
Boll, Franziska (European University Viadrina Frankfurt (Oder)): "Intra- and interpersonal comparability of gestures and speech in consecutive interpreting"
Marcantonio, Daniela (Università della Svizzera italiana): "Emblems across Culture. How Italian-Germans and German-Italians use their multimodal code."
Makoveyeva, Alina (Moscow State Linguistic University): “FAMILY stereotype in adults' and children's spoken discourse in Russian: A multimodal study”
Gilquin, Gaëtanelle (Université catholique de Louvain): "NESSI: A multimodal corpus of New Englishes"
- Czeszumski, Artur (Osnabrück University); Chiara Carrera (Osnabrück University); Basil Wahn (Osnabrück University) & Peter König (Osnabrück University & University Medical Center Hamburg-Eppendorf): "Does the social situation affect how we process feedback about our actions?"
- Krajcik, Chelsea (SOAS, London): "Placement and removal verbs in Joola Kujireray - using gesture to explore semantic information."
Saturday, 10 June 2017, 2:30-5:30pm, Parallel Sessions
Panel 8, Room B01 CREATIME: Creativiy and cognition in the representation of time across modalities. Chair: Anders Hougaard.
2:30pm: Págan Cánovas, Cristóbal (University of Navarra): Brief Introduction to CREATIME: Time in the Creative Mind
3pm: Alcaraz Carrión, Daniel (Lancaster University) & Javier Valenzuela (University of Murcia): "Temporal co-speech gestures: A comparison between spatial and non-spatial temporal expressions"
3:30pm: Gordejuela, Adriana (University of Navarra): "Film flashbacks and the conceptualization of time"
4pm: Piata, Anna (University of Geneva & University of Neuchâtel) & Cristina Soriano (University of Geneva): "Spatial representations of time and emotion: Evidence from PoetiCog"
4:30pm: Olza, Inés (University of Navarra); Javier Valenzuela (University of Murcia); Daniel Alcaraz Carrión (Lancaster University); Cristóbal Pagán Cánovas (University of Navarra): "The Red Hen Lab’s NewsScape Library and Gesture tagging in the CREATIME project: Methodology, theoretical implications and contribution to machine-learning"
5pm: Pagán Cánovas, Cristóbal (University of Navarra); Inés Olza (University of Navarra); Javier Valenzuela (University of Murcia): "Conceptual integration templates for time: converging evidence from big multimodal data"
Panel 9, Room E26 Poetics & Artforms 1. Chair: Marianna Bolognesi
2:30pm: Suchan, Jakob & Mehul Bhatt (University of Bremen): "Obsessed by Symmetry: A Multi-Modal Study on the Perception and Interpretation of Symmetry in the Films of Wes Anderson"
3pm: Berov, Leonid (Osnabrück University): "Mental Simulation for Multi-Modal Story Telling"
3:30pm: Krug, Maximilian (University of Duisburg-Essen): "Rehearsing in theatre: Collective achievement of an interactional system through the coordination of multiple activities"
4pm: Císlerová, Eva (Czech Technical University in Prague): “Architecture: Language and Paralanguage”
4:30pm: Lüdtke, Jana & Arthur M. Jacobs (Freie Universität Berlin): "‘QNAing’ Shakespeare: Tools and predictions for neurocognitive poetics"
5pm: Liao, Zhenggang (Jilin University): "The Cognitive Interpretation of Basic Color Words in the Yellow Emperor Bible"
Panel 10, Room 104 Multimodality & Culture. Chair: Jeremy Page
2:30pm: Jurewicz, Joanna (Warsaw University): "The Dancing Śiva and the Work of Mind. Cognitive Analysis of the Religious Image"
3pm: Wabende, Scholastica (Moi University): "Gender and Power as Constructed in Bukusu Circumcision Discourses"
3:30pm : Paliichuk, Elina (Borys Grinchenko Kyiv University): "Human Trafficking Media Representation: An Iconic Perspective"
4pm: Bilkic, Maida (University of Bern): "Sensing the war: The multimodal production of public memory in Bosnia-Herzegovina"
Panel 11, Room 106 Social Media & Art. Chair: Kimberley Grogan
2:30pm: Sadowska-Dobrowolska, Katarzyna (University of Maria-Curie Skłodowska): "Multimodality in the translation of humour in comics"
3pm: Ibrahim, Wesam Mohamed Abdel Khalek (Tanta University): "A Mulitmodal Analysis of Political Cartoons in Post-Revolution Egypt"
3:30pm:Libura, Agnieszka (Wrocław University): ""Any Minuten now... Wörk!" Playing with stereotypes in Polandball memes"
4pm: Borkent, Mike (SSHRCC Postdoctoral Visiting Fellow; University of Calgary): "Multimodality and Medium-specificity: The Case of Temporal Inferences in Comics"
4:30pm: Albano, Mariangela (University Sorbonne Nouvelle, Paris 3): "Adaptive memory in case of linguistic contamination: a comparison of the reception of TV News in France after FIFA World Cup in 2014 and after November 2015 Paris attacks"
5pm: Horst, Dorothea (European University Viadrina, Frankfurt (Oder)): "The embodied and multimodal dynamics of metaphor in campaign commercials"
Panel 12, Room 107 Body, Face, Gesture, Space 2. Chair: Marie-Louise Brunner
2:30pm: Schönherr, Beatrix (Universität Innsbruck): "Some Functions of Prosody and Gesture in Conversations on Stage"
3pm: Ambrus, Laura (Eötvös Loránd University): "The role of gestures in the interpretation of digital memes"
3:30pm: Trujillo, James P. (Radboud University Nijmegen); Julija Vaitonyté (Radboud University Nijmegen); Irina Simanova (Radboud University Nijmegen); Harold Bekkering (Radboud University Nijmegen) & Asil Özyürek (Radboud University Nijmegen & Max Planck Institute for Psycholinguistics): "Kinect and gesture research: a validation study of automatic feature coding"
4pm: Sukhova, Natalya Vitalyevna (Lomonosov Moscow State University; National University of Science and Technology "MISiS", Institute of Linguistics, Russian Academy of Sciences.) & Yulia Vladimirovna Nikolaeva (Lomonosov Moscow State University. Institute of Linguistics, Russian Academy of Sciences.): "On Defining Cephalic Gesture Categories"
Panel 13, Room E25 TV, news, documentaries, & advertising. Chair: Karen Pearlman
2:30pm: Jaki, Sylvia (University of Hildesheim): "Extending the notion of documentary: A multimodal analysis of the docudrama series Roman Empire: Reign of Blood (2016)"
3pm: Fonseca, Paula (Polytechnic Institute of Viseu & Centre for the Study of Education, Technologies and Health) & Esther Pascual (Zhejiang University): "Reality, Fiction and Fictivity: Multimodal interaction blends in The Daily Show with Jon Stewart"
3:30pm: Vajrabhaya, Prakaiwan (RWTH Aachen University): "Not so smooth criminal: Pointing gestures in crime news report in Thai newspapers"
4:30pm: Cichmińska, Monika (University of Warmia and Mazury in Olsztyn, Poland): "Global and local multimodal metaphors in (television) series"
Panel 14, Room 215 Methods 1. Chair: Stefan Diemer
2:30pm: Giangreco, Ivan; Luca Rossetto; Claudiu Tanase; Heiko Schuldt (University of Basel): "vitrivr: A Multimedia Search System supporting Multimodal Interactions"
3pm: Gonzalez-Marquez, Monica & Jerome Bergmann (RWTH-Aachen University): "MsquareC: A Python tool for coding multi-method data"
3:30pm: Shukla, Abhinav (IIIT Hyderabad & CCExtractor) & Carlos Fernandez (CCExtractor): "CCExtractor - Towards extracting all international formats of caption text and building a holistic subtitle processing tool"
4pm: De Smedt, Tom (Textgain); Guy De Pauw (Textgain) & Walter Daelemans (university of Antwerp): "Real-time text mining applications"
4:30pm: Brenger, Bela; Daniel Schüller; Christain Beecks & Irene Mittelberg (RWTH Aachen University): "Methods and tools for motion-capture data analysis: Advances in exploiting multimodal corpora"
5pm: Brône, Geert; Bert Oben & Kurt Feyaerts (University of Leuven): "The grounding function of eye gaze in interactional humor. A corpus-based study using eye-tracking data"
Sunday, 11 June 2017, 2:30-4:30pm, Parallel Sessions
Panel 15, Room B01 Multimodal expression patterns. Chair: Olga Iriskhanova
2:30pm: Cienki, Alan (Vrije Universiteit Amsterdam & Moscow State Linguistic University): "Utterance Construction Grammar (UCxG): An approach to constructions as variably multimodal"
3pm: Hirrel, Laura (University of New Mexico): "Multimodal Constructions Incorporating Cyclic Gestures and Finger-spread Handshapes"
3:30pm: Kibrik, Andrej A. (Russian Academy of Sciences & Lomonosov Moscow State University); Olga V. Fedorova (Lomonosov Moscow State University & Russian Academy of Sciences) & Julia V. Nikolaeva (Lomonosov Moscow State University & Russian Academy of Sciences): "Russian multichannel discourse"
4pm: Lelandais, Manon & Gaëlle Ferré (University of Nantes): "The multimodal expression of subordination in spontaneous conversation"
Panel 16, Room E26 Methods 2. Chair: Wiebke Ahlers
2:30pm: Nirme, Jens & Henrik Garde (Lund University): “Computational camera placement optimization improves motion capture data quality”
3pm: Pollaroli, Chiara (Università della Svizzera italiana): “An integrated method for the analysis of multimodal tropes and their argumentative-rhetorical function”
Panel 17, Room 104 Methods 3. Chair: Alexander Bergs
2:30pm: He, Xu (Jacobs University Bremen): "Audio Processing Pipeline for NewsScape Datasets."
3pm: Gullapuram, Shruti (IIIT Hyderabad & Red Hen Lab) & Abhinav Shukla (IIIT Hyderabad & CCExtractor): "News Shot Classification and Affect Recognition in Videos"
3:30pm: House, David (KTH Royal Institute of Technology, Stockholm); Gilbert Ambrazaitis (Lund University); Simon Alexanderson (KTH Royal Institute of Technology, Stockholm); Otto Ewald (Lund University) & Anneliese Kelterer (Lund University): "Temporal organization of eyebrow beats, head beats and syllables in multimodal signaling of prominence"
4pm: De Beugher, Stijn (KU Leuven Eavise): "A semi-automatic annotation tool for unobtrusive gesture analysis"
Panel 18, Room 106 Discourse & interaction 2. Chair: Michael Borkent
2:30pm: Lackner, Helmut K. (University of Graz); Ilona Papousek (University of Graz); Christian Rominger (University of Graz); Geert Brône (University of Leuven); Bert Oben (University of Leuven) & Kurt Feyaerts (University of Leuven): "Mapping patterns of interactive alignment in and across verbal and physiological behaviour. An empirical study."
3pm: Müller-Viezens, Johannes (TU Chemnitz): "Performing speech under copresence - Multimodal communication and its conceptualizing in face-to-face role-playing games"
3:30pm: Senkbeil, Karsten (University of Hildesheim): "Multimodality in intercultural communication: a cognitive-pragmatic approach to intercultural transfers of multimodal cultural texts"
Panel 19, Room 107 Body, Face, Gesture, Space 3. Chair: Anna Piata
2:30pm: Micklos, Ashley (Max Planck Institute for Psycholinguistics): "Multimodal repair initiation in silent gesture communication games: Eye gaze, facial expressions, and timing"
3pm: Jehlicka, Jakub & Eva Leheckova (Charles University Prague): "Functions of beat gestures in spontaneous interaction: evidence from English and Czech"
3:30pm: Zima, Elisabeth (University of Freiburg); Geert Brône (University of Leuven) & Clarissa Weiß (University of Freiburg): "On the role of eye gaze in competition for talk. Simultaneous starts in triadic interactions."
Panel 20, Room E25 Poetics & Artforms 2. Chair: Tonia Sperling
2:30pm: Bonifazi, Anna (University of Stuttgart): "The multimodal poetics of weeping from Homer to the Beatles"
3pm: Engh, Line Cecilie (The Norwegian Institute in Rome & University of Oslo): "Transforming Group Identities through Liturgy"
3:30pm: Caponetto, Paolo G. & Elisabetta Vinci (University of Catania): "Multimodality and Cultural Heritage: the Lamberto Puggelli's archive and the 'Teatro Machiavelli'"
Panel 21, Room 215 Constructions. Chair: Jacek Woźny
2:30pm: Ferre, Gaelle (University of Nantes): "Weather Reports as Multimodal Constructions"
3pm: Hampe, Beate (University of Erfurt); Irene Mittelberg (RWTH Aachen); Mark Turner (Case Western Reserve University) & Peter Uhrig (University of Erlangen): "Multimodal constructions? There-constructions ‘in the wild‘"
3:30pm: Averina, Anna (Moscow Teacher Training University): "Modal particles in combination with modal words as multimodal constructions in German"
Panel 22a, Room 105 Part 1 Multimodal Communication & Psychology. Chair: Mariangela Albano
2:30pm: Vasiliki Kondyli (University of Bremen); Mehul Bhatt (University of Bremen) & Jakob Suchan (University of Bremen): "People-centred design precedents for wayfinding. Learning from multi-modal analysis of visuo-locomotive experience."
3pm: Wahn, Basil (Osnabrück University) & Peter König (Osnabrück University & University Medical Center Hamburg-Eppendorf): "Is Attentional Resource Allocation Across Sensory Modalities Task-Dependent?"
Panel 22b, Room 105 Part 2 Metaphor & Metonymy. Chair: Mariangela Albano
3:30pm: Pereira de Souza, Aline & Beatriz Arruda-Doná (State University of São Paulo): "Metaphor and metonymy providing text comprehension"