Empirical Multimodality Research
豆瓣
Methods, Evaluations, Implications
Jana Pflaeging / Janina Wildfeuer …
简介
This volume advances the data-based study of multimodal artefacts and performances by showcasing methods and results from the latest endeavors in empirical multimodal research, representing a vibrant international and interdisciplinary research community. The collated chapters identify and seek to inspire novel, mixed-method approaches to investigate meaning-making mechanisms in current communicative artifacts, designs, and contexts; while attending to their immersive, aesthetic, and ideological dimensions. Each contribution details innovative aspects of empirical multimodality research, offering insights into challenges evolving from quantitative approaches, particular corpus work, results from eye-tracking and psychological experiments, and analyses of dynamic interactive experiences. The approaches and results presented foreground the inherent multidisciplinary nature and implications of multimodality, renegotiating concepts across linguistics, media studies, (social) semiotics, game studies, and design. With this, the volume will inform both current and future developments in theory, methods, and transdisciplinary contexts and become a landmark reference for anyone interested in the empirical study of multimodality.
contents
Preface and Acknowledgements | V
PartI: Introduction
Jana Pflaeging, John A. Bateman, and Janina Wildfeuer
Empirical Multimodality Research: The State of Play | 3
PartII: ChartingPathsforEmpiricalResearch: Theoretical and Methodological Reflections
John A. Bateman
Dimensions of Materiality | 35
John A. Bateman and Tuomo Hiippala From Data to Patterns | 65
Barbara Tversky and Angela Kessell Thinking in Action | 91
Ralph Ewerth, Christian Otto, and Eric Müller-Budack
Computational Approaches for the Interpretation of Image-Text Relations | 109
PartIII: EmpiricalInroads:
Case Studies and Results
Andreas Rothenhöfer
“I can’t see why you’re laughing”: Multimodal Analysis of Emotionalized Political Debate | 141
Christian Mosbæk Johannessen, Mads Lomholt Tvede, Kristoffer Claussen Boesen, and Tuomo Hiippala
A Corpus-Based Approach to Color, Shape, and Typography in Logos | 159
VIII | Contents Hartmut Stöckl
Pixel Surgery and the Doctored Image | 187 Jiaping Kang and Zhanhao Jiang
Multimodal Discourse Analysis Based on the GeM Model | 211 Loli Kim and Jieun Kiaer
Conventions in How Korean Films Mean | 237 Dušan Stamenković and Janina Wildfeuer
An Empirical Multimodal Approach to Open-World Video Games | 259 List of Contributors | 281
Index | 285