Multimodality in Higher Education Settings

What is it about?

As a field of scientific enquiry, multimodality came into its own in the 1990s. Pioneering studies by Roth (1994), Kress and van Leeuwen (1996), Lemke (1998), van Leeuwen (1999), and Martinec (2000) are examples of early research in this area that highlighted the key contribution of non-verbal communicative resources (e.g., gestures, visuals, music, bodily movement, facial expression) and their interplay to the construction of meaning during social interaction. Since then, there has been a growing interest in the study of multimodal phenomena among scholars with various disciplinary orientations and different perspectives, ranging from theory to application. Indeed, according to Jewitt (2009, p. 12), “multimodality can be understood as a theory, a perspective or field of enquiry or a methodological application”. We are concerned with pedagogical applications of multimodality in the context of English language teaching in higher education, with particular attention to both fostering and leveraging multimodal literacy among adult English language learners. Multimodal literacy is now a widely established concept that refers to the ability to successfully engage with texts that integrate different semiotic resources. It is defined by Walsh (2010) as the ability to construct meanings through “reading, viewing, understanding, responding to and producing and interacting with multimedia and digital texts” (p. 213). This definition is clearly inspired by the notion of multiliteracies, which was originally formulated by New London Group in 1996. This was a group of scholars (including Norman Fairclough, James Paul Gee, and Gunther Kress) who met to propose ways to change teaching and learning paradigms in response to changing forms of communication. They stressed the need to go beyond the traditional interpretation of literacy in terms of the capacity to read and write (Gee, 1996), and urged educators to utilize new technologies to enhance multimodal literacy (New London Group, 1996). Thus, multiliteracies can be seen as drawing from (1) New Literacies Studies that interpret literacy as a sociocultural (vs. cognitive) phenomenon, driven by changing social practices involving digital communications and new media (Gee, 1996; Street, Pahl, & Rowsell, 2011) and (2) Multimodal Studies that apply new theoretical and methodological frameworks for analysing communication which integrates modes beyond verbal language (Kress & van Leeuwen, 1996; O'Halloran & Smith, 2011). The concept of multiliteracies is now widely applied in general education and the need for educators to develop multiliteracies among learners is beyond dispute (Jewitt & Kress, 2003; Royce, 2002). Multimodal literacy has also become highly relevant for language teaching thanks to an increasing awareness that the multimodal approach can help students learn to exploit semiotic modes beyond verbal language (e.g., visual, gestural, spatial) to both understand and produce texts in the target language more effectively (O'Halloran, Tan, & Smith, 2016). It may also heighten their awareness of the target culture, particularly in relation to different styles of non-verbal communication across cultures (Busà, 2010, 2015). Of particular interest in the context of language learning is how non-verbal elements reinforce and/or add meaning to verbal expressions and thus facilitate comprehension, but also how multimodal approaches can serve to motivate learners (Shih, 2014). Over the years, there have been some studies pointing to the advantages of using techniques and resources that highlight non-verbal modes, such as images, gestures, and facial expressions, during language teaching (cf. Guichon & McLornan, 2008; Sueyoshi & Hardison, 2005), thereby supporting the important role of multiple semiotic resources in the construction of meaning.

Featured Image

Why is it important?

Shows pedagogical applications of multimodality in the context of English language teaching in higher education, with particular attention to both fostering and leveraging multimodal literacy among adult English language learners. Multimodal literacy has become highly relevant for language teaching thanks to an increasing awareness that the multimodal approach can help students learn to exploit semiotic modes beyond verbal language (e.g., visual, gestural, spatial) to both understand and produce texts in the target language more effectively (O'Halloran, Tan, & Smith, 2016). It may also heighten their awareness of the target culture, particularly in relation to different styles of non-verbal communication across cultures (Busà, 2010, 2015).