AAU logo

Multimodal (inter)action analysis of video for the study of human actions & practices

The Course

Multimodal (inter)action analysis of video for the study of human actions & practices

This PhD course explores multimodal analysis from a multimodal mediated perspective. With mediated discourse theory (Scollon, 1998, 2001) building the theoretical core, multimodal (inter)action analysis (Norris, 2004, 20111) explores the actions that people perform and links these to the practices that the actions draw upon and re-produce or change.

More specifically the course will examine the following areas:

- Video data and analyse the lower-level, higher-level and frozen actions that participants perform.

- Using the site of engagement (Scollon, 1998) as analytical tool, the actions are then further analysed as linked to practices.

- The term practice is here used in Scollon’s (2001) sense and is defined as an action with a history.

The course delves into analysing video data from a multimodal mediated (inter)action analytical point of view. The students will learn the basic units of analysis, the mediated action (Wertsch, 1998; Scollon, 1998), the lower-level, higher-level and frozen mediated action (Norris (2004), the site of engagement (Scollon, 2001; Norris, 2014), and practice (Scollon, 2001).

With this theoretical and analytical background, the students will then be lead to conduct their own analysis either using their own data or using data made available by VILA.

The workshop will form a pre-conference activity of the annual Association of Visual Pedagogy Conference and will be supported through resources provided through DIGHUMLAB.


Sigrid Norris, Auckland University of Technology, AUT, Auckland, New Zealand



General set up: Students will be asked to bring 10 minutes of their own raw data if they have data collected or to use raw data provided by VILA;

10:00-11:00 Welcome and presentation by Prof Norris outlining some methodological concepts and the units of analysis that the students will work with

11:00-12:00 Students will begin working with data (in small groups) – conducting a general analysis of the data will be led by Prof Norris

12:00-12:45 Lunch break

12:45-14:15 Students will continue to work with the same data (in small groups) – conducting a more detailed analysis of the data will be led by Prof Norris

14:15-15.15 Students will present their work to the entire group

15.15-16.00 Discussion and wrap up

16.00 Finish



Norris, Sigrid. 2017. Scales of action: An example of driving & car talk in Germany and North America. Text & Talk. 37(1): 117-139.

Norris, Sigrid. 2016. Concepts in multimodal discourse analysis with examples from video conferencing. Yearbook of the Poznań Linguistic Meeting, 2 (1): 141–165, ISSN (Online) 2449-7525, DOI: 10.1515/yplm-2016-0007, September 2016. de Gruyter Open.

Norris, Sigrid. 2014. The impact of literacy based schooling on learning a creative practice: Modal configurations, practices and discourses. Multimodal Communication 3(2).

Pirini, Jesse. 2014. Introduction to multimodal (inter)action analysis. Interactions, Images and Texts: A Reader in Multimodality. Berlin, Boston: de Gruyter Mouton.

Scollon, Ron. 1997. Handbills, Tissues, and Condoms: A Site of Engagement for the Construction of Identity in Public Discourse. Journal of Sociolinguistics. 1(1): 39-61.

Suggested Further Reading

Norris, Sigrid. 2004. Analyzing Multimodal Interaction:  A Methodological Framework.  London: Routledge.

Scollon, Ron. 1998. Mediated Discourse as Social Interaction. London: Longman.

Scollon, Ron 2001. Mediated DisocurseL The Nexus of Practice. London: Routledge.

Wertsch, James V. 1998. Voices of the Mind: A Sociocultural Approach to Mediated Action. Cambridge, MA: Harvard University Press.

ECTS: 1 – active participation and analysis of data on the day


16 June 2017


Rendsburggade 14, 9000 Aalborg, Room TBA

Deadline is 1st of June.

The Course is free of charge. You can cancel your registration until 1st of June. Cancelation from 2nd of June will be charges with a fee of DKR 500,-


Program Director Lektor Lars Birch Andreasen
Phone 9940 2361

PhD secretary Jeanette Arboe
Phone 9940 3062