Sounding Media: Designers and Users

Sounding Media: Designers and Users

2014/2015

MA Media Culture

Faculty of Arts and Social Sciences

Maastricht University

Coordinator & main teacher

  • Dr. Alexandra Supper
Intended learning outcomes (more on programme level)

The course deepens the understanding of the field of sound studies already acquired in the module Sound Technologies and Cultural Practices. Where the Sound Technologies module focused on musical practices, the focus of this course will be sounds of a non-musical nature.

The course also picks up the questions of the shifting boundary between real and virtual, first addressed in the course on Real Virtualities, by exploring the working of sounds in, for instance, video games and science-fiction movies.

By investigating how non-musical sounds are involved in new media practices – such as Web 2.0 – the course explores a new angle for thinking about the same developments that were dealt with in module 1, Transformations in Media Culture. The course also offers a new perspective on the mp3 – one of the central technologies for the developments discussed in the Sharing Cultures modules – by scrutinising the assumptions about the listeners that are built into this media format.

Methodologically, the course builds upon previous experiences with qualitative empirical research, especially ethnographic approaches which were first introduced in the modules Sharing Cultures and Sound Technologies and Cultural Practices.

Learning objectives (course specific)

In this module, you will:

  • deepen your understanding of the interdisciplinary field of sound studies;
  • acquire insights in the role of non-musical sound in media experience, use and design;
  • learn to evaluate the quality of sound designs from a user perspective;
  • learn to reflect on the sonic conventions and assumptions about users that are built into these sound designs
Objective statement (course description)

A beep alerts you to the fact that you have received a new e-mail. The rattling sounds of a machine gun draw you into the experience of a first-person shooter game. An audio guide helps you make sense of the objects in a museum.

These are just a few examples of how sound is involved in our experience of media. Very often, we take sounds such as these for granted – we may not even properly notice that they are there, let alone reflect on how they inform and shape our experiences. For a long time, scholars in fields such as philosophy and media studies have focused on understanding our world as a “visual culture,” pointing to the dominance of the sense of sight in our societies and to the many visual metaphors that are deeply engrained in our language. In the last few decades, however, this has started to change – a growing number of scholars have called into question the dominance of vision, and have requested more scholarly attention for practices of hearing and listening.

The increasing attention paid to sound is not just a scholarly matter, however. On a more practical level, a growing industry is dedicated to the design of sounds, not just for video games and action movies, but also for websites, consumer appliances and museum displays. Sounds are everywhere in our mediated surroundings – and even where there is (almost) silence, more often than not, that silence was deliberately created rather than the result of an inadvertent absence.

In this course, we pay close attention to the non-musical sounds that surround us in our everyday environments. We consider, in particular, the technologies and media that shape, and are shaped by, our aural experiences and expectations. In doing so, we want to focus on practices of sound design and media use. In particular, we want to explore how design and use intersect. How, for instance, are particular imaginations of the listener (as attentive or distracted, as hearing or deaf, etc.) inscribed into certain technologies and media? And what kind of sonic conventions and icons do designers make use of to communicate meaning through sound?

Type of course :

Skills course

Target group :

Master’s students

Pedagogical approach:

Problem-based Learning

Activities

We will begin our explorations by attending to the different ways and contexts in which we listen to non-musical sounds. As we will find in the first assignment, different professional groups have developed their own modes of listening to sounds that are relevant to them. In the second assignment, we turn our attention to a professional group that design sounds for other users to listen to – sounds used in films, games, ads or consumer products. As we will see, sound designers have developed particular conventions and procedures for the design of sounds, but also for the testing and evaluation of them. In the third assignment, we will zoom further into these sound design practices by exploring some of the sonic conventions that sound designers use to create links between sounds and meanings. In the fourth assignment, we will turn to studying how sound is related to issues of accessibility and participation: for instance, how sound can break down, but also create, barriers of access for users with visual or hearing impairments. Indeed, technologies and media often already have particular imaginations of the user inscribed in them – not just whether they are hearing or deaf, or seeing or blind, but also whether they are individuals or collectives, or whether they are technologically competent or inept.

It is this issue – the assumptions that designers make about the eventual users of technologies – that we will turn to in the fifth assignment. Finally, in the sixth assignment, we will turn to the boundary between noise and silence, and discuss how sound designers work not just on creating sound, but also silences.

Lectures and Skills Trainings

Week 1:
1. Introductory lecture by course coordinator – thematic introduction,

practicalities.
2. Skills Training 1: Discussion of methodological approaches to studying users

(based on selected readings)

Week 2:
1. Lecture by Tom Langhorst (sound designer)

  1. Skills Training 2: Semi-experimental designs / discussion of your research designs
  2. Skills Training 3: Video analysis (with Jessica Mesman)

Week 3:
1. Consultation hour

Week 4:
1. Consultation hour

2. Final Presentations

In the first skills training, we will discuss the experimental methods used by sound designers to evaluate the quality of their designs. In preparation for this meeting, attend the lecture of sound designer Tom Langhorst, and read the following texts:

  • Bruno L. Giordano, Patrick Susini and Roberto Bresin (2013), Perceptual Evaluation of Sound-Producing Objects, in: Karmen Franinović and Stefania Serafin (eds.): Sonic Interaction Design (Cambridge: The MIT Press), pp. 151-154. (D)
  • Guillaume Lemaitre, Olivier Houix, Yon Visell, Karmen Franinović, Nicolas Misdariis and Patrick Susini (2009), “Toward the design and evaluation of continuous sound in tangible interfaces: The Spinotron”, in: International Journal of Human-Computer Studies 67:11, pp. 976-977 & 980-993. (Skip sections 1 and 2!) (E)
  • Christian Heath and Dirk Vom Lehn (2008), “Configuring ‘Interactivity’: Enhancing Engagement in Science Centres and Museums”, in: Social Studies of Science 38:1, pp. 63-91. (E)

The first two texts give an idea of the kinds of methods that sound designers often use to evaluate the quality of sound designs in their laboratories. Compare these methods to the qualitative, ethnographic methods that you have encountered in the previous module. The third text is an example of a qualitative study, using video analysis, about a museum display (although not focusing on sound). What are the benefits of these different methodological approaches? What different kinds of research questions can be addressed with the different approaches? We will discuss these questions during the skill trainings, but come prepared by thinking in advance about how these laboratory methods of sound designers compare to the field research of ethnographers.

In the second skills training, we will consider how it is possible to combine elements of qualitative and experimental research. Before the tutorial, read the following text:

  • Lina Dib, Daniela Petrelli and Steve Whittaker (2010), Sonic Souvenirs: Exploring the Paradoxes of Recorded Sound for Family Remembering, In: ACM International Conference on Computer Supported Cooperative Work, Savannah, Georgia, USA, 6-10 February 2010, pp. 391-400. Available online: http://shura.shu.ac.uk/2908/1/CSCW2010- sonic_souvenirs.pdf (E)

Think about how the authors combine experimental and ethnographic elements in their research, and about how you can fruitfully combine them in your own research project. Before this second skills training, prepare a research design for your own project. The first half of the training will be dedicated to discussing the combination of experimental and ethnographic research more generally, while the second half will be devoted to discussing your own research designs.

In the third skills training, we will be acquainted with the method of video analysis.

In addition to the meetings of the skills training, you will keep a research diary, detailing the progress of your project, which you post on the blog of one your team- members.

In week 1, your research diary should mention the names of your teammates, the topic of your research project, and a few sentences explaining the assumptions or interests that led you to pick this subject. This research diary is due on Wednesday at noon.

In week 2, your research diary should describe your research design and strategy. This research diary is due on Monday at 9am.

In week 3, your research diary should describe first results of your research and some ideas about the interpretation of these results. If applicable, you can also describe refinements of the research design posted in the previous week. This research diary is due on Friday at noon.

Assessment of learning:

As the final assignment, you will be working in teams of 2-3 students to conduct a user study of a specific sound design and prepare a research report together. The requirements for the project are:

  • User study of the effectiveness and/or experience of the sound of

– An audio guide or museum display

– A piece of software or an app

– A household appliance (e.g. electric toothbrush, coffeemaker, vacuum cleaner, camera)

– A videogame

– An assistive technology (e.g. comparison of different screen readers) o An informative (rather than artistic) sound installation

– An interface (e.g. tom-tom, alarm system)

  • Through a qualitative, but semi-experimental study in which the users conduct a particular task in a natural setting.
  • The semi-experimental design consists of a comparison between two similar products that differ primarily in their sound design, or of the same product with different sound settings or the sound being turned on/off.
  • Observations, interviews/focus group, video analysis
  • Team work in groups of 2-3

The detailed requirements for the assignment will be explained in the introductory lecture and are recapitulated on the last page of the course book.

You will also give a short presentation of your results in the final week of the course. Although this presentation will not be part of the assessment for the final assignment, it does count towards the participation component (+/- 0.5 points) of your final grade.

Effect (witness account, evaluation of the course)
Additional biblio sources

Assignment 1 – Listening to non-musical sounds

  • Alexandra Supper & Karin Bijsterveld (under review), Sounds Convincing: Modes of Listening and Sonic Skills in Knowledge Making. (D)
  • Barry Truax (2001 [1984]), Acoustic Communication (Second ed., Westport: Greenwood), pp. 19- 27. (D)

And read two of the three following cases:

  • Tom Rice (2012), Sounding Bodies: Medical Students and the Acquisition of Stethoscopic Perspectives, in: T. Pinch and K. Bijsterveld (eds.): The Oxford Handbook of Sound Studies (Oxford, New York: Oxford University Press), pp. 298-319. (LRC)
  • Cyrus Mody (2005), “The Sounds of Science: Listening to Laboratory Practice”, in: Science, Technology & Human Values 30 (2): pp 175-198. (E)
  • Stefan Krebs (2012), “Sobbing, Whining, Rumbling” – Listening to Automobiles as Social Practice, in: T. Pinch and K. Bijsterveld (eds.): The Oxford Handbook of Sound Studies (Oxford, New York: Oxford University Press), pp. 79-101. (LRC)

Assignment 2 – Sounds just right: sound design practices

  • Karin Bijsterveld, Eefje Cleophas, Stefan Krebs & Gijs Mom (2014), “‘Selling Sound: Sensory Marketing in the Automotive Industry”, in Sound & Safe: A History of Listening Behind the Wheel (Oxford: Oxford University Press), pp. 140-168. (LRC)

And read about two of the three following domains:

  1. About Product Sounds and Sonic Branding:
  • Daniel M. Jackson (2003), Sonic Branding: An Introduction (New York: Palgrave MacMillan), pp. 1- 10. (D) and
  • Elif Özcan & René Van Egmond (2008), Product Sound Design: An Inter-Disciplinary Approach? In: Undisciplined! Design Research Society Conference, Sheffield Hallam University, Sheffield, UK, 16-19 July 2008. Available online: http://shura.shu.ac.uk/531/
  1. About Film Sound:
  • William Whittington (2007), Sound Design & Science Fiction (Austin: The University of Texas Press), pp. 14-37, 93-114. (LRC)
  1. About Game Sound:
  • Karen Collins (2008), Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design (Cambridge: The MIT Press), pp. 85-106. (D) and
  • Mark Grimshaw (2012), Sound and Player Immersion in Digital Games, in: T. Pinch and K. Bijsterveld (eds.): The Oxford Handbook of Sound Studies (Oxford/New York: Oxford University Press), pp. 347–366. (LRC)

Online Resources:

Assignment 3 – Auditory icons and earcons: the meaning of non-musical sounds

  • R. Murray Schafer (1994 [1977]) The Acoustic Designer, in: The Soundscape: Our Sonic Environment and the Tuning of the World (Rochester: Destiny Books), pp. 237-245. (LRC)
  • David Oswald (2012), “Non-speech Audio-semiotics: A Review and Revision of Auditory Icon and Earcon Theory”, in: Proceedings of the 18th International Conference on Auditory Display, Atlanta, pp. 36-43, available at: http://www.david- oswald.de/downloads/Oswald_ICAD2012.pdf (E)
  • Paul Robare & Jodi Forlizzi (2009), “Sound in Computing: A short History”, in: interactions 16:1, pp. 62-65. Available online: http://dl.acm.org/citation.cfm?id=1456218 (E)
  • Axel Stockburger (2003), “The Game Environment from an Auditive Perspective”, presentation at Level Up, Digital Games Research Conference, Utrecht. Available online: http://www.stockburger.at/files/2010/04/gameinvironment_stockburger1.pdf (E)

German speaking students should read (instead of Schafer)

  • Georg Spehr (2008), Funktionale Klänge: Mehr als ein Ping, in: H. Schulze (ed.), Sound Studies: Traditionen, Methoden, Desiderate (Bielefeld: transcript), pp. 185-208. (D)

Optional further reading:

  • Ute Jekosch (2005), Assigning Meaning to Sounds – Semiotics in the Context of Product Sound Design, in: J. Blauert (ed.), Communication Acoustics (Berlin, Heidelberg: Springer), pp. 193-221. (D)
  • Stephen Brewster (2008), Nonspeech Auditory Output, in: A. Sears and J. Jacko (eds.), The Human- Computer Interaction Handbook (New York, London: LEA), pp. 247-264. (D)

Assignment 4 – Sound and disability: from assistive technologies to universal design

  • Stuart Blume (2010), The Artificial Ear: Cochlear Implants and the Culture of Deafness (New Brunswick: Rutgers University Press), pp. 1-8, 58-84. (LRC)
  • Kate Ellis & Mike Kent (2011), Disability and New Media (New York: Routledge), esp. pp. 1-8 and 29-43. (E)
  • Mara Mills (2010), “Deaf Jam. From Inscription to Reproduction to Information”, in: Social Text, 28 (1), pp. 35-58. (E)
  • Graham Pullin (2009), Design Meets Disability (Cambridge: MIT Press), especially pp. 1-3 and 87- 109. (LRC)

Optional further reading:

  • Michele Friedner & Stefan Helmreich (2012) “Sound Studies Meets Deaf Studies”, in: Senses & Society 7 (1), pp. 72-86. (E)
  • Jonathan Sterne (2001) “A Machine to Hear for Them: On the Very Possibility of Sound’s Reproduction”, in: Cultural Studies 15 (2), pp. 259-294. (E)

Online Resources:

Assignment 5 – Imagining the listener

  • Jonathan Sterne (2012), MP3: The Meaning of a Format (Durham: Duke University Press). (LRC)

Optional further reading:

  • Nelly Oudshoorn & Trevor Pinch (eds.) (2003), How Users Matter: The Co-construction of Users and Technology (Cambridge: MIT Press). (UL)

Assignment 6 – Noise & Silence

  • R. Murray Schafer (1994 [1977]), The Industrial Revolution, in: The Soundscape: Our Sonic Environment and the Tuning of the World (Rochester: Destiny Books), pp. 71-87. (LRC)
  • Jonathan Sterne & Natalie Zemon Davis “Quebec’s Manifs Casseroles are a call to Order.” The Globe & Mail, May 31, 2012. Available online: http://www.theglobeandmail.com/globe- debate/quebecs-manifs-casseroles-are-a-call-for-order/article4217621/

And one of the following texts:

  • Emily Thompson (2002), Acoustical Materials and Modern Architecture, in: The Soundscape of Modernity: Architectural Acoustics and the Culture of Listening in America, 1900-1933 (Cambridge: MIT Press), pp.169-229. (LRC)
  • Karin Bijsterveld, Eefje Cleophas, Stefan Krebs & Gijs Mom (2014), “It Shuts with a Comforting Sound”: Closing the Car Body, in: Sound & Safe: A History of Listening Behind the Wheel (Oxford: Oxford University Press), pp. 21-72. (LRC)
  • Mack Hagood (2011), “Quiet Comfort: Noise, Otherness, and the Mobile Production of Personal Space”, in: American Quarterly, 63(3), 573-589. (E)

Optional further reading:

  • Karin Bijsterveld (2008). Mechanical Sound: Technology, Culture, and Public Problems of Noise in the Twentieth Century. Cambridge: MIT Press. (UL)
  • Shannon Mattern (2007). “Resonant Texts: Sounds of the American Public Library”, in: Senses & Society 2(3), 277-302. (E)
  • E.P. Thompson (1992). “Rough Music Reconsidered”, in: Folklore 103 (1), pp. 3-26. (E)

 

Posted in ,