How do people understand stories during face-to-face communication?

What is it about?

In face-to-face communication, people often rely on the speaker’s gestures and mouth movements to understand their message. This study looked at whether these cues—gestures and mouth movements—improve understanding when listening to someone speak, both in clear conditions and when it's noisy. We also explored whether the setting of the conversation (i.e., clear or noisy) affects how the speaker uses these cues. Participants watched videos of a speaker telling stories and then answered questions about what they heard. In the videos, the speaker either used gestures or kept their hands still and either wore a surgical mask or had their mouth visible. In two experiments, we changed the conditions: in one, the speaker told the stories in silence while participants listened in either clear or noisy conditions (similar to watching TV at home or in a noisy café). In the other, the speaker also had background noise, making it more like a conversation in a busy place. We found that gestures helped people understand the stories, regardless of how well they could hear or whether the speaker was wearing a mask. Mouth movements were most useful when it was harder to hear. Overall, both gestures and mouth movements are important for understanding speech, but they help in different ways. These results show how important nonverbal cues are for following conversations, even in difficult listening situations.

Featured Image

Photo by Olga Serjantu on Unsplash

Photo by Olga Serjantu on Unsplash

Why is it important?

This study sheds light on how nonverbal cues, such as gestures and mouth movements, contribute to understanding spoken narratives. What makes this study novel is that it examines the combined effects of gestures and mouth movements on comprehension across different communicative environments.

Perspectives