What is it about?

This article explores how adding sensory experiences like smell and touch to traditional multimedia (like video and audio) affects how much people enjoy the content. With the rise of digital and mobile devices, media companies are looking for new ways to engage users. The Internet of Things (IoT) allows these companies to go beyond just sound and visuals to create more immersive experiences. The study examines how combining different senses (crossmodal correspondences) influences the overall experience. Through experiments involving sight, sound, smell, and touch, the researchers found that simply adding these sensory elements to match the video content does not necessarily enhance the user experience. The paper aims to understand better how to design multimedia content that genuinely improves enjoyment by stimulating multiple senses.

Featured Image

Why is it important?

Understanding how different senses affect our experience with multimedia is important for several reasons: (1) Enhanced User Engagement: By discovering the best ways to integrate multiple senses, media creators can develop more immersive and engaging content. This can lead to a more enjoyable and memorable experience for users. (2) Innovation in Media: The research encourages innovation in how we consume media. Moving beyond traditional audio-visual content to include senses like smell and touch can open up new possibilities for entertainment, education, and virtual reality. (3) Personalized Experiences: Insights from this study can help in creating more personalized media experiences. By understanding how different people respond to various sensory combinations, content can be tailored to individual preferences, enhancing satisfaction and retention. (4) Practical Applications: Beyond entertainment, the findings can have practical applications in areas such as therapy, training, and simulation. For instance, multi-sensory experiences could improve learning outcomes or assist in rehabilitation processes. (5) Market Advantage: Companies that successfully implement multi-sensory media can gain a competitive edge. Offering unique and superior user experiences can attract and retain customers in a crowded market. (6) Technological Advancement: This research can drive the development of new technologies and devices designed to deliver multi-sensory experiences, spurring growth in related industries and advancing the field of human-computer interaction. (7) Understanding Human Perception: The study contributes to our broader understanding of human perception and how our senses work together. This knowledge can inform various fields, from psychology to product design.

Perspectives

From my perspective, this article represents an exciting frontier in the evolution of multimedia experiences. Here are a few key points that stand out: (1) Holistic Engagement: Traditional media has predominantly focused on visual and auditory stimuli. Exploring how other senses, such as smell and touch, can be integrated opens up a more holistic approach to engagement. Imagine watching a cooking show and being able to smell the food being prepared or feeling the texture of materials in a documentary about textiles. This can create a more immersive and emotionally resonant experience. (2) Challenges of Integration: The study highlights that simply adding sensory elements to match visual content doesn't automatically improve user experience. This suggests that there's a nuanced art and science to effectively integrating these senses. It challenges creators to think deeply about how different sensory inputs interact and how they can be harmonized to enhance, rather than distract from, the core content. (3) Potential for New Genres: This kind of research could give rise to entirely new genres of media. For example, therapeutic content designed to reduce stress could use calming visuals, soothing sounds, relaxing scents, and gentle tactile feedback. Educational content could become more interactive and engaging by involving multiple senses, thereby potentially improving learning outcomes. (4) Technological Development: The implementation of multi-sensory media will likely drive advancements in technology. Devices capable of emitting scents or providing tactile feedback in sync with visual and auditory content will become more sophisticated and mainstream. This could spur innovation in consumer electronics, gaming, virtual reality, and more. (5) Commercial and Social Impact: Companies that harness these insights can create standout products in a crowded market, offering unique experiences that competitors can't match. On a social level, this technology could improve accessibility for individuals with sensory impairments by providing alternative ways to engage with content. (6) Ethical Considerations: With the ability to manipulate multiple senses, there comes a responsibility to consider the ethical implications. Sensory overload, privacy concerns related to personalized content, and ensuring the well-being of users are important factors that need to be addressed as this field develops.

Dr. HDR. Frederic ANDRES, IEEE Senior Member, IEEE CertifAIEd Authorized Lead Assessor (Affective Computing)
National Institute of Informatics

Read the Original

This page is a summary of: A study on the quality of experience of crossmodal mulsemedia, September 2018, ACM (Association for Computing Machinery),
DOI: 10.1145/3281375.3281387.
You can read the full text:

Read

Contributors

The following have contributed to this page