What is it about?
The paper describes a system for multimodal speech acquisition using electromagnetic articulography (EMA), 3D computer vision system and dedicated microphone array. The system is the first one where the acoustic camera was applied for speech assessement. Examples of using the system are also given.
Featured Image
Why is it important?
The system is important besause combine electromagnetic articulograph, advanced vision system composed of fast-speed video cameras and high resolution acoustic camera. This combination gives an opportunity to non-invasive speech assessment (without electromagnetic articulograph) using only video system, acoustic camera and programs for speech inversion.
Perspectives
The work on writing this article gave me better understanding of multimodal speech acquisition proces. I hope that it will help me in more effective and precise designing of speech inversion methods I deal with.
Dr Robert Wielgat
University of Applied Sciences in Tarnow
Read the Original
This page is a summary of: Multimodal speech data acquisition with the use of EMA, fast-speed video cameras and a dedicated microphone array, June 2016, Institute of Electrical & Electronics Engineers (IEEE),
DOI: 10.1109/mixdes.2016.7529777.
You can read the full text:
Resources
Multimodal speech data acquisition with the use of EMA, fast-speed video cameras and a dedicated microphone
The presentation which better explain our work described in the paper. The presentation was delivered on MIXDES 2016 at Łódź in Poland.
Polish Language Pronunciation Analysis Using 3-dimensional Articulography
This is the link to our project where you can find this and other publications on Polish speech analysis by the electromagnetic articulography.
Electromagnetic midsagittal articulometer systems for transducing speech articulatory movements
Probably the first work on the electromagnetic articulograph. Principle of operation has been not changed so far.
Trajectory mixture density networks with multiple mixtures for acoustic - articulatory inversion
One of the examples where EMA recordings are supported by audio recorder. A method for acoustic to articulatory speech inversion is presented in the paper as well.
An Audiovisual Talking Head for Augmented Speech Generation: Models and Animations Based on a Real Speaker’s Articulatory Data
The article describes 3D model of talking head obtained from MRI and video images. In order to animate talking head, positions of EMA sensors were used.
Contributors
The following have contributed to this page
