What is it about?
This paper presents a new game mechanic driven by artificial intelligence to visually assist users in their movements through the Unity Game Engine, Unity MI-Agents, and the HTC Vive Head-Mounted Display. We discuss how deep reinforcement learning through Proximal Policy Optimization and Generative Adversarial Imitation Learning can be applied to complete physical exercises from the same immersive virtual reality game. We examine our mechanics with four users through protecting a virtual butterfly with an agent that visually helps users as a cooperative “ghost arm” and an independent competitor.
Featured Image
Photo by Liam Charmer on Unsplash
Why is it important?
What if we could train a virtual robot arm to guide us through physical exercises, compete with us, and test out various double-jointed movements? Our results suggest that deep learning agents are effective at learning game exercises and may provide unique insights for users. Immersive Virtual Reality applied to exercise games has a unique potential to both guide and motivate users in performing physical exercise. Advances in modern machine learning open up new opportunities for more significant intelligence in such games.
Perspectives
Our long term goal is to develop an at-home recovery game that uses machine learning to adapt exercise difficulty and assistance. Subsequently, we plan to explore more machine learning algorithms and input parameters such as biofeedback and musculoskeletal simulation to inform of gameplay progression.The incorporation of predictive runtime models to identify muscle weaknesses may further aid in custom movements for an individual user to help maximize their exercise by ensuring the targeted muscles are being used for a given movement. To this end, there are more butterflies to learn from as we continue working towards achieving greater physical intelligence.
Aviv Elor
University of California, Santa Cruz
Read the Original
This page is a summary of: Deep Reinforcement Learning in Immersive Virtual Reality Exergame for Agent Movement Guidance, August 2020, Institute of Electrical & Electronics Engineers (IEEE),
DOI: 10.1109/segah49190.2020.9201901.
You can read the full text:
Resources
Contributors
The following have contributed to this page
