RUI Establishing bidirectional principles of motor learning and rehabilitation in virtual reality

In this RUI proposal, we seek to expand the knowledge of the bidirectional relationship between users and virtual reality (VR) systems to maximize learning of motor skills via two feasibility studies. Through understanding the user experience while learning in VR, we can develop more targeted training practices that fully leverage both human and VR adaptation. In the physical rehabilitation domain, VR provides unique avenues for patients to practice in an accessible, affordable, and engaging manner. Recent innovations in quality, availability, and affordability of VR systems have made it possible for these experiences to be more immersive than ever, and the current generation of systems is being used to train rehabilitative skills related to balance, gait, and fine motor skills required for daily living activities. Despite the growing adoption of this platform for rehabilitative care, however, there are still many unknowns related to its application, including how users interact with VR on a sensorimotor level, how virtual environments (VEs) can adapt to meet the training needs of users and users in turn undergo reciprocal changes in performance, and how to best design VEs to maximize real-world performance. With the exponential growth in the adoption of VR technology, it is important to keep pace with our scientific understanding of how people interact with these systems. Two VR design factors that a developer can manipulate are physical fidelity and cognitive fidelity, which are influential for motor learning and virtual-to-real-world transfer. Physical fidelity pertains to how well a VE visually, proprioceptively, and haptically mirrors the real world, while cognitive fidelity relates to how well a VE induces psychological demands similar to those experienced in the real world. However, there has yet to be a study that systematically evaluates how these two fidelity domains affect user experience (e.g., user?s sense of presence) and downstream perceptual and biomechanical effects. Compounding these relationships, the VR experience may differ between users, and thus it is crucial to also understand how a VE can adapt to best meet their learning needs. Given these gaps in knowledge, the goal of this study is to determine the feasibility of adaptive VEs with differing design features for driving real-world transfer of rehabilitative skills. We propose systematic investigations of two tasks generalizable to rehabilitation cases: 1) a locomotor obstacle negotiation task and 2) a custom-designed forearm and hand prosthesis control task. For both tasks, we will implement eight-week training programs, allowing us to examine how user-VR interactions are shaped through repeated exposure. In addition to comparing VEs with varying physical and cognitive fidelities, we will compare adaptive and static VR platforms to evaluate the bidirectional relationship between the user and the VR during rehabilitation. We will use this training program to evaluate the degree to which adaptive VE design leads to real-world outcomes with the following objectives: Objective 1. Evaluate the impact of varying physical and cognitive fidelity in an adaptive virtual environment on motor learning and real-world skill transfer of a novel locomotor task. In Study 1, we will train participants on a task where they practice walking quickly through an unpredictable environment with obstacles in their path, either in VR or the real world. For those in VR, we will systematically manipulate the VEs so that people train with differing levels of physical and cognitive fidelity. We will measure gaze and biomechanical gait strategies, subjective user experiences, and task performance on a cognitive task during real-world baseline, retention, and transfer tasks. Objective 2. Evaluate the impact of varying physical and cognitive fidelity in an adaptive virtual environment on motor learning and real-world skill transfer of a novel forearm and hand prosthesis control task. In Study 2, participants will learn to operate a custom-designed forearm and hand prosthesis with a foot controller during an object sorting task in VR or the real world. Similar to Objective 1, users will train in adaptive VEs with varying levels of physical and cognitive fidelity while we quantify gaze and biomechanical reaching and grasping strategies, as well as performance on a cognitive task, while measuring subjective user experience during real-world baseline, retention, and transfer tasks.