Parallel Adaptation: Switching between Two Virtual Bodies with Different Perspectives Enables Dual Motor Adaptation
Adrien Verhulst, Yasuko Namikawa, and Shunichi Kasahara
IEEE ISMAR 2022
Abstract
Virtual Reality (VR) lets us experiment embodiment. Here we investigate dual embodiment under the prism of dual motor adaptation. We asked participants (N=21) to perform reaching motions in VR with opposite gradual visuomotor perturbations. The participants sequentially switched between 2 VR bodies and had to adapt to the VR body’s perturbation (up to + 15deg for the VR body A, and – 15deg for the VR body B). We then designed a 2×2 within-subject study: 1 factor being the perspective (1st person or 3rd person), and 1 factor being the head rotation (without head rotation before the reaching motion or with head rotation before the reaching motion). We found that by providing strong visual cues between bodies (alternating symmetric perspective and/or symmetric head rotation), participants had little awareness of the perturbations, good adaptation, and large aftereffects in both VR bodies. Those elements are consistent with implicit dual adaptation. In contrast, a naive 1st person’s perspective resulted in little to no adaptation with a high cognitive load and no aftereffects.
Can one person adapt to two bodies at once?
In VR, one person can embody two bodies at the same time, a state called dual or parallel embodiment. Each body may call for different movements: in a taller body, for example, we have to reach lower. Embodying two bodies would then mean keeping two internal models for the same movement, one for each body, and switching to the one for the body in control. Whether people can keep two internal models for the same movement remains controversial. Learning two opposite perturbations in quick succession is known to be hard because they interfere, and when people do manage it, they tend to rely on a conscious strategy that is cognitively demanding.
We studied the question through motor adaptation. If people adapt to each body’s perturbation implicitly, without a conscious strategy, the adaptation is tied to the body they control, which brings us closer to evidence of dual embodiment. We asked whether cues that differ between the two bodies could make this possible.
Reaching with two virtual bodies
Twenty-one participants made reaching movements in VR, sliding a controller across a real table from a start point to a target. They controlled two avatars, each in its own virtual room, and on every trial there was about a 50% chance of switching to the other body. Over the trials, each body’s virtual hand was gradually rotated away from the direction of the real hand, up to +15° for one body and −15° for the other, so adapting to one body meant moving the other way in the other. When the rotation is finally removed, an error in the opposite direction (an aftereffect) shows that the movement itself was updated, one of the signs of implicit adaptation.
A pre-study with a single body first confirmed that this adaptation is implicit in VR. The main study then varied two cues between the bodies in a 2×2 design: the perspective (first person, or a third-person view over the shoulder, mirrored between the two bodies) and the head rotation (whether the participant had to turn their head, in opposite directions for the two bodies, to see the target).
Different viewpoints let both bodies adapt implicitly
When the two bodies were seen from different viewpoints, participants adapted to both opposite rotations implicitly, with little awareness of them and clear aftereffects in each body; with the same natural first-person view in both bodies, they barely adapted. At the end of the full rotation, reaching in the first-person, fixed-view condition was still off by about 10 to 11° of the 15°, against about 4 to 6° in the other three conditions. When the rotation was removed, those conditions showed aftereffects in the opposite direction, about 3 to 6° on average over the first ten trials, while the first-person, fixed-view condition showed almost none. With the mirrored third-person views, the error then decayed gradually back to baseline, following an exponential curve as implicit adaptation does.
The first-person condition also cost more than accuracy. The rotation raised the workload more in 1pp-fix than in any other condition and lowered both body ownership and agency, whereas in 1pp-move and 3pp-fix it raised body ownership.
Why the natural view failed with two bodies
With a single body, the same first-person view had led to implicit adaptation in the pre-study: the error grew more slowly than the rotation, only 7 of the 21 participants reported noticing the perturbation, and the workload stayed low.
The paper explains the reversal by how easily the errors can be noticed. With two bodies seen from the same natural view, the participant can compare the reaching of the two bodies, whose rotations differ by 30°, and so detects the error and responds with a conscious strategy. When each body is seen from a shifted, mirrored viewpoint, comparing the two would take a mental rotation, so the errors drive adaptation without being noticed.
Hiding the error, within limits
A stronger cue is not always better. Turning the head made the task harder: 3pp-move required more workload than 3pp-fix (NASA-TLX 10.7 against 8.18) and lower body ownership than the first-person conditions, and in 1pp-move one body fitted the gradual decay of implicit adaptation better than the other, a mix of implicit and explicit adaptation. The paper therefore suggests limiting how well people can perceive the errors, for example by shifting the viewpoint, but only within what the task allows; otherwise the difficulty of the task outweighs the advantage of implicit adaptation.
Looking ahead, the paper aims to show that implicit adaptation updates two internal models, one for each body, while explicit adaptation updates only one, and to use the ability to update two internal models as a measure of dual embodiment.
Reference
Adrien Verhulst, Yasuko Namikawa, and Shunichi Kasahara. 2022. Parallel Adaptation: Switching between Two Virtual Bodies with Different Perspectives Enables Dual Motor Adaptation. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 169–177. https://doi.org/10.1109/ISMAR55827.2022.00031
BibTeX
@inproceedings{10.1109/ISMAR55827.2022.00031,
author = {Verhulst, Adrien and Namikawa, Yasuko and Kasahara, Shunichi},
title = {Parallel Adaptation: Switching between Two Virtual Bodies with Different Perspectives Enables Dual Motor Adaptation},
booktitle = {2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
year = {2022},
pages = {169--177},
publisher = {IEEE},
doi = {10.1109/ISMAR55827.2022.00031}
}Figures from the paper (Verhulst et al., IEEE ISMAR 2022). © 2022 IEEE.