Visual-Tactile and Visual-Motor Influences on Virtual Body Ownership

The vast amount of research on the rubber hand illusion uses visuotactile synchronous stimulation to induce the illusion. This means that sight of the rubber hand being touched is synchronous temporally and spatially with the tactile stimulation felt on the corresponding (hidden) real hand. It has also been shown that the illusion can be induced with visuomotor stimulation - meaning that (in this case) the virtual hand moves synchronously with the movements of the corresponding (hidden) real hand.

For most of our work on virtual full body ownership we have relied on visuomotor effects, where the virtual body moves synchronously with the real body. This is accomplished through real-time motion capture, so that participant movements are mapped to the corresponding movements of the virtual body. The virtual body is seen directly by looking towards it through the head-mounted display, and also in a virtual mirror.

Which of these two methods of stimulation is the most powerful in inducing the body ownership illusion - visuotactile or visuomotor? In a recent paper  (PDF) we describe an experiment that addresses this question.

Participants were in a reclined position and saw their full virtual body from a first person perspective through a head-tracked, wide field-of-view head-mounted display. As they moved their leg (A) the corresponding virtual leg would move synchronously or asynchronously (B). When the experimenter tapped a leg with the wand (C) the participant would see a virtual ball tapping the corresponding position on the virtual leg (synchronously or asynchronously). Hence we had a 2 by 2 experimental design (synchronous movement or asynchronous combined with synchronous tapping or asynchronous) where these were delivered alternately.  There were 60 participants in a between groups design - hence each group of 15 experienced just one of the 4 combinations of these two factors.

The experiment was organised so that every subject first experienced for a while the best possible setup - that is visuomotor and visuotactile synchrony. Then after some questions had been answered they experienced one of the 4 conditions. Then questions about body ownership and agency were again answered.

Based on the questionnaire responses visuomotor synchrony outweighed visuotactile in producing the illusion.

However, unusually, we also attempted to measure not only what generated the illusion but also what extinguished it. Here we used a method first proposed in the study of presence in virtual reality (the sensation of being in the place depicted by the virtual environment displays). This method is called ‘breaks in presence’. The assumption is that the normal state is for the illusion of presence to occur, but occasionally it breaks for various reasons (errors in rendering or tracking, physical entanglement with cables, bumping into a Cave screen, or just spontaneous switches in attention or perception). From an indication by participants about when each break occurs it is possible to estimate an overall probability of presence (the proportion of time the participant had this illusion). Here we adopted the same idea except that instead of presence we considered the illusion of body ownership, and participants reported when the illusion vanished.

We found that a break in body ownership could be caused equally by asynchronous visuomotor or visuotactile stimulation. Hence while synchronous visuomotor was paramount in generating the illusion, the number of breaks that occurred did not differ between visuomotor and visuotactile asynchrony.

We also recorded skin conductance and heart rate. This was in order to measure the response to a sudden event that took place. The picture above shows that participants had their real (and virtual) legs resting on a table. At the end of all the stimulation the table suddenly pulled away. Participants tended to react with an involuntary response to stop their legs from falling, and this showed up in both heart rate and skin conductance changes. Moreover these changes were positively correlated with a questions about how stressed they had felt at that moment. However, there were no differences in the physiological changes amongst the four conditions of the experiment - the event was equally arousing under all conditions. We believe that this is because seeing a virtual body that coincides spatially with your own body is already enough to produce a body ownership illusion. Additional synchronous multisensory stimulation only adds to this.

By looking at breaks in the body ownership illusion we were able to assess subjective ownership through time as well as at the end of all the stimulation. To obtain this information we used the same method as in the ‘breaks in presence’ work - that is we only asked participants to indicate when the illusion broke, and not when it started. For presence this procedure makes sense, because if we ask people to report when they become ‘present’ in the virtual place the very requirement to report this may disrupt it. Asking them to report when the illusion breaks is not the same, since, of course, the illusion has already broken. However, with hindsight it is probably possible to ask people to report when an illusion of body ownership kicks in without disrupting the illusion. Indeed we did this in an earlier paper (PDF) (for different reasons - we were interested in estimating time for the rubber hand illusion to start). Although based only on reporting breaks our statistical method can estimate the probability of being in the illusion of ownership state, in future work we will also try out the idea of asking participants to report when the illusion starts as well as when it ends.

Finally our method also includes an approach that may overcome some problems in subjective assessment of the body ownership illusion. Normally researchers ask participants in experiments under different conditions to report things like ‘How much did you have the feeling that the (virtual / manikin) body was your body?’. But participants naïve to this idea (as they should be) have no real clue what we are talking about. In everyday life we do not go around thinking "Oh my body feels like it belongs to me." As is the case with presence, the special qualia attached to a 'body ownership illusion' is to have that feeling of ownership even knowing that it is an illusion - that the virtual body is obviously not really your body. 

Now especially in control conditions (e.g., asynchronous) we are asking them to report on something that they do not know about - yet of course they will always give some answer to a questionnaire. This is especially problematic in within-group studies where we ask people to report the strength of the illusion in both an experimental (e.g. synchronous) condition and in a control (e.g., asynchronous condition). But these are not balanced in the sense that the order of the conditions does matter. Experiencing first an asynchronous condition and then a synchronous one is really very different from the other way around - since when the synchronous condition is experienced first participants know what you are talking about with respect to ‘body ownership’ and therefore can more appropriately evaluate the asynchronous condition. No amount of counter balancing can overcome this, and anyway it violates a fundamental assumption behind the statistical analysis (by ANOVA) of within-group designs - that all orders of stimuli delivery are equivalent.

Here what we did is give all participants the experience of the best setup that we could offer (within the constraints of that experimental design) to induce body ownership. Hence when later we ask questions about their responses to the experimental conditions they have already experienced the ‘best’ setup, so that they have an experience against which they can compare.

Questionnaires alone are never the best method of measurement. But they can be improved through asking people to compare their responses to different setups; but first demonstrating as a baseline the closest we can get to inducing the strongest response.

Kokkinara, E., and Slater, M. (2014). Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion. Perception 43, 43 – 58. (PDF).

YouTube Video

This research was conducted as part of the VR-HYPERSPACE project.