Presence at SIGGRAPH 2010

One of the problems with ‘presence’ has always been that of measurement – since ‘place illusion’ (the sensation of being in the place depicted by the VE displays) and ‘plausibility’ (the sensation that what is happening is really happening) are both qualia, they are feelings that cannot be directly measured.

Some years ago I put forward an analogy with colour science - in the now defunct online journal Presence Connect – see [1].

For example, the sensation of seeing the colour ‘red’ is also a qualia, much beloved of scientists and philosophers who study consciousness. In colour science there is a physical function, the wavelength distribution, that describes the energy distribution of light emitted or reflected by a surface patch. Yet what we see is not simply some simple function of this physical energy distribution but a complex, and not completely understood interplay between the physics of colour and our perceptual systems. Moving on from there to the sensation of ‘red’ and our consciousness of seeing red is another far leap into the science of consciousness.

With respect to ‘presence’ the physical basis is the type of immersive system used, its properties and capabilities. This physical basis then becomes transformed into our perception and action within an alternate reality. Again how this transformation occurs is a problem for significant study.

There is a successful quantitative and predictive theory of colour science, that shows how an ‘average observer’ is likely to respond to patches that emit light with specific energy distributions. This success is partly built on the psychophysics of colour matching experiments. We applied an analogy of this idea to ‘presence’ (Place Illusion and Plausibility) and carried out an experiment to show how this idea could work. This has just been published at SIGGRAPH 2010 [2].

This shows how to create functions that predict how the ‘average participant’ would respond to particular system configurations in terms of presence. The paper provides a methodological counterpart to the earlier theoretical paper [3].

There is an associated youtube video, and a version of the presented slides is given below.

1. Slater, M., A note on presence terminology. Presence connect, 2003. 3(3).

2. Slater, M., B. Spanlang, and D. Corominas, Simulating Virtual Environments within Virtual Environments as the Basis for a Psychophysics of Presence. ACM Transactions on Graphics (SIGGRAPH) (TOG), 2010. 29(3): p. Paper: 92.

3. Slater, M., Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philos Trans R Soc Lond, 2009. 364(1535 ): p. 3549-3557.

Presence in a Another Body

We recently published a paper on how we used virtual reality to give people the illusion that their body was temporarily that of another (First Person Experience of Body Transfer in Virtual Reality). In particular we were able to show that to some extent some men can be given the (temporary!) illusion that their body is that of a girl.

The image shows an overview of the scenario. The experimental participants were located in or near the seated girl's body, via wearing a head-tracked wide field-of-view head-mounted display. In one condition of the experiment, when they looked down towards their body they would see the girl's body instead, as if their eyes were located in the same position as hers. If they turned to look to their left, they would see the body of the girl in the virtual mirror, with its head movements matching their own. When the standing woman stroked the arm of the seated girl, they would feel this on their own arm.

This paper seems to have generated a lot of interest judging from press reports and the PLoS ONE metrics.

Some of the comments about the paper are quite funny, and it seems to have awakened kinky fantasies in some men. Other comments are off the mark believing (spurred on by some journalistic takes on the article) that we were transforming men into women. Indeed one angry comment on the paper notes that to give men the experience of being female we would have to also simulate all of the social conditions that women are subject to, and also physiological processes such as menstruation, etc..

I agree very much with this. We were not giving men the experience of what it is like to be a woman! We were generating an illusion that is very hard to describe or understand unless you have experienced it. It is an illusion that the different body that you see when you look down at ‘yourself’ in virtual reality, or when you see ‘yourself’ in a virtual mirror, is somehow your body (even though you know it isn’t).

The brain seems to be quite liberal in deciding what is part of your body, and such illusions have shown that it is not difficult at all to trick the brain into believing that something is part of your body when it is not. It is important to realise that this ‘trick’ does not happen at the cognitive level, that is you never believe that the fake body or body part is really part of you. Rather it is at some lower perceptual and proprioceptive level that you don’t have much conscious control over that this happens. So it is like ‘presence’ in virtual reality - you know for sure that this is not your body, but nevertheless it feels like it is.

These illusions stem from an initial insight by Botvinick and Cohen known as the ‘rubber hand illusion’, and see also the New Scientist video with Olaf Blanke. Here people experience the illusion that a rubber hand is their hand. However, no one would then go on to say that people who experience the rubber hand illusion know what it is like to have a rubber hand! Similarly we do not claim that a male who has the illusion that his body looks like a female one knows what it is like to have female body. These are different things at quite different logical levels.

I hope you find the article interesting, and on the PLoS ONE web page you can see a video which gives some idea of how it looks to the experimental participant. However, the only way to really know how the illusion feels would be to experience it. Welcome to Barcelona.

The Illusion of Violence

If you see some people fighting in the street, how do you respond? Some people may intervene and try to stop the fight, others may try to get help, others may just watch, join in one side or the other, or do nothing but get away as quickly as possible. How people respond to violence is an important topic in social psychology, and the issue goes back to a case that happened in the US in 1964, the violent murder of a young girl while apparently 38 bystanders stood by and did nothing. There is controversy over what actually happened, about whether there really were these bystanders, but anyway a whole area of research was opened up in social psychology to try to understand this ‘bystander effect’. One of the problems in studying this is that, of course, it is impossible to carry out experimental studies of how people respond to violence that they might encounter by chance in a public place. The social psychologist Dr Mark Levine of the University of Lancaster has come up with some ingenious ways to try to study this problem, and actually his view of how crowds behave is not quite so negative as the popular view that crowds are bad and that they encourage or don’t prevent violence. A few years ago Mark and I talked about the possibility of using virtual reality to study this issue, since in VR one can set up apparently contingent violent confrontations between virtual people and then study how real people respond to this.

We know from research into presence that people do, in VR, tend to respond as if situations and events were real. (On this topic, my paper published in the Proceedings of the Royal Society, mentioned in earlier posts, is freely available online until the end of February).

So if we place people in a virtual reality where there are virtual characters who start arguing and fighting with one another, then this might prove an interesting way forward for the experimental study of the topic of the bystander effect.

Actually one reason why we did the virtual reprise of the Stanley Milgram obedience experiment was that we needed to convince sceptical EPSRC project reviewers that it is the case that people do tend to respond realistically. (On this point too another paper has been published recently using brain imaging to try to understand what happens when people do experience that Milgram paradigm). The EPSRC were convinced, and Mark Levine, Prof. Jian J Zhang and I obtained project funding, with this as the major application.

We carried out a first pilot study on the issue of people's responses to violence in virtual reality which has now been published. Actually this did not portray violence but aconfrontation between two football fans that would eventually lead up to violence. The results were very encouraging, and we are now carrying out a full study.

Other News

On the topic of ‘response as if real’ there are some other recently published papers from our group that might be of interest. The first examines whether illumination realism makes any difference when people experience standing over a virtual pit. Does it help to have dynamic shadows and reflections? A pre-publication version is available online.

The second shows that when people walk across a narrow beam in virtual reality they tense their back muscles to avoid falling. This is in press with IEEE Transactions on Visualization and Computer Graphics, and is available as a pre-publication version.

The third shows that how people scan their eyes over a scene in virtual reality follows a pattern that is similar to what might occur in reality.

The fourth shows that people exhibit behaviour consistent with proxemics theory when virtual characters break into their virtual space, which is in press with ACM Transactions on Applied perception

Research Posts Available

I have several jobs available in the group in Barcelona. Take a look at www.event-lab.org over the next few days and you will see the adverts appear.