Action in Cyprus

Then almost immediately after returning from California I had to go to Cyprus (Limassol) for a conference called ‘Virtual Reality Sofware and Technology’ (VRST 2006). What happened was that it turned out that I was one of the Chairs of this conference – apparently I had agreed to this but forgot. Then I thought – well if I’m Chair I will have to go, so we submitted a short paper.

The paper was about the impact of different types of rendering quality on ‘presence’ in virtual reality. What do I mean by rendering quality? – well in computer graphics you might display the scene just with lines (called ‘wire frame’) or with flat shaded polygons, or with textured polygons … or ultimately with objects that appear to be shaded correctly according to the light distribution in the scene. The latter is very hard to do – and until recently was impossible to do in real-time. But we had a way to do a limited version of this ‘global illumination’ – based on ray tracing - and the question was: what difference would it make?

You might think that the answer is obvious – the more realistic that a scene looks the more likely it is that people will feel themselves to ‘be there’. But actually the evidence to date doesn’t support that idea – there have been experiments that suggest that it makes no difference (google Zimmons rendering quality). The difference with our research was that we not only had a level of global illumination, but one where the shadows and reflections actually move. We had people looking down over a virtual pit – and they would see the shadows and reflections of their virtual body move in correspondence with their real body (well, to some extent, since we only can track their head and one hand). Although this experiment was completed a few months ago, to date I’ve only had time to analyse the questionnaire data – there are many questionnaires that attempt to measure presence and we used one of these. I call the type of presence that you can get from the use of a questionnaire ‘reported presence’. So our study showed that ‘reported presence’ seemed to be greater on average for those people in our experiment who experienced the pit room with the ray tracing rendering method (which has shadows and reflections) compared to those who experienced a method of rendering that did not include second order lighting effects (i.e., shadows and reflections). It seemed to make a difference. But is it only because there is dynamic feedback, or is it because it simply looks more realistic, or both of these? We have to wait for the next experiment.

But if there’s ‘reported presence’ is there also … unreported presence? Why ‘reported presence’ and not just ‘presence’? Well actually there is ‘unreported presence’! For example, more than 10 years ago when I was at Queen Mary College and we had one of the first types of Head-Mounted Display we had some London firefighters in for a demonstration. The environment we showed them was the pit room. There were two of them, and when they noticed the virtual pit, each of them visibly shook, and stepped back – they had a very strong behavioural response to what they were seeing. But … when we asked them afterwards about their experience (informal ‘reported presence’) in particular whether they had any sense of ‘being there’ the replies were … ‘Oh no mate, felt nothing, no nothing happened’. Their words contradicted what we saw with our own eyes.
Presence as far as I’m concerned is quite uninteresting if it is limited simply to what people will tell you about their experience after the event – no matter how well ‘validated’ the questionnaire; and you can’t ask them during the event because that in itself could destroy the experience. So presence is to do with how people respond to events and correspondingly how they are able to act within a virtual reality (or even in a mixed reality) – it is their response and activity in relation to virtual sensory data (how exactly it is produced is not important – but if you want to think of it as computer generated on computer controlled displays that is fine).

Obtaining their answers to some questions after they have completed the experience is ok, and can be useful. But what is more important is how they respond at many different levels: their physiological responses (e.g., heart rate and heart rate variability, temperature, electrodermal activity – skin sweat response – even measures of brain activity such as EEG). Then there is their automatic behaviour – like do they duck automatically if a virtual object flies towards their face? Then there is deliberate behaviour – like they decide to pick something up and look at it. Then there are thoughts ‘I wonder what’s going on over there?’ and emotions…. Basically the total response of the person to the situation and unfolding events. (And this is why the ‘pit room’ provides such a useful scenario for testing people’s responses within a virtual reality – since we know the types of response that people would normally have in reality).

So in virtual (or mixed reality) we substitute (or augment) real sensory data with virtual sensory data. The question is whether people respond to this as if it were real.

And … obviously there are different levels of this ‘presence’ – since as we saw in the firefighters example, they may exhibit high ‘presence’ in one field (automatic behavioural in that case) but low ‘presence’ in another (reported presence). High presence means that they are ‘high’ across the board.