In the Presence of a Rat

BEAMING

The TV series Star Trek originally popularised the
idea of instantaneous transportation of people to distant places. This involved decomposition of human bodies and objects into their atomic
constituents, and then reconstruction at the remote location. Clearly such technology is perhaps centuries away even if it could ever be
realised. Today there is an alternative technology that aims to realise the same idea but through quite different means. This is the idea of ‘beaming’ as digitally transporting a representation of yourself to a distant place, where you can interact with the people there, as if you were there. It is achieved through a combination of virtual reality and teleoperator systems. The visitor to the remote place (the destination) is represented there ideally
by a physical robot. Such rapid transportation to distant locations, where you have the strong feeling to be there, and where the local people in the destination experience you as there has many economic and practical advantages. It is a step beyond approaches such as video conferencing which do not give participants the physical sensation of being in the same shared space, and certainly not the physical capability to actually carry out actions in that space.

The European BEAMING project has achieved early examples of this goal, in relation to humans beaming to distant places and interacting with people there. In one example, Dr Mavi Sanchez-Vives in our event Lab in Barcelona, digitally beamed to London to be interviewed by Rory Cellan-Jones, BBC Technology Correspondent. Or Josep Corbella of La Vanguardia in Catalunya beamed to London to carry out an interview with me about the BEAMING project. Moreover in the framework of this project a system for remote medical assistance for patients at home that can be “transported” to the hospital is also being developed. 

Alien Worlds

In our new paper  published in PLoS ONE the beaming idea is extended by showing how it is possible to beam to even what might be considered as ‘alien worlds’. We show how a person can be ‘beamed’ into a rat open arena, where the person interacts with the rat as if it were another person, and the rat interacts with a rat-sized robotic representation of the distant person. This not only shows the range of this technology, but also provides a new tool for scientists, explorers or others to visit distant and alien places without themselves being placed in any kind of danger, and importantly, to be able to see animal behaviour in a totally new way - as if it were the behaviour of humans.

A combination of several different technologies were used (1) Virtual reality places people in an alternate computer-generated world where they can look and move around, carry out tasks - all at life-size and perceived in stereo 3D. People generally have the illusion of presence - that is of being in the computer-generated space as if it were a real space. Thus the human participants in the system were in a virtual reality lab at the event Lab which is at the Mundet campus of the University of Barcelona. The rat was located about 12 km away in an animal care facility in Bellvitge. (2) Tracking technology was used to track the movements of the rat in its arena, and the tracking data was transmitted over the internet to the computers running the virtual reality simulation in Mundet. This tracking information was used to control a virtual human character (an avatar) that represented the rat. Hence whenever the rat moved its avatar moved too, in a representation of the rat arena but scaled up to human size. Hence the human participant shared the virtual arena (which looked like a room with some pictures on the walls) with a humanoid avatar. (3) Teleoperation technology in conjunction with tracking was also used. The movements of the human in the virtual reality were also tracked, and this tracking data was sent to computers in Bellvitge which controlled a small robot that was located in the rat arena. So whenever the human moved in the virtual space so the robot moved in the rat space.

Interaction Between Human and Rat

So putting all this together - the rat interacted with a rat sized robot that represented the remotely located human, and the human interacted with a human sized avatar that represented the remotely located rat.

In order to make the rat interested in the robot, a small tray attached to the robot body had some jelly on it, which the rats had previously eaten in some training sessions. The humans in a small study interacted with the rat. They had to learn to entice the rat-avatar to go near some specific pictures in the virtual reality. The humans didn’t know about the jelly on the robot, but they typically learned that the rats would follow them, so they had to draw the rats out of their normal behaviours (staying in the corners and staying close to the edges) to get them to move around so that both could stand by the same picture.

Overall the study showed that that the system technically performed well and that there could be an interesting interaction between the animal and the human and remotely located beings at different scales.

At Different Scales

A very important part of this is that this system shows that it is possible for people to interact with creatures at very different scales from the human. As discussed in the paper it would be possible to set up a similar system for humans to visit worlds of vastly different scales - insects, birds, and so on. Seeing these animals in a transformed way (for example as human avatars as we did in this paper) might give insights that would otherwise not be possible. However, our fundamental reason for doing this work is to show how it is technologically possible. It remains to be explored what insights might be obtained from this type of human-'alien' interaction.

This paper has been a long time in the making. It was started as part of the PRESENCCIA project, where the idea was formed and some initial work was carried out. It was then fully realised during the BEAMING project, with contributions of technology that had been developed under other projects mentioned in the paper - in particular the avatar work developed in the ERC TRAVERSE project. 

The research was carried out by the following groups:

event Lab, University of Barcelona, Barcelona, Spain

Cortical Networks and Virtual Environments in Neuroscience, Barcelona, Spain

Fachgebiet Augmented Reality (FAR), Munich, Germany

g.tec Medical Engineering

UCL Computer Science