You Are Here Read online

  Although combining real-life events with virtual ones within a desktop environment offers many interesting opportunities for us to understand our connections to the planet, the onus would be on us to deliberately seek out such experiences and to do a considerable amount of work to relate what we experience to our own lives in any sensible way. A more promising approach would be to embed pieces of virtual life into our everyday experiences. If, on my walk to work every day, I must pass a solid sculpture in a park that renders the quantity of remaining sea ice in a tangible and direct way by varying its size and shape, then a fact of a remote environment will, over time, worm itself into my consciousness and become a part of my inner landscape. In the manner of the best in ambient computing, the sea ice indicator will not be in my face as a blaring beacon of guilt but will form a part of the deep background of my geographic experience as I walk a familiar route.

  In our Research Laboratory for Immersive Virtual Environments (RELIVE) at the University of Waterloo, we build richly detailed simulations of buildings and landscapes that rival those offered by less immersive environments like Second Life.10 The models begin with a skeleton of the positions of the main features of a virtual world, but fine details of visual texture and lighting can be added to generate compelling illusions of real spaces. Participants in our experiments view these simulations by wearing headsets that contain small screens much like those used on laptop computers.

  Specialized sensors keep track of the user’s movements as they walk around the room, allowing computers to update the images that appear on the screens so that users feel completely immersed in the simulation. Virtual reality researchers speak of an ephemeral quality called presence, by which they mean the quality of a virtual experience that leads one to believe at some level that one is immersed in a real space. Though nobody could really forget that these virtual environments are not the same as physical space, the best of such environments can produce a remarkable depth of presence. One virtual reality designer has produced a simulation of a car so realistic that he cannot persuade himself to drive this car over a virtual cliff. In our laboratory, we can generate a convincing illusion of a deep pit that suddenly appears in the floor of an ordinary-looking art gallery. Though we can coax ourselves to walk to the edge of the pit and look down, even experienced users walk gingerly, and physiological monitoring equipment betrays our rapid heart rates and sweaty palms if we force ourselves to “walk the plank” across the pit.

  So what happens when people are placed into such virtual spaces? In addition to feelings of anxiety and arousal when challenged to drive over cliffs or jump into chasms, we are surprisingly quick to accept such alternative universes as the real thing. There are many informal signs of such deep immersion. For one thing, virtual navigators soon forget that real, concrete walls lie outside the realm of their helmets, and we must guard their safety closely to prevent them from banging into the walls. In addition, after users have worn the helmet for a few minutes, removing it gives rise to a “popping” sensation as they discover and remember the real world that they had just left. It’s difficult to describe in words the relationship between the physical and virtual worlds experienced by users who are jacked in to our equipment. In a way, it is like a dream state. Some of us, when we dream, become aware that we are dreaming and so, for a fleeting time, can be simultaneously aware of our dream selves involved in a narrative and of ourselves as sleeping bodies lying in bed. Those who have “willed” themselves out of a precarious situation in a nightmare will be familiar with this sensation. Some have been able to cultivate this kind of dual awareness, or “lucid dreaming,” perhaps even finding ways to manipulate dream content as it takes place. The virtual reality experience is a little like this sensation. Though the graphic worlds we see are of very high quality, we can never forget completely that they are embedded in a real-world context.

  In many of our studies, we stand beside our volunteers and talk to them as they work in virtual space. It is not hard to move one’s sense of self back and forth between the physical room containing the experimenter’s voice and a virtual realm filled with parks, trees, buildings, and avatars, but, as yet, nobody knows what consequences might ensue from longer-term chronic exposure to highly immersive virtual settings. This is a very important question, for such technology, so far mostly confined to research centers, is clearly destined for people’s homes in a few short years.

  More careful measures of behavior in virtual environments suggest that people tend to move and linger through our digital spaces in much the same way as in real spaces, and that the rules of space syntax that work so well to predict movement through buildings and cities appear to hold up in virtual spaces as well. Most interesting, the same disregard for angle and distance that can confuse residents of real space also applies in virtual spaces, and it can be used to interesting effect. In one study conducted at a large virtual reality laboratory at Brown University in Providence, researchers produced a large virtual maze that contained what they called wormholes. These holes were virtual portals that catapulted observers from one position in the maze to another remote location.11 To get some idea of the effect, imagine walking out the front door of your house and discovering yourself back inside the house, somewhere near the back door. The simulation was designed to conceal the presence of the wormholes from the virtual travelers. As the whole space was unfamiliar to them, and one area looked much like another, this was not difficult to arrange. In navigation tests, not only did the volunteers fail to notice the wormhole effect at all but their routes through the virtual space suggested that they had little understanding of the connections between the arrangement of these holes and the real spatial relationships of different parts of the maze. When asked to find shortcuts from one location in the maze to another, the routes they chose suggested that they had been oblivious to the geometry of the room, computing location and path using topology alone.

  Much of our research at RELIVE relates to the testing of principles involved in the design of built spaces in houses, buildings, and cities. A skillful designer can produce a model of a proposed building or streetscape that can then be presented to users to measure their reactions. Not only can we ask participants whether they like what they see but we can monitor their movements to see which parts of our digital spaces attract their eyes and then their feet. All the while, sensors record the reactions of their bodily systems to what they experience. Such systems can be used powerfully to understand how human minds process space, but they can also help solve interesting practical problems. Thomas Seebohm, an architect at the University of Waterloo’s School of Architecture and a pioneer in the use of such visualization methods in the design process, used large-scale detailed simulations of entire city blocks to underpin the public consultation process in urban-planning decisions. Though drawings and tabletop scale models are of some help in letting people imagine how construction projects might transform their lived spaces, more immersive environments that allow people to walk through detailed virtual mock-ups of a new building or development project can be used to provide a much more comprehensive vision of alternative futures.

  In our laboratory, we are also designing virtual structures whose size and shape adapt over time to reflect the preferences and interest of the observer, as measured by their movements and their physiological state. This symbiotic dance of human user and virtual space should help us to define and optimize the shape of dwellings and workplaces, and might also help us to understand how different types of spaces can be adapted to individual preferences. Another local architect and artist, Philip Beesley, is one of a group of individuals with an interest in what has been called responsive architecture, in which buildings might sense the movements and perhaps even the physiology of their occupants, and adjust their properties accordingly to yield maximal comfort.12 On a simple level this could involve such things as adjusting ambient temperature to coordinate with the body temperature of an occupant, but it might also be possible to ad
just the shape and appearance of space to enshroud the user in a zone of comfort, ease, and security.

  High-powered graphics, surround sound, and compelling narratives can take an occupant of a virtual place a long way away from the real world, but what can make such digital constructions crackle with life is the presence of other beings. In another demonstration of Jane Jacobs’s “life attracts life” dictum that even she might never have imagined, one of the surest ways to boost a feeling of presence in a virtual world is to share that world with other people. This is one reason for the huge success of efforts like Second Life and World of Warcraft. What such metaverses lack in immersive realism they make up for by being social places in which one can walk up to other beings for conversation, laughter, dancing, dating, hugging, and even a romp in the bedroom if the virtual stars are just right. These opportunities for social interaction can so draw a user into a metaverse that they may sometimes feel they never want to leave it, even though their entire embodiment is invested in a few hundred colorful pixels on a small screen on a laptop. Now imagine that such shared presence can be enjoyed by users of a more powerful, more immersive experience such as the one provided by helmet-based virtual reality, in which users wear a headpiece containing a visual display and a pair of headphones that essentially “lock” them into the virtual world and exclude the actual physical world that surrounds them.

  At Waterloo’s RELIVE, along with a few other such installations, two users in different physical locations can don headsets and meet one another in a shared virtual space. Like Second Life, users meet one another’s virtual embodiments in the form of digital avatars, but these avatars can be rendered in incredible detail with realistic skin textures, clothing, and gestures. When the users wear additional sensors (such as special gloves that allow the main computers to track the positions of hand and fingers), their digital avatars can interact physically with one another. And though the technology for truly realistic bodily interactions (such as handshakes) is still very much in development, it is possible to outfit such sensors with some degree of feedback so that an occupant of a shared virtual space can not only “reach out and touch someone” but both those doing the touching and those who are touched actually feel a physical sensation on their skin that corresponds to the virtual action.

  If such technology was to be made widely available, the consequences for social interaction are simply mind boggling. For example, teleconferencing has been touted as a means of improving the efficiency of communications by allowing a group of individuals to meet in a shared auditory space. But anyone who has ever participated in a conference call knows that when the only information available consists of words, pauses, and tone of voice, it can be difficult to keep track of the thread of conversation, let alone to leverage the normal dynamics of a group experience to promote a clear set of actions. There is no better way to make one appreciate the complexities of group interactions than to reduce everyone to disembodied voices. Adding a video link is scarcely better, as the effect is really just to add to the melange of voices a set of slightly dynamic headshots of people sitting at tables. Natural movement, body language, and eye contact are entirely missing.

  Now imagine sitting in a room that exists only in the shared mind spaces of the members of the meeting. It is possible not only to see three-dimensional embodiments of other members of the group but to talk to them, watch their gestures, facial expressions, and patterns of eye movements, and make eye contact. Though there are some technical hurdles left to work out to make such meetings a common reality (the biggest one perhaps being Internet connections of sufficient bandwidth to allow the real-time transfer of data from one place to another), many of the most difficult challenges have been overcome.

  The widespread possibility of such virtual interchanges could revolutionize many business practices, and the use of immersive environments in conjunction with social networking staggers the mind. Imagine being able to go to a souped-up version of an Internet café where you could convene a meeting of friends scattered across the globe, sit in a circle, converse, share music or a movie, and perhaps even hold hands and hug. In a sense, nothing that I have just described is particularly new. People have been going on cyber-dates for years, and there is plenty of evidence that even text-based cyber-sex using an email program can be compelling enough for people to leave stable marriages to take up with new partners whom they have never met in the flesh. New, though, is that such interchanges will be possible using technology that will provide for undreamt-of levels of realism and presence.

  Now go one step further, and remember who we are and where we have come from. When we send our avatars down fiber-optic cables to meet up with avatars representing friends, business partners, political counterparts, or potential lovers, what guarantee is there that we must convey accurate representations of ourselves? Everyone knows of people who can woo with words, make love to a camera, or otherwise convey enhanced versions of themselves that are, in some way, larger than life. How much more potential for interpersonal havoc might there be when such “transformed social interactions,” as psychologists have called them, can be carried out by using technology to manipulate the features of lifelike, solid-looking avatars who might even possess warm virtual flesh and a beating virtual heart? Again, this is not a particularly new question. All our experiences are mediated by something, whether it is text, image, or just the distorting channels of our own senses, cognitions, and emotional biases. Jim Blascovich, a pioneer of the use of virtual reality to study human interactions, puts it simply. “Everything is virtual,” he says.13 But what is new is the extent to which it will be possible to recruit sophisticated technology to plant false beliefs in others using not only word and image but movement, body language, facial expression, and something approaching flesh-to-flesh contact in a virtual environment. Once such technology is more widely available, Pandora will be out of the box, and past experience suggests that any attempt at regulation will be fraught with difficulty and will probably fail. It may be that our best hope of preventing such technology from eroding the quality of genuine physical human interaction is to act now to study and understand both the scope and the limits of its virtual counterpart.

  Although equipment that can produce deeply immersive experiences of interactions with virtual partners is likely to be confined to specialized facilities for the foreseeable future, improvements in computer graphics power and speed are rapidly bringing decent-quality immersive virtual reality experiences to within the reach of consumer budgets. A baseline system for generating helmet-based digital worlds with some motion tracking can be had for about $20,000, around the same price as a high-end home theater. The much more affordable Nintendo Wii, with its wireless controller, is perhaps the thin edge of the wedge for consumer-grade virtual reality environments meant to be installed in homes. This controller gives users the ability to immerse themselves physically in a script by holding a wand whose movements through three-dimensional space are incorporated into such things as virtual tennis, golf, and combat. Like Second Life and similar ventures, we adapt so well to developments such as these because of our extremely flexible context- and view-based understanding of physical space. Our minds leap joyfully from one kind of space to another with scarcely a backward glance because they are built to absorb the shocks of a world put together one brief, disconnected glimpse at a time.

  The advent of such technologies in our homes presents both great opportunities and great risks. If a game like StarCraft can cause players to forget to feed themselves or to go for days without sleep, then what happens when we can invite Attila the Hun into our living room to engage in swordplay with a weapon whose heft we feel in our hands? At the very least, we will need to come to a fuller understanding of the elements of game play that contribute to addiction. But we will also have to take seriously the issue of how to ensure that the lines between physical and virtual reality are drawn cleanly enough to prevent social havoc.

  Understanding more a
bout how human beings, especially young ones, engage with immersive virtual worlds could help us to arrive at a rational rating system for regulating the sales of such software and educating users about the potential risks. As virtual environments become more common and compelling, we will need to address questions about the extent to which we should expect normal social mores and legal frameworks to apply in the virtual realm.

  Such questions are already being raised in Linden Lab’s Second Life. When the world was initially set up, residents who refused to adhere to a code of conduct that included provisions designed to prevent harassment of other members were incarcerated in “the corn field,” a dark corner of a simulated farmer’s field in which nothing happened. Avatars were sentenced to various lengths of forced stays in the field. More recently, a lawyer has sued Linden Labs for violating trade practices in a land deal where the land in question consisted of simulated real estate in Second Life. Though this might be seen as a publicity stunt, such issues will arise more frequently as we begin to use virtual realms as legitimate extensions of our lived spaces and invest time, emotion, and, perhaps most important, money in our virtual lives. Imagine how these kinds of issues could multiply in virtual realms in which we can see, hear, and feel one another with much higher degrees of fidelity than are currently possible.

  In spite of the challenges and risks, wider availability of good-quality immersive virtual experiences in homes holds great promise. Educators will have tremendous opportunities to take advantage of an inherent appetite for game play in children, providing them with knowledge and experience that would be hard to acquire in any other way. Imagine a game like the hugely popular geography quest “Where in the World is Carmen Sandiego?” rendered in full immersion and in three dimensions. In institutions such as universities and perhaps larger community centers, more powerful immersive virtual reality setups could generate precisely detailed walkthroughs of ancient Rome or Shakespeare’s Globe Theatre in which students could learn by talking to local “residents,” participating in group activities, and manipulating virtual objects with the look and feel of the real thing. Applications of such technologies are limited only by the imagination of the content developer.