The Encounter pioneers a new kind of high-tech theatre
During the Sydney Festival, night after night, a rainforest will grow inside the Opera House drama theatre, then vanish.
Every person who sees the one-man show, The Encounter, will hear rain falling, feet running on moist earth, and mosquitoes buzzing in their ears.
Simon McBurney in The Encounter.
The sound effects arrive through headphones worn by each audience member. At performances in London and New York, people would sometimes swat their necks to dislodge the bugs that seemed to have landed just below their ears. With the help of binaural recording technology, which replicates human hearing, The Encounter plunges everyone in the theatre into a 3D world made of nothing but sound.
Some of it is pre-recorded. The mosquitoes, for instance, are a colony at the London School of Hygiene and Tropical Medicine. Many of the rainforest noises were collected in the Amazon, where the story is set. Others are created live on stage, using bottles of water and a device shaped like a human head that "hears" through a microphone in each ear canal, creating an illusion of space and distance.
Marco Perry, Bjork's sound designer, at Bjork's virtual reality exhibition at Carriageworks. Credit:Louise Kennerley
Simon McBurney created the show and originally performed it (his role will be played in Sydney by Richard Katz). The play is based on Petru Popescu's book, Amazon Beaming, which recounts the story of National Geographic photographer Loren McIntyre, who in 1969 lost his way deep in the Amazon. He stumbled across the Mayoruna tribe and, unable to communicate with them in English or Portuguese, he claimed he began exchanging telepathic messages with the headman.
McBurney wanted to give every audience member a sense of McIntyre's isolation. How better to do that than to lock each person into a private world of noise? "For me, the form and the content are inseparable," McBurney says. "It became essential to tell it this way … my main worry was that people would see it as a gimmick, but in 15 seconds they've forgotten about it and become immersed in what's going on. The quality of sound is so good."
Thanks to binaural technology, the story seems to grow inside the audience's brains. The boundary between reality and fiction begins to blur, just as it did for McIntyre. The story, and the way it's told, forces questions about the reliability of our senses.
Artists have always created imaginary worlds, but now there is a suite of technical tools that can make that process a little more literal. Binaural sound has existed in various forms since the 19th century. Other, newer technologies – virtual and augmented reality, and 360-degree video – have grown sophisticated enough to plunge the user into an immersive universe of sound and vision.
Wonder.land Installation in the Wolfson Gallery at the National Theatre. Credit:Richard Davenport
Traditionally, audiences experience art from the outside: looking at a painting, watching a play, listening to an opera. But immersive technology makes it possible to get inside the work and, perhaps, participate in it.
At Florida's Dali Museum, for instance, one exhibition allowed visitors to don virtual-reality headsets and step through the canvas of Salvador Dali's Archeological Reminiscence of Millet's Angelus. Inside the 3D digital universe, viewers could soar through a recreation of the painting's desert landscape, exploring its crumbling ruins or lifting their gaze to its twilight sky.
"[Immersive] technology allows you to tell stories in radical new ways," says Peter Tullin, co-founder of the REMIX Summit events, which explore the intersection of art and high-tech innovation. The most recent Sydney event drew almost 2000 people.
"You get creatives in tech to talk to creatives in culture to see what they can do together," Tullin says.
Few cultural institutions – and fewer artists – have the skills or the funds to tackle immersive projects alone. Recently, Icelandic singer Bjork invited director Jesse Kanda to film inside her mouth to make an immersive video. The piece, which premiered at Sydney's Carriageworks in June, placed a VR-goggled audience on Bjork's virtual, spittle-flecked tongue, looking up and around at teeth and gums as she sang. The singer was at Carriageworks for two weeks to make sure everything was in place. There were questions about how to move people through the space, how to show VR newbies what to do, how to keep the headsets clean.
Then there were the technical demands. It helped that Carriageworks was collaborating with Intel. It often works that way when arts companies take on expensive, high-tech projects. The institution gets expertise and gear; the tech giant gets cultural kudos.
"It is still very resource intensive," says Lisa Havilah, the director of Carriageworks. "It's interesting that artists like Bjork are making work like this. She's a leader in the area and, if she's exploring it, it seems likely that other people will jump on board … it's within our reach, but at the moment there are not a lot of artists demanding it for their work."
Nevertheless, there is a kind of immersive-technology gold rush happening in industries spanning gaming, journalism and the arts. It's as yet unclear what might be found in these artificial universes. But no one wants to miss out.
The technology has yet to hit the mainstream. While there was a flurry of enthusiasm for augmented reality when Pokemon Go first came out, it dimmed with the game's popularity. It's possible to experience virtual reality at home with the right technology, but headset sales seem to be relatively low. Since the April launch of the HTC Vive (one of the few headset models with publically-available sales figures) 140,000 have been sold. Not bad for a pricey niche product, but hardly the 660,000 iPhones sold every day.
However, if it feels risky for cultural institutions to invest in immersive technology, it's a gamble that the world's most powerful businesses are also taking. In 2014 Facebook acquired Oculus, a leading maker of virtual-reality goggles and software, for $2.6 billion. New employees at Oculus get a copy of Ernest Cline's science-fiction novel, Ready Player One, which imagines a world in which social networking happens in VR. Google has developed a cheap headset made from plastic lenses and cardboard, used in conjunction with a smartphone, as well as a more sophisticated version called Daydream. Imaginary realities are becoming big business, and those in the business of imagination would be foolish to overlook the opportunities.
Google, in particular, is encouraging artists and cultural institutions to explore these new worlds. The company's Cultural Institute began as a vast online collection of images and objects from museums and galleries. Increasingly, though, it is creating immersive experiences: virtual walk-throughs of museums including the Art Gallery of NSW, and ultra-high-resolution images that allow you to zoom in close enough to see every brushstroke in a masterpiece.
The British Museum has worked with Google's Cultural Institute to create close-up views of its art works that can be experienced online and through the Daydream headsets. "To see that vivacity of colour coming through in VR, you get a really tangible sense of how it was created," says Chris Michaels, the museum's head of digital. "That's a very powerful thing."
The challenge is to exploit the possibilities of the new technology without allowing it to distract from the live experience. The museum was an early adopter of augmented reality in its educational program. When school groups visit the museum's famous Parthenon gallery, students clutch tablets equipped with the specially designed app, A Gift for Athena. Image recognition allows the app to identify ancient Greek sculptures in the room, and weave them into an animated story on the screen.
Sydney will get a taste of the museum's high-tech approach in the touring show, Egyptian Mummies, which will allow Powerhouse Museum visitors to virtually peel back the ancient wrappings to reveal the bodies beneath.
Is it a risk that in these shows visitors spend more time gazing at a screen than at the exhibits? "It's a massive risk and a destructive one if we don't work out how to do it correctly," Michaels says. "The point of coming to a museum is to see and experience the collection. So let's acknowledge that people will carry their phones in their hands, and let's make sure that when people need to call on that phone, they're using it to add something on top of their experience."
Michaels hopes virtual reality will help take the collection to people who cannot visit in real life. The museum is preparing to launch an app for the Oculus Rift headset, which allows users to explore ancient objects and arrange them in their own virtual gallery.
That's no doubt an appealing prospect for anyone who has ever had to elbow past tourist hordes to see the exhibits. But it reveals something about immersive media that is at odds with most live cultural experiences: virtual universes are lonely. If Facebook gets its way, more social networking in virtual reality won't be far off. But, at least for now, the technology locks the user away in a wondrous universe for one.
Earlier this year, in the foyer of London's National Theatre, audience members sat on prop toilets covered in green slime, wearing virtual-reality headsets that blinded them to the real world and took them on a steep dive down a rabbit hole rendered in 360-degree neon-coloured graphics. At the bottom was a garden of pulsating purple mushrooms. In the sky, a glowing yellow cat with flaming whiskers and a sharp Cheshire grin was singing a song.
The National Theatre built this experience to complement its stage show, wonder.land. In the musical, with songs by Blur's Damon Albarn, teenaged Aly hides from school bullies in the toilets and escapes to a magical digital universe on her phone.
During a five-month run, 90,000 people plunged down the virtual rabbit hole. That's more than saw the show on stage.
The VR experience had none of the communal feeling that grows in a theatre full of people, but the NT's head of digital development, Toby Coffey, believes that is a temporary setback. To him, immersive technology feels inherently theatrical. "In 360-degree video, everything is on show. You can look at whatever you want. That's very theatrical and it's also a creative challenge; you've got to guide the gaze … theatre designers, directors, writers and actors are more instinctively used to working in the spatial form, whereas filmmakers have the focus and security of the frame. Film is the format that gets output by the software, but the process and the staging is more like theatre."
Coffey can imagine a day when the technology might be used as part of a live performance, but he's not sure how. "What we can say is that we know how it won't be used, which is 1200 people sitting in the theatre with a headset on for two hours," he says. "That is in no way what we are trying to do."
Humans have used technology to tell stories since the first artists smeared pigments on cave walls. High-tech tricks make art look and feel different, but do they fundamentally alter the process of spinning a tale?
The Encounter has been hailed by many as an utterly new kind of theatre, but McBurney is cautious. "I always find the technology discussion a bit of a red herring," he says. "Artists are scavengers. We use whatever is to hand to tell the story." The tools now available are about to expand their ability to tell their stories in unimagined ways.
The Encounter is at the Sydney Opera House from January 18-28 and the Malthouse Theatre, Melbourne, February 2-10.
Most Viewed in Entertainment
Please explain
Our weekly podcast giving you insight into the stories that drive the nation.
Listen now