Virtual Reality and Augmented Reality are power tools for telling stories, for improving learning, for bringing distant world experiences, both real and imaginary, into your hands and brain.
Virtual Reality(VR) is powerful because it enables us to feel physically immersed in realities we could never otherwise experience. This visceral sensation of believing that we’re somewhere else is called ‘presence’. It opens up new possibilities for storytelling, deeper ways of understanding ideas and experiences.
Augmented Reality (AR) experiences are powerful because they keep you in the real world but alter that environment in unique ways.[1] The choice of which technology to use depends on your objectives and your audience.
-
-
-
- VR is already being successfully deployed for remote training and education, for image modeling, therapy and tourism.
- It can also bring you to a museum exhibition, place you at a historic event or an archeological site, immerse you in a theatrical production, take you to a vision of a future park or cityscape, or a dreamscape.
-
-
Producers, directors and artists interested in how to tell these kinds of stories and make art in VR have already learned some techniques and considerations to keep in mind. Scroll down to see what we’ve been doing. Click here to skip ahead to our Guide to Creating Immersive Storytelling in VR.
Here at the MediaCombo VR/AR Lab, we’ve been experimenting with these technologies since 2016 to explore new ways of telling stories with immersive media and we’re sharing what we’ve learned with you.
Virtual Reality
TRACING PAINT
We’re in production on a new VR experience, Tracing Paint: The Pollock-Krasner Studio in VR, commissioned by the Pollock-Krasner House & Studio in East Hampton. We’ve recreated the studio exactly as it looked when Jackson Pollock was painting there in the 1950s, and then later when it was Lee Krasner’s studio in the ‘60s. Tracing Paint is designed to be an on-site experience but will also be available for the Oculus Quest.
WE ARE NATURE
We Are Nature: Living in the Anthropocene was a groundbreaking exhibition at the Carnegie Museum of Natural History, that closed in 2018. Our 3D 360º tour is open 24/7 for potential funders, educators, exhibition designers, and anyone who wants to visit this exhibition to learn more about the human impact on climate change. Read the story here.
LAKE BAIKAL
Our VR first project was Lake Baikal: The Science and Spirituality of Extreme Water, MediaCombo partner Michael Owen traveled to Moscow, Irkutsk, and Lake Baikal, to produce three programs for Google Pioneer Expeditions in March of 2016. Michael and his Russian crew used a camera rig built around 9 GoPro cameras. They filmed on the frozen Lake, at the Involginsky Datsan, the center of Russian Buddhism, and in surrounding towns. The result was a short, impressionistic 360º film intended as a teaser for a longer VR documentary about the environmental issues affecting fresh water everywhere.
Room Scale
Now we’re using scientific data to design a room-scale experience about the Lake, which will allow users to compare the depth of the Lake with the height of the Empire State Building and its length with that of Manhattan!
Augmented Reality
THE 1907 TOUR: PIERPONT MORGAN’S LIBRARY REVEALED
You can immerse yourself in a world that seamlessly connects 1907 and 2020 in this AR audio tour for The Morgan Library & Museum in New York. The tour is a deep dive into aspects of the magnificent architecture and decorations and his unrivaled collection of rare books, and shines a spotlight on his librarian, Belle Da Costa Greene, and on Morgan’s personal life.
If you’d like to know more about what we can do, or have an idea you’d like to talk about, please get in touch!
One of the most frequently asked questions we hear is: Virtual or Augmented Reality, which technology to choose?
The answer is always: It depends on the experience you want to offer.
Slip on a VR headset and you’re suddenly transported to a virtual place that feels as real as the physical space you’re in. That’s called presence. When the experience is over you can feel as if you’ve physically participated in the virtual world, and that memory can last for months. In part that’s because you have agency to direct your gaze and see everything that’s happening around you in every direction.
Pick up an Augmented Reality enabled mobile device and suddenly you can see people and objects in your physical space that aren’t actually there. Yet you can move up to them and around them – you can interact with them as if they were real. And you can interact with sounds too, just as you would in real life – moving toward them to hear them more clearly, or away from them until they disappear. This mix of real and virtual is informative, uncanny, and captivating.
The primary differences between the two technologies are that VR is immersive but isolating (though that will change). AR, or MR (Mixed Reality), on the other hand, locates people firmly in the real world where they can share information and reactions and reveals things they can’t see, but it’s not as visceral an experience.
Let us know if you want to start a conversation about this.
Virtual Reality Best Practices
–
How do we create the best possible VR experience?
–
How do we create the best possible VR experience?
Once you’ve affirmed that VR is the right technology to tell the story, there are five main components to consider to create the best possible VR experience:
Narrative, Environment, Cognitive Impact, Social Impact and Technology.
Narrative Considerations
- Plot, narrative arc, and characters that elicit emotions from viewers, are basic elements of storytelling, and are as important in VR as any other medium.
- Virtual Reality provides the opportunity to tell a story where things are happening in 360º around the person in the headset, not just in front of them. This immersive quality is what makes the medium so powerful. But most viewers are not used to following a story in the round so the user experience design (UX) needs to provide guidance about where to look and focus your attention.
- UX should also draw the viewers’ attention to the important objects and story points, so they spend less energy constructing the story and more energy receiving it.
- If your story can’t take advantage of the 360º world then don’t use VR.
Environment (created with visuals and sound)
- Whether the world is 360° video or computer generated there will be a lot to see outside a viewer’s field of view. Give them time to look around at their surroundings and explore before the story really gets underway.
- Fictional worlds created with computer graphics don’t have to be visually perfect to feel real. The human brain can fill in a lot of details if the story and characters are well developed.
- Sound design is all important! Sounds are what you see with your ears so creating a soundscape adds to the sense of immersion of being in a real world. Also, simple audio cues can alert visitors to turn their gaze away from what’s directly in front of them, so they don’t miss what’s going on behind or beside them.
Cognitive Concerns
- Viewers will automatically adopt a role because they are inside an experience. They may only be observers, or you may assign them a specific role. You can hint at this showing arms or legs that signal who they are, or place them in something like a cockpit or a wheelchair or whatever is appropriate to their character. There are different ways to do this but the more the viewer feels connected to the action, the stronger the impact of the story will be, regardless of whether you’re producing a social impact documentary or creating a fictional world.
- Producers have a responsibility to warn users about violent or emotionally charged situations that may be occurring in the virtual world, before they put a headset on, because an immersive virtual experience can feel like it’s really happening to the visitor.
Social Impact
- With documentary or social impact VR, it’s important to remind visitors that the VR headset is serving as a proxy. The virtual experience is not the same as the real thing.
- Yet it can still inspire users to take action in the real world to address the issues.
- Pre- and post-experience guidance is advisable. Visitors often want to talk about the visceral scene they’ve witnessed, to share their reactions, ask questions, process their feelings.
Technology
- The ultimate VR experience occurs when the user can physically move about in a virtual environment composed of computer-generated imagery (CGI). This is known as a room scale experience and requires a VR headset that provides 6DOF (Degrees of Freedom), the ability for the visitor to move forward, backward, left and right, up and down in the virtual space, mirroring movement in the physical space. This requires tracking devices that can tell where the headset is and what the viewer is looking at. As of June 2020, the Oculus Quest is the only headset that offers inside out tracking for a 6DOF experience without a cable or “tether” to a computer with a powerful graphics card. There also headsets such as the Oculus Rift S, the HTC Vive, and the Varjo that can deliver better quality visuals but they all must be tethered, to an external computer.
- Projects that are composed of 360º video footage, rather than CGI worlds can be viewed in 3DOF headsets. These offer the ability look up, down and around 360° but from a single point. Even though the visitor can’t move through the video space, they can still have an extremely immersive seated or standing experience.
- You can also make your content directly available to visitors on their own devices to be viewed with Google Cardboard, either via YouTube 360º or an app downloaded from app stores.
If you want to know more about immersive storytelling, please get in touch. We’re here to help!
