EXPLORING
for Entrepreneurial Journalism and is currently finishing up a masters degree at the CUNY Graduate School of Journalism, where he has focused on filming and editing video for the web, investigative data reporting with SQL and Excel, and reporting on race, tech employment, business and the economy. Before journalism, Matt worked on videos and interactives in the exhibitions department at the American Museum of Natural History.
Ally Kirkpatrick
Editor
Leslie Martinez
Layout and Design
Table of Contents
INTRODUCTION 4
ACKNOWLEDGEMENTS 5
WHY NOW?
7
STATE OF PLAY
9
TIMELINE 17
BUSINESS MODELS AND CORE COMPETENCIES
19
WHEN WILL CONSUMERS CATCH ON?
28
I confess — the title for this report was appropriated. In a blog post in August 2015, NYU computer scientist Ken Perlin wrote that he found himself struggling with the terms ‘Augmented Reality’ and ‘Virtual Reality’. Like many he imagines a future in which the distinction is no longer necessary. The phrase he suggests instead is “future reality,” to describe a research agenda agnostic to any one set of technologies.
This summer NYC Media Lab commissioned Matthew MacVey, a Tow-Knight Center for Entrepreneurial Journalism fellow at the CUNY Graduate School of Journalism, to write a report on augmented and virtual reality, and he struggled with how to deal with this distinction as well. Perlin’s post came at just the right time— we call this report Exploring Future Reality.
But this report is not written for experts like Ken Perlin, Steven Feiner or others quoted here — if you’ve been to SIGGRAPH, there is likely nothing new here for you. Instead, it is tuned to a different constituency— those in the media industry who are now beginning to consider how to experiment with augmented and virtual reality, using tools for the capture, creation, distribution and consumption of new experiences that seem to improve by the month. The report tries to detail the main technical challenges and opportunities for media makers, and how to set up a good experiment. And, it attempts to summarize what is known about the market opportunity in the near term.
Many questions remain: which technologies will win, what companies will dominate the space, how soon consumers will seek out these experiences in large numbers. But what is clear to anyone who has had a chance to experience these media is that we are undoubtedly at the beginning of something, and it is time for those in industry to be deliberate in the exploration of what is possible.
“So our experiments center on putting people into experiences together — using
technology that is now expensive, but that will at some point in the future become cheap and widely available — and asking questions about social interaction, play, learning, culture,” Perlin wrote. “All those messy but wonderful things that any communication technology is really all about, be it a stone tablet or a smartphone.” Exact trajectories are difficult to determine, but the general direction is clear- future reality is drawing nigh.
Justin Hendrix Executive Director
NYC Media Lab
Debra Anderson
The New School for Public Engagement Dan Archer Empathetic Media Sean Atkins Discovery Communications Matt Bell Matterport Steve Beres HBO Jason Black RRE Ventures Kris Budelis KitSplit Nicolas Burtey VideoStitch Conal Byrne Discovery Communications Tipatat Chennavasin Independent Creative Director Paul Cheung The Associated Press Jens Christensen Jaunt Craig Dalton DodoCase Steve Dawson Razorfish Nonny De La Peña Emblematic Craig F. Douglass Contact Controller Interfaces Piers Fawkes PSFK Steve Feiner Columbia University Chuck Fletcher Razorfish Nathan Griffiths The Associated Press Luke Hamilton Razorfish Sam Harrell NBC Universal Media Labs Matt Hartman betaworks Michael Ibrahim NBC Universal Media Labs Roger Kenny
Wall Street Journal
Karl Krantz SVVR Kyle Li Parsons School of Design Sanjay Macwan NBC Universal Media Labs Francesco Marconi The Associated Press W. David Marx Google Nick Peck Bloomberg LP Ken Perlin NYU Fergus Pitt
Tow Center For Digital Journalism Dakota Powell VR Lab & Hackathon / LoNyLa - TimeWave Julian Reyes Fusion Stone Roberts Refinery29 Frank Rose Columbia University Alex Rosenfeld Vrideo Ana Rosenstein betaworks Bob Sacha CUNY Graduate School of Journalism Mariana Santos Fusion Adrian Saravia Fusion Juan Pablo Santillan Vantage TV Helen Situ NextVR Ryan Staake
Pomp & Clout
Samantha Storr
Vrse
Gavin Teo
Comcast Ventures
YingLi Tian
CUNY City College
Rob Weiss
YouVisit
Ray Velez
Razorfish
Aaron Vinton
Pomp & Clout
ACKNOWLEDGEMENTS
I think that immersive
3-D content is the
obvious next thing
after video.
”
-Mark Zuckerberg*
As recently as 2012, augmented and virtual reality was anything but “obvious” to most of the world. But work by researchers and developers in industry and universities in the three years since has refined the technology to record, create and view immersive content. Tech companies have led the way investing in augmented and virtual reality. Now they’re joined by global media companies such as Disney, which recently led a $65 million investment round in Jaunt.
›
“In 2015, leading technology and media companies, Comcast and NBCUniversal among them, realized that augmented and virtual reality is an exciting new medium. I expect this holiday season to support multi-million unit sales of consumer head-mounted displays (HMDs). Once demand is validated
for hardware, content creators will surely follow— starting with games and then expanding into
animation, live events, and film, which is why this is a highly relevant space for media and distribution companies,” said Gavin Teo, principal at Comcast Ventures.
Augmented and virtual reality hardware and content, it seems, are nearer than ever before to the consumer mainstream. Since 2014, notable examples of high profile augmented and virtual reality projects abound. Vice and the United Nations partnered with Chris Milk’s Vrse production company to produce “Clouds Over Sidra,” a spherical video that immerses the viewer in the daily life of a 12-year-old Syrian refugee in a Jordanian refugee camp. The International Champions Cup partnered with streaming virtual reality company NextVR to stream a match between Manchester United and FC Barcelona to the Samsung Gear VR. In late August of 2015, Discovery Communications launched its Discovery VR platform with a slate of virtual reality content they will grow over the next year. In November 2015 the New York Times launched the NYT VR app and distributed Google Cardboard viewers to a million home delivery subscribers.
Augmented and virtual reality appeals to media companies for its broad range of possibilities in both form and content. “We perceive virtual reality as not just a venue for advertising,” notes Steve Beres, west coast vice president of media and production operations at HBO. “We’re as interested, if not more so, in it as an opportunity for storytelling.”
Augmented and virtual reality is presently accessible to many consumers via their smartphones.
WHY NOW?
media producers must adapt to physical production concerns and experiment with cameras, software, distribution systems and displays. They also face decisions over whether to develop their technical talent in house, or build external partnerships. Media companies also face the challenge of auditing, evaluating, and adapting their core competencies, in order to decide which assets translate best to an augmented or virtual reality experience.
Evaluating existing content and migrating assets to an augmented or virtual reality platform remains one of many hurdles for media companies. While augmented and virtual reality promises to be a rich storytelling vehicle it is, as a nascent medium, still host to more craft than art. How will the visual language of augmented and virtual reality emerge, and who will lead that stylistic charge? Consider, for example, the creative evolution of film. “When Edison was making his early films in the 1890s, he had most of the technology needed to make a Hollywood feature film,” said Ken Perlin, of the NYU Media Research Lab. “It took decades to work out over the shoulder shots, two-shots and editing. It’s not about technology.”
The investment of time and resources in forging new augmented and virtual reality content may seem steep, but many media producers see first-person media as an important opportunity for immersing audiences in creative content as never before. Take, for example, a scene from HBO’s “Game of Thrones” virtual reality experience: Viewers find themselves in the virtual world of the show standing perilously close to the edge of a cliff. A physical response is inevitable, the viewer’s legs tense up, bracing against the seemingly unending drop. “You know you are standing in the middle of a conference room in Santa Monica, but your brain has that connection,” said Steve Beres. “You remember it like you did it, not like you watched it.”
When audience members recall a virtual reality experience it’s often as an active, lived experience. While one might remember a good film, that memory is filtered from a fixed, distant perspective. In virtual reality, the viewer is in the midst of the story. “Virtual reality brings up fuller types of feeling,” said Juan Santillan, CEO of virtual reality live streaming company vantage.tv. “One being intimacy. I’m here at the concert, very close to my favorite singer. That type of closeness and intimacy is creating higher ranked emotions with users than what was being done before virtual reality.” What this intimacy and closeness can achieve remains to be seen. Yet one interesting point persists: With virtual reality, the transaction between an audience and the story is fundamentally changed – and pared down. Established mediums, even when storytelling from a first person perspective, still rely on the removed - and unmoved – perspective of their viewer. Virtual reality bridges that gap, and in inspiring a lived memory makes this frontier of storytelling a shared
When audience members recall a VR
experience it’s often as an active, lived
experience.
”
There is more money and talent invested in virtual reality than ever before. More than $100 million was invested by venture capital firms into virtual and augmented reality in each of the first two quarters of 2015, according to CB Insights. Rothenberg Ventures has invested the most between 2012 and 2015. Other investors include Andreesen Horowitz, Google Ventures, Comcast Ventures and Qualcomm Ventures. Facebook and Google are acquiring and supporting many virtual reality
companies as well.
“It’s not just venture capital money. It’s also the brands and the consumer electronics companies
investing heavily,” said Jens Christensen, CEO of Jaunt. “There’s a whole ecosystem that’s being built up to deliver a really great experience to the end-user.”
A case in point is the Oculus headset: When it first debuted to the public through a Kickstarter campaign in 2011 it seemed to break new ground, combining many features thought necessary for a smooth virtual reality experience. Features like low latency and a wide field of vision were refined and delivered in one relatively cheap device.
For many, Oculus revived a long dormant interest in VR. A late spring 2013 demo of the Oculus led Matt Bell, founder of Matterport, to move from web virtual reality distribution to a company with a wider scope. “I realized the Oculus Rift would be the perfect medium for experiencing our content,” said Bell. “Because we have this incredible amount of immersion plus the ability to move around. The feeling you would get inside a Matterport model would be fantastic.”
Oculus was acquired by Facebook for $2 billion in March 2014. A consumer release is scheduled for early 2016. More than 200,000 developers self-identify as focusing on Oculus development, according to an announcement at the Oculus Connect developers conference in September 2015.
Even with limited rollouts and a soft-launch for those headsets that have been released, more than 2 million virtual reality devices are expected to be sold this year according to mobile industry research company CCS Insights. There are more than 260 startups working on virtual reality software,
hardware and platforms listed on startup investment website AngelList.
STATE OF PLAY
There’s a whole ecosystem that’s
being built up to deliver a really
great experience to the end-user.
”
Oculus Development Kit 2 demo. Source: Softlight Photography
“
Virtual and augmented reality experiences can be distributed in several different ways, each method presenting its own challenges. If content is downloaded it requires significant storage. In the Vrse app library, for example, “Clouds Over Sidra” is 600 MB and “Walking New York” is 700 MB. Some projects are even larger. The download size for the Gannett / Des Moines Register’s “Harvest of Change” series is greater than a gigabyte. Viewing virtual reality on a headset through a web browser is currently not fully implemented in any browser. An experimental version of Firefox and Chrome, installed with WebVR, allows some content to be viewed with an Oculus developer kit.
Distribution
“Clouds Over Sidra” is available in the Vrse app that is available in the iOS and Android store.
“The Nepal Quake Project” by RYOT in the Vrideo library.
“Harvest of Change” from Gannett and the Des Moines Register.
A Paul McCartney concert video produced by Jaunt.
“Is the Nasdaq in another bubble? ” by the Wall Street Journal.
An in-store virtual reality kiosk at the North Face, which features base jumping in Yosemite and Moab. (Christopher Gregory/For The Washington Post)
Content can be on a platform that is app
or
browser based
.
Content can be downloaded or streamed
in a
app library
.
Content can be
downloaded
.
Content can be played at a
kiosk
.
Content can be
rendered
on a webpage.
Content can be delivered in its
own app.
Current Bandwidth and Mobile Device Limitations
A large amount of bandwidth is needed in order to download all of the data necessary to render a full virtual reality environment. HD video is usually 1,920 x 1,080 pixels, which is the equivalent of more than 2 megapixels. Much more data is needed to create a high-resolution 360 degree video where the viewer can look in different directions.
Virtual reality content is usually
downloaded to a device before playback. Wi-Fi or mobile network connections must be high speed in order to accommodate live streaming of virtual or augmented reality content. However, file sizes and formats may evolve quickly to reduce download times. Right now most virtual reality content downloaded to a device is stored much like an MP3 in a music library. The larger file size of virtual reality files is a limitation on how much content can be downloaded at a time and how much space is available on the hard drive of someone’s smartphone or computer.
As internet service providers offer faster download speeds, the ability to stream content from virtual reality delivery platforms will open up. “We’re moving towards an era with better streams and better cameras. The content is going to get more and more bandwidth intensive,” said Alex Rosenfeld, founder and CEO of virtual reality streaming platform Vrideo. “We support up to 4K right now. We could, in theory, support 8K as well. But that’s limited by the internet speeds that most people have access to, and also the quality of video that most people’s computers or mobile phones are able to support.”
Technological Developments on the Horizon
The entire technology ecosystem for virtual reality is still evolving. Display resolution will increase from 2K to 4K, and eventually upwards to 8K. This will enhance the clarity of the virtual reality world at hand, improving perception of details such as text.
Also, the price of fast central processing units (CPUs) and graphics processing units (GPUs) for computer based virtual reality headsets will decline, meaning more people will have computers in their homes
that can support smooth virtual reality playback. “There are all these elements that need to move somewhat in sync between the camera technology, internet speeds, the quality of content that computers and mobile phones are able to support,” noted Rosenfeld.
The current generation of smartphone-based virtual reality works with phones that weren’t designed with virtual reality in mind. Smartphone based experiences stand to improve once the needs of virtual reality begin to inform mobile hardware design.
Industry experts anticipate technological advancements that will improve user experience. “There’s a clear sightline to really amazing head-mounted displays in 2 to 5 years,” said Jason Black, analyst at RRE Ventures. “With wider field of view, no screen-door effects, incredibly low-latency, amazing tracking. There’s already some amazing tracking systems out there that almost perfectly render your head movement.”
Content downloaded to a smart phone is easily accessible, but limited by storage.
Source: Softlight Photography
Advancements in virtual reality content must made in concert with those in consumer computer hardware. Source: Softlight Photography
There are two types of head-mounted displays (HMDs) for virtual reality. The first affixes a
smartphone to a headset directly in front of the viewer’s eyes; Google Cardboard and Samsung Gear VR, for example. The other headsets have a display built in like the forthcoming Oculus, HTC Vive and Sony Playstation VR. The refresh rate, resolution and latency for these displays all have similar specifications.
Samsung Gear VR
The Samsung Gear VR is a smartphone enabled headset that works in conjunction with either of the Samsung Note, Galaxy S6 or S6 edge phones. The headset connects to the phone via microUSB. Since the smartphone is held in the device, a touchpad is provided on the side of the headset as a user interface. The Gear VR was first released in fall 2014 and an update was released with new phone compatibilities in March 2015. A team led by John Carmack at Oculus developed the Gear VR with Samsung.
Early versions of the Gear VR were called the Innovator’s Edition. A consumer release and marketing push for Gear VR rolled out in November 2015.
Google Cardboard
First announced at Google’s I/O Developer’s conference in 2014, Google Cardboard is a software development kit that allows people to create virtual reality applications that work with a smartphone and a cardboard viewer that consists of a simple phone holder and two lenses.
Google estimates that millions of smartphone holders that meet Google Cardboard specifications have been produced. Google has developed the API and the specifications for the viewer, but Google doesn’t sell the cardboard viewers. Google has created a “Works with Google Cardboard” badge and QR code that indicates that viewers are compatible with the Cardboard app. In order to participate, manufacturers send work samples to Google’s Mountain View office.
Google released software developer kits for Android and Unity, a development platform for creating 3-D and 2-D games and interactive experiences.
Customers have trouble finding content
The surplus of distribution channels for virtual reality content, paired with only minimal consistency as to what appears where, poses a challenge for audiences seeking content that interests them. For now, users discover virtual reality content by touring an app store or via targeted promotional outreach.
The current limited amount of virtual and augmented reality experiences being produced means early adopters can find out about new experiences through the media buzz for each new piece launched. The pace of new releases is slow enough that audiences have to time to research each new release and learn how to download it. As content production increases, this discovery system will seem anachronistic.
That said, YouTube has enabled 360 degree videos in its video platform, meaning that one of the biggest discovery platforms for traditional video is now involved in helping people find content that can be viewed on its Google Cardboards. Vrideo and Littlestar also offer virtual reality content libraries, and Vrse works with publishers to provide virtual reality pieces in its Android app.
Oculus
The Oculus has shaped the trajectory of recent interest in virtual reality. There is a large development community working on games, media and software for the platform. As of September 2015 more than 100,000 Oculus developer kits have been released and more than 200,000 developers registered. Oculus is rumored to be the best funded and most staffed virtual reality effort currently in development.
The combined price for an Oculus compatible computer and the Oculus display itself is anticipated to start at approximately $1,000 dollars. This price is significantly higher than most items in the gaming accessories category and more in the ballpark of 3-D televisions or Google Glass.
A camera and sensors on the Oculus headset tracks a user’s position to replicate movements in the virtual space as they move through the physical space.
The Oculus will require a reasonably powerful computer for playback. The minimum system requirements are an NVIDIA GTX 970 or AMD 290 GPU, an Intel i5-4590 CPU and 8GB of RAM. A computer with those specifications currently costs around $1,000 out of the box. If a potential Oculus customer needs to upgrade their computer to use the headset, this will add to the price of the device. The Oculus will initially run only on the Windows operating system. Microsoft and Oculus have partnered to offer an Xbox One controller with every headset of the consumer version that is sold. These controllers mean that the Oculus will have more opportunities for input from the user than smartphone based virtual reality experiences. This could mean a much richer type of content will be available for the Oculus, but if media experiences aren’t able to offer a deep interaction users may be left tantalized and disappointed by the promise of the Oculus. To address the lack of content, Oculus has started its own content production unit called Oculus Story Studio.
Sony PlayStation VR
The Sony Playstation VR, called the Sony Morpheus early in development, is a headset that will work with the
Playstation 4 gaming console. Since the Playstation 4 is one of the most popular gaming consoles the Playstation VR will have a large potential audience of users.
HTC Vive
The HTC Vive is a collaboration between HTC and the Valve company that runs the popular Steam gaming service, which has built a platform for content creators. The Vive was used for an experience at the 2015 US Open tennis tournament, produced with American Express. It allowed fans to play a virtual game of tennis with Maria Sharapova.
Production
There are three approaches to creating content for a virtual reality world: you can animate a 3-D environment, record video, or take a hybrid approach by creating a 3-D environment with photos and video incorporated.
For each stage of the virtual reality production process there are different companies working on the devices and software. While not comprehensive, the list below is an attempt to highlight the different types of cameras and devices gaining traction within each segment.
Cameras
Cameras that record in 360 degrees can capture either time (360 video), or space (a 3-D environment). In both cases, the camera is placed on a tripod. The camera arrays usually have six or more cameras pointing horizontally and one up. Audio recorders are also typically incorporated into the camera rig.
GoPro and 360Heros
GoPro cameras are frequently used in camera arrays for capturing 360 video. The GoPro wide-angle lens makes for image overlap when stitching the video together in post-production. The relatively low-price of the camera helps keep costs down when six or more cameras are required for a full array.
There is no standard way to use GoPros in creating 360 video content. Producers often use 3-D printed holders to put create a ring of GoPro cameras. Some companies are beginning to sell pre-made camera arrays.
GoPro recently announced its own array of cameras, called Odyssey, that will work with Google’s forthcoming Jump end-to-end video processing system. The Odyssey consists of 16 cameras in an array designed to produce stereoscopic video and ease the syncing process. The Odyssey is currently only available through an application process.
Matterport
The Matterport camera captures a space and turns it into a virtual reality environment. The result is 3-D recreation of a room a viewer can explore. The Matterport camera has been adopted by architectural firms that want to capture spaces for their clients, but it has also been used by media companies like The Associated Press and the Detroit Free Press. The Matterport camera captures 360 photos from a variety of locations throughout a space. Those 360 images are then stitched together into one large 3-D image file. The camera capture is controlled by an iPad, which also provides an initial processing of the 360 image. The .obj 3-D files and textures can be downloaded from Matterport for use in other environments such as Unity.
Dropping in on “Seinfeld” with the
Associated Press
Any ‘90s TV fan will recognize Jerry Seinfeld’s sitcom apartment. Now, for the first time, viewers can walk around the famous set, exploring details down to Jerry’s CD collection and the cereal boxes on the kitchen shelf. This virtual environment, captured in 360 degree video on a recreation of the set, was created as a promotion for Hulu. The Associated Press published this tour, its first experiment in 360 video, in
To explore the virtual “Seinfeld” apartment visit: http:// interactives.ap.org/2015/seinfeld-apartment/
Jaunt
Jaunt has worked in conjunction with media companies such as Elle, ABC, and The North Face. In June 2015 Jaunt released the NEO camera system, which can capture depth information along with image information. With depth information, a viewer is not only able to change the angle of the video that they see, but also can move front to back and side to side within the virtual space. The NEO system is aimed at creating high-quality, cinematic VR footage.
Audio
Audio capture and editing is very important to the creation of a compelling immersive experience. Sounds can be used to direct the viewer to things that aren’t immediately within the field of vision. Locational sound reinforces the immersive nature of using head-mounted displays (HMDs). “With the 16 by 9 world my mantra has always been 50 percent of what you see is what you hear,” said Stone Roberts, VP of programming at Refinery29. “For VR it’s probably that 70 percent is what you hear. Maybe even higher. Directional audio and sound design. That is your editor. The user of these headsets is effectively the editor. When they choose to turn their head they’re making the cut by changing their perspective.”
Syncing and Stitching
A 360 camera is not one lens and one sensor, but instead an array of cameras working together to record the scene from multiple points of view. Once the different angles from each camera are recorded, they must be brought together into one image in post-production through a process called syncing and stitching.
First the video from each camera in the array must be put into sync with the others, often using a sound, flash or light on set as a cue. Once the video from each camera is playing back in the same time, the images from each camera in the array can be processed to find the overlap, and then can be combined into one complete shot. This can be done on a server or on a local computer.
Kolor and VideoStitch are the two most widely used stitching softwares. Photoshop is also used to touch up 3-D stitching. Many producers say that stitching software still needs improvement.
The Google Jump workflow allows for processing in the cloud with the Jump Assembler. The details have not been specified pending the camera’s release in late 2015 or early 2016.
Matterport offers an end-to-end capture and processing pipeline and charges for hosting the finished 3-D environment. As of August 2015, the monthly cost ranges between $49 and $149 dollars per month depending on the number of models hosted and how many users are granted access to the back end of the content. Matterport offers discounts for yearly subscriptions.
These services present a few potential trade-offs: Since processing is matched to the camera, it can be tweaked to match the specific quirks of that camera array. On the other hand, there may be a smaller team working on this software and it may not be updated as frequently as stitching software that is open source or covers a wider range of camera types. Also, there may be less of an incentive for the developers to make processed 3-D files compatible with other editing software, although at this point content from Matterport can be downloaded to use in a Unity environment.
Limits on capturing 3-D environments
Commercial 360 cameras cannot capture depth as they capture video. This significantly limits the type of experience created with recorded media in comparison to media that is rendered graphically.
Recorded content is limited to 360 degree video with one perspective, or to static images that users can explore. Some have questioned the length of time a user can be engaged with 360 video given these limits.
Rendered
The other way to create content is through animation. Animated 3-D environments can be a cost effective way to create immersive virtual reality experiences where the user moves through a space and events are unfolding around them.
Visual effects artists or 3-D artists may be enlisted to finish producing this type of experience to make it look realistic. However, up until those finishing touches, most of the experience can be developed and prototyped using low-fidelity graphics that don’t require a 3-D artist.
Game Engines - Unity and Unreal
Unity and Unreal are two game development platforms that are also used to create virtual reality environments. Working with Unity can be more complicated because it is not a web development language. Some companies are adapting to this challenge, such as
Razorfish, which has been successfully training front-end developers to work in Unity.
Unreal is regarded as having the highest quality graphics and lighting effects, while Unity content runs more smoothly on less powerful hardware.
As the audience for virtual reality evolves, so will the population working with programming languages for developing
virtual reality content. The more developers working in these environments, the more improvements and new libraries will be added to Unity and Unreal. Environments will become more efficient, easier to use, and will offer new features. It will become cheaper and easier to develop content. In turn, producers and developers won’t need specialized skills to participate.
Gannet used Unity to create “Harvest of Change” with The Des Moines Register, in which users learn about the changing economics of farming by exploring and engaging with a century-old farm. The National Press Foundation recognized the production team as the first-ever recipients of the ‘Best Use of Technology in Journalism’ award.
Digital news outlet Fusion partnered with graphic journalist and virtual reality producer
Dan Archer of Empathetic Media to create Ferguson Firsthand, a rendered experience based on the shooting of Michael Brown in Ferguson, MO. Archer recreated the street where Darren Wilson shot Michael Brown in 3-D, allowing the audience to explore the event through eight different eyewitness perspectives.
In Ferguson Firsthand viewers can explore the scene by navigating with keystrokes and cursor movement. Colored beacons indicate the location of eyewitness perspectives. View the project by visiting: http://static.fusion.net/ ferguson-vr/
Graphics Libraries - Web GL and three.js
WebGL is a graphics library that allows 3-D graphics to be rendered in web browsers without extra plug-ins. No applications. No downloads. A user simply goes to a web page and the 3-D environment loads up, ready to explore. Projects made in Unity 5 can be exported to run in WebGL.
Javascript developers can create web browser friendly immersive 3-D environments with three.js.
Roger Kenny, Senior Engineer at Dow Jones, used this library in conjunction with the D3.js statistical and graphic library to create an interactive 3-D rendering of the 2001 NASDAQ stock bubble for The Wall Street Journal.
Kenny was experimenting with the three.js javascript library and D3 when he realized that he could map a dataset in three-dimensions without much difficulty. After discovering the workflow, Kenny produced a 3-D chart of the Dow Jones average and refined the color scheme. Although finishing the textures for the environment and finalizing the design took a few weeks, the project was much less time and resource intensive than other types of computer graphics or 360 video. “We’re trying to make it as compelling as possible with the lowest-common denominator hardware,” said Kenny.
The finished piece was deployed for Google Cardboard as well as mobile and desktop browsers, and was met with an enthusiastic audience. “People all over the world were tweeting videos of themselves riding the Nasdaq curve.”
People all over the
world were tweeting
videos of themselves
riding the Nasdaq
curve.
”
“
A CONSUMER’S PERSPECTIVE
RECENT EVENTS
SEPTEMBER
Oculus raised more than $2 million for its virtual reality development kit on Kickstarter.
APRIL
Google launched a prototype of Google Glass, a pair of glasses with a small head-mounted augmented reality display, for $1,500 dollars.
MAY
Oculus started shipping its first development kits and opened Oculus Development Center.
MARCH
Facebook bought Oculus for more than $2 billion.
JUNE
Google Cardboard virtual reality platform announced at Google I/O conference. Cardboard viewers handed out to conference attendees.
DECEMBER
• Samsung Gear VR Innovator Edition, a smartphone powered head-mounted display which uses Oculus technology, made available for $199.
• Rothenberg Ventures announced the River virtual reality startup incubator.
APRIL
“Use of Force,” an immersive
documentary by Nonny de la Peña, was featured at the Storyscapes program of the Tribeca Film Festival.
OCTOBER
Magic Leap, a company developing augmented reality and virtual reality, announced investment of $542 million in a Series B financing round led by Google Inc.
FUTURE RELEASES
• Oculus Consumer Edition • HTC and Valve Vive
• Microsoft HoloLens to be released to developers at $3,000 per device.
JANUARY
• Google announced that it will stop producing Google Glass, but will continue developing the project. • The Sundance Film Festival
featured virtual reality and
immersive content – including work by Vrse, Empathetic Media and Felix & Paul – in the New Frontier installations.
APRIL
• HTC and Valve announced Vive partnership.
SEPTEMBER
• Sony announced the Playstation VR virtual reality headset for the PS4 gaming console.
• Jaunt announced investment of $65 million in Series C financing round led by The Walt Disney Company, Evolution Media Partners and China Media Capital.
NOVEMBER
• Gear VR Consumer Edition released by Samsung and Oculus.
• Microsoft announced development of the HoloLens augmented reality headset.
• “Clouds Over Sidra,” a virtual reality film about a Syrian refugee camp in Jordan, premiered at the World Economic Forum in Davos, Switzerland and is made available in the Vrse application and on iTunes.
MAY
• Google announced that more than 1 million viewers that are compatible with the Cardboard SDK have been distributed. Cardboard iOS SDK announced, making it easier to develop Cardboard apps for the iPhone.
• Oculus Connect 2 Developer Conference: Oculus announced more than 200,000 people have registered for the Oculus Developer Center.
• The New York Times launched NYT VR and announced a partnership with Google Cardboard, distributing 1 million cardboard VR headsets to Sunday home delivery subscribers.
Although many head-mounted displays (HMDs) are not yet publicly available and consumer usage is limited, some media companies have already found ways to bring in revenue from virtual and augmented reality content.
Advertising and Cross-platform Marketing
The most common way to advertise with virtual reality has been through sponsorships. For example, Refinery29 worked with Samsung as the brand partner in creating its travel series content. It’s an
arrangement that benefits both parties. The media company gets to experiment with different methods of storytelling in virtual reality. The company that is advertising gains association with the innovation and excitement of virtual reality. “Immersive advertising is going to be huge,” said Alex Rosenfeld, founder and CEO of virtual reality streaming platform Vrideo. “If you’re a life long brand marketer, it’s a dream come true to be able to immerse viewers in the world of your product or service.”
Many media companies have been very successful at creating media ecosystems across multiple channels that spread from television, to websites, social media, and, in the case of companies like NBCUniversal, theme parks. These companies have been able to make deals with advertisers based on these ecosystems. Now media companies will be able to add virtual reality to the advertising platforms they offer.
So far, in-experience advertising and commerce has been limited, but there are opportunities for product placement. “What will be important is to bridge the experience to how it can impact commerce,” said Ray Velez, chief technology officer at Razorfish. “Better resolution is going to help drive more sales. You don’t get much better resolution than these types of devices.”
Sales to consumers
The next way to monetize virtual reality is by selling the content directly. Because the devices are expensive and consumers are investing in them, there is an expectation that consumers may spend more money on content. Producers are afraid of the low-price trend in the iOS and Android mobile app stores. The low barriers of entry for app development and the app market for smartphones led to small prices being paid and the most typical monetization strategy being in-app purchases.
In-experience purchases, like shoppable fashion virtual
reality content, could be a good option for virtual reality. One comparable business is the virtual economy in the Second Life metaverse, which is worth more than $22 million dollars according to Re/Code. That money is spent on things like virtual clothing and real estate for the online avatars of Second Life users.
The other possibility for selling the content directly is through subscriptions. Just as Hulu, Netflix, Amazon and HBO have grown their over-the-top video subscription offerings, there could be similar offerings for virtual reality as well. Valve offers the Steam game store that will offer content for the Vive when it is released. Once head-mounted displays are connected to these devices, usage for streaming could increase significantly.
BUSINESS MODELS
AND CORE
COMPETENCIES
Immersive
advertising
is going to be
huge.
”
“
One example of a more comprehensive subscription service that might be akin to a virtual reality subscription offering is the Xbox subscription service, Xbox Live Gold. It offers a range of features from chat and media sharing to recording and game downloads. It has been very successful for Microsoft and has shown a resilient 30% year-over-year engagement increase in 2015, even as console sales declined. The first generation Xbox launched in North America in 2001 and the online game play platform Xbox Live was launched in 2002. Xbox consistently lost money for Microsoft until Xbox Live gained great adoption and the network functionality was made standard in the 2010 version of the Xbox 360 console.
Narrative
Media companies know how to make engaging narratives. They have the talent and experience in making compelling content on their own and in partnership with other companies. Narrative is very important to virtual reality content. People still need a reason to keep their headsets on and to find out where to look in an immersive environment.
Skill in producing engaging narratives can be offered to other companies as well. Just as many media companies like the New York Times and Vice have built creative services business, that model can be applied to virtual and augmented reality content production as well.
Existing brands
Fiction and Non-fictionVirtual reality experiences tend to gain the most traction when a user can engage with familiar content in a new way. For example, Sixense has developed a virtual reality experience where people can use lightsabers. “The demos with the biggest lines and the most excitement are the ones that allow people to go into an environment that they are already familiar with in some way,” said Karl Krantz, who runs the Silicon Valley Virtual Reality meetup. “Letting people go into the Millenium Falcon, Star Trek, or Game of Thrones – I think a lot of people will buy into virtual reality in order to do that.”
Live Events
Coverage of live events presents a valuable asset for existing brands. Offering a front-row view of a sporting event or a fashion show is lucrative. “If you’re comparing this to a $300 dollar ticket, there’s probably some price point you would be willing to pay to virtually sit in that seat. It makes sense to charge much less to many more people,” noted Matt Hartman, director of seed investments at betaworks. NextVR is developing a pipeline to stream events live to virtual reality displays. The company has experimented with on-site streaming of events from the NHL, NASCAR, a European league exhibition soccer game, and more. These events were then viewed remotely in New York, Los Angeles and Vancouver. Helen Situ, product lead at NextVR, noted that the emotional connection between fans and their favorite players and teams could lead to more ticket purchases and merchandise sales. “For rights owners, the thing they are most excited about is a deeper sense of connection,” said Situ. “Already we see that people feel much more engaged, which could lead to more action in commerce.”
Consumer Trust
Virtual reality offers the possibility of transporting people into places with a greater degree of verisimilitude than ever before. It also offers the possibility of deceiving people in a way that has never before been seen. This means that there will be an even greater need for trustworthiness and ethics in virtual reality production. Brands that are established as trustworthy and that can offer their experience in making ethical calls about media production will be able to gain an edge in consumer trust and will avoid the fallout of making the wrong decision during production in this complicated new medium.
Discovery VR
Discovery VR launched in late August 2015 with content available across iOS, Android, and the Samsung Gear VR Milk app. It is also viewable on a self-contained website and on YouTube. The library will be a platform for content from across the holdings of Discovery Communications, which owns cable channels including Discovery Channel, TLC and Animal Planet.
Discovery VR contains a library of content, all 360 video. The videos bring established Discovery brands and shows like “Survivorman” and “Mythbusters” to virtual reality. Discovery worked with Littlestar to create the app and to embed the content on the Discovery VR website.
Road Map for Experimentation
1. What editorial content makes sense? Put the viewer in the middle of the action and give them something to look at.
In many cases, inaugural virtual reality productions are collaborations between hardware companies and advertisers. For media companies, the smaller scope of an advertising project means less cost and less risk. For brands, it’s an opportunity to garner media coverage.
“VR can be utilized for introducing the user to all manner of subjects and stories,” said Samantha Storr, executive producer at Vrse. “From you-had-to-be-there moments to humanitarian crises, our collaborations with the UN – Clouds Over Sidra and Waves of Grace – have proved how powerful VR can be in connecting world leaders to the major issues they preside over. Rather than supplanting an older medium, VR is a new way of thinking about story, spatiality, and character exploration. There are documentary applications, narrative adventure applications, and everything in between.”
For Discovery VR, there were two important factors in selecting content. The first was to work with the production teams that were also starting to experiment with virtual reality independently. In this case, the “Mythbusters” production team was already excited about virtual reality. The second consideration was the content itself. Conal Byrne, senior vice president of digital media at Discovery Communications, noted that the challenge was in “trying to find intellectual property where the location was as big a character as the talent. We thought that would be a good place to start for virtual reality.” Discovery VR created content for “Gold Rush,” set in Yukon, Alaska and “Survivorman,” which is filmed in remote locations all over the world.
2. Video or computer graphics? Try it all.
Media companies well equipped to create video content are poised to make a smooth transition into producing 360 video for head-mounted displays. However, there are some aspects of virtual reality that can’t be tapped into with video. “360 video limits the mobility of the viewers to the point of view of the rig, even in movement on a predetermined path,” said Adrian Saravia, digital creative director at Fusion. “Computer graphics environments allow the possibility to move at will and in any direction which again, depending on the experience, can only be achieved with Unreal or Unity engines.”
Although producing interactive content with game engine software requires the greatest number of new skills for media companies, it’s worth the investment since these platforms —the Oculus, Vive and Morpheus— will use interfaces and tracking that present the interactivity that virtual reality has to offer.
Rather than supplanting an older medium,
VR is a new way of thinking about story,
spatiality, and character exploration.
”
3. Pre-Production and Pre-Visualization
“Great VR is immersive, first and foremost,” said Samantha Storr, executive producer at Vrse. “You have to feel like you’re within a new space. That comes down to a lot of things: art direction, story, character and environment interaction, and a robust editing process. The full 360º environment should look seamless, and the depth and dimensions of this virtual world should cohere with the real one.”
Many producers have found that virtual reality production is somewhat like producing for theater. The shot unfolds as scenes such that stage direction, not editing, leads the attention of the audience.
The time-intensiveness of rendering, syncing and stitching virtual reality footage is an added incentive to get the footage needed in only a few takes. To address this issue, Stone Roberts at Refinery29 had the talent he was working with rehearse a scene many times before filming.
4. Setting up your scene: Production Techniques and Challenges
The whole location should be considered when capturing 360 assets. “When you’re shooting in a 3-D space, you can’t crop anything out. You’re really capturing everything,” says Matt Bell, co-founder of Matterport. “When you go in to take a photograph, you can frame exactly what you want in the photograph and outside the photograph.” Anything in the shot that can’t be cleaned up on location can be tackled in post-production, usually taking a greater amount of time than addressing it on set.
One major challenge is keeping lighting even as the space is captured. The lighting needs to remain somewhat consistent and not have too much contrast between light and dark in order for the 360 image of the space to appear consistent on screen.
“VR production typically consists of a smaller crew,” noted Storr. “Instead of having your director, producers, grips, and electrical hiding directly behind camera, a VR crew has to hide in another room or behind a wall to be out of shot. Tools like boom poles and lighting equipment can’t be in the shot either, so VR production typically utilizes a handful of people that can multi-task to make sure the shot is set, which cuts down on production costs.”
For cameras that capture motion, early experiments have shown that it is best for the camera to remain still or move at a very slow and steady pace. Vice had success moving the camera in its piece documenting a protest in New York City. Similarly, the North Face virtual reality experiences have shots that involve a 360 camera gradually moving while suspended from a drone.
Instead of having your director, producers, grips, electrical
etc. hiding directly behind camera, a VR crew has to hide
in another room or behind a wall to be out of shot.
”
“
Fusion
In spring 2015, in an effort to get ahead of the curve on virtual reality production, Fusion created in-house teams working on 360 video capture and CGI interactives. Producers have worked with the investigative team, adding virtual reality to their news packages. “Right now it’s little satellite experiments. Our team is trying to embed themselves in some of these bigger projects in the field,” said Adrian Saravia, a digital creative director at Fusion who is leading virtual reality projects. The Fusion team is looking for not only what devices get the biggest audiences, but what
demographics are on what devices and platforms. “Some of the most interesting things are looking at real-time data sources coming into a VR space. Creating graphical experiences around video feed or data feeds that can produce experiences that build themselves in front of you.”
16x9 vs VR
“Production alone is typically less costly, depending on the shoot, than 16x9,” said Samantha Storr, executive producer at Vrse. “The heavy costs of VR arise from camera equipment. Shoots can require anywhere from 8 to 32 cameras or more, including corresponding lenses. Rigging is often tailored specifically to the shoot at hand. Post-production often swallows more than half of the budget.”
Many producers expect that costs will come in line with 16x9 production soon. “It may not be one-to-one in regards to a 16x9 production world, but it’s pretty close. If it’s not there yet, within the next six months to a year, it should be,” said Stone Roberts, producer at Refinery29. “It’s a different approach than you would take with a traditional production, but there’s not much additional you
the lenses and the overlap of the images needed for stitching. When capturing stereoscopic video, the subjects need to be twice that distance, at least four feet, or even further from the camera to appear undistorted to the viewer on playback.
5. Production Costs
The production costs for virtual reality vary considerably depending on the method used to make the content and the creative direction behind the shoot. However, there are some trends emerging.
For captured content, many producers have compared the cost to a high-quality 16x9 shoot. The producers we spoke to for this report estimated a cost-per-minute of virtual reality production of anywhere from $8,000 to $100,000. The more computer generated imagery, the higher the costs tend to be. The price to create a virtual reality experience with graphics as high-quality as those in an animated Hollywood feature film could cost well above $100,000 for a minute of footage. There are opportunities to save money by reusing computer graphic assets between productions. For example, NBCUniversal took an animated model of a dinosaur from “Jurassic World” and used it to create a virtual reality experience for the Samsung Gear VR.
Many 360 video projects use only one camera rig and a few takes. This reduces the budget for camera operators. However, the degree of pre-planning for a scene means that there needs to be more time and money put in the budget for working with the talent ahead of time in rehearsal. Preparing a set for 360 video capture can require many resources because it must be lit evenly across every angle.
6. Syncing
Once the footage has been filmed, the first step in post-production is to sync the footage from the various cameras. A signal captured on each camera is used as a marker to match video clips to each other. There are three common methods of synchronization: flash synchronization, which is often the most accurate, sound synchronization, and motion synchronization.
Advances in 360 camera rigs
are coming soon that will allow the camera start and stop time to be locked in the camera. Until then, syncing will continue to be a trying process. “One of the most painful things today is synchronization,” said Nicolas Burtey, CEO of virtual reality post-production software company VideoStitch. “It’s
something we can do by algorithm, but if it’s done directly in the camera it’s much easier to handle.” Once the video is in post-production, stereoscopic video takes two to three times longer to sync, stitch and render than monoscopic video.
7. Narrative is essential in virtual reality and requires a new approach
To keep a viewer interested in a virtual reality content, the viewer needs to be hooked into the story and needs to know where to look within the 360 degrees to follow the next phase of the story.
Fergus Pitt, Senior Fellow at the Tow Center for Digital Journalism at Columbia Journalism School, found that 360 video without narrative can only hold someone’s attention for a short amount of time. “Narrative remains important,” said Pitt. “Humans respond to narrative in a way that they don’t respond to other types of things.”
Samsung and Refinery29 collaborated on a project called “Fashionably Bound”, which captured fashionable people from cities around the world as they shared their favorite local places. This was the first virtual reality piece that Refinery29 produced and the team found that leading the audience through a narrative required a paradigm shift from 16x9 video production. “Virtual reality storytelling as a device is very different,“ said Stone Roberts, VP of programming at Refinery29. “It’s like going from being a sculptor to being an oil painter. They are very different mediums in how the story evolves.” One of the challenges for captured content is that figures cannot come within 6-8 feet of the camera. This puts characters at a distance and makes it harder to get the audience drawn into the moment. “You have to construct a scene and work with your on-camera talent to interact with this, 95 percent of the time, immobile camera,” said Roberts.
Traditional camera angles and film grammar don’t translate to virtual reality. Roberts thinks of every turn of the head as a cut in the film. “You’re really forced to direct with carrots every 15-20 seconds within each story,” said Roberts. “That allows the user to hang on. You need to bait them to be the editor and investigate in new ways.”
Audio is very important to delivering the experience. In the case of the Refinery29 experience, audio is used to tell people where to move their head. “Using sound to incentivize the viewer to turn and interact is the most effective method of storytelling within this medium,” said Roberts.
Still developing: Navigation
Navigation in virtual reality has yet to be standardized. It can be difficult for users to select items, move within a space, and choose from menus. When a viewer is using a head-mounted display (HMD), they don’t have access to the keyboard, mouse or touchscreen that they are used to using for interaction.
Nick Peck gave in depth consideration to user navigation when developing his prototype for a virtual reality Bloomberg terminal. Peck deemed it was too much to say, “Not only do you have to get a hedge fund guy to put a virtual reality headset on his head, but now he has to pick up a video game controller and toy around.” For Peck, the question becomes: “How do we make the transition from ‘here’s what I do today’ to ‘here’s this new experience’ as seamless and easy as possible?” A scene from Milk VR’s Berlin 4 Things Get Weird, published by Refinery29.
guidance given to the viewer needs to be tweaked for each experience. “The beauty of each individual story is that you can tailor it. That learning curve has been really interesting for me,” said Archer. “How much leeway do you want to give the user? In the first Ferguson build for example, we didn’t include breadcrumbs along the street and people were kind of paralyzed by the amount of freedom they had. It’s about finding a sweet spot.”
8. Distribution: platforms and apps
“The big challenge right now is the distribution part of it,” said Adrian Saravia, a digital creative director at Fusion who is leading virtual reality projects. “The fact that it’s so fragmented and each platform or device has its own process and marketplace. Even for us opening accounts in the Oculus space and the Samsung marketplace, that in itself has been interesting to see all of the caveats. The low-hanging fruit for us, as a TV network and such a video-centric digital platform, is video: YouTube,” said Saravia. “The fact that it has the most commercially available experience makes it a lot easier.”
Producers can also create their own applications to host their virtual reality content. An independent application allows companies to maintain more control over how people find and playback their content. Software engineers for these applications will have to contend with the demands of virtual reality on device hardware, bandwidth and file size. “The upper limit of bandwidth and speed is always being pushed by service providers,” said Gavin Teo, principal at Comcast Ventures. Teo noted that there are limits to the CPU and GPU of mobile devices, but there are ways to make great work within those limitations. “Those minimum restraints, you can solve for them with this current generation of hardware if you are a thoughtful and experienced content creator.”
Platform Wars
Display devices like the Oculus, Sony Playstation VR, Gear VR, and the Vive from HTC and Valve differ in the machines that they work with (smartphones, gaming consoles and personal computers), the type of interfaces (camera sensors, trackpads) they use and how the content is distributed.
Google Cardboard and Samsung VR, the two smartphone enabled virtual reality headsets, are the best match for the type of video that media companies can make with 360-degree cameras and the type of linear narrative that media companies are used to developing.
The Oculus is the most hyped upcoming platform, but it requires a computer more powerful than most people have sitting at home or in their laps.
The media created by capturing video content or developing rendered content will be able to be used with all of these systems. But once you get to the state of distributing the content the processor and display device have more limits on what you can do. For example, at this point the Cardboard system isn’t ideal for long content because the holder must be held with your arms, can cause abrasions on the nose, and the storage on a smartphone is limited.
On the narrative side, the HTC Vive is offering a unique interface that will not be available with other systems. This means that the way input is used to move around and make decisions within an experience will be different on that device.
If you want to put your content out on a device like the the Sony Playstation VR or the HTC Vive, you will be entering into a space where most of the audience is consuming games. Some media brands can be used in games, but to make other types of media to use on these devices you will be swimming upstream by competing with game companies that spend a lot on each project, have expertise in marketing to gamers and are incumbents on the devices.
At this point, developers can choose to make content for the platform that their stories fit best. Soon additional information will be available about what demographics and audience sizes are to be found on different platforms and using different display devices.
9. Measure your results
“For traditional video you look a lot at time-spent on the video, unique views or shares. In virtual reality, maybe the metric will be repeat viewings of a single clip,” said Conal Byrne, senior vice president of digital media at Discovery Communications. “You feel like you have missed the other 180° or 270° degrees of the content so you go back and watch it again.”
10. Talent and workforce development
Most of the skills of video producers will translate to virtual reality production. The medium is so new
that there aren’t many experts. However, given that there is so much to be discovered about how to best use virtual reality, there is a lot at stake in hiring producers who know how to test the bounds and find the best use cases for a new medium.
Media companies that haven’t produced animated content before may have to create new positions in order to produce some types of animated content. These animators need knowledge of 3-D modelling and experience developing content for the 3-D gaming engines Unity or Unreal. “If you want to do anything serious, you have to think about the same sort of skill set as an independent game firm,” said Roger Kenny, senior engineer at Dow Jones.
Some companies have been training their programmers to work with game development engines instead of hiring new employees with game development skills. “There are lots of conceptual leaps when you go from browser based to 3-D, but we’ve started to make that shift,” said Ray Velez, chief technology officer of Razorfish.
The opportunity to start a groundbreaking company in the new space of virtual reality has attracted visual effects artists. For many 3-D designers the choice is between taking a job and starting their own companies. “There is a talent war happening right now for the good animators and the good visual effects guys,” said Karl Krantz, who runs the Silicon Valley Virtual Reality meetup. “Right now, this is the birth of a new medium… When you have a shot at being the next Pixar, why work for somebody else?” For the time being, the supply is limited and demand is growing quickly as a wider group of firms want to produce 3-D content.
Visual effects and animation talent is mainly concentrated in the Los Angeles area. Jaunt opened their content studio in Los Angeles, Oculus has an office in Irvine, CA, and NextVR is located in Laguna Beach, CA.
Dakota Powell started the Virtual Reality Lab New York after moving back to New York from London. Powell has found that New York has a lot to offer virtual reality producers. “I think that we get
different strains of virtual reality going,” said Powell. “Because it’s not a one industry town. All of the different industries that are looking to explore virtual reality are doing it here: non-fiction, advertising, all that. This is a great place to be.”
There is a steady supply of new 3-D animation talent coming out of New York City’s art schools and, in some cases, this talent can be found overseas as well.
There are lots
of conceptual
leaps when you
go from browser
based to 3-D
”
“
Right now, this
is the birth of a
new medium...
When you have
a shot at being
the next Pixar,
why work for
somebody else?
”
11. Partnerships in Innovation
Companies have to decide between spending money to create the capacity for making virtual and augmented reality content in-house and working with outside production companies and hardware companies. “There’s innovation that comes from within, which is reinventing your internal process and reinventing the business,” said Francesco Marconi, corporate strategy analyst at the Associated Press. “We’re doing that, but we’re also taking a different perspective, which is to innovate through partnerships and collaborations.”
The majority of virtual and augmented reality hardware and software is still in the testing phase. The only way to start using it and learning about it is to form partnerships. “Our approach when we partner or when we invest in these organizations is not necessarily financially driven, we’re not a venture capitalist,” noted
Marconi. “We identify companies or technologies that can augment or improve the Associated Press’ current capabilities. Our perspective is that the Associated Press, as a major supplier of news, should take a leadership position in identifying these new technologies that can later be used by our members, clients and partners.”
New York Times
In November 2015 The New York Times announced a partnership with Google to deliver a million home delivery subscribers free Cardboard VR viewers. Subscribers were then able to access virtual reality content and immersive video from the Times via the NYT VR app.
The first film released by The Times is “The Displaced”, a documentary following the stories of children displaced by war in South Sudan, eastern Ukraine and Syria.
A screen capture from “The Displaced” featuring 12 year-old Hana, a
Partnerships with academic labs offer industry partners insight into advancements in virtual and augmented reality. Here, a researcher at Columbia University Computer Graphics and User Interface Lab demos Augmented Reality for Maintenance and Repair (ARMAR).
Signs of growing interest in immersive storytelling
To get a sense of how and when consumers will be viewing virtual and augmented reality content we can look at the spread of virtual reality experiences on smartphones, apps and developer kits as well as the sales of analogous technologies like the Microsoft Kinect camera, an accessory for the Xbox.
The success of immersive theater experiences like “Sleep No More,” a site-specific rendition of
Macbeth that has been running in New York since March 2011, points to a wider cultural readiness for immersive storytelling. “At the same time that we continue to be hooked on screens and in frames, there’s a really powerful movement to get rid of the frame,” said Frank Rose, digital culture critic and Senior Fellow at Columbia University’s School of the Arts.
Smartphone usage has increased the most for categories that will incorporate virtual reality content: gaming, video, and sports.
Within the last year many firms working within virtual reality have seen interest double or grow exponentially. “It’s an order of magnitude of greater interest than a year ago,” said Jens Christensen, CEO of Jaunt.
The growth in interest is met with a growth in access, too. Christopher Mims, in an article in the Wall Street Journal, noted that with the help of Cardboard many consumers already have VR viewers in their pocket. They just need a reason to use them. New content from high-profile media outlets provides that reason.
The New York Times, in partnership with Vrse, released “The Displaced”, the first of their recently announced virtual reality film series that introduces viewers to the lives of three refugee children from South Sudan, eastern Ukraine and Syria. In association with this release is the announcement of a VR app The New York Times created with virtual reality
studio IM360, as well as the distribution of 1 million Google Cardboard views to home subscribers.
Over 1 million cardboard virtual reality viewers had been distributed by the Google I/O Android developers conference in late May 2015, double the 500,000 that had been distributed in December 2014. Cardboard was first launched at the Google I/O conference in mid-2014. Interest in events about virtual reality has also been growing. The Silicon Valley Virtual Reality conference has sold out both years of its existence, 2014 and 2015. Attendance went from 400 people in 2014 to 1,500 in 2015.
Virtual reality is receiving significant media attention, including an August 2015 Time magazine cover story and an October 2015 Vanity Fair feature on Mark Zuckerberg that focused on Facebook’s investment in Oculus. It has also gained attention at industry events like CES and academic conferences like SIGGRAPH.