10 May The Senua Trailer in VR
To start off, an important note: To be clear the release of the Senua Trailer in VR doesn’t mean that Hellblade will be a VR experience. We’re testing things out in VR and it makes sense for us to use the assets that we had available to us while experimenting.
At GDC this year, at the Epic keynote, we released a real-time video of a scene from Hellblade which you can see here:
In a cool twist, we drew back the curtain mid-trailer, to reveal that it was being performed live by Melina on stage. She was being performance captured live on stage and rendered in real-time in Unreal Engine 4. In case you missed it, here is how we did it:
We worked in partnership with Epic Games, Cubic Motion, 3Lateral & iKinema to make this happen and it really was a serendipitous coming together of minds wanting to do something cool led by Kim Libreri, CTO of Epic.
While performance capture has been done before in films like Avatar and our very own Heavenly Sword, it has never been done live on a character of this kind of fidelity before. At the end of the demo I suggested that this would be especially pertinent to VR.
So while the Hellblade team is small, there are 100 people at Ninja Theory. A handful of these have been experimenting with VR. After GDC, we now had a great looking scene and we wanted to get it up and running in VR. Today, we are in a position to share it with you as a stereoscopic 360° video form:
Getting it working in VR required quite a bit of work. The main challenge was optimising it to run at 90fps without losing too much in the way of visual fidelity. It’s now mostly hitting 90fps with some exceptions so we’re not quite ready to release the PC build of the scene to the wild.
In the original scene, the cameras were captured on set and tweaked in Maya but, as you would expect, in VR it felt quite nauseating to quite a few people. So the cameras were re-authored in matinee from scratch.
It is possible to eliminate nausea by having static cameras but we felt that you also eliminate part of the drama by doing so. So what we have is a camera path that tries to preserve the intent of the original camera while avoiding some of the main pitfalls of VR-based camera motion, namely accelerations in movement or rotation. Anyway judge for yourself and let us know how you feel about it on twitter @NinjaTheory.
What was really remarkable is the feeling of Senua’s presence in VR. When you can lean in and see individual eye lashes up close, it is stunning. But when she looks into your eyes, most people experience an eerie feeling of self-consciousness which is a clear sign that your brain believes that she is real. However that illusion is shattered when you move your head away from her and she looks into blank space.
To get around this, we added head and eye tracking to Senua so that she follows you when she is talking to camera. Her head moves independently from her eyes, just as it would in a real person, and her eyes flit between your own eyes and mouth procedurally, giving the impression that she is looking at your face.
We have been creating rich worlds with strong characters in real-time for many years now and VR is a great medium to expand on our heritage into the future, especially in areas where performance capture, procedural animation and interactivity intersect.
We are pretty excited about bringing that future closer!