Notes on sequential art in VR
Here's some of the thoughts I shared with my team as Feature Developments Lead on my last project. It was a VR narrative piece that was released on the Oculus Store in June 2018.
The format below wasn't strictly followed but drove a lot of the other design documents I authored throughout the project, including VR storyboards, feature specifications and animatics review notes.
Note: it was copy and pasted from Google Docs, so please excuse the broken formatting.
-----------------------------
[VR comics] is a virtual reality experience where users are immersed in comics narratives. The story is conveyed through scenes acted out by sequential art and animated characters accompanied by fully voiced dialogue, sound effects and dramatic music scores.
As an interpretation of the comics medium, the main mode of storytelling is sequential art, or a series of visual elements arranged in a deliberate sequence. One example of such a visual element are 3D comics panels, or “Live Windows”. These Live Windows can be accompanied by speech bubble and onomatopoeia visual effects which materialize in the VR space to deliver the comics experience's essential literary element.
Users are invited to peer into the diorama-like Live Windows and at times even literally step into the characters' world to experience the scene from a first-person point of view.
As the user views each scene, they will be unlocked and made available in a a scene select hub. The user will thereafter be able to experience any unlocked scene at their convenience.
While [VR comics] borrows conventions from cinema and animation, it is not a VR short film like Henry or Dear Angelica where the user must sit and watch the entire production from front to back. Linear animated sequences will be leveraged to express things that only motion can express, but they are only one element of the experience.
Rather than relying entirely on temporal pacing, pace can also be established by the speed by which a user navigates their gaze across compositions purposefully arranged in the VR space. This is similar to how a comics artist controls pacing by arranging their panels on a page. Consider this example:
An image like the one above may be interpreted as a single moment in time.
However, we can guide the audience’s sense of time by introducing visual elements which are consumed sequentially across space. The panel above accomplishes this using dialogue bubbles.
Another way we might illustrate the concept of space as time would be to imagine a tourist making their way through a Hindu temple or an ornate cathedral whose bas reliefs and stained glass windows come to life, depicting a sequence of events:
Characters and environments can be represented as 3D models illustratively rendered in comics's hallmark monochrome aesthetic. If a character were to stand perfectly still the image would be indistinguishable from a hand-rendered ink illustration. The narrative should feel fluid, with visual and aural cues guiding the user's attention through living images artistically arranged in the VR space.
At times the user should have the experience of reading an animated comics whose panels fill the VR space; at times the user should also feel as though they have literally stepped into the comics world. Regardless, the experience of traversing between these two spaces must feel clear and deliberate yet seamless and unobtrusive.
In this space the user interacts with the comics medium itself rather than the world represented in the narrative. These interactions are abstract rather than emulative, such as advancing the narrative’s progress by turning a virtual page, choosing between two branches in the narrative, or bookmarking the user’s progress. The space must be distinct from others by a characteristic aesthetic -- a monochrome post-process, for instance.
The comics space is the world which the comics characters occupy. The user can enter this space by being enveloped by a Live Window for example. This space is distinguished by the presence of characters and environments from the narrative. The comics space should also be somehow visually distinct, if not only subtly, from the user space: a subtle duotone post-process, for instance.
The experience of stepping into the comics
Vivid three-hundred sixty degree 3D environments
Intuitively connected comics and cinematic narrative elements.
comics / sequential art storytelling
Narrative delivered with sequential art compositions
comics-like reading experience with speech bubbles & onomatopoeia
Narrative rhythm is directed by purposeful, sequential arrangement of compositions in VR space.
Content navigation supported by utilitarian interaction
Intuitive motion controller input
Scene playback features (pause, skip)
Replay value added by allowing users to experience the content in different ways
Discoverable content spread across the VR space during playback.
Additional modes of playback, experience
User gaze dependent playback elements
Branching content, optional scenes
Narrative supported by the animation of 3D subjects
Deliberate, authored pacing
Controlled viewing experience
Narrative rhythm is established by carefully timed animation sequences
Immersive elements enhanced by novel interaction
Character acknowledgment of user presence by reacting to user gaze
Grasping and moving panels and speech bubbles with motion controllers
Grasping and moving small props in the comics space
Non-features (“Things We Won’t Do”)
Exposition not reflected in the source material
Original material should support the essential experience in a meaningful way (storytelling grammars) or be warranted by necessity of the platform (VR acclimation grammars).
Animation for animation’s sake
Animation should be used when direction wishes to express something that only animation can express.
*Panels from Scott McCloud’s Understanding Comics are represented here for reference only. It’s a great book, and anyone with interest in sequential art should go buy it along with McCloud’s other texts.