New research shows how brain-computer interaction is changing cinema
by Polina Zioga
Shutterstock/TatianaShepeleva
Over the past few years, we have seen the extraordinary development of neural prosthetic technologies that can replace or enhance functions of our central nervous system. For example, devices like Brain-Computer Interfaces (BCIs) allow the direct communication of the brain with a computer. The most common technique applied in these devices, is Electroencephalography (EEG) – a recording of the electrical activity along the scalp.
These technologies are mainly used in health, but our new research shows how they are changing the future of cinema too.
This is no coincidence. Artists have been among the pioneers in the use of these technologies, developing creative applications since their first emergence in the 1960s. Early examples include Music For Solo Performer (1965) by Alvin Lucier, which is considered the first performance using EEG technology. Interactive artworks, like the Brainwave Drawings (1972) by Nina Sobell and installations like Alpha Garden (1973) by Jaqueline Humbert also illustrate how the art world paved the way.
Interactive filmmaking
During the same period, the first interactive film was presented. The comedy Kinoautomat (1967), which was created in Czechoslovakia, allowed the audience to vote on what should happen next by pressing buttons. Since then, acclaimed filmmakers, such as Peter Greenaway, have been advocating the new possibilities of interactive technologies in cinema.
More recently, the film industry is showing interest in emerging technologies, such as Virtual Reality (VR). A milestone in this direction was the special award presented by the Academy of Motion Picture Arts and Sciences Board in 2017 to Carne y Arena directed by Alejandro G. Iñárritu. Carne y Arena is a VR installation which was said to be opening “new doors of cinematic perception”. This follows on from the work of an increasing number of festivals (like Berlinale and the Venice Film Festival), filmmakers and researchers who are investigating the potential of using new interactive technologies in cinema.
Among the most recent innovations are new wireless Brain-Computer Interfaces, which are now available in the market as low cost headsets. They are already used in computer games and the arts, but more recently they have been applied in interactive filmmaking as well. For example, Hollywood studios, like Universal and 20th Century Fox have released interactive versions of their films, where the spectator can control key moments of the plot with the use of a BCI headset.
Multi-brain interaction
Our research sheds new light on how our brains can control a film or live cinema event, not just for home entertainment but also in a cinema theatre. And it shows how this can be used to bring to the audience a new, engaging and collective experience.
More specifically, we developed a new system that allows multiple brains to interact. This system was used in a new live cinema event that we created for this purpose and presented at the CCA: Centre for Contemporary Arts in Glasgow.
Illustrative example: The passive multi-brain EEG-based BCI system. Zioga et al. 2016 (Images of human heads originally designed by Freepik).
For the first time, the technology enabled one performer and two members of the audience to interact simultaneously. Using their passive brain-activity – separately or jointly – they were able to control aspects of the live projected film. Their transition from relaxed to more alert cognitive states was visualised as a shift from colder to warmer tints. This created the constantly changing colour of the live projections and set the overall atmosphere of the narrative.
The event was also a neuroscience experiment in a real-life environment with the presence of public audience. This allowed us to obtain data from the participants, which have provided us with important results. They showed they were able to understand which parts of the event were controlled by their brain-activity and how. These particular scenes made a special impression on them. And at the same time, their attention and emotional engagement were also increased, while having the feeling of being “connected”.
‘Enheduanna―A Manifesto of Falling’ Live Brain-Computer Cinema Performance at CCA: Centre for Contemporary Arts Glasgow. Polina Zioga/Catherine M. Weir.
Cinema’s new horizon
Our experiment provides new tools and methods for creating interactive films with the use of the brain-activity of the spectators. It proves how the use of brain-computer interaction in cinema can enhance the audience’s perception and engagement. And even more, it opens a new horizon of possibilities. Audiences in the future will be empowered to immerse themselves and collectively control a film through their combined brain-activity.
Together with studies looking at the effect of films on the spectators’ brain-activity (neurocinematics), these new possibilities will push further the motion picture arts and sciences. They will also advance our understanding of how we collectively engage, collaborate and compete in emotive environments and situations.
Polina Zioga is Director of the Interactive Filmmaking Lab at Staffordshire University.
This article was originally published on The Conversation, a content partner of Sci Fi Generation.
So I guess this will be the first published post of the Kinopto blog-there do exist some unpublished musings on the intricacies of setting up a tech business, giving up the day job and raising finance but those are for another time, for now I wanted to focus on the reason for Kinopto's existence-film.
More specifically, Indie film. Whilst, again, the technical details of our system are beyond the scope of this post, Hollywood in general, and DCI specifications specifically, prevent any great level of innovation in the cinema technology market.
That's not to say that technical advances aren't made, just that they are made within the confines of DCI and therefore limited in what they can achieve. In reality, much of the innovation is DRIVEN by DCI. The buzz words in the industry at the moment are '4K', 'High Frame Rate' and 'Integrated Media Block'. All of these are solid technological advancements, but will they fundamentally change the industry for the better? I would argue not. I would also argue that true innovation should be for the benefit of an industry, and ultimately society, as a whole and not serve the interests of one section to the detriment, financially or otherwise, of the other.
Regular technological improvements such as this benefit equipment manufacturers. It is the model which has built Samsung, Apple et al into some of the most valuable companies in the world. Is the iPad 4 radically different from the iPad 3? No, but they still sold 3 million of them in the first 3 days. These minor improvements create an upgrade culture in consumers which is, in the most part, mutually acceptable. Consumers are happy as they feel, or are seen to be, at the cutting edge. Of course, the manufacturers are happy as they have a repeating, cyclical, constantly replenished customer base.
However, whilst it is this model that cinema technology innovation is following, I am dubious as to whether the pattern seen in consumer electronics can, and will, be successfully repeated and that is because it is not the feature set of the iPad 4 that 3 million people rushed out to buy, it is the package as a whole. Whilst a consumer will proudly say to their peers that they own the 'latest' iPad, will they proclaim with the same enthusiasm that they attend 4K cinema screenings, or that they just saw High Frame Rate? No, they won't. Cinema technology isn't as sexy as consumer electronics, and the benefits of the package as a whole are less clearly defined.
In addition to this, innovators in cinema technology first have to convince the middle-man, the cinema, that this upgrade is beneficial to them, that they will get a return on their investment. Business-to-Business is very different from Business-to-Consumer. Consumers measure their Return on Investment in an intangible way, in 'happiness', 'enjoyment', 'cool-ness'. Businesses, in this case Cinemas, measure it in Pounds, Shillings and Pence.
For me, technical innovation in Cinema shouldn't be focussing on minor advancements in feature sets, but looking at how we can give the end user, the audience, increased feelings of 'happiness', 'enjoyment' and 'cool-ness' to keep them coming back.
In my next post I'm going to look at what we can learn from nearly every other form of media, from Music, to Books to Journalism where it is indie, and self publishing, that is driving change. Isn't it about time cinema caught up?
----
We are going to be migrating this blog to www.kinopto.com/blog very soon....so keep and eye out over there-and don't forget to follow us on Twitter @kinopto
