28 Jun Perception & Creativity
A few months ago, I thought about writing a blog post about designing characters and the importance of silhouette, colour and movement but I felt like I would be retreading ground that has been covered elsewhere by countless artists. But now, I do feel compelled to return to it from a different perspective as my understanding of human perception has shifted.
We have been working with Professor Paul Fletcher, Psychiatrist and Professor of Health Neuroscience, from Cambridge University for a few months now on Hellblade. One of the first things Paul did was talk to the entire team about the latest understanding of the brain when it comes to perceiving the world. While this article represents my own views and understanding, much of it is informed through my exposure to Paul’s ideas and knowledge.
In particular, Paul suggested a book called The Predictive Mind by Jakob Hohwy. It isn’t a particularly easy read but the concept behind it is very clear. It theorises that our brain is a predictive machine that uses prior knowledge to statistically predict and model what we are seeing. In effect our brains filter the noisy input data of our senses to extract meaning using our prior experience, memories and knowledge and this results in what we “see”.
The simplicity of the idea belies its significance as it gives a deep insight in how we each understand the world from our own unique perspective. It explains the age-old view that “reality” is fluid, contextual, subjective and one that can be misdirected.
Recently I came across this post which is doing the rounds. It describes how neural networks using prior expectations can see things that are not there, somewhat akin to dreaming.
The images are stunning in their beauty but what they say is about perception is fascinating: the neural net is creating abstract meaning from random noise and, in the process, creating images we could consider to be art.
I sent this to Paul who noted that it reminded him of the paintings of Louis Wain, an artist who suffered from psychosis. Here we see his paintings of a cat at various stages of psychosis:
The images of the neural nets also reminded me of the paintings of Francis Bacon whose portraits captured people from several perspective at once as well as incorporating a sense of movement:
It occurred to me that the process behind the neural net pictures, the predictive mind theory and many abstract artists may be connected and might give us some understanding on how we see the world around us.
Curious about what was going through Francis Bacon’s mind, I watched an in-depth interview with him. It is a fairly long interview but it is fascinating:
The interviewer presses Francis to explain the meaning behind his paintings and the message they convey. He responds repeatedly, that he wants to capture the subject as directly, as raw as he can and that there is no moral or literal meaning to his work. He seems to have been looking for a truthful representation of his subjects that is clearer to him than the literal view of them.
This makes sense to how I see and judge characters. When I look at a character critically, I look at it from several abstract perspectives each of which I judge separately: silhouette, colour, contrast, texture, lines, curves, movement. Mostly I do this unconsciously but a good artist can very deliberately design and justify these elements to create a whole impression. While the process for an artist can be arduous, the impression from the viewer is immediate: “looks cool!’ or “looks boring, derivative”. I think that creating interesting characters or worlds, requires a deeper understanding and connection with our ability to see in abstract ways.
I believe we all naturally see the world abstractly before seeing it literally. This is more than a philosophical musing. Consider that in our waking lives we see remarkably little of the world around us:
- Our focus point in our retina sees only a tiny fraction of what’s in front of us, probably 1% or so, the rest being a blur. So that means our brain fills in the gaps to give us what seems like a complete picture.
- The time it takes for light to hit our retina to us “seeing” something is around 100ms which in the context of a typical 30fps game, is a 3 frame delay. This means our brain features a form of temporal correction and prediction of movement to make this lag appear to disappear.
- Every time we move our eyes, our visual processing shuts down giving us temporary blindness so that we don’t see streaking images as our eyes move. The blindness is estimated to be around 10ms or 0.3 frames. This could add up to about 30-45 minutes of blindness a day yet we feel that the world is a continuous experience.
What we literally see or hear is informed by prior experience. There are simple illusions that demonstrate this such as the McGurk effect seen here:
The magic of our brain makes us believe we see an objective world that appears complete, but this simply cannot be the case based on such little information. The inescapable conclusion is that most of what we see, hear and experience is in fact a simulation created by our brain without which, we simply could not function and make sense of the world.
A powerful demonstration of our brains ability to simulate worlds and experiences happens every night when we sleep. Based on no sensory inputs, we create and inhabit worlds that can feel just as rich and real as waking life. But it does seem that without the regular sensory inputs to anchor the simulation, the mind’s eye is free to meander and take us on bizarre fantastical journeys. Perhaps during dreams, we shut down the higher level, resource-hungry, literal functions of our brains and live in the abstract space entirely.
Whereas I suspected that we do not see the world in pictures, I am now convinced that we see the world constructed from layers of abstract information: silhouette, depth, colour, contrast, movement, orientation, time, edges, corners, recursive patterns and so on, each representing a dimension space of understanding. Combined with prior memories, these form impressions.
Impressions in this sense of the word are unique signatures, like a snowflake, a hierarchical construct that links the mass of multi-dimensional data into a cohesive form that matches our prior experience: what we see and feel.
When we see or remember someone or something, we sieve through our brains for the impression that best encapsulates the mass of abstract data and from there reconstruct what we see based on our memories, albeit not in an accurately manner, but closely enough.
From these impressions we apply higher level meaning and concepts – words in fact – to create a literal space of understanding.
I wonder if artists, film-makers, writers, designers, musicians, and scientists attempt to capture deep-seated impressions from their lives and try to translate these as directly as possible from the abstract space. If successful, that abstract representation resonates with the viewer’s experience: the art appears truthful to us, giving us specific impressions and we recognise these as effortlessly as our ability to make sense of the chaos in our world.
To make sense of the world, our mind is not interested in predictable impressions. We are optimized and tuned to ignore the mundane and predictable in order to focus on new interesting and unusual impressions. Originality therefore is ever evolving and creativity must find new ways of drawing attention away from the known.
So to take things back to characters, the literal space gives us very little room for us to delve deeper into our own understanding of a character. It feels superficial, empty, predictable, and doesn’t fire the deeper layers of our brain. There is no deeper concepts or impression to focus on:
Characters or portraits that delve below the surface and take a more abstract form can be endlessly fascinating as we delve into the abstract space to find a point of view for understanding them:
(“Saturn devouring his son” by Goya)
(“Pale Man” from Pan’s Labyrinth)
(Artist: Jonas Burgert)
(Artist: H. R. Giger)
These characters play with things that are familiar: depth, movement, orientation, location, body parts, silhouettes, texture, in unexpected ways to draw attention and illicit a deeper emotional impression from us.
Perhaps the creative process involves delving deeper into the abstract space, where many concepts have no associated words to describe them. Associations between abstract concepts can form and combine free from the rigid constructs of our language and perceived reality. Most of the resultant combinations feel meaningless but some connect with a medley of our prior experiences to affect us with new impressions that are hard to convey in words.
Once a new impression has been found, an artist will use their skill, sometimes obsessively, to convey it by visualizing the abstract concepts and in some cases will produce works of great originality and powerful resonance. Sometimes, the art doesn’t resonate with our own life experience and appears meaningless but sometimes it fires up our own impressions and feels like a revelation.
Our accumulated lifetime of impressions may explain why we appreciate abstract art more so as we get older as we have a richer tapestry to draw from whereas as kids we prefer the more basic, predictable and repetitive portrayals of cartoons. It may explain why some characters are boring while others are deeply compelling. A compelling character makes us want to understand them, to dig deeper into their story and origin.
I had not expected neuroscience, neural nets and art to be natural bedfellows and but perhaps it is not surprising given they are all derived from the mysteries of the mind. This framework for creativity and perception does seem to make sense to me and is relevant to the next phase of development we are embarking on: telling Senua’s story through her eyes based on her memories.
In Senua’s case, I see her as someone who has unconsciously rejected the world she once believed in and has delved deeply into an abstract state of mind where her experiences appear chaotic and symbolic, informed by her past memories and experiences. When she sees something it can fire up impressions based on her memories that in turn reconstruct hallucinatory visuals from which we can derive meaning.
If we can convey specific impressions, based on her memories, through symbolic and abstract imagery, there may be a chance to connect, understand and empathise with her at a deeper level than traditional expository scenes can achieve.
To put this into practice, we are going to start developing the story from Senua’s point of view in abstract ways. At the same time, we will start experimenting with ways to deconstruct visual images into separate components: colour, contrast, depth and so forth. Based on Professor Paul’s experience in the field, and from talking to people who have experience hallucinations, we may be able to go some way in approximating the strange world of abstract perception.
If creativity is about exploration of the unknown, then this could be an interesting path to follow. We will keep you posted if and when we start getting results.