Synaesthesia in Storytelling
You see video and you hear audio. But have you ever seen sound, or heard video? This is the concept of synaesthesia, where constant stimulation of one sense can lead it to become intertwined with sensory responses to another sense. Some people (though an extremely few amount) say they associate a certain smell when they see a color, or a slight tone when they see a certain image. It's an extremely interesting phenomenon that can be used in storytelling or any kind of media for an incredible effect.
Most of us have probably never experienced it before, but in a small sense you could say synaesthetic elements have been around and experienced for a long time. Music videos all have images that correspond to a certain sound (such as a dance move to a certain beat), and if the same movement keeps being interconnected to the same sound, your brain will begin to correspond that sound to that visual moment. Though not a complete synaesthetic experience in the phenomenon sense, some music videos can trigger a minimal synaesthetic experience. One of my favorite videos for this is Daft Punk's "Around the World":
From a storytelling perspective, synaesthesia can be an amazing tool to make the interactivity more significant. A story benefits from being memorable and a synaesthetic element makes the experience intimately memorable for the person experiencing it.
Most of us have probably never experienced it before, but in a small sense you could say synaesthetic elements have been around and experienced for a long time. Music videos all have images that correspond to a certain sound (such as a dance move to a certain beat), and if the same movement keeps being interconnected to the same sound, your brain will begin to correspond that sound to that visual moment. Though not a complete synaesthetic experience in the phenomenon sense, some music videos can trigger a minimal synaesthetic experience. One of my favorite videos for this is Daft Punk's "Around the World":
daft punk- around the world from Victor Carvalho on Vimeo.
From a storytelling perspective, synaesthesia can be an amazing tool to make the interactivity more significant. A story benefits from being memorable and a synaesthetic element makes the experience intimately memorable for the person experiencing it.
Comments
There are many well known artists, musicians and thinkers where this phenomenon is intricately related to their work, since that is the way they perceive their world. From Wassily Kandinsky or Olivier Messiaen to Goethe, Nabokov, Aphex Twin, Duke Ellington and Nikolai Rimsky-Korsakov etc.
A classic example in the movies, where as you suggest, is used as a tool of engagement or interactivity is the early Disney animation Fantasia, an animated feature with no words, only sound and its relationship with color and music, a great risk in this medium, became a great commercial success, perhaps because so many people related to the experience