Solved: Synesthesia and Digital Art Critical Essay

Essay 1: Synesthesia and Digital Art

Synesthesia is an exceptional neurological condition that connects two senses in the brain. When people with this condition experience something with one of their thoughts, another corresponds by picking it up. Approximately only 4% of the world’s population has the synesthesia gene, and the odds are higher with artists at 25% per 100 artists (Jeong & Kim, 2019). In this way, digital artists translate perception modalities from one sensory domain to another due to specific properties of the synesthetic experience. Specifically, it appears to them as a pre-verbal direct experience of the environment. Therefore, synesthesia is a persistent and persuasive trope in the digital arts. It provides some enticing parallels with the effects and techniques of audiovisual practice. Ryoichi Kurokawa is a digital artist who describes Synesthesia through the quality of time-based, interactive, and multimedia artwork.

Ryoichi Kurokawa is a Japanese artist. For the last 15 years, he has been a forerunner of audiovisual art. He works in different mediums to unite image and sound into a new language. Currently working and living in Berlin, he progressively explores the symbiosis between nature and technology and chaos and order. Furthermore, he is the recipient of the 2010 Ars Electronica Golden Nica prize in the category of Digital Sound and Music Art (Hawkins, n.d.). Some of Kurokawa’s ongoing projects include the New Realities Exhibition in Barcelona and Alpha-ville LIVE at BFI with Sonic Cinema. For the latter, the artist has released three limited artworks, which he operates as a composer of image and sound. He pairs noise pulses with intense, strobe-like glimpses of practically-recognizable imagery. While seemingly chaotic, these works conceal complex fundamental structures and patterns, ultimately revealing themselves as sensibly constructed explorations of movement and sound.

Based on his works, Ryoichi Kurokawa employs different media, including live performances, video, recording, and installation, to generate unique audio-visual sound pieces. He has been presenting and pioneering this globally since 1999. His works involve architecturally reconstructing the audiovisual phenomenon. In this way, he composes time to sculpture with the digitally generated structures and field recordings (“Ryoichi Kurokawa,” n.d.). Kurokawa considers the image and sound like a single unit and terms his work as time-based sculptures. His works are compositions in the sense that they consist of symphonies of sound recorded, produced, as well as imagined. Apart from that, he combines action with computer-generated aesthetics and video material to change how spectators view the familiar.

Furthermore, Kurokawa presents and invents audiovisual language where simplicity and complexity combine and alternate in fascinating synthesis. He achieves whether recordings in a field in amalgamation with computer-produced structures, such as glitch minimalism, co-occur in harmony in an unharmonious frightening world of destruction and war. In essence, this is also possible whether the recordings of waterfalls form an almost reverential and spiritual stillness around the viewer or simultaneously obliterate into white noise (“Ryoichi Kurokawa,” n.d.). Kurokawa, in a sense, is a digital artist who can be considered an exemplary trendsetter in his field. He can further be regarded as an aesthetic-defining pioneer who points the way for others. The astounding diversity of his assessments highlights remarkable productivity that appears to flow without end. More so, he is a genre-transcending digital artist who progressively transforms and confounds stereotyped notions regarding the depiction of electronic image expression.

Another work by Kurokawa is “unfold.” It is a large principal piece and audio-video installation that dominated the cinematic space. It consisted of three imposing monitors that together shocked the audience’s senses with an endless display of color and uncomfortable static noise. The film plays with all concepts of scale and design, contrasting monochromatic and colorful, loud and large, and silent and small. Geometric 3D shapes, movement, dots, and lines appeared through a pointillist cloud of stardust, turning into mountainous space and deep ravines (Hawkins, n.d.). While others took full advantage of the intensity of the provided film, in some cases, it was very jarring to viewers who could not remain for the entire screening period.

Kurokawa describes “unfold” as a constrained surface that is formed as a pure synaesthetic experience. It is a visual masterpiece that explores light and color, with radial and linear gradients pouring in light segments from one screen into the other, blending and mixing them. The reliable descriptions prompted sensations of taste in the audience, which is not unusual because the color red is known for provoking thirst and hunger. By using the power of intense shades, Kurokawa wants the viewers to experience a dynamic spectrum of the senses (Hawkins, n.d.). In “unfold,” the spotlight moved directly toward the sound, substituting the video footage with still prints, compelling the viewers to engage in a more one-on-one expedition into the space soundscape using headphones. Kurokawa employed scientific logic to demonstrate his interpretation of how one can transform pure energy from outer space into measurable visuals and sounds. He achieved this by applying raw experimental data as the point of departure.

In an interview with Kirsten Hawkins, Kurokawa provided various responses concerning his works as a digital artist. For example, Kurokawa stated that his works offer phenomena that could be shared concerning synaesthesia. Additionally, he indicated that he uses constrained and unfolded surface installations to give viewers surprise and pleasure. In the interview, he further specified that for the usage of projection and display, he attempts to liberate himself from conventional methods and find new possibilities within the frame boundaries. At the same time, Kurokawa argued that he did not intend to compare synaesthesia and astronomy in his work. Instead, with “unfold,” he used astrophysics data, and simultaneously, the piece offers a synaesthetic experience (Hawkins, n.d.). In this regard, Kurokawa asserts that providing such an experience is critical to the idea of most of his works. Nevertheless, the focus of the constrained surface is purely on this neurological occurrence by employing only color gradient as pure light. This is because it disregards all aspects that convey a meaning that would produce a sharper experience of chromesthesia.

Furthermore, in the interview, Kurokawa noted that he is not a musician but aspires to compose by designing time in space. Apart from that, sound is an essential element for him since most of his works are audio-visual pieces. Concerning his ground. Alt work, Kurokawa responded that it has both visual and musical elements; otherwise, the visual and sonic components would be disconnected. Regarding his Octfalls project, which has compelling moving images of waterfalls, Kurokawa responded that space is very momentous in his installation, and it impacts the piece a lot. In this way, the characters of unusual space might offer a good outcome (Hawkins, n.d.). Nevertheless, Kurokawa says that he does not expect it to provide any additional value. A project displayed in different characters in space could easily give a more distinct impression. Kurokawa argues that some of his plans do not require much space linked to knowledge. However, some should be exhibited under specific well-designed conditions to ensure the art piece operates as efficiently as intended.

Unlike psychology which considers synesthesia to be a natural reaction that is an innate endowment of people. It presents a visual voice to humanity’s inexpressible, complex, and hidden experience (Van Leeuwen, 2016). Kurokawa demonstrates that artists can use synesthesia to combine different digital mediums and art. In this way, they can show their creative work, stimulating the audience’s senses and offering a perfect synesthesia experience.

Faced with extreme images, people can, for the most part, close their eyes. Nevertheless, the sound is different. Whether one likes it or not, it permeates the body. It can irritate or soothe. It can also bore the listeners or bombard them. Despite the outcome, its vibrations are enveloping and unavoidable (Balsom, 2019). Kurokawa demonstrates that sound is fundamentally crucial to artists’ engagement with moving images, especially in the contemporary moment. In digital art, although the works of synesthetes have been categorized as abstract, Kurokawa demonstrates that the artists perceive what they have constructed as absolutely realistic. Without a doubt, Kurokawa has further shown that even though they are the only ones who can see their vision, synesthetes express what they experience. Kurokawa, like many synesthetes, works with sound because it has vibrant, moving visuals and not because it is a more generally experienced category of synesthesia.

A major misconstruction is that synesthetes operate from only one of their joined perceptions. As an illustration, a sound will generate an image of one or more moving colored shapes (Balsom, 2019). Nevertheless, as Kurokawa demonstrates, the experience by synesthetes can as well be a mixture of different perceptions that transpire at the same time. They are then combined to become one experience; in the same manner, the front door, windows, and roof become the image of the house. Artwork, such as the ones by Kurokawa, which feature colored and moving shapes, are often generated from sensory triggers, such as smell, touch, and music. Apart from that, they can be produced from non-sensory triggers, such as the personalities of shapes and/or colors of letters. Irrespective of the triggers or triggers utilized, the commonalities of the form of ways observed in photisms seem to be the same.

Conclusion

All in all, those with the gift aspire to share it with the world, which is where using synesthesia to make art comes in. Without a doubt, Kurokawa has demonstrated that synesthesia makes artists more creative. This is because while others attempt to capture their visual insight of time in painting, those with synesthesia paint the abstract image they perceive from hearing a sound. As an artist, Kurokawa has also demonstrated that the results of an artwork created with synesthesia are amazing because, apart from the more vibrant colors, the concepts become deeper than before.

References

Balsom, E. (2019). Cinema as synaesthesia: How to listen to artists’ films. Retrieved 16 October 2019, from https://frieze.com/article/cinema-synaesthesia-how-listen-artists-films.

Hawkins, K. (n.d.). Making sense of it all. An interview with Ryoichi Kurokawa Digicult: Digital Art, Design and Culture. Retrieved from http://digicult.it/news/making-sense-interview-ryoichi-kurokawa/.

Jeong, W. U., & Kim, S. H. (2019). Synesthesia visualization of music waveform:‘kinetic lighting for music visualization’. International Journal of Asia Digital Art and Design Association, 23(2), 22-27.

Ryoichi Kurokawa. (n.d.). Retrieved from https://www.seditionart.com/ryoichi-kurokawa.

Van Leeuwen, T. (2016). A social semiotic theory of synesthesia?-A discussion paper. HERMES-Journal of Language and Communication in Business, (55), 105-119.

Essay 2: Synaesthesia in Digital Art

Synaesthesia was originally derived from the Greek words σύν (syn, meaning union) and αἴσθησις (aesthesis, meaning sensation)(Best, 2003). In the traditional meaning, it is a term of psychology, which means a sensory stimulation, or the cognitive approach spontaneously causes another perception and cognition. I learned synaesthesia from a Chinese singer. His creation experience is a classic example of synaesthesia; most of his songs are created by the different scenes that he has seen. He said that Tuscany is his favorite place because his mind was full of melody when he looked at those sceneries. Thus, synaesthesia is an innate endowment for humans. For example, when people hear some sounds, their first thought may be a specific color; or when people recall the past, their first thought can be a smell.

With the thriving of digital technology nowadays, there are increasing artworks gradually digitizing. Digital arts emphasize the interaction between the artwork and the audience, creating an immersive experience. Synaesthesia is right on this requirement because it challenges the stereotypes of perception. Some artists use the concept of synaesthesia to present their work in digital art, which visualizes sounds and music or combines art and virtual technology. Synaesthesia in digital art provides a better sensory experience for the audience.

Ryoichi Kurokawa is a Japanese new media artist that mainly works on synaesthesia experience. He said that he wants to provide an interactive and immersive experience for the audience in his work that perfectly combines audio and visual. He uses various digital mediums such as video, device, recording, and live shows to create audiovisual works, stimulating the audience’s senses (Hawkins, n.d.).

Ryoichi Kurokawa tried many of his audiovisual works. He has two typical digital artworks, <Rheo: 5 horizons> and <Octofalls>. These are great examples of how audiovisual artworks can create a synaesthesia experience for the audience in digital art.

<Rheo: 5 horizons> expresses the change of objects in the physical state. The things of this world are flowing and shifty, but in daily life, people are hard to feel the change in objects and spacetime. He takes advantage of this fact, which combined sounds and scenes to show the movement, direction, speed, form, and color of objects and spacetime, which made audiences generate synaesthesia and meditation experiences. He tried to present that digital art can stimulate viewers’ senses to provide a synaesthesia experience for them (“RYOICHI KUROKAWA,” 2010).

For <Octofalls>, it stands for one of a thousand ways to defeat entropy. Ryoichi Kurokawa recorded many sounds from nature and extracted the waterfall sound, representing the natural attribute. Then he combined those sounds and eight huge HD screens that provided a synaesthesia experience and deeper immersion for visitors as if they were in nature (“RYOICHI KUROKAWA,” 2011).

As mentioned above, the concept of synaesthesia in digital art differs from its psychology theory. In psychology, synaesthesia generally considers a natural response that is an innate endowment for humans. However, in digital art, artists combine different art and digital mediums to show their creative works, stimulating audiences’ senses and providing a perfect synaesthesia experience.

Reference List

Best, S. (2003). synaesthesia (1) | The Chicago School of Media Theory. Retrieved 26 September 2019, from https://lucian.uchicago.edu/blogs/mediatheory/keywords/synaesthesia-1/

Hawkins, K. Making sense of it all. An interview with Ryoichi Kurokawa • Digicult | Digital Art, Design and Culture. Retrieved 26 September 2019, from http://digicult.it/news/making-sense-interview-ryoichi-kurokawa/

RYOICHI KUROKAWA. (2010). Retrieved 26 September 2019, from http://www.ryoichikurokawa.com/project/octfalls.html

RYOICHI KUROKAWA. (2011). Retrieved 26 September 2019, from http://www.ryoichikurokawa.com/project/octfalls.html

Kohli, K. (2017). Smelling the color is possible in Synaesthesia- A rare case reported [Image]. Retrieved from https://speciality.medicaldialogues.in/smelling-the-colour-is-possible-in-synaesthesia-a-rare-case-reported/

Nance, R. (2013). Octofalls – Sound and Visual Waterfall Installation [Image]. Retrieved from http://5thingsilearnedtoday.com/blog/2013/5/13/octofalls-sound-and-visual-wa