When I was first approached to collaborate on OLED, particularly the Cinematic Sound OLED panel, I was more curious about what it was than the type of work I had to do or the deadline. Soon enough, I was able to find this out through the documents that came with the project proposal.
Cinematic Sound OLED uses the screen itself as the diaphragm of a speaker to create sounds directly, as opposed to employing separate speakers. The structure of LCD panels, the favored system in existing TVs, have too many layers of optical sheets that make it too thick and stiff to act as a diaphragm. But the self-illuminating technology of OLED allows the single-panel to sufficiently emit light, which provides an extremely slim design and flexibility. This enables it to function as a diaphragm and makes Cinematic Sound OLED technology possible.
While existing TVs position the speakers on the back lower part of the panels, with the sound reflected back from the floor to the viewers, you can hear the sound directly from the front surface of the TVs with Cinematic Sound OLED technology. High pitch sound is distorted by obstacles and deflection. Each time it encounters an obstacle and deflects off of it, much of the energy is lost and the sound becomes dull. Cinematic Sound OLED has improved this issue significantly. In fact, when I actually installed and tested it, it played much clearer high-pitch sound than other TVs without a separate sound bar. Another important strength is that the sound is coming projected forward instead of from the bottom, which enhances the immersive experience. This alone is a significant appeal to most consumers.
But to me, a media artist, the greatest advantage was being able to map the sound directly on the screen. Though this isn’t commercialized yet, I’ve tried out the prototype in the test lab at LG Display. It had additional exciters that act as heightened speakers to differentiate left, right, up, and down. It enables the positioning of the sound origin as if I am designating coordinates on the screen. There have been similar attempts that used four speakers, but nothing as effective as Cinematic Sound OLED in creating sound intuitively and directly, corresponding exactly to the beautiful visuals. I was captivated right away.
The theme of my media project was also sound image matching (corresponding the location on the screen and sound). It was challenging and the concept of the video had to be limited to the two-dimensional plane, but it was fascinating to try something in a whole new sensory level. As I prepared to materialize my ideas, it gave me the kind of excitement that I hadn’t felt in a long time.
At the initial stage of the work, I contemplated which tools to use and how to approach it. For matching the location of the sound exactly on screen, the traditional animation style would be quite limited. In other words, the existing method is very labor intensive, which requires me to designate each location for individual sound. It wasn’t something appropriate for this amazing new technology.
I needed a coding tool that analyzes sounds, calculates the panning (positioning of sound in space) in reverse, and generates video on that same location. Frameworks such as Processing and open Frameworks, programming tools like Max(Jitter) of Cycling’74, and 3D software such as Cinema 4D that I used for creating a book trailer for Kim Young-ha’s novel Diary of a Murderer were other options, as they support Python scripting. But my final choice was Houdini, a graphics content creation tool with node programming, optimal for procedural (creating results through continuous computation based on given conditions and input) methods of video coding, while it also enables high-quality rendering. It was ideal for this project.
A screenshot of coding on Houdini
While considering the visual concept of the video, the most important part of this project, I hoped it would portray the shape and position of the sound on the screen. Since the Cinematic Sound OLED panel uses the screen as a diaphragm, I set the Chladni pattern (a phenomenon in which a certain pattern is created by vibrating when sound is connected to the diaphragm and sand is sprayed on the diaphragm) as the basic unit of the video. I looked up the research on the Chladni pattern, wrote several trigonometric codes, and executed it in Houdini to generate the actual Chladni pattern according to the frequency of the sound (pitch).
Chladni pattern (Source: http://nigelstanford.com/Cymatics/)
Chladni pattern generated by coding
The initial concept was to create abstract art or a structure by combining these Chladni patterns—similar to what happens in my mind when I arrange and mix music—but the visuals seemed a bit unrefined. I went with the current concept because this was more intuitive and suitable for matching sounds and images. (I was a little hesitant to discard the initial concept, so some of it remains around the middle of the sequence. If you are curious about what it was like, you can imagine it by looking at that section.)
Initial concept image
Because of the nature of sound image matching, we needed a device to give it more character and a stereoscopic effect to the inevitably flat visuals. As the objects created by sound were systematically organized and arranged like an organism, we set the lighting and texture to make them look like microorganisms viewed through a microscope. The brilliant colors and the contrast of the black background allowed me to take advantage of the perfect black and the vividness of high chroma, the major characteristics of the OLED panel. The color is enhanced, and the texture of the objects is emphasized by the infinite contrast ratio, which highlights the stereoscopic vitality of the visuals.
The objects on the screen indicate where the sound is emitted
The sound part was also much different from my other projects. Since sound image matching was a key concept, it was important to keep the position of the sound constantly changing and to make sure that each individual sound is clear and distinct without overlaps. In particular, I had to create a sound image that is not only distinguished from both the left and right, and the top and bottom parts of the Cinematic Sound OLED panel being produced, I had to install two more speakers in addition to the existing two speakers and monitor for a total of four speakers. At the same time, I aimed to maintain a familiar style rather than something difficult or experimental.
Working with OLED TV and 4 monitoring speakers
Though I gave a brief summary above, I tried and learned a lot of things for the project that took about two months. Anybody can view and enjoy the completed video on a PC or a mobile phone. However, in order to experience the intended concept of sound image matching, a TV or a monitor equipped with a Cinematic Sound OLED panel is required. In fact, to distinguish the top and bottom sounds, you should watch it on the prototype version. During the course of the project, it made me rethink many issues about art and technology. This was both a task and a challenge for me, rather than a constraint.
Working with OLED TV
One of the important roles of art is to awaken new senses. It must give you clues about a certain subtle feeling that you didn’t even know existed before, and can’t explain easily. It allows you to update the aesthetic standards based on those clues and to broaden any boundaries gradually. One of the central values that we look forward to and try to reach in art is to induce us to feel and enjoy more in our perspective and appreciation of the world. We have passed the period of great heroic leaps that explosively expanded the world of art.
Now, we are amid a multiplicity of colorful little footprints. As we often encounter the slogan, “There is nothing new under the sun,” more attention is being paid to the value of “rediscovery” with less and less emphasis new discoveries. Still, I believe that we are and must be moving into the realm of new senses, albeit slowly. Someday, such small advances will be combined to make the next big leap possible. Art and technology are now at the forefront of pioneering these new senses. At a point where nothing seems to be new, the advent of new technology is enabling the experience of new senses. Instead of being broadcast live around the world at Madison Square Garden, these tiny first steps are quietly taken in labs and in studios.
