Chromakopia:
Interactive Audio Visualizer
An educational project to learn the ins and outs of Touch Designer,
while producing fan content for a favorite album of mine.
The Beginning
Sprouting from sketches during a meeting, the idea for an experimental audio-visual experience centered around Tyler, The Creator’s most recent album Chromakopia was born.
Immediately I knew I wanted to expand listeners’ experience into the visual as a way of interacting with the music in a physical space, and I wanted to create an opportunity to learn Touch Designer as a way to expand my skillset.
2. Initial Learning
Histogram
Tutorial
Starting off where most great things do, I began searching YouTube for tutorials with a visual end goal in mind. Knowing how massive Touch Designer’s potential could be, I wanted to start with tutorials that directly aligned with my end goals.
I quickly picked up on Touch Designer’s node based system, and this tutorial began teaching me what would become the most valuable: different ways of interpreting the audio and tying it to different parameters. As far as the color changing, the camera movements, the live updated type, and the visual effects, that was all independently sourced or figured out on my own.
Freeform Line
Tutorial
Following the Histogram, I wanted to experiment with more instancing, visual effects, and different forms of audio reactivity. The best pieces I learned from this had to do with filtering different metrics from the audio input like its relative speed, and this taught me rendering techniques and post processing visuals that I would retro actively apply to the Histogram.
The free for line was something I had wanted to include into the final product, but it was right around this point I’d run into the largest obstacle long the way: optimization and performance. With the instancing and the post processing, it started to slow down already.
3. Next Steps
I wanted to consider all options when it came to deciding on an album to adapt for this project. I dug through my personal favorites and scored and ranked them to find the best for the project.
2. I also wanted to organize all my visual inspiration for the analogue display aesthetic I wanted to capture in this project as well. I wanted to utilize the modular grid style to incorporate multiple visualizers.
Taking what I had began learning from my time in the Touch Designer YouTube trenches, I began organizing my thoughts in Figma as a way to sketch the next iteration in Touch Designer. I threw together the first layout here too.
1. Final album selection process.
2. Visual inspiration for the layout and visual style.
3. The sketch of the final layout done in Figma.
4. Getting Sticky
The ASCII Mask
The ASCII Mask was a tremendous benchmark in my learning progress as it was the most complicated audio-visual asset. This became the most time consuming and resource consuming step towards the finale as well.
4. Last Pieces and Performance
The Kinect
The final piece was the inclusion of a Microsoft Kinect as the primary point of interaction. Using a Kinect 2 I made a simple audio-reactive displaceable background that would allow users to see themselves in the music. While being the key feature, this would also become a key obstacle. The Kinect 2 was a major resource-user that then ended up limiting the performance of the final piece and restricting other simulations or visualizers.
The Final Product
The final version was projected on a wall where users could interact with the projection via the Kinect 2, and the visuals reacted to the music they were hearing. Overall, I am only somewhat satisfied. Limited by time and hardware, my end product only became a fraction of what I had envisioned, but this project was invaluable in showing my capacity to quickly learn anything necessary to bring my vision to life.