Overview

Even though this article is a bit of a whirlwind I do recommend reading it in full and in one sitting, but who really has the attention span for that anymore, so here's a little clickable bullet point list of what's coming:

• HYDRACTRL & my VJ setup
• Why VJing?
• Resolume
• TouchOSC
• RESOCTRL
• Efdemin "Good Winds"
• TouchDesigner
• HYDRACTRL features
• Annotated VJ setup

HYDRACTRL

Ok, but what is HYDRACTRL you might ask. Well, it's exactly what it sounds like. It's a control surface for Hydra.

And Hydra? Hydra is a live code-able video synth and coding environment that runs right in your browser. It's an open-source project by Olivia Jack that was inspired by analogue video synths.

I've been an absolute fan of it ever since I learned about it and have been trying to spread the love through a flash-intro during a Creative Coding Amsterdam meetup last year.

And now this love has brought me to a point where I built HYDRACTRL to fill the live-generated-audio-reactive-visuals hole in my VJ setup. The last big gaping hole that needed to be filled if I'd ever want to call myself a respectable VJ. Or so I told myself.

But what does that actually look like? Well, here's my point of view:

On the left you can see the laptop running HYDRACTRL with the Korg NanoPad2 connected. What all the other stuff is, we'll get back to later, but for now I want to take a moment and talk about why I'm actually doing all of this.

Why?

I'd argue that's a good question to ask when you find yourself pouring considerable amounts of time and/or money into something. And just like with many other of life's great questions, it is best answered by a Futurama scene:

For those unfamiliar, this is Fry trying to play the holophonor. He doesn't quite know what it all means, but he knows it's important to him. Which kind of hits home.

Similarly I dream of a way to bring the ideas from my head into the world. Ideally in real-time and dancing to music. Unfortunately, we haven't invented anything holophonor-y yet and even in the whole Futurama universe only a few people can play it and they're pretty bad at it. So that was not an option.

Resolume

What's the next best thing? That is exactly the question I asked myself back in late 2021 when we had pinned our first date for the D&LY A/V project.

I figured I definitely wanted a way to play and mix video, so that was a good place to start. The decision to go for resolume came easy as my setup is currently Windows based, plus I had a decent amount of (dusty) experience with the software from back in the late 2000's.

TouchOSC

Of course, if I ever were to become a video virtuoso, I didn't want to be doing that with a mouse and keyboard. I wanted a more intuitive and playful way of interacting with the software, which soon led me to TouchOSC. The people over at resolume even had a starter TouchOSC file linked in their OSC documentation. Happy days.

At least, they were happy days in the beginning. I went through a couple of iterations of this interface but soon came to realize that creating these interfaces was rather tedious work. Configuring the right OSC routes for every little control required drilling down into fiddly multi-level menus. I'm sure there was probably a more efficient way to do what I was doing, but that was not for me as a TouchOSC non-expert. And the more time I spent with it, the less I wanted to become an expert.

Don't get me wrong, TouchOSC is a great piece of software and you can make wonderful interfaces with it to use on your tablet, but my inability to efficiently iterate on these interfaces bugged me more and more. (For those interested, you can find all iterations of these TouchOSC interfaces on my GitHub)

And then I saw this:

Wait, what? That's been there the whole time? And they even link a repo that's actually the entire application's UI rebuilt in React?! What! That was literally in front of my nose the whole time, yet I didn't bother looking at it.

I think I cried a little bit about all the time I had spent fiddling with TouchOSC. Time I could’ve used to build a Resolume webapp instead. Time I was never going to get back. But then I also remembered that we cannot change the past so the best thing we can do is accept it and figure out what the next best step is. But don't take my word for it, Ilya Sutskever can tell you as well.

RESOCTRL

So I started building RESOCTRL. That's a control surface for Resolume, but I guess you could've figured that one out yourself. The first thing I implemented was a clip selection overview. And this time I could actually have thumbnails of the video as buttons! I was very chuffed about that. And still am in fact. This is how it looks like as of June 2025:

Next up? Well, while building a playful interface I found myself rhythmically selecting clips and realizing that I only have two hands and one head. Or in other words, while I was selecting clips to the beat, I wasn't able to do anything else. What if there were a playful way of recording these sequences? Right! 💡I needed a sequencer!

A first naive implementation of this sequencer was ready by the time we had our gig at OT301 in 2024, but I never ended up using it. Conclusion: 1. the sequencing interface wasn't exactly as playful as I'd thought, 2. The whole tool had some other annoying issues that needed fixing and 3. I needed to practice more. Way more... It is a hard balance to strike between building an instrument and learning to play and clearly I was still a way off.

But as babies turn into toddlers, so did my broken sequencer turn into something that actually worked. A big breakthrough came when I visited a friend who's got hardware sequencers set up in his home studio. That's when I realised I was trying to reinvent the wheel. The wheel in this case being a step sequencer. Something very ubiquitous in the audio production scene and has a very intuitive interface. This is how it looks like:

Picture a vertical column of buttons lighting up in a swooping motion from left to right. In time with the beat. And you can turn each button off and on as you like, while each row has a sound linked to it. Or in my case, a video.

Efdemin "Good Winds"

The final push to get this RESOCTRL sequencer thing in good shape was when I got inspired by the track "Good Winds" by Efdemin. That whole album is in fact quite inspiring and do recommend, but this one track gave me a clear vision on how I could use my sequencer and in doing so effectively put it to the test. Only building features that you know how you're going to use them or why you need them has also been an important lesson over the years.

Below you can see what that looks like in one of the better tryouts. One of the main elements is the base video layer switching back & forth every 2 beats. Something you don't want to be doing by hand. At least not for 6 minutes...

Great. That was working pretty well. What's next? Well, I'd been looking into expanding the effects available in Resolume but was unhappy with what I found in their proprietary effects shop, nor did I want to invest in their -also proprietary- plugin builder. I wanted to be able to just plug in shaders as I desired. It seemed like there was a way but you had to do very strange dances, jump through a bunch of hoops and sacrifice your firstborn. None of which were things I wanted to do, so I kept looking for alternatives.

TouchDesigner

I had been meaning to learn TouchDesigner for a while, but it feels very daunting once you get in there which had kept me at bay until then. But now no longer. I had a very simple and well defined little thing I could build with it: a post-processing studio. I would take my Resolume output and add some effects.

Which effects? VHS, scan-lines, RGB delay and glitch effects of course. What else? If you've been wondering why a lot of my recent stuff has had that good old 80s feel to it, now you know. This is what that looks like inside of TouchDesigner:

It is at this point that I had the space to start thinking of how to incorporate real-time graphics. My first idea was to integrate Synesthesia, but playing around with it I realised I was just entering yet another walled garden. Even though it's a lot easier to extend and plug in custom shaders, it still is a proprietary piece of software. And 400$.

And then I realised that Hydra was actually perfect for this job. Except that it was missing some essential features for my purpose. This was also around the time that I'd been trying to think of a project that would be a good fit to explore this whole vibe-coding thing all the cool kids were talking about. This seemed like the perfect fit.

HYDRACTRL

So let's have a closer look at HYDRACTRL.

So what were these missing features I'm talking about? The main ones were scene management, MIDI (NanoPad) integration, a code editor with code completion and integrated Hydra documentation. If you're interested (excited?:) by any of this, I recommend the github page I set up to get a full feature overview.

The prompt-coding experience went pretty smooth and within about a week's worth of work spread across a month I ended up with a functional version I can actually play around with. The total Claude Code bill for this project added up to around 60$, though I will add that around 15$ of that was spent trying to unsuccessfully integrate different code editors. Until I pointed it to the hydra repo as a reference codemirror implementation which worked like a charm.

There's a whole bunch more lessons I've learned from this project, but as this has been a pretty hefty one already, I'll keep that for next time.

RTVPS

Instead, let's wrap it up by revisiting that VJ setup POV picture from earlier. I'm not sure about the name though. In fact, I think I might start calling it my very own real-time video production studio. Or RTVPS. Or maybe not.

Anyways, here's the same picture from above, but annotated with all the stuff we've been talking about.

In case you have ideas, questions and/or remarks, please do get in touch. I'd love to here them!

Thank you for reading!
& until next time.

David