Building custom midi interfaces with arduino for use in in Ableton and TouchDesigner (Part One)

So recently it was my friends birthday and we were considering what to get him as a present. As he’s all about music production myself and Miyu thought it’d be fun to build him his very own MIDI controller.

Since his name is Dominique We decided to call the midi controller the “Domicon”

Domicon Box Art, this is the finished device complete with all the random references one could need.

Stage 1, Design

We figured that the controller should be a simple 4 rotary pots and 4 buttons controller as we had those bits and bobs lying about already and the board was to be an atmega 32u4  based as this allows the device to show up as a native mouse/keyboard and the Midi Library we planned to use happened to be based on that. We happened to have an Arduino Pro Micro lying around too so we figured we’d use that.

So our ingredients became the following:

4x Aussel 10K OHM B10K Linear Potentiometer
4X Random Arcade buttons we had lying around, the came from a kit similar to the linked
1x Kookye Arduino Pro Micro

We opted to 3D print the casing in two parts, one was the lid with embedded logo and holes for our buttons/knobs and the second was the bottom of the box which just included a small hole in the side for the USB cable to pop out of. We measured up our components and came up with the following 3D model:

We printed a test plate to make sure everything fitted through the holes and amazingly enough all our measurements were pretty spot on so we went ahead and set our Ultimaker away printing.

Stage 2, Wiring

I will confess to being absolutely useless at anything practical. The general gist was all the components hooked up to the arduino directly with no need for resistors or anything like that. 4 Analog pins were used for the potentiometers and 4 digital pins were used for the buttons. The potentiometers had to be wired to ground and vcc but the buttons just needed wiring to ground so that was all pretty straight forward. Miyu did all the wiring and soldering/heatshrinking so I didn’t end up breaking anything and everything felt very solid.

Stage 3, Programming

We now had a product but we still needed to program the board to receive information from the components, convert that into some sort of message that the computer can understand and then send that as midi data.

There’s a great library called MIDIUSB that makes this all quite trivial. There’s even an example on there with some basic functions to get you going. I personally sent everything as control changes but it’s pretty easy to send Note commands too. I kept the note functions in even though I’m not using them just in case I decide I want to expand or change anything in future.

Here’s the final code:

#include <frequencyToNote.h>
#include <MIDIUSB.h>
#include <pitchToFrequency.h>
#include <pitchToNote.h>

// initialize everything

int slider1val = 0;
int a0val = 0;
int slider2val = 0;
int a1val = 0;
int slider3val = 0;
int a2val = 0;
int slider4val = 0;
int a3val = 0;

int b2 = 0; 
int b2toggle = 0;
int b2down = 0;

int b3 = 0; 
int b3toggle = 0;
int b3down = 0;

int b4 = 0; 
int b4toggle = 0;
int b4down = 0;

int b5 = 0; 
int b5toggle = 0;
int b5down = 0;

// these are the functions from MIDIUSBs demo, I left the noteOn and noteOff in even though we aren't using them.

void controlChange(byte channel, byte control, byte value) {
  midiEventPacket_t event = {0x0B, 0xB0 | channel, control, value};
  MidiUSB.sendMIDI(event);
  MidiUSB.flush();
}

void noteOn(byte channel, byte pitch, byte velocity) {
  midiEventPacket_t noteOn = {0x09, 0x90 | channel, pitch, velocity};
  MidiUSB.sendMIDI(noteOn);
  MidiUSB.flush();
}

void noteOff(byte channel, byte pitch, byte velocity) {
  midiEventPacket_t noteOff = {0x08, 0x80 | channel, pitch, velocity};
  MidiUSB.sendMIDI(noteOff);
  MidiUSB.flush();
}


void setup() {
  // I've opened the serial port for logging reasons. You don't need that line really but I just left it in.
  // At this point we also set the pin modes for our 4 buttons. They are set to pullup and are on digital pins 2, 3, 4 and 5.
  Serial.begin(9600);
  pinMode(2, INPUT_PULLUP);
  pinMode(3, INPUT_PULLUP);
  pinMode(4, INPUT_PULLUP);
  pinMode(5, INPUT_PULLUP);
}

void loop() {
  // lets get some data from our buttons via digitalRead and throw them into some variables
  b2 = digitalRead(2);
  b3 = digitalRead(3);
  b4 = digitalRead(4);
  b5 = digitalRead(5);

  // lets check if the button is down, these are toggle buttons so we use b2toggle to remember what state the toggle is in. We do this for each button
  if (b2 == 0) {
    if (b2down == 0) {
      if (b2toggle == 0) {
         // the control change function sends on the same channel but with a different pitch on each button. The velocity is 127 as these are simple "on off" arcade buttons.
         controlChange(0, 5, 127);
         b2toggle = 1;
      } else {
        controlChange(0, 5, 0);
         b2toggle = 0;
      }
      b2down = 1;
    }
  } else {
    b2down = 0;
  }

   if (b3 == 0) {
    if (b3down == 0) {
      if (b3toggle == 0) {
         controlChange(0, 6, 127);
         b3toggle = 1;
      } else {
        controlChange(0, 6, 0);
         b3toggle = 0;
      }
      b3down = 1;
    }
  } else {
    b3down = 0;
  }

     
  if (b4 == 0) {
    if (b4down == 0) {
      if (b4toggle == 0) {
         controlChange(0, 7, 127);
         b4toggle = 1;
      } else {
        controlChange(0, 7, 0);
         b4toggle = 0;
      }
      b4down = 1;
    }
  } else {
    b4down = 0;
  }

     if (b5 == 0) {
    if (b5down == 0) {
      if (b5toggle == 0) {
         controlChange(0, 8, 127);
         b5toggle = 1;
      } else {
        controlChange(0, 8, 0);
         b5toggle = 0;
      }
      b5down = 1;
    }
  } else {
    b5down = 0;
  }
  

  // now lets deal with our sliders. These are on analog pins 0, 1, 2 and 3.
  // first we read them via analogRead and then if the value has changed from the last time we send the MIDI command. 
  // This is so we don't just spam MIDI data all the time constantly (it's a right pain when using MIDI learn if all sliders are pushing)
  // We do this for all sliders

  a0val = (int) analogRead(0)/8;
  if (a0val != slider1val) {
    slider1val = a0val;
    controlChange(0, 0, a0val);
  }

  a1val = (int) analogRead(1)/8;
  if (a1val != slider2val) {
    slider2val = a1val;
    controlChange(0, 1, a1val);
  }

  a2val = (int) analogRead(2)/8;
  if (a2val != slider3val) {
    slider3val = a2val;
    controlChange(0, 2, a2val);
  }

  a3val = (int) analogRead(3)/8;
  if (a3val != slider4val) {
    slider4val = a3val;
    controlChange(0, 3, a3val);
  }

}


 

The Final Domicon

 

May TouchDesigner 099 Audio Reactive VJ Effects Pack

I thought it’d be cool to maybe start releasing some of my TouchDesigner VJ content from over the years…then 099 came out and suddenly my whole library was pretty much null and void. Slowly I’m going to start updating my library. Here’s some effects to start off with. Mainly CHOP to SOP stuff in this including a little Particle trick that Markus showed me all those years back… and a bit of geometry with noise too.

You can download the .toe from my bitbucket >>> here <<<

The enviroment maps in this pack were found at www.mrbluesummers.com and it’s a really good resource to use.

Remapping

Remapping is simple and powerful and there’s one aspect of it quite a few people may not have noticed yet.

A remap image looks something like this:

remap

and the GLSL code to remap (aka the Remap TOP in TouchDesigner) looks something like this:

vec4 remap = texture(sTD2DInputs[1], vUV.st);
vec4 image = texture(sTD2DInputs[0], vec2(remap));
fragColor = image;

What does the above code do? It takes two texture inputs and uses the second textures red and green colour channels as the uv coordinates for the first texture. It’s commonly used by people who use the stoner component as it means they can do their mapping using the stoners render pipeline and then simply remove the whole mapping tool from the system leaving only the remap texture in place.

Whats very useful about the stoner is that inside its render Geometry COMP it actually gives us another GLSL MAT which simply renders out a 3d objects UV coordinates into red and green colour channels. In the stoner it’s only used for perspective corner pinning and grid warping however there’s no reason to use that shader just with the stoner, we can also use it with anything else we like. For example lets take a sphere with noise and apply this material and we end up with:

remap2

…and using our Remap TOP we get the following…

remapsphere

So what can we conclude from this? Quite simply that if you really wanted to you could have the same remap setup you have with the stoner in CamSchnappr, there are some caveats to this but if you can get around them it’s a super powerful setup.

Note: You need to keep your remaps as 32-bit images. 8-bit just isn’t enough to get any kind of smoothness and even with 32 bit you’ll get some jaggies on the edges. To avoid jaggies I tend to add an edge ring into my shader to alias my remappings. If you’re into glitch effects a really cool effect is to take a highly compressed h264 remap image and then use the remap with your video and you get all kinds of datamoshy loveliness.

Optimisation in TouchDesigner

So for the past few months I’ve been putting together a bit of a realtime MediaServer System called Mara. As this is more of a personal project than a work project (our workflow is d3 with touchdesigner capturing in) I went in all guns blazing, no planning and building it as I went along. Every time I saw something I didn’t like with the system I’d simply rebuild the whole thing and because of this process it’s taken a bit longer but is much much neater than if I’d just kept adding and adding to it.

More recently I’ve finally completed most of my system and am now in Optimisation mode.

So here are some of the common tricks that people use when optimising, whether all of them are really beneficial is up for discussion but I’ve been implementing them anyways.

Get rid of references

This is not something I discovered myself but having lots and lots of expressions and references in parameters is not as efficient as exporting to or scripting to those parameters. I decided to do a bit of a test on this and couldn’t really prove it to be true I tried out Idzs test on the TouchDesigner help group, it’s 100x more efficient. Unless you have a huge system you probably needn’t bother with this. Nonetheless I’d be foolish to not just get on and have it in the system. It’s also nice to keep things “stateless” too whereby any op can set a value on any other op quickly. For sliders this isn’t advisable as running a script every frame can get quite heavy.

The 2×2 trick

Back in the olden days when VJ’ing on my macbook pro with 256mb of gpu memory or whatever it had the only thing that would ever be quick was the rate at which your fps dropped as you loaded lots of generative effects. Movies are fine as they can be preloaded and unloaded at your hearts desire however this unfortunately doesn’t seem to be the case for generator TOPs. The solution is fairly simple. If we aren’t using a generator TOP we set its resolution to 2×2 or something very low and suddenly we get all our GPU memory back. There is one caveat to this though, the stutter when loading your entire network into memory is very ugly and can cause problems, so on more modern cards it’s probably advisable to just leave your generative bits and bobs loaded into memory if you can. You can also split things across a couple of TouchDesigner instances and this will certainly help out a bit too.

COMP once

All those overs, multiplies and composite TOPs are never doing you any good really. A very elegant solution to this would be to check out Keith Lostracco’s  Archo-Ps? GLSL composite which uses one op for all your compositing needs. My own method is pretty much the same as that example but I’m generating my GLSL TOP from a bunch of table values, basically python writes the GLSL shader for me. The GLSL TOP scales with the system then and it’s much more manageable.

Maras Timeline

Realtime timelines can be a tricky one to get running fast.

Keep your UI separate

I’d always suggest keeping the UI on another machine or in a seperate instance. It’s handier and much more scaleable if you need to fork out to other machines. It forces you to be a bit more disciplined and helps you when you need to start putting previz tools etc in. I’ve been very careful to take care of the little details in the ui too such as making sure TOPs scale with the UI (but not using expressions) and making sure that CHOPs are kept to a minimum. Only one type of UI element really needs a CHOP and that’s a slider, sometimes even they don’t need them.

mara, 3d viewer, stage, toucdesigner

Maras 3d Viewer is pretty heavy with all of it’s opacity nonsense etc and billion and one switches.

One window to rule them all

Multiple windows are slower than having one big window. The general method most people seem to use is to have their containers and set their positions to wherever they need them in the window. For Mara I’m just throwing in a d3 style feed mapper so you can throw things wherever you want and split them up nicely.

TOP colour channel trick

Got a black and white movie? Then head to the common page and set your pixel format to 8bit fixed (R) and you’ve just removed everything except the one channel you need. This wont work with transparency but is really useful for fast UI elements.

Keep away from SOP transforms and materials

Why transform in a SOP when you can transform in a Geometry COMP? The reason for this being that transforms on the COMP supposedly happen on the GPU whereas transforms on the SOP level happen on the CPU. (or so I’ve been told) I’m actually very guilty of using transform SOPs because I’m a lazy noodler but it’s definitely one worth noting.

Applying a material to a COMP is much faster than to a SOP too.

SOP to DAT is also apparently slower than a SOP to CHOP so for SOP deformations etc too you’re better off using a CHOP than a DAT.

General Rendering Efficiency

Got more than one camera on one scene? Then use render selects rather than having 5 render TOPs. They’re much more efficient.
Projecting your project onto a building? Set your anti-aliasing to 2x rather than 4x and you’ll pretty much half your memory load.
Transparency seems to be a wee bit heavier than you’d like. Try and keep away from it
Use instancing in the Geometry COMP rather than the copy SOP.

Chain reorganisation

Only cook what you need to. there’s no point in cooking more than required so if you have a level thats cooking and it can go further up the chain so it no longer needs to cook then put it further up the chain.

Anything I missed?

This article is kind of a work in progress so if there’s anything I missed you’d like to share let me know and I’ll add it in.

TouchDesigner Beginner Workshop in London

workshop

Image courtesy of ab30 on the Derivative Forum.

It’s time for another London workshop. This is a two day 12 hour workshop on TouchDesigner with Richard Burns on 18th/19th June 2016. The price for the full event is £160. To purchase a ticket you can use the form below below details. This workshop will focus more on creation of content and VJing than building of systems which the last one was quite strongly using. This workshop will be held at Projection Artworks new headquarters.

Note: If the event is sold out then you can register interest by emailing richard@richard-burns.com. Also there is the comments/suggestions section of this post.

Day One (6 hours) – Introduction to TouchDesigner 088

– TouchDesigner user interface and an introduction to operators.
– Working with different operator families, generators and filters.
– A simple project using all operator families to create an audio reactive visual.
– Using Live controllers, Gamepads, MIDI devices and OSC.
– An introduction to Python and how Python works within TouchDesigner.

Day One Evening

On the evening we will likely go to the local pub to talk about projects, general catchup etc. The invitation to this extends to anyone and everyone who’d like to come as this’ll likely be a big meetup. We’ll be in The Vine from 6.30 in Kentish Town.

Day Two (6 hours)- TouchDesigner 088 Techniques & Workflow

– Hooking TouchDesigner up to Resolume via Spout
– Movie Playback and rendering techniques for rendering loops.
– An in-depth look at SOPs and what can be achieved with them.
– An overview of the Animation Component and keyframing techniques.
– An in-depth look at audio analysis using CHOPs.

Location, Date and Time: 

Date: 18th-19th June
Time: 10.00am-6.00pm both days.
Projection Artworks
Unit 400, Highgate Studios
53-79 Highgate Road
London
NW5 1TL

Requirements:

A Laptop with TouchDesigner FTE 088 and mouse
It would also be recommended to have a copy of Resolume for part of this workshop although it is not vital
Knowledge from First things to know about TouchDesigner

 

Purchase Here:

this workshop has now finished.

Keeping in Touch 03 – SOP Tricks

In this video I go through some SOPs and explain a few tips and tricks you can use when working with 3d objects in TouchDesigner.

Keeping in Touch 02 – Particle Attractors & Kinect

Spoutkit Version 1.0

I had a little idea today, wouldn’t it be great if I could give people who don’t use touchdesigner visuals which they can use in their own applications. If there was a little control panel that had some generative visuals in that could then be sent out via spout and picked up in other programs such as arena and people who don’t know TouchDesigner could still go ahead and use generative clips. Well…here’s what I mocked up tonight….Spoutkit:

Capture

 

Requirements:
Windows (sorry mac people although apparently there may be a way to send from windows to mac using >>>TCPSpout<<<)
TouchDesigner FTE. For 1080p you will need a commercial licence. Only the last build found >>here<< and above will work. (version 30580)
Spout which can be downloaded >>here<<

To run the file just extract the .zip and double click spoutkit.toe. This will open up a control panel, on the left hand side are your parameters, audio input, effectors etc and at the bottom is a bank of 5 different visuals which can be loaded. Once loaded move the effector sliders around to see what they do. To set the resolution to 1080p as opposed to 720p just use the toggle at the top right of the screen.

The project will automatically send out a spout texture handle named “spoutkit” which you can select in the program you wish to use.
It will use whatever the default audio input is as an audio input.

I’m going to expand on this idea over the weekend and make it a bit more useful, this is more just a proof on concept at the moment really, heck I wrote the whole thing in T-Script.

>>>DOWNLOAD HERE<<<

For information on how to install spout within resolume see the following video:

Abstract Still Pieces

I’ve been playing about a bit trying to do some t-shirt designs and things for myself since I can’t seem to find many online that are nice geometric shapes and so on…anyways I got completely distracted with all that and ended up doing these little SSAO pieces this evening.crablikething foxheadthingwithbg treething

 

Maybe I’ve been playing a bit too much pokemon or something but they seem to be kind of Fire, Grass and Water/Ice based. Anyways I’ve attached the .toe files so people can have a look at how they’re done.

 

WARNING: These files are HUGE, like…massive 4k renders running in realtime with an SSAO TOP in there too. I recommend pausing the timeline or turning off cooking and if you want to modify them turn the SSAO TOP off until you’re ready to render, the grass one I made can get into some really cool variations with the feedback.

Download .TOE Files

I’m probably going to throw them up on RedBubble too as prints/t-shirts etc, why not…just set one up at redbubble.com/people/visualesque

 

On a final note, theres a keeping in touch coming very soon. I’m currently laptopless as I’ve leant it to a friend for a VJ set.

Richard

Keeping in Touch 01

×