Beginners guide to do music with gestures

Recently some peole introduced me to different technologies one can use for the purpose of music creation, and involving other people in the process. Experiencing that it is difficult to understand some of the terms (or even just figure out that the terms / technologies them selves exist), I've decided to write a little post on my findings so far.

I'm currently involved in a project called Spectrogram, of which uses some of these technologies, so I will use that project as an example of one way of using these technologies, and try to explain some words / objects that are good to get familiar with, if you are interested in that kind of stuff.

Spectrogram is a collaboration with interaction designer Sebastian Rønde Thielke, about doing concerts where people are able to participate, and affect the music, using technologies such as smartphones and TouchOSC. 

We want to play concerts where it's important that it is you who are present, because the music would develop different otherwise. 
We want to use the smartphone as a social media, making people able to socially interact where they are, not having to pay attention to virtual social connections.

In our video we use a few different tools, that make us able to communicate with peoples smartphones, and make them work with our syntheseizers and drummachines (of which we “programmed” our selves, using a visual programming launguage called Max/MSP/Jitter). 

• TouchOSC is a smartphone app (currently available on iPhone and Android), that make you able to communicate using OpenSoundControl or  just OSC (the succeeder of MIDI), to a lot of different software. We use it to send data between the smartphones and our computers running Max/MSP.
• A cool thing about TouchOSC is the interface (which has been very influenced by the Jazzmutant's Lemur) that you are able to “design” your self. They guys from hexler.net have coded “a dial”, and “a slider” etc. and made you able to choose color, placement in your interface, size and such. It let you decide which kind of numbers it sends out (using WiFi), and if you have a software on your computer that can receive this data, you can map it to whatever you want (ie. Sliders, dials or effects in your music software – or in short DAW).
• Around 2.00 minutes in the Spectrogram video, I linked to above, you see a guy in a swinging chair. We used Nintendo Wii-motes and OSCulator to make him able to control different filters and pitch in our syntheseizers. 

We placed a wii-mote underneath the swinging chair. The wii-mote is super good at  detecting weather it is rotated in the Y-, X- or Z-axis, and if it is accelerating from  one speed in a room to another. You can use the very good (and cheap) software  OSCulator, to read the data from your wii-mote, and route the data os OSC  messages to whatever software you have on your computer (or another device  using WiFi). 

I've just started using the Microsoft Kinect to control synth's, DAW's and max-patches in ways mentioned before. If you are interested and have a mac on snow leopard, you can check out these tutorials:

Tutorial 1   

use this one, and when you need to change the .xml files from the OpenNI/Samples/Bin/Release folder, use the second tutorial.

Tutorial 2    

With the kinect you are able to map your self out, and read each point in the sceleton to control something, in ways similar to what i've described with the wii-mote. I will do a tutorial on this subject, whenever I've gotten more familiar with it. (Just the installation took a while). 

I hope that introduced you to a few new items and subjects for further exploration. Just for the fun of it, try going on youtube and see the amazing people are doing with some of the stuff I mentioned above.

// lmnk