Building an instrument

While developing a new instrument to control image and sound, we were looking at traditional instruments' properties:

• Complexity: Most actions on any instrument have repercussions on more than just one parameter. When plucking a string of a guitar, the player is controlling amplitude, timbre and duration. When hitting a key on a piano, three strings are starting to oscillate with again different durations, timbres and amplitudes resonating through the body of the piano
• Learning curve: For any instrument, you need to invest a certain amount of time to learn how to play it. When your instrument does not produce the wanted result, don't change the instrument. Start practising.

Our 3-axis accelerometer has the beautiful property of outputting the acceleration measured on each of its three axes, so any change of orientation results in more than one parameter being modified. Acceleration being measured can also mean two things:

• Orientation: Slowly changing the orientation of the accelerometer can give us a reference to Earth's horizon, since Earth's gravitation is downward acceleration.
  
• Change in motion: When a change of movement occurs, acceleration is being measured. This can give us an idea of the properties of movement.

To have this data streams available to us, we are using the raw data of three axis, low-pass filtered data for orientation and high-pass filtered data for change in motion. We implemented these (recursive) filters in a Max/MSP patch to be quickly linked to whatever parameter we want.
On the image on the left you can see the original stream at the top, the LPF-filtered stream on the bottom left and the HPF-filtered stream on the bottom right.


Now that we have this data, what to do with it?

Looking again at traditional instruments, we were looking for existing digital instruments that already have an inherent complexity and see what happens when we hook up some of their parameters to our accelerometer.

Reaktor by Native Instruments is a synthesizer-building tool that comes bundled with a lot of ready-made synths. Skrewell is one of them, and has the properties of being in a way very hard to understand. So we hooked up our accelerometer to it and started playing....

And now we need to practice...

But the results are already promising!