AV Sync Tutorial

My first tutorial so please allow for teething problems

The basis of this method is that you use a vst plug in in Live to send values to your chosen visualist app.

So you need any VST Plug in capable of sending OSC or Midi values to your app.

I’ve been using Plogue Bidule but for the purposes of this tutorial I’m going to use a Pluggo vst as its free.

You must install Pluggo from Cycling 74.

Place the vst plug in into your live plug ins location. Download the vst here: windows version or the osx one.

Open Live
Drag a Warped Mov file into the first audio slot.
It makes it a lot easier if its warped.
If not then warp it and then export it to video and audio (quicktime photoJpeg maybe) .

If you do this you might want to just import the wav file into Live when you’re done. The important thing is that the video and audio are synched before you start.

Drag the Gloveosc vst into the audio track. On a Mac I’ve had a few problems with Live rejecting the plug in but it seems OK with Leopard & Live 7.
Go to the clip view.
Click on the envelope button.
In the envelope window choose gloveosc and parameter 1
Then draw a clip envelope starting at the bottom left of your clip and going to the top right so that it reaches the top at the end of your clip.

Double click at the top of your audio track and unfold the Gloveosc plug in.
Drag the parameter slider to the far right.

Start playing the clip and make sure the plugin is transmitting values. You should be able to see the parameter value changing in the plugin window.

Open Isadora.
Make sure the OSC port number is set to 2000 (or whatever the port the plug in is transmitting too)
If it isn’t reset it and restart Isadora.
OK first step drag an OSC Listener actor into Isadora.
Start playing the clip in Ableton Live. This should change the values in Isadora. If it doesn’t go back over the previous steps.

Now import the mov file which corresponds to the clip playing in Live.
Chain a movie player actor to a projector and set the speed and volume to zero.
Link the value of the OSC Listener to the first value input of a calculator actor.
Set the operation to divide and the second value to 16383.
Link the output to another calculator actor which multiplies this value by 100.
Connect this to the position input of the movie player.

Show the stage in Isadora (control G) and start playing the clip in Live. It should shift the playhead of the movie in Isadora.
I have found that you may need to slightly adjust the endpoint of the clip envelope in Live to make sure the audio and video sync accurately all the way through.
If you’ve warped the clip correctly and the audio and video of your file are in sync this should me no problem.

Its worth saying here that the frequency Live passes parameter values to its plug ins is dependent on your plug in buffer size so

if you want to achieve a decent frame rate, you have to set this to as small a value as you can (Preferences>CPU)

Its also worth saying that the timing of each frame  is not 100% accurate – ie if you are sending 25 values a second to Isadora

these may not be at exactly even time intervals.

Watching the video play though I cannot tell. You can test it by watching the video window in Isadora and Live at the same time

(you have to drag the video into the arrangement view but trigger the playbutton in clip view for this)

Now duplicate the clip in live a few times and adjust the start,end points, create different loops, use follow actions.

The video output from Isadora should keep sync with the audio.

You can increase the speed, use redo and undo, move the loop envelope with the arrow keys and tons of stuff I haven’t thought of I’m sure.
Play about till your hearts content. Add extra tracks, effects (you can link them to visual effects using midi feedback and an external controller)

If you want to use more than one instance of the plug in at the time then you will need to transmit values on a separate channel.

Here’s another version of the plugin which has an extra parameter for the channel.

For Quartz Composer I think each value has to be sent to a different port with OSC (?) so another version would have to be made.