[Pd] Video Looping

I recently met a friend of a friend who had some ideas for a project using looped videos. He'd been trying to find a way of doing it with hardware and despite asking around all over London he couldn't find anyone who knew a way of doing it. I reckoned it could be done somehow using software although I'd never worked with video clips before (I tend to work the other way around - using data from my webcam to control patches using OpenCV).

I downloaded a couple of clips from archive.org and cranked up Pd. [pix_film] could play the clip and I could scrub through it easily just using a [number] and I could use a [metro] and [mod] to play back any number of frames at any point in the clip at any speed. Great, but no sound. One of the major goals is to take four clips that can be played at variable rates which will create "music" - the main criterium for choosing a clip is that the sound should be interesting and loop well.

After a bit of research it seemed that the audio from the clip I was using had to be extracted from the .mov file and then synchronised to the video in the Pd patch. I came up with the patch at the top of this post in the end. There are some redundant parts of the patch not relevant to the goal but it works. The clip can be slowed down and speeded up, the section of the clip that we want to loop can be chosen by selecting start and stop points (currently these relate to audio frames). This is fed into a [snapshot~] which has to be banged repeatedly - a bit of bad maths later and the video will loop. As the loops are going to be very short (around a second or two) they stay in sync even though the maths is out by 0.5 of a frame per second (in this case).

One problem is that the clip will not play backwards without stuttering, which is a bit of a pain as it would be great to be able to play the clips in reverse at any speed as well as doing the same thing forwards. Further experimentation is needed.

Another problem is that I would really like to be able to play four clips in the same [gemwin]. If that wasn't enough there needs to be a library of different clips that can be changed on the fly; not only that, the patch needs to be interactive in a way that makes sense to an audience unfamiliar with Pd - at the moment it's looking like I will use OpenCV to achieve this but I am not entirely sure how...