not long ago i was tackling again the problem of getting nice sync between visuals and audio and tried a bunch of different things. i made this tool to make manually syncing pretty efficient. it shows you the waveform or amplitude or other visualizations and you draw a curve on top of it with some nice controls for duplicating points and stuff. that still took much too long so then i tried some stuff with rendering out fft and other data to images and using graphics stuff i was more familiar with to process them. here's a pretty bad example of that.

anyhow what i wound up finding and using for the demo n-0505 is an awesome web service called echonest. it's crazy. you upload your mp3 with the music and it analyzes it for a bit then you can download xml data with tons of info you can sync to. the most useful by far is some data that tells you essentially how many beats per minute and the time of the first beat. that's covered like 99% of syncing for me as i set absolutely everything to start on beat indices or pulse with waves centered at the beats and things like that. suuuper simple!

logo.gif