This last month I’ve been slaving over a new tool that allows me to vj in REAL TIME based on images.
It samples pixel data that is used to drive a fluid simulation with image recognition layered on top.
Custom vector maps are derived to let elements `search` each other.
The results can be displayed using fields, particles , geometry and image data.
My intention was to play with the structure of an image, find it’s hidden agenda.
The image is decomposed in to key elements and from there on recreated using various fields of data.
The results can be quite, ehm, trippy.
It runs stable at around 60 fps, where it can draw over 100.000 particles + ordered geometry and scalar fields.
All is written in C++ using OpenGL, with the help of OpenFrameworks and OpenCV. Houdini is used to pre-process certain images.
Multiple images can be loaded in when it is launched and everything is controlled through my own midi interface on the spot.
It has to be noted that it wasn’t easy but very rewarding!
The reason for doing this is that I’ve always been disappointed by visuals presented at parties.
Most often it comes down to pre-rendered shapes with some glow, synced to a sloppy beat.
I had my first gig demo’ing the software last thursday, during the Amsterdam Dance Event.
As ‘Lange Man’ I tried my best to make the visuals work at the Green Room event, where friends hosted the Nachtwerk stage. The application ran stable for about 5 hours, without dropping the frame-rate. So far reactions were very (very) positive and can’t wait to use it again.
My plan is to improve the application over time, implementing a custom UI and new features that would allow it to run even faster, process larger amounts of data and be more versatile. I’m waiting on some footage that was shot during the night and am working on a small demonstration video. For now I just have some screen grabs.
Hopefully I can release the source code in some usable form this month, so anyone interested can dive in and see what was done or how to do such a thing.
First off let me thanks my girlfriend Maartje Willemijn Smits for shooting some beautiful images to work with.
I’d also like to thank MEMO for getting me started with a very open and scalable fluid solver solution, much appreciated!
