I was recently thinking about art and film and photography–three things that are inextricably intertwined. I wondered how I could combine the three in a somewhat unique way. My thinking cap on as I walked my dog, Winston, the other night, I started to consider that a film, after all, is at it’s most basic level, a series of thousands of photographs, flying by on the screen at 24 or 30 frames per second. I wondered if there was a way to see all of these frames at once. At first, I considered what a film would look like with each frame layered on top of the other with a transparency set at a low number, such as 5%. My mental exercise quickly told me that this would, after just a relatively few frames, turn into a grey mush as each frame’s color combined with the others. Then I started thinking about what it might look like presented as individual frames arranged in a giant grid in one image, perhaps each frame’s average color making up one pixel of the overall image. This sounded interesting, but then I started thinking about the technical aspects of doing this and decided to defer that project for a rainy weekend. My final idea–much easier to implement from a technical perspective, was to take each frame of a film and compress it to an image one pixel wide by some arbitrary number tall (in this case, 720px). This would allow color averaging in only one dimension rather than two, a much easier proposition. My first task was to choose a movie to experiment with. I happened to have a copy of 127 Hours on hand. (And here’s where I confess that I’m not 100% sure on the legalities of this). I went about extracting each frame from the film, using an application called Virtual Dub to export out a JPG of each individual frame–all 166918 of them. On my desktop machine, this took about an hour and left me with roughly 2GBs of images (I wisely limited the output resolution of each JPG to 592 x 320, figuring this would be plenty to get the gist of the horizontalcolor mapping for each frame.
My next task was to resize each of these images from 592px wide to only 1px wide. I pondered this for a bit, wondering how to best accomplish this task. I briefly considered using batch automation in Photoshop CS5 to do the resizing, but a bit of experimenting quickly showed that while PS is an awesome application, it just has too much memory overhead to do the job–it would crash pretty quickly. I then thought about writing an app in .Net to do the work for me, but I thought that there had to be a way to do it without having to do any programming, as I knew that I’d have to do some coding later and wanted to put that off for a bit. Then I remembered Irfanview, a great little image utility that has tons of batch operation options and is pretty powerful, but also maintains a small memory footprint. I fired it up and got to work resizing, knowing that the application’s bicubic resampling on resize would handily average all the horizontal colors across the screen into a single vertical line. Less than half an hour later, I had a folder full of 166918 images, each 1px wide by 720px tall.
My final task was to combine each of these images–each derived from one frame of the movie–sequentially into one final image. Since Irfanview and no other image editor that I know of would do this automatically, I had to roll my own utility. Firing up VB.Net 2010, I quickly hammered out a function to do the job:
Geeky, eh?
My app looped through each image in my folder, appending the frames together sequentially. Unfortunately, I quickly discovered that there’s only so many images it could append before the app really started eating up my 8GB of RAM and would slow to almost nothing. This started me thinking and I decided to break out the job into parts that I could combine later in Photoshop. So, I divided the files up into five sets and started processing each set. But then I started thinking about the size of the final image. If I maintained my size, it would balloon to 120 megapixels in size…not something I wanted to deal with (though, not a size that’s totally insurmountable from a processing side in Photoshop). I made the decision to only use every 10th frame, as that would reduce my processing by an order of magnitude and still give me 3 lines on my final image from each second of film (okay, so 2.97 frames per second for the really detail-oriented)–plenty of color detail for my needs.
I processed the five sets of images and, after about 45 minutes, I had five .png files. Five minutes in Photoshop later, I had one giant image that was 16690px x 720px. Nice, but not very practical to view as it was so wide. Resizing (and fully realizing that I would lose liner detail as you can’t really make the individual lines representing each frame any narrower than one pixel), I finally had a nice image in a photographically-grounded 4×6 ratio.
End result: 127 Hours in one image:
