Max is a Pushover is a digital media art installation created and exhibited
in February 2008 by
artist Max Kazemzadeh and programmer
at the University of North Texas Artspace in Fort Worth, Texas.
The UNT Artspace FW gallery was open from September 2005 through March 2008 and was host to eleven avant-garde exhibitions.
Max is a Pushover is a digital fusion
of real space with virtual space.
The ProgrammingThe code for Max is a Pushover was written in C++ for Windows. I was intrigued by the problem of capturing video from DirectShow, performing vision computation frame by frame, then injecting the finished frames into a Direct3D animation stream. The challenge is that video input comes very slowly, on the order of 15-20 frames per second for a standard el-cheapo webcam, whereas the animation frame rate is typically 60 frames per second. This version also has pixel shaders, although they are used in this demo. Although the software could easily have been written using one of the standard vision programming APIs for artists, it would not have been able to match our screen resolution and frame rate.
The following diagram shows the input from the video camera and the output to the screen as blue boxes. These devices are asynchronous and operate at dfferent frame rates. We maintain two pairs of buffers shown in the diagram as black boxes, one pair for video processing (top) that should be in system memory for processing by the CPU but can be in video memory for processing in the GPU if desired, and one pair for animation (bottom) that should be in video memory. The video camera asynchronously renders to one of each pair (called a back buffer), while the other is used for video processing (top) and animation (bottom). When the video camera finishes loading a new frame into the back buffers, they are page-flipped with the corresponding front buffers. Synchronization is achieved using critical sections around very short pieces of code. The red boxes denote computation, the upper red boxes representing visual processing tasks such as edge detection, and the lower red box representing animation tasks such as sprite rendering. Notice that the lines on the left side of the diagram may process as slowly as 12 frames per second, whereas the lines on the right side of the diagram may process as quickly as 60 frames per second.
An obvious question is "Why do we need 4 buffers?" The video processing needs a separate buffer from the animation so that the two tasks do not interfere - there is no use in performing edge-detection on sprites, for example, and each of these needs two buffers to cope with anynchrony, so that be back buffer is being filled while the front buffer is being processed.
The ArtMax started with some frames of a video of himself walking and jumping.
which he then edited into a series of sprite frames that looked like this: