Final Project: Interactive Audiovisual Interface

March 13, 2016

 

 

Our group began the final project with an idea to create an interactive music 

experience. We played with different ideas on how to interact with the musical 

environment including color and shape cues and DJ-ing with dance. Ultimately, we 

decided to try to modify audio and visual output with both shadow movement and 

pressure pads. 

 

The basic idea was to project a video piece onto a two-way projection screen and 

have people stand in front of the projector to create a shadow. That shadow would then 

be read by a webcam behind the screen, and the shadow’s actions would be translated to 

a change in the video output. Simultaneously, four pressure pads connected to a makey-

makey board will result in audio output change. 

 

This system design catered to individual strengths of the members of the group. 

Gus’ main focus was designing and writing a max patch to read visual input from the 

webcam in order to modify video. He also mixed the audio for the piece. Logan focused 

on the video art piece, designing and editing the visuals in final cut pro. Annie designed 

and constructed the pressure pads. 

 

 

 

As with any system, we ran into more than a few snags along the way. From a 

half-baked idea to implement LEDs into the foot-pads, to last minute coding bugs, we all 

ended up learning a great deal about each other’s specialty through team troubleshooting. 

In creating the pressure pads, Annie spent a lot of time choosing materials to build with, 

stumbling upon the frosted corrugated plastic as an alternative to plexiglass when the 

plan was to light the pads internally. The next challenge was to ensure that the circuit 

would remain broken until the pad was pressed, while having a reliable connection at that 

point. In coding, Gus first had to find ways to practically realize our vision of sensing 

shadow input via webcam. Next, he spent time researching and field testing the options 

for sensing systems and taught himself various programming techniques needed to 

finalize the max patch. Ultimately, the largest hurdles included incompatibility on his 

computer (resulting in an untestable patch), an unanticipated bug in the final presentation 

– We theorize that the jitter object was at fault in the end, failing to output. Logan spent 

his time designing and editing the visual output feature of the instillation. Major 

troubleshooting on his end included rendering video to be compatible with the max patch 

as well as a last minute scramble to edit in white bars (for sensing) when the code failed 

to overlay them. 

 

In the end, our instillation was extremely successful in sensing pressure cues and 

subsequently mixing audio output, and the video added to the overall aesthetic and 

ambient quality of the experience despite not reacting appropriately to the shadow-cue.

Please reload

Recent Posts

January 13, 2016

January 6, 2016

Please reload

© Psyndicate 2020

This website powers the conservation

of the Amazon Rainforest. Learn More