November 24, 2007
Reading for November 27th, are now posted. Enjoy!

October 2, 2007
To upload your thoughtless acts, create a new assignment page like any other lab. You'll see "Thoughtless Acts" listed as one of the assignment options.

May 24, 2008
This site has been archived and is no longer editable. Stay tuned for the next version, coming in the fall!

Bubblegum Sequencer

Project Members: 
Andrew McDiarmid
Hannes Hesse
Sooyeon Han

Problem statement

Since some time in the 1980's, step sequencers have been a popular tool to create and edit music, particularly drumloops. Drumcomputers like the Roland TR-808 have relied on this method heavily.

A step sequencer consists of a number of positions (typically 16) that represent points in time in a soundloop (in this case 16 note positions comprising one measure). By turning on or off these discrete positions (often represented as buttons with LEDs to indicate their status), the user places a sample at that particular point in the loop.

If there is only one physical row of buttons available, a selector typically changes the track so that the user can edit several instruments just by switching tracks.

When played back, a running light often indicates the current playback position of the loop.

However, only being able to directly manipulate one track at a time can be limiting. Software sequencers have long solved this by simply adding additional tracks as rows in the sequencing grid. Only recently, hardware like the Mononome were developed to address this. The Mononome consists of several rows of LED-lit buttons that can be used to control software step sequencers.

Proposed solution

We aim to make step sequencers more physical by using tangible objects to represent samples. Our project builds on the work of Durrell Bishop's Marble Answering Machine and Patten, Recht, and Ishii's AudioPad by mapping musical samples to colored marbles (or likely gumballs), which can be set into holes arranged in a grid on a tabletop. As in software sequencers, columns in the grip map to beats (time) in the loop. A camera placed below the surface of the table reads the colors in each column to generate the loop. Rows in the grid, however, no longer needed for sample assignment, can be mapped to other MIDI data, for example velocity, so that placing a marble higher on the grid will generate a louder sound.

While the mapping between samples and colored marbles is not ideal, such physical embodiment does provide differentiation that arbitrary assignment of samples to rows in a GUI or LED grid lacks. Users can assign samples to colors of their own choice (provided each color chosen is distinguishable by the camera from the others), and remember, for example, that black marbles are kick drums, and red ones snare drums. Mapping to color rather than some recognizable iconography requires some arbitrary memorization, but provides much more flexibility for customization.

Sketches / designs


Technical implementation

We plan to implement this system through a combination of computer vision and MIDI messages.

The table consists of a board with holes in it, so that the marbles are visible from below. Below the surface, a vertically mounted camera takes a continuous picture of the table. The image is fed into a computer vision system which divides the image into a grid and (possibly after some segmentation) computes average the average hue of each hole (proper lighting will be required). The system transforms the state of the board into a two-dimensional data structure and sends it to the next software layer to process.

The next layer can either consist of an improvised step sequencer or of a transformer that forwards the state of the board as MIDI messages to another music application, thus acting as a plugin for existing software.

bubblegum-sequencer.jpg61.92 KB
STEPS.gif37.83 KB
gumballs.jpg218.1 KB
gumballs.jpg304.38 KB


Observation - Great

Observation - Great observation of the abstract representation of the current step sequencers to the drum music. Adding a tangible aspect to this user interface seems like a great idea.

System -- Fun and playful idea to map music samples to colorful bubble gums! The size of the balls seems to matter. I can imagine having a handful of bubble gums in both of my hands and sequencing them. In your sketch, however, it looks like the balls are the size of billiards balls, which might allow only one or two balls to be manipulated at once. The main question is: are you maximizing the fun and playful aspect of the multi-colored balls? The table with holes acts as guide to snap the balls into places, but what about other playful movements people may make with bubble gum sized balls? Tossing, bouncing, hitting, juggling, etc? Can you drop a bag full of balls onto the table and create an accidental master piece? Are the balls supposed to be always neatly arranged on the grid?

Related Work -- A bit old but in case you havent done so yet, look at Musical Play Pen: tml AniMusic

Powered by Drupal - Design by Artinet