IPoXP Final Project Proposal
IP over Xylophone Players
Members: Stuart, Emily, and Yoon
Introduction
IP over Xylophone Players (IPoXP) aims to construct an Internet connection between two computers using xylophone-based Arduino interfaces, with human operators transmitting data by striking designated keys. Inverting the traditional mode of human-computer interaction, the computer uses the human as an interface to communicate with another computer.Overly-Technical Background
All messages sent on the Internet follow the Internet Protocol (IP), meaning that they are separated into packets, each of which are made up of a string of bits -- 1s and 0s. IP is founded on the idea that these bits can be transmitted via any medium -- copper wire, fiber optic cable, radio waves, etc. -- as long as you have a standard way to encode and decode them.When the Arduino is established as a serial-based network interface in each computer, the operating system automatically sends outgoing Internet traffic in packets as strings of ASCII characters to the Arduino, and all ASCII characters the Arduino sends to the serial port are treated as incoming packets. The Arduinos turn each 8-bit ASCII character into two 4-bit hex characters, which correspond to one of 16 notes on the xylophone.
Image 0: Here is one of the smallest IP packets, a ping request, with its constituent elements decoded (top) and represented as a series of hexadecimal characters (below). Because of the standardization of IP, all of that information at the top is compressed into 84 digits. For example, the blue highlighted portion represents the source IP address, my computer at 192.168.1.21, which appears as the hex characters c0 a8 01 15 starting with the 53rd digit. An ethernet cable might send these as 32 1s and 0s, high and low pulses of electricity, while a modem might have to only send two tones to represent this entire IP address, if it has 65536 distinct tones. For our interface, the IP address 192.168.1.21 would be transmitted as 8 notes.
Artifact Overview
Image 1: Materials. To bring our Internet Protocol over Xylophone Players (IPoXP) project to life, we propose using two xylophones, 4 xylophone mallets, 32 Piezos, 32 LEDs, two Arduinos, 4 multiplexers, two computers, and lots of wiring.
Image 2: Functional diagram of the basic units of the interface. Two Arduinos are connected via USB/serial link to two laptops running a Unix-based OS (Linux or OS X), one with access to the Internet and one without. Each Arduino is connected to their local Xylophone's LED ports and the remote Xylophone's piezo sensors.
Storyboard
Image 3: When a packet is to be sent, Computer 1 translates the packet into a string of ASCII characters, which it sends to the Arduino over the USB/serial interface. The Arduino converts each 8-bit ASCII character into two 4-bit hexadecimal values, which correspond to the 16 lower keys on the xylophone. Arduino 1 will light up a LED on the corresponding Xylophone 1 key for the first digit in the packet.
Image 4: When the user strikes the key, it vibrates and activates the piezo sensor mounted on the bottom of the key on Xylophone 1, which is sent to an input on Arduino 2. Arduino 1 senses that a key has been pressed (method to be determined -- sensor in the mallet?) and lights up the key corresponding to the next hex character in the packet.
Image 5: The piezo sensor on xylophone 1 is linked to a (multiplexed) input on Arduino 2, which is connected to Computer 2 via serial/USB. Arduino 2 decodes each pair of inputs as a single ASCII character, and stores it in its buffer.
When the packet is finished, an ‘End of File’ key will be used to signal that there are no more characters in the packet. Arduino 2 will send the string of ASCII characters over serial/USB to Computer 2. If Computer 2 sends a reply to the packet, this process repeats in the reverse direction: Arduino 2 will convert the next packet message into hexadecimal and light up the corresponding LEDs on Xylophone 2 in turn. The process repeats until the packet exchange is complete. If a user strikes the wrong key, this EOF key can also be hit to end the packet early, which will generate a malformed packet, a manageable error in any modern network stack.
Current Progress and Changes from Original Proposal
Image 6: This is a photo of our LED display prototype. We originally planned to mount an LED at the top of each key, but due to the configuration of the xylophones that we purchased, there wasn’t room to do so. We opted to add a map above the keys on the lid of the xylophone in order to help the user identify which key to strike without getting in the way of the other keys or the player’s mallet.
Image 7: This photo shows the bottom of the xylophone case through which we bore sixteen holes for the piezo sensor leads and soldered them to a multiplexer. One long ground wire was stripped and tapped horizontally across one side of the case.
Image 8: This image shows the xylophone keyboard with multiplexed inputs hooked up to an Arduino. Piezo sensors were taped to the bottom of each white key using transparent tape.
Installation and User Experience
This project is primarily an art installation which we envision living in a museum like the Computer History Museum in Mountain View where it can be used, appreciated, and understood by an older audience. By contrast, putting the installation in the Exploratorium would be a less than ideal home because the target Exploratorium patron would probably be too young to have the patience or desire to play the xylophone keys in the correct order instead of extemporaneously--as children do in normal play, thereby missing the point of the installation and spoiling the child’s fun.
Although we cannot completely control the amount of enjoyment people might experience while playing the xylophone IP installation, the intended user experience is meant to be mechanistic and austere. This effect will be achieved through the use of monochromatic LEDs to instruct the player which keys to strike and a monochrome display of the messages as they are stored and sent between the two computers. We want the player to see the packets as we imagine computers see them, as unsentimental electrical impulses. We are also hoping that the farrago of notes coming from the xylophone as it is played to add to the sense of dystopia.
