Saturday, May 14, 2011

Mobile Phone Choir

Nicolas and I have been working on the mobile phone choir as a part of the CHI2011 Interactivity demos that took place throughout this past week. We met a lot of really cool people and it was a great experience (1st CHI for both of us). Hopefully we'll be able to take the ideas generated at the conference and develop the system further.

Now a short description of how the system works:

Each mobile phone (in this case, an iTouch or iPhone) is running a voice synthesizer that generates a single note. The player (or singer, or whatever you'd like to call him/her) can control the vocal effort and pitch using the X-Y position on touch screen, and the vocal tract shape by tilting the device. Each device in the video is "tuned" to a certain voice (in this case modelling the spectral characteristics of a soprano, alto, tenor and bass voice, respectively). By default, the center pitch of each device is set so that combined, the 4 voices create a C major chord.

Another app runs on the iPad (controlled by me in the video below). This app is called the "Director" and it sends harmony information to each of the devices. For each chord that the director selects, the individual notes for that chord are sent to each device. This way, the overall harmonic decision is made by the conductor. However, each individual voice has the option to deviate from the selected center note, and Nicolas shows that around halfway through the video.


More on this later...