- runs on ANY mobile device with a front facing camera and 2-way video application (such as Skype)
- uses optic coupling to provide a novel interface between the mobile device and drive control hardware
- does not require any other mobile app for operation
- costs less than $100 in parts (excluding iPod touch) if you build it yourself
Tippy allows the user to achieve telepresence by leveraging powerful, existing mobile devices quickly and easily. Tippy can be easily packaged into other formats using different sized devices (smartphones, handhelds and tablets) without custom mobile applications and the control scheme can be adapted to control any kind of external hardware.
For more information, take a look at our conference paper HERE.
To see Tippy in action, check out the following video:
If you're interested in finding out more, please find our contact info here!
Nicolas and I have been working on the mobile phone choir as a part of the CHI2011 Interactivity demos that took place throughout this past week. We met a lot of really cool people and it was a great experience (1st CHI for both of us). Hopefully we'll be able to take the ideas generated at the conference and develop the system further.
Now a short description of how the system works:
Each mobile phone (in this case, an iTouch or iPhone) is running a voice synthesizer that generates a single note. The player (or singer, or whatever you'd like to call him/her) can control the vocal effort and pitch using the X-Y position on touch screen, and the vocal tract shape by tilting the device. Each device in the video is "tuned" to a certain voice (in this case modelling the spectral characteristics of a soprano, alto, tenor and bass voice, respectively). By default, the center pitch of each device is set so that combined, the 4 voices create a C major chord.
Another app runs on the iPad (controlled by me in the video below). This app is called the "Director" and it sends harmony information to each of the devices. For each chord that the director selects, the individual notes for that chord are sent to each device. This way, the overall harmonic decision is made by the conductor. However, each individual voice has the option to deviate from the selected center note, and Nicolas shows that around halfway through the video.
So Vincent, Benny and I have been working on a cute little project over the past few weeks. It's a compact telepresence robot, similar to the one that the famous Johnny Lee wrote an instructional for on his procrastineering blog. The key factor of our design is the optical coupling between the mobile device and the robot control hardware: in our implementation, we use Skype for two-way video but embed the control signals in the video stream as well - thereby reducing the amount of development (in terms of hardware and software interfacing). We've submitted an ICEC demo paper along with the following video, which should do a slightly better job at explaining how it works:
We hope to get together an instructional, and a DIY kit soon so people in the community can play around with the idea (if the video isn't enough to get you going :)
A few points on my personal motivation/comments for this project:
1.) Robots are cool; telepresence robots are also cool.
2.) Mobile devices are so powerful these days that you can do so much with them. However, for the DIY hobbyist/hacker, it's not that easy to interface these devices with custom hardware that you build. For example, on an iOS device (iPhone, iTouch, iPad etc), you are quite limited when it comes to interfacing hardware. Even if you do have a developer license, it's still pretty hard to get a physical connection to the serial port working (at the risk of voiding the warranty and blowing up the device, etc). Using platforms such as Android does away with the license hurdle, but the hardware interface is still not easy. This project is essentially a demonstration at a quick and dirty (but working!) solution to this problem. There are many ways that it can be improved (e.g. modulating the visual signal, auto-calibration of the sensors, etc etc), but it should provide a good starting point.
3.) I'm surrounded by a bunch of amazing people with different skills and really appreciate the opportunity to work with them. This project was definitely worth the evenings and weekends we spent at home or in the lab tinkering away. Good job guys!
