- runs on ANY mobile device with a front facing camera and 2-way video application (such as Skype)
- uses optic coupling to provide a novel interface between the mobile device and drive control hardware
- does not require any other mobile app for operation
- costs less than $100 in parts (excluding iPod touch) if you build it yourself
Tippy allows the user to achieve telepresence by leveraging powerful, existing mobile devices quickly and easily. Tippy can be easily packaged into other formats using different sized devices (smartphones, handhelds and tablets) without custom mobile applications and the control scheme can be adapted to control any kind of external hardware.
For more information, take a look at our conference paper HERE.
To see Tippy in action, check out the following video:
If you're interested in finding out more, please find our contact info here!
Nicolas and I have been working on the mobile phone choir as a part of the CHI2011 Interactivity demos that took place throughout this past week. We met a lot of really cool people and it was a great experience (1st CHI for both of us). Hopefully we'll be able to take the ideas generated at the conference and develop the system further.
Now a short description of how the system works:
Each mobile phone (in this case, an iTouch or iPhone) is running a voice synthesizer that generates a single note. The player (or singer, or whatever you'd like to call him/her) can control the vocal effort and pitch using the X-Y position on touch screen, and the vocal tract shape by tilting the device. Each device in the video is "tuned" to a certain voice (in this case modelling the spectral characteristics of a soprano, alto, tenor and bass voice, respectively). By default, the center pitch of each device is set so that combined, the 4 voices create a C major chord.
Another app runs on the iPad (controlled by me in the video below). This app is called the "Director" and it sends harmony information to each of the devices. For each chord that the director selects, the individual notes for that chord are sent to each device. This way, the overall harmonic decision is made by the conductor. However, each individual voice has the option to deviate from the selected center note, and Nicolas shows that around halfway through the video.
So Vincent, Benny and I have been working on a cute little project over the past few weeks. It's a compact telepresence robot, similar to the one that the famous Johnny Lee wrote an instructional for on his procrastineering blog. The key factor of our design is the optical coupling between the mobile device and the robot control hardware: in our implementation, we use Skype for two-way video but embed the control signals in the video stream as well - thereby reducing the amount of development (in terms of hardware and software interfacing). We've submitted an ICEC demo paper along with the following video, which should do a slightly better job at explaining how it works:
We hope to get together an instructional, and a DIY kit soon so people in the community can play around with the idea (if the video isn't enough to get you going :)
A few points on my personal motivation/comments for this project:
1.) Robots are cool; telepresence robots are also cool.
2.) Mobile devices are so powerful these days that you can do so much with them. However, for the DIY hobbyist/hacker, it's not that easy to interface these devices with custom hardware that you build. For example, on an iOS device (iPhone, iTouch, iPad etc), you are quite limited when it comes to interfacing hardware. Even if you do have a developer license, it's still pretty hard to get a physical connection to the serial port working (at the risk of voiding the warranty and blowing up the device, etc). Using platforms such as Android does away with the license hurdle, but the hardware interface is still not easy. This project is essentially a demonstration at a quick and dirty (but working!) solution to this problem. There are many ways that it can be improved (e.g. modulating the visual signal, auto-calibration of the sensors, etc etc), but it should provide a good starting point.
3.) I'm surrounded by a bunch of amazing people with different skills and really appreciate the opportunity to work with them. This project was definitely worth the evenings and weekends we spent at home or in the lab tinkering away. Good job guys!
A friend is doing a project that involves building a robot that finds and blows out candles placed within a course. They are thinking about using an infra red filter on a webcam to locate the candles, and asked me for suggestions. I know one could purchase filter sheets that serves this purpose, but thought there must be cheaper options/quicker options. First I thought about using red/dark red coloured cellophane, as the frequency is close enough to infra-red and maybe with enough layers it'll block out most visible light. Also, where to find the right coloured candy wrappers? Arts and crafts stores? There MUST be a cheap and simple way to get an infra red filter... (without having the crack open a TV remote or a Wii-controller - all these devices have infra red filters on the front).
After a quick google and 2 minutes of tinkering, I sent an excited message back to my friend with the following images:
The solution: Exposed film negatives!! It turns out developed film that has been exposed to light is a great infra red filter. The bit at the beginning of the roll is usually exposed when you put it into the camera, unless you load it in the dark. (I remember sometimes trying to do this under a blanket to save the first few shots of a roll... this way a roll of 36 can get you 38~40 shots, if you're lucky). You can see the tea-light candle and the film strip in my hand in the above image. And here is what the captured image looks like with the film strip taped onto the front of the web cam:
The solution was so quick, cheap and simple that I had to post about it, right away! :-)
Now the problem of course is the lack of availability of film these days... luckily I kept all my photos/negatives from before...
For our EECE541 project, we're building a 3D webcam chat system. I've been working on the headtracking portion, and decided this would be a neat way to demonstrate the results:
Ingredients:
-Processing sketch -Arduino Microcontroller -Servo -Two webcams: one for head tracking, one for the view. One is slightly hacked to fit onto the servo.
The system demonstrates three concepts:
1. Simple pixel-based 1D head tracking - take the difference between a static background and a live view, threshold it, and return the horizontal value of the top-most pixel
2. Fishtank AR/VR systems: utilizes the viewer's position to render a scene. Kinda like this.
3. Arduino/Processing: An awesome platform to work with hardware and software to prototype new ideas. Servos, webcams, image processing... all done with a couple lines of code!
A really simple controller for Google Earth using the Wii-mote. Simply maps a few buttons and actions to keyboard keys using GlovePIE. This was one of the first youtube videos I ever uploaded... somehow forgot about it...
