While likely not originally designed for typical IoT platforms, it is possible to play a MIDI keyboard in node-red via the GUI dashboard using a relatively simple setup.
For this somewhat unusual exercise, you will need:
- A Rasbperry Pi (can do this on a desktop platform, but whats the fun in that?)
- A USB-MIDI capable Keyboard. (You can use a USB-MIDI adapter with an older MIDI keyboard without USB of course)
- Install of the most recent node-red, with the dashboard. The only additional node you'll need is node-red-midi.
For further details check out the detailed writeup here.
Here's what it looks like in action. The computer monitor shows the flow as well as dashboard UI. Excuse my rats nest under the desk.
Some neat features:
- Works on any platform in the browser
- Allows concurrent connections, so more than one person can play with it at the same time on different devices
Some obvious limits I can think of:
- limited UI from node-red-dashboard for this purpose; a row of buttons is not a great interface for an instrument
Here's what I've been working on at Infusion lately: documenting the Raspberry Pi based plant watering system that measures the moisture content of the soil and triggers a relay-driven valve to water the plant:
Control software was built using node-red, with a FRED endpoint to talk to the cloud-hosted node-red instance that allows a public dashboard to be displayed (public dashboard available here).
One of the awesome things about node-red is that you can easily view the dashboard from a browser, and with FRED you can access it anywhere:
I fiddled for a good part of the day trying to find a low-latency local network video streaming solution to implement a "FPV"-like control for the robot using a RPi camera module. It turns out a bunch of existing solutions (like VLC streaming etc) have really high latencies. This is OK for things like a security camera, but not so much for realtime control.
Finally, this rather hacky solution using netcat appeared to work best:
1. On the client (viewer/controller, running on a Mac) side, run:
netcat is used to listen and send; raspivid grabs the video and pipes it to netcat, and on the other side netcat pipes the received data and spits it into mplayer.
This was the only way I could get less than 1s of latency on the video feed, although it still wasn't great. Having a good USB wifi dongle also helped a bit. Finally, changing the dimensions of the video using raspivid actually made it much slower, probably due to software resizing of each frame on the fly compared to just sending out the raw feed from the camera.
Next steps: get some more interesting sensor info from the PiShield, and get a separate power source since there seem to be random system freezes when the motor and Pi are both running off the same power bank.
UPDATE Oct 26:
with the advice of a very helpful redditor who suggested gstreamer, I gave it a shot this evening.
On Raspbian, gstreamer1.0 is currently already in the official repos, so no need to add custom sources as these earlier instructions.
For OSX, I just downloaded and installed the latest compiled version from the official source. It also handily tells you that by default the path for the commands are at
/Library/Frameworks/GStreamer.framework/Commands/
From here, we can either have a tcpserver on the Pi that a client can connect to, in which case:
Mac (note explict path to gst-launch-1.0 command, since I haven't added it to path):
/Library/Frameworks/GStreamer.framework/Commands/gst-launch-1.0 -v tcpclientsrc host=IP_OF_RPi port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! osxvideosink sync=false
Now UDP would be faster, and in this case it would be similar to the netcat example where the RPi would define the UDP "sink" using the IP address and PORT of the viewer (on the Mac), which would yield the following:
RPi (note destination should be explicitly defined by the source this time):
Eye-balling the two versions, I feel like the UDP version is slightly snappier, but considering that my network is not very congested, I have a feeling the difference wouldn't be as much as if I had a lot of traffic running. Regardless, this is a HUGE improvement over the existing solution! Should try to do some timing tests to see if I could get a better measure of the actual latency - perhaps using a photo of a stopwatch like this guy did (which was one of many sources I originally consulted for the netcat solution above as well)! finally, here's a test on the actual video latency of the setup above: looks like its somewhere between 150 and 200 ms.
FINAL UPDATE (Jan 2017)
Yet another option, is to use RPiCamWebInterface. In the end I didn't get around to do the latency measurement, but it feels *almost* as fast as the previous solution. The other bonus is the client user interface is far nicer!
Here's a sample video showing the stream in action:
I have a habit of back filling this blog, typically in a sneaky way such that when you visit, it looks like the content has been gradually updated over the years. However, currently due to PhD stuff, family/baby stuff, and various other things, I haven't managed to backfill adequately to get rid of the obvious gaps, so this post will look somewhat unexpected... but here goes: after starting this project about two years ago with Infusion Systems, the PiShield is finally live on Kickstarter!!
