FaceOSC and AIR working example

I got a couple of questions about how to make an AIR app listen for FaceOSC, as shown in these two previous posts, so I've put together a basic 'one class' working example in Flash Builder.



What do you need?
FaceOSC (mac only, already included in my download project in the faceOSC folder)
Osculator (mac only)
This as3 Tuio library to make it easier listening for the OSC messages from FaceOSC (also already included in the src folder)

Actually you don't need Osculator to make the FaceOSC to AIR protocol work, but you need it to open the FaceOSC-Osculator.oscd file - in the osculator folder - that comes with FaceOSC in order to see all the OSC addresses that Kyle used for the facial gestures and expressions. Note that you cannot run the Osculator file and the AIR app simultaneously as they use the same port.

I also exported the project as an AIR app, which you can find in the air folder. A certificate is included, the password is 1234. Basically, you just need to open up FaceOSC and publish the Flash Builder project, or run the AIR app. You can download the example project here.

FaceWriter, typing text with your face - FaceOSC & Flash (Air)

Here's another video that shows an Adobe AIR app I've been working on. It listens for the data from FaceOSC, a great tool by Kyle McDonald; read more about it in my previous post.

This AIR app enables you to type text by moving your head and using some facial gestures.
It's quite basic for now, allowing you to add text, add spaces, do a backspace and clear the entire textfield. Functionality could be added of course, such as saving the final text to a local file or e-mailng it.



The hardest thing is to keep it all stable and precise, as you tend to change the orientation of your head quite easily. This would require a more advanced way of calibrating, and some functionality that lets you re-calibrate once in a while.

Anyway, I'm not sure if this could be benificial to anyone - I was thinking of people with spinal cord injury, but I'm far from familiar with that area. If you have any ideas, please drop me a line.

UPDATE 11-12-2011: I've put together a basic working example of an AIR project that listens for FaceOSC messages. You can find it here.

Face gestures, FaceOSC and Flash

The coolest thing I saw last week was a video by Kyle McDonald , who released a brilliant tool called FaceOSC, which handles facial gestures. FaceOSC is based on Facetracker by Jason Saragih.

With the help of this open source AS3 Tuio library I made an AIR project listen for the OSC messages sent by FaceOSC. OSC stands for Open Sound Control, although this example has nothing to do with sound.



The code to receive the OSC messages is quite simple. First, create a new OSCManager, add a listener, and start the manager. I found out that I had to use port 8338 by opening the osculator that comes with FaceOSC.

var oscManager: OSCManager = new OSCManager(new UDPConnector("0.0.0.0",8338,true),null, false);
oscManager.addMsgListener(this);
oscManager.start();

Then, in a function called acceptOSCMessage, check what kinds of OSC packets are coming in by checking the message address:

public function acceptOSCMessage(oscmsg:OSCMessage):void
{
//To see the different addresses(i.e. references to the gesture values)
//trace("address: " + oscmsg.address)

if( oscmsg.address == "/pose/orientation" )
{
//roll = oscmsg.arguments[2] * rollFactor;
//speed = oscmsg.arguments[0] * speedFactor;
//your code here
}

}

To make this last function work you need to implement IOSCListener.

The 3D scene is from the book Papervision3D Essentials.

UPDATE 11-12-2011: I've put together a basic working example, which you can find here.