Face gestures, FaceOSC and Flash

The coolest thing I saw last week was a video by Kyle McDonald , who released a brilliant tool called FaceOSC, which handles facial gestures. FaceOSC is based on Facetracker by Jason Saragih.

With the help of this open source AS3 Tuio library I made an AIR project listen for the OSC messages sent by FaceOSC. OSC stands for Open Sound Control, although this example has nothing to do with sound.



The code to receive the OSC messages is quite simple. First, create a new OSCManager, add a listener, and start the manager. I found out that I had to use port 8338 by opening the osculator that comes with FaceOSC.

var oscManager: OSCManager = new OSCManager(new UDPConnector("0.0.0.0",8338,true),null, false);
oscManager.addMsgListener(this);
oscManager.start();

Then, in a function called acceptOSCMessage, check what kinds of OSC packets are coming in by checking the message address:

public function acceptOSCMessage(oscmsg:OSCMessage):void
{
//To see the different addresses(i.e. references to the gesture values)
//trace("address: " + oscmsg.address)

if( oscmsg.address == "/pose/orientation" )
{
//roll = oscmsg.arguments[2] * rollFactor;
//speed = oscmsg.arguments[0] * speedFactor;
//your code here
}

}

To make this last function work you need to implement IOSCListener.

The 3D scene is from the book Papervision3D Essentials.

UPDATE 11-12-2011: I've put together a basic working example, which you can find here.