Monday, July 11, 2011

Face gestures, FaceOSC and Flash

The coolest thing I saw last week was a video by Kyle McDonald , who released a brilliant tool called FaceOSC, which handles facial gestures. FaceOSC is based on Facetracker by Jason Saragih.

With the help of this open source AS3 Tuio library I made an AIR project listen for the OSC messages sent by FaceOSC. OSC stands for Open Sound Control, although this example has nothing to do with sound.



The code to receive the OSC messages is quite simple. First, create a new OSCManager, add a listener, and start the manager. I found out that I had to use port 8338 by opening the osculator that comes with FaceOSC.
var oscManager: OSCManager = new OSCManager(new UDPConnector("0.0.0.0",8338,true),null, false);
oscManager.addMsgListener(this);
oscManager.start();

Then, in a function called acceptOSCMessage, check what kinds of OSC packets are coming in by checking the message address:
public function acceptOSCMessage(oscmsg:OSCMessage):void
{
//To see the different addresses(i.e. references to the gesture values)
//trace("address: " + oscmsg.address)

if( oscmsg.address == "/pose/orientation" )
{
//roll = oscmsg.arguments[2] * rollFactor;
//speed = oscmsg.arguments[0] * speedFactor;
//your code here
}

}

To make this last function work you need to implement IOSCListener.

The 3D scene is from the book Papervision3D Essentials.

UPDATE 11-12-2011: I've put together a basic working example, which you can find here.

4 comments:

  1. hi jeff, that's a good reference between faceOCS and flash !

    Would you let me know where you find the raw data property like "/pose/orientation" ?

    ReplyDelete
  2. Oh .. I got it.
    when we trace "oscmsg.address" should display more sting like "/pose/orientation" , right ? but any doc / api for property ?

    thanks !
    Louis

    ReplyDelete
  3. hi Louis,
    you can open the osculator that comes with FaceOSC to see what the addresses/properties are, that's all, no docs :)

    Jeff.

    ReplyDelete
  4. Good morning.
    Thank you for your articles and blogs.
    I was reading the facial recognition and have a doubt.

    With this program we could recognize a person who was in front of a web cam in real time?

    Thank you very much for your work
    that God fill you with light.
    best regards

    ReplyDelete