Here's another video that shows an Adobe AIR app I've been working on. It listens for the data from FaceOSC, a great tool by Kyle McDonald; read more about it in my previous post.
This AIR app enables you to type text by moving your head and using some facial gestures.
It's quite basic for now, allowing you to add text, add spaces, do a backspace and clear the entire textfield. Functionality could be added of course, such as saving the final text to a local file or e-mailng it.
The hardest thing is to keep it all stable and precise, as you tend to change the orientation of your head quite easily. This would require a more advanced way of calibrating, and some functionality that lets you re-calibrate once in a while.
Anyway, I'm not sure if this could be benificial to anyone - I was thinking of people with spinal cord injury, but I'm far from familiar with that area. If you have any ideas, please drop me a line.
UPDATE 11-12-2011: I've put together a basic working example of an AIR project that listens for FaceOSC messages. You can find it here.
one of the most useful things you could add would be the simulation of actual keypresses. e.g.: http://stackoverflow.com/questions/1918841/how-to-convert-ascii-character-to-cgkeycode
ReplyDeleteDo you mean simulating actual keypressses to use them to control other programs and your native computer system? And simulating the mouse, would be great...that's what osculator does, right?
ReplyDeleteexactly, once you are simulating actual keypresses then you don't have to worry about implementing things like "saving" and "emailing" the text.
ReplyDeletemaybe just controlling the mouse and having a simple "keyboard app" would be best :)
So that would for instance mean typing with a keyboard app and having the result text typed into another program. I could listen for FaceOSC messages in the Keyboard app, send osc messages from keyboard app to osculator, and from there simulate the keyboard. Not sure if I would run into port/socket problems, but worth a try :). The mouse is another thing, as I can only use the FaceOSC orientation values once...
ReplyDeleteActually I would only need the mouse to open another program, activate it, and give it commands (e.g. send email or save). So I can implement a command in the keyboard to osculator flow that switches between virtual keyboard and desktop. The air app doesn't have to be activated to work properly.
ReplyDeleteI wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your site to check out the new stuff you post. curso de mecanografia
ReplyDelete