Thursday, February 25, 2010

Remotely controlling a desktop AIR app with an Android phone

AIR 2.0 has support for UDP sockets, which allows you to send and receive messages using the Universal Datagram Protocol. This means that you can communicate with other applications without the need for an extra server.

I used RemoteDroid for this demo, which is an Android app that turns your Android phone into a remote control, using your own wireless network. Originally, the app gives remote access to native applications on your computer and simulates the mouse - moving it and pressing left and right buttons - but you also need to download a server on your computer to make that work.

However, since AIR 2.0 supports UDP, you don't need an extra server to make your Android phone communicate with an AIR app. In this video, I made the AIR app listen for UDP messages sent by the Android app, to control a paperplane in a Papervision3D scene. So basically, you can turn your Android phone into a game controller.

RemoteDroid has just been open sourced, which gave me a chance to take a look at the source code of the Android app. I added the accelerometer data of the phone to it (locally, I didn't contact the owner of the project yet), and in the video you can see how I:
- control the speed of the paperplane by pitching the phone
- steer the paperplane by rolling the phone,
- zoom in and out by pressing the left 'button' on the phone's touch screen
- change the camera mode to a random perspective by pressing the right 'button' on the touch screen
- change the camera perspective to the Third Person, First Person and default perspective by pressing the 1, 2 and 3 keyboard keys respectively.



The Papervision3D scene in the video stems from chapter 6 of the Papervision3D Essentials book.

Of course there are other apps than AIR that support UDP. For instance, I also managed to control a Unity file this way, so another video will follow soon.

19 comments:

  1. Nice! I did the same with a Nintendo DS (http://bit.ly/5Gl0j0), UDP opens new possibilities of interactions with AIR, awesome! :D looking forward to your next video.

    ReplyDelete
  2. hey, that's nice man, I hadn't seen that one yet :)

    ReplyDelete
  3. http://recombu.com/news/android-phone-turned-into-adobe-air-game-controller_M11492.html

    ReplyDelete
  4. Original and creative... Like it. I need to study more about UDP and look forward to your next video that you have mentioned.

    ReplyDelete
  5. This post is fantastic. I'm using UDP to simulate touch events from the iPhone to the Flash platform. http://multitouchup.com/?p=428

    I'll have to check out the source code of Remote Droid to add an android option for the simulator.

    ReplyDelete
  6. Love the Sifl & Olly song inclusion.

    ReplyDelete
  7. Great work Jeff!

    Does this take multiplayer into account by any chance?

    I'd be interested in knowing whether there were any scenarios in which it could ;p

    ReplyDelete
  8. hey Rick,

    I guess that would be possible, as the source ip address is being sent with the datagram packet. And you probably can work with different udp sockets for different clients - if needed.

    Haven't tried none of that yet though :).

    ReplyDelete
  9. Hello!
    I just looked your video about remote control application written in Adobe Air.
    I want to ask if you are not going to lay out the source of your project?
    If not, then perhaps you advise me some resources or books to study the topic.
    Oh very much want to repeat your experiment. I am a Actionscript programmer, but in this subject know nothing yet.
    Thanks!

    ReplyDelete
  10. hi Кирилл,

    lack of time kept me from open sourcing or explaining this thing, but a good starting point is the Datagram class, and getting familiar with byte arrays.

    I hope I have some time over the next months to clean it up and make it publicly available.

    ReplyDelete
  11. amazing work jeff! you have inspired me to start scratching the surface w/Unity

    ReplyDelete
  12. hoi Ewoud,

    if you mean RemoteDroid, it's available on the Android market. However, the edited accelerometer version isn't, and neither is the Air app.

    Jeff.

    ReplyDelete
  13. Hi jeff

    I have remote Droid already but why can't i download the acceletrometer one? You can put it on this website, can you?

    Ewoud

    BTW. Your the first developer that is reading his own comments

    ReplyDelete
  14. AIR 2.0 is incredible I was waiting for an idea like this because it doesn't need extra server, I recommended you this it'll do your life easier.

    ReplyDelete
  15. This is pretty cool. Inspired me to try and add support in my Air app to be remote controller using RemoteDroid.

    One question, when I recieve the message in my AIR app, I just see:

    Received from 192.168.0.14:36671> /mouse
    Received from 192.168.0.14:36671> /keyboard

    How do you get more info like mouse coordinates or keycode etc?

    Thanks,

    ReplyDelete
  16. Those are awesome! I kinda got but any help will be great. Thanks for sharing this informative post. Keep posting...

    ReplyDelete
  17. Technology nowadays never seizes to amaze me!

    ReplyDelete