New MSAgent engine for Android

Can you imagine Merlin, Genie, and all of the other MSAgent characters flying around on an Android device? Well, now you can, because I have begun work on an MSAgent engine for Android.

I have been able to get Merlin to appear in a test app running in an Android emulator.  And on modern Android screen resolutions, he doesn’t take up a whole lot of screen space, which is a plus, so he might actually look good on phones and tablets alike.

I have some basic pieces working:

  • load ACS file from device/SDCard
  • an asynchronous request queue (TODO: canceling requests, syncing requests between multiple characters)
  • playing animations (TODO: stopping/interrupting animations)
  • show/hide, w/ state animations
  • moving the Character around the screen, either with touch input (dragging) or in code, w/ state animations
  • gesturing, w/ state animations

Although it is possible to display a Character on top of the app’s UI, and even on top of Android screens when the app is in the background, I did come across some odd quirks in managing the Character’s top-level window.  This might limit MSAgent to just in-app usages.  At least the character would stay on top of the app UI (good for tutorials and such).  Will keep experimenting with it.

I did start work on TextToSpeech output, but discovered some shortcomings with Android’s TTS framework, the most important of which is that there are no notifications of when individual words/phonemes are spoken.  That means words in the speech balloon cannot be highlighted while the character is talking (unless the text is broken up into individual words and feed into the TTS engine one at a time, which is likely to cause lag).  That also means that proper mouth overlays cannot be displayed.  This is a big issue for user experience.  Not sure if these will be solvable without resorting to third-party solutions.

I am trying to make it as functional as possible.  I’m not really expecting this to be a full implementation like on Windows, just due to some shortcomings of Android and differences between how Android and Windows work in general.  But it is an interesting challenge to see how far I can take it, it gets me back into playing with MSAgent, and it helps further my learning of Android app development.

I will try to put together a little video soon to show what I have so far.

When I have an Android library that is viable outside of my development environment, I will probably put it up on SourceForge or GitHub or something.

8 thoughts on “New MSAgent engine for Android”

  1. hello,

    That seems great!
    Im looking forward to test it.

    Also, do you know about any project/framework I can use to start animating the characters in an intelligent way, replicating some of the human behavior. For instance, I would like the character to say something when it starts depending for instance of the hour of the day; the weather; if it was already open in that day; etc… Then, to do or say something in other situations.

    I started thinking about spliting its beahvior in groups and inside each group define some actions.

    But I want to achieve this in a way I can make it grow in the future and be easily to add more actions. I looked at prolog, but it is difficult to get a server and it is not a ready framework. I believe someone already started something similar, but I didnt found nothing…

    thank you,

    sérgio

  2. I was wonder if you have some advance on this project.
    As soon as you have some, please let us know, I get some kids programs that i want to back to life. The kids love the intercative character and learn a lot with his virtual friend.

    Regrads

    1. The project has been on hold for awhile due to lack of time to work on it. I want to get back to it some day. Keep an eye on this blog for future activity.

  3. Hello my Brother,

    In 2020, will you continue with this project? We support the new version of msagent, we would love to see this new version with these features mentioned, congratulations

    Thanks!

Leave a Reply to Remy Lebeau Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.