OSGi on AppEngine?

As many people know, Google recently announced support for Java on its AppEngine platform. As a connoisseur of OSGi, the first natural thing that came across my mind was quickly I could get OSGi running on it. Well, I’m happy to announce that we are part way there. I have OSGi running locally on the AppEngine SDK using the Equinox servlet bridge. I’ve managed to get a basic environment running with Equinox (and some bits of p2) and the Knoplerfish console:

appengineosgi

This wasn’t easy and was full of problems. If you read the AppEngine documentation, the sandbox is quite restrictive. First, there were a few places we were setting the context classloader and AppEngine doesn’t like that. I managed to hack the Equinox servlet bridge and the Equinox http service to not do that (it’s only needed in certain cases). Second, there’s a JRE “whitelist” of classes you can access that can cause trouble. I think this isn’t much of a problem if we define a custom OSGi execution environment for AppEngine. I attempted to do this since in Eclipse 3.5, we support the definition of custom execution environments. However, I found that the current whitelist doesn’t seem to be complete (maybe someone from Google can help). If you solve those problems, you can get Equinox working locally within the AppEngine SDK (as seen in the screenshot above). Here’s a zipped project if you’re interested (see the Equinox OSGi shared launch configuration and just go to localhost:8080/console).

In reality, this is only half of the battle. If we actually try to deploy what we have created above, things will fail. The most complicated problem I came across with running OSGi on AppEngine was you can’t write anywhere to the file system. Equinox and other OSGi frameworks need a place to write its state.  I think we can get around this if we write a custom storage adaptor hook in Equinox that uses the AppEngine storage facilities, however, that’s going to take some effort.

That’s it for now. I plan on making some more progress next week… are there any other people that want to see OSGi running on AppEngine? Have any other people out there been experimenting with OSGi and AppEngine? Did you hit the same roadblocks? Do you want to collaborate?

14 Comments
  • Posted at 2:48 pm, April 10, 2009

    Waoouw,

    That sounds exciting ! How much cost to code the adapter to use JDO ?

    Thanks,

  • Posted at 4:14 pm, April 10, 2009

    Nice work, will be cool to see it running on the actual AppEngine 🙂

    BTW, you also can’t create threads in the actual AppEngine sandbox (it’s primarily request driven along with a cron scheduler service for admin tasks, each request/cron task is limited to 30s) – but there’s probably a fair bit of OSGi functionality that would still work inside a single thread…

  • Pascal
    Posted at 5:01 pm, April 10, 2009

    How did you work around the limitation of thread creation? Equinox needs a slew of them for the purpose of event delivery.

  • Posted at 5:32 pm, April 10, 2009

    Chris – are you sure this will work? Last I read about AppEngine it was a way to fit Web apps onto the Google infrastructure, which is all highly customized, for “automatic” scale out. Aren’t the Google APIs sort of required?

    Eric

  • Marcel Offermans
    Posted at 6:17 pm, April 10, 2009

    This all sounds a bit like the restrictions Google Android has (which also places some restrictions that complicate running OSGi). Abstracting the OSGi bundle cache itself to use a different API sounds possible, but the bundle data store is built on top of the File abstraction, which is not that good an abstraction. 😉 Running OSGi in cloud like environments is definitely something I’m interested in though.

  • Posted at 6:21 pm, April 10, 2009

    I don’t know, I just imagined the Google APIs would create some difficulties, but if they provide a JDO adapter maybe they have anticipated the issues.

    Ok, well, interesting experiment! I look forward to hearing more about it.

    Thanks,

    Eric

  • Jochen Krause
    Posted at 10:34 pm, April 10, 2009

    Chris,

    This looks really interesting. I would love to see Equinox / OSGi running on the app engine – obviously RAP too 😉

    Jochen

  • Jochen Hiller
    Posted at 10:12 am, April 12, 2009

    Hi Chris,

    cool. I will support you making OSGi running on Google AppEngine, if its really possible !?
    I think the threading limitations will be the most difficult part to address when bringing an OSGi implementation on GAE.

    Jochen

  • Brandon
    Posted at 4:42 pm, April 13, 2009

    Have you tried the Prosyst Android OSGi platform on Android yet? Works pretty well though still in development it seems.

  • Neil Laurance
    Posted at 5:49 am, April 15, 2009

    Nice one. Look forward to the next installment. Will have to download your sample project and have a tinker.

  • Posted at 7:55 am, May 9, 2009

    Hi Chris,

    I have also started my experiments with getting OSGi to run on AppEngine.

    http://drop.io/appengine/

    Because of the restrictions imposed there, I do not think it is possible
    to get a complete OSGi-R4-compliant framework, but we might want to settle
    for a reasonably useful subset thereof. Useful as in: Can run (many) OSGi bundles
    without the need to modify them.

    Three big roadblocks so far:

    1) no threads: This means that we cannot start bundles or propagate events
    asynchronously. Everything has to happen on the main thread. I can live with that.

    2) no files: This means no bundle cache, and no persistent bundle state.
    I think we have to give up on bundle state (persistent or not) anyway, because
    in App Engine, we are spread over multiple JVM. So our whole application
    (including the framework) has to be mostly stateless. Dynamically starting bundles
    for example is not useful unless it happens across all servers.

    3) SecurityManager does not like bundle classloaders.
    You can construct OSGi classloaders on AppEngine (I have that part of Felix working),
    but code running on them is much more severly restricted than code
    running on the “main” classloader. Reflection for example does not work at all.

    http://code.google.com/p/googleappengine/issues/detail?id=1503

    1)+2) can be dealt with in the framework, but 3) seems to be a problem.
    Neither the Pax Web Extender Whiteboard (needs reflection) or the Equinox HttpService Servlet
    (needs Thread.currentThread().getContextClassLoader) bundles can be
    used (unmodified) because of that.

  • satya
    Posted at 3:13 pm, May 21, 2009

    I am also trying to run my osgi web app on google appengine but with no success yet. But after seeing your blog I am now hopeful about it.

  • duduzerah
    Posted at 9:21 pm, July 5, 2010

    Very good!!!

    And this is the final release? Any improvement after this article?