OSGi on AppEngine?

As many people know, Google recently announced support for Java on its AppEngine platform. As a connoisseur of OSGi, the first natural thing that came across my mind was quickly I could get OSGi running on it. Well, I’m happy to announce that we are part way there. I have OSGi running locally on the AppEngine SDK using the Equinox servlet bridge. I’ve managed to get a basic environment running with Equinox (and some bits of p2) and the Knoplerfish console:

appengineosgi 300x188 OSGi on AppEngine?

This wasn’t easy and was full of problems. If you read the AppEngine documentation, the sandbox is quite restrictive. First, there were a few places we were setting the context classloader and AppEngine doesn’t like that. I managed to hack the Equinox servlet bridge and the Equinox http service to not do that (it’s only needed in certain cases). Second, there’s a JRE “whitelist” of classes you can access that can cause trouble. I think this isn’t much of a problem if we define a custom OSGi execution environment for AppEngine. I attempted to do this since in Eclipse 3.5, we support the definition of custom execution environments. However, I found that the current whitelist doesn’t seem to be complete (maybe someone from Google can help). If you solve those problems, you can get Equinox working locally within the AppEngine SDK (as seen in the screenshot above). Here’s a zipped project if you’re interested (see the Equinox OSGi shared launch configuration and just go to localhost:8080/console).

In reality, this is only half of the battle. If we actually try to deploy what we have created above, things will fail. The most complicated problem I came across with running OSGi on AppEngine was you can’t write anywhere to the file system. Equinox and other OSGi frameworks need a place to write its state.  I think we can get around this if we write a custom storage adaptor hook in Equinox that uses the AppEngine storage facilities, however, that’s going to take some effort.

That’s it for now. I plan on making some more progress next week… are there any other people that want to see OSGi running on AppEngine? Have any other people out there been experimenting with OSGi and AppEngine? Did you hit the same roadblocks? Do you want to collaborate?

14 Responses to “OSGi on AppEngine?”

  1. Cyril Lakech says:

    Waoouw,

    That sounds exciting ! How much cost to code the adapter to use JDO ?

    Thanks,

  2. Nice work, will be cool to see it running on the actual AppEngine :)

    BTW, you also can’t create threads in the actual AppEngine sandbox (it’s primarily request driven along with a cron scheduler service for admin tasks, each request/cron task is limited to 30s) – but there’s probably a fair bit of OSGi functionality that would still work inside a single thread…

  3. Pascal says:

    How did you work around the limitation of thread creation? Equinox needs a slew of them for the purpose of event delivery.

  4. Chris – are you sure this will work? Last I read about AppEngine it was a way to fit Web apps onto the Google infrastructure, which is all highly customized, for “automatic” scale out. Aren’t the Google APIs sort of required?

    Eric

  5. @Pascal I haven’t worked around the thread issue yet as I’m still having trouble getting the full framework booted due to Google’s stranglehold on classloading:

    java.lang.VerifyError: (class: org/eclipse/equinox/servletbridge/FrameworkLauncher$ChildFirstURLClassLoader, method: getPermissions signature: (Ljava/security/CodeSource;)Ljava/security/PermissionCollection;) Illegal type in constant pool

    @Eric I am not sure if everything will work, this has been more of an experiment. Who says you can’t write web apps on top of GAE that run on OSGi? The Google APIs are going to be required for me in order to get the state persistence working properly in Equinox. Since I can’t write anywhere on “disk,” I need to write a custom adaptor hook that uses JDO apparently.

  6. Marcel Offermans says:

    This all sounds a bit like the restrictions Google Android has (which also places some restrictions that complicate running OSGi). Abstracting the OSGi bundle cache itself to use a different API sounds possible, but the bundle data store is built on top of the File abstraction, which is not that good an abstraction. ;) Running OSGi in cloud like environments is definitely something I’m interested in though.

  7. I don’t know, I just imagined the Google APIs would create some difficulties, but if they provide a JDO adapter maybe they have anticipated the issues.

    Ok, well, interesting experiment! I look forward to hearing more about it.

    Thanks,

    Eric

  8. Jochen Krause says:

    Chris,

    This looks really interesting. I would love to see Equinox / OSGi running on the app engine – obviously RAP too ;-)

    Jochen

  9. Jochen Hiller says:

    Hi Chris,

    cool. I will support you making OSGi running on Google AppEngine, if its really possible !?
    I think the threading limitations will be the most difficult part to address when bringing an OSGi implementation on GAE.

    Jochen

  10. Brandon says:

    Have you tried the Prosyst Android OSGi platform on Android yet? Works pretty well though still in development it seems.

  11. Neil Laurance says:

    Nice one. Look forward to the next installment. Will have to download your sample project and have a tinker.

  12. Thilo says:

    Hi Chris,

    I have also started my experiments with getting OSGi to run on AppEngine.

    http://drop.io/appengine/

    Because of the restrictions imposed there, I do not think it is possible
    to get a complete OSGi-R4-compliant framework, but we might want to settle
    for a reasonably useful subset thereof. Useful as in: Can run (many) OSGi bundles
    without the need to modify them.

    Three big roadblocks so far:

    1) no threads: This means that we cannot start bundles or propagate events
    asynchronously. Everything has to happen on the main thread. I can live with that.

    2) no files: This means no bundle cache, and no persistent bundle state.
    I think we have to give up on bundle state (persistent or not) anyway, because
    in App Engine, we are spread over multiple JVM. So our whole application
    (including the framework) has to be mostly stateless. Dynamically starting bundles
    for example is not useful unless it happens across all servers.

    3) SecurityManager does not like bundle classloaders.
    You can construct OSGi classloaders on AppEngine (I have that part of Felix working),
    but code running on them is much more severly restricted than code
    running on the “main” classloader. Reflection for example does not work at all.

    http://code.google.com/p/googleappengine/issues/detail?id=1503

    1)+2) can be dealt with in the framework, but 3) seems to be a problem.
    Neither the Pax Web Extender Whiteboard (needs reflection) or the Equinox HttpService Servlet
    (needs Thread.currentThread().getContextClassLoader) bundles can be
    used (unmodified) because of that.

  13. satya says:

    I am also trying to run my osgi web app on google appengine but with no success yet. But after seeing your blog I am now hopeful about it.

  14. duduzerah says:

    Very good!!!

    And this is the final release? Any improvement after this article?

14 responses so far

Written by . Published in Categories: EclipseSource News, Planet Eclipse

Author:
Published:
Apr 10th, 2009
Bookmark: