In addition to demonstrating the power of combining DS with ECF 3.0′s support for distributed OSGi services, the tutorial has two other very cool aspects:

• It was created by a community member after using these technologies to build their own system
• The tutorial will soon be contributed to the ECF documentation section available in the ECF wiki area

Enjoy!

We’ve added to our distributed OSGi documentation (with examples+source) for ECF 3.0/Galileo:

• Distributed OSGi Services with ECF
• Getting Started with ECF’s RFC119 Implementation
• Getting Started Using the ECF Remote Services API

We have API layering so service programmers can choose the simplest appropriate mechanism for their system requirements, while still providing access to ‘lower-level’ concerns when necessary:

• failure handling
• synchronous vs. asynchronous remote method invocation
• serialization and wire protocol
• and so on…

If you have any questions or comments, please let us know on the ECF mailing list.

What does this mean? Well, you can now create distributed implementations of OSGi services using JMS or other messaging frameworks for distributing messages to other OSGi frameworks. This is all possible because of ECF’s provider architecture that allows the creation of a distributed EventAdmin implementation that can use a variety of wire protocols (e.g., XMPP, ActiveMQ).

Please let email the ecf-dev list if you have any questions!

In this post, I would like to jump out of the description of RFC 119 and talk about how the implementation of RFC 119 and ECF remote services fit together…as our implementation of RFC119 is layered on top of the ECF remote services API.

I think of remote procedure call as a leaky abstraction.  For those that haven’t read Joel Spolsky’s The Law of Leaky Abstractions, I highly recommend it.  The reason I would say that it’s a leaky abstraction is that although transparent RPC *looks* like a local method call, it’s clearly not under some situations:

1. The remote call is very slow (or blocks) because of a network problem
2. The remote call cannot complete because the network fails/goes down
3. A parameter to the remote method call cannot be serialized…e.g. trying to make a remote call e.g. like this:  serviceProxy.setWorkspace(IWorkspace workspace…why would passing a workspace to a method call be a problem?)

There are others, but I think these are the most compelling.

Note that all three of the above are runtime issues…i.e. they can happen at runtime for lots of reasons that have nothing to do with the semantics of the RPC itself.  In the case of 1 and 2 they cannot be prevented.  And they are likely to happen for non-trivial procedures.

Note also that since OSGi services are method calls on some service interface (a pojo), that remote services will also have the above issues.  It doesn’t matter what your remoting implementation is, they will all be subject to such problems.  Unfortunately, we (the distribution system implementers) can’t prevent it.

So, to me this means remote procedure call is a leaky abstraction…because even though it looks like normal/local/in memory method call, there are occasions where the ‘truth’ about networks leaks out.

So what to do?   Well, I think there are several things to do, both from the service designer’s viewpoint (i.e. those defining the service to be remoted) and the distribution system implementer’s viewpoint (i.e. people that implement distribution infrastructures…like me).

From the service designer’s viewpoint you could design all of your services to prepared for 1, 2, 3 above…and/or document them as having these properties.  This can/does definately help.  But it is a major pain, and you can end up having services that are more complex (especially if they are used locally as well as remotely).

From the distribution system implementer’s viewpoint I feel one thing to do is what Joel describes in his paper as what TCP did for IP..layering.

That is the approach we’ve taken with implementing RFC 119…as the implementation of transparent remoting as specified in RFC119 is implemented on a non-transparent/explicit remoting API (ECF remote services).  I think this is nice, because it allows/supports more use cases:

“I want to create a simple remote service, as easily as I can and have it work”  (use RFC 119)

“I want to create a remote service that knows about or at least responds properly to the remoting leaks (1, 2, 3, etc above), and not simply crash/block/fail when the service is used”  (use ECF remote services)

So, we’ve implemented RFC119 itself using ECF remote services…and layered transparent remoting on top of a non-transparent remoting runtime API.  This gives choices to both service designers and service consumers about how much they want/need to care about the network leaking into remote OSGi services.   That is, if they care about the leaks they can do something about them, but if they (or the service consumers) don’t care about such leaks they can have a standard way of publishing, discovering, and receiving remote OSGi services.

Please contribute any ideas for names to the bug…amusing or not.

Over beers at EclipseCon, Ian Skerrett suggested to me that we needed a name for this work…something other than ‘ECF RFC 119 Implementation’ .

So, we’ve opened a bug to identify a name for ECF’s distributed OSGi implementation.  Please join the bug, and give us your ideas for a good, clear, understandable name.

The news for today…I’m rather excited to report…is that we now have a working implementation of RFC 119, with support for both discovery and distribution parts of the specification.  This will be part of the ECF 3.0/Galileo release, and of course we will be talking about it in Markus’ and my tutorial, as well as some other talks.

What’s So Cool About Yet Another Spec Implementation?

In my view, what makes this implementation interesting and useful is that we’ve used the abstract ECF discovery and remote services APIs to implement this spec, meaning that without doing any other work, RFC compliant distributed OSGi services are supported using the following transports

Distribution

• r-OSGi (http)
• XMPP
• ECF Generic
• Skype
• Java Messaging Service (JMS)
• JavaGroups

Discovery

• Service Location Protocol (RFC 2608)
• Bonjour/Zeroconf/DNS-SD

And, of course any other transports that people are willing to create, as the ECF remote services API is completely open.

Our desire/hope is that others will implement ECF providers built from other protocols/transports of their choosing (both commercial and open source)…with the payoff to them being that they will then automagically have a RFC 119-compliant implementation…since all ECF providers are now RFC 119 compliant (and will remain so, as we will update the impl as the spec changes).  All the existing implementations done by the ECF team (e.g. r-OSGi, ECF generic, etc) are open source and so may be used/reused to create new providers as desired.

Another thing that I think is cool is that even though RFC 119 specifies using transparent proxies for accessing remote OSGi services, it allows (and even encourages) implementations to support non-transparent/asynchronous remoting (i.e. using one-ways, asynchronous rpc, and futures).  ECF’s remote services API already has support for futures, and asynchronous/non-blocking remote invocation via its IRemoteService interface.

With our current impl of RFC 119, clients can access the IRemoteService instance (in addition to or instead of the proxy), via a standard service property on the ServiceReference.  This gives clients a runtime choice of whether to access a remoted service via a proxy…or via asynchronous/non-blocking techniques like futures and one-ways.

One final thing…the size cost for the ECF remote services API is fairly small (<60k of code) next to the size of the implementations…making it possible to use in smaller (as well as larger) execution environments.

ECF now has a set of plugins that serve as a bridge between the RFC 119 spec (see part 1) and ECF’s discovery and remote services API (note:  all the following plugins begin with ‘org.eclipse.ecf’ and can be found in /cvsroot/rt/org.eclipse.ecf/framework/bundles at dev.eclipse.org):

1. osgi.services
2. osgi.services.discovery
3. osgi.services.distribution

The osgi.service bundle only contains the classes that are included in the OSGi 4.2 compendium, i.e. interfaces such as DiscoveryProvider, DiscoveredServiceTracker, and DistributionProvider.  These interfaces are defined by the OSGi Enterprise Experts group, and are part of the RFC 119 specification.  The implementation is from the OSGi Alliance, as are other parts of OSGi that are implemented in Equinox.

RFC 119 is divided into two parts:  1) service discovery; 2) service distribution.  Service discovery is optional, and service distribution is required.

The osgi.services.discovery bundle implements the RFC 119 discovery and uses ECF’s discovery API to implement protocol independent network service discovery.   There are two existing provider implementations of the ECF discovery API right now:  zeroconf/bonjour and SLP…so users will be able to publish their services automatically over both SLP and zeroconf discovery mechanisms with a single service publication.

Similarly the osgi.services.distribution bundle implements RFC 119 distribution using ECF’s remote services API.   When this is complete, it will mean that without any additional coding, remote services will run immediately on the following transports:

1. r-OSGi/http
2. TCP/ECF generic
3. XMPP/XMPPS
4. Skype
5. JMS – ActiveMQ, Weblogic JMS
6. Others under active development:  EMF/CDO, http-with-rest

Further,  the ECF remote services API also allows non-transparent access to remote services, in order to support asynchronous remote method invocation, one-ways, futures, explicit failure handling, and other features not part of RFC 119 at this point.

Note also that other providers that implement the ECF remote services API will automatically be RFC 119 compliant…i.e. they have to do no work to be compliant with RFC 119 for transparent remote procedure call.  Further, with ECF remote services, if clients wish to use non-transparent remoting…in combination with or in exclusion to transparent remoting…that is also available.

We’re closing in as rapidly as we can on having this all ready and working…and EclipseCon 2009 is coming up fast…so stay tuned.