As part of our next release cycle, the Eclipse Communications Framework (ECF) project is going to implement the Google Wave Protocol as described in this enhancement request. Please join the request if you are willing to contribute to this effort, or would like to use the implementation.
This will make it easy for Eclipse and any Equinox, RCP or RAP-based application to interoperate with other Google Wave applications.
I’m going to take a moment to thank some folks for some outstanding contributions. For those that weren’t aware, for over a year, ECF has contributed it’s filetransfer API and implementations to P2 and the Platform, to use for http/https/ftp/file downloading during installs/updates.
Over the past year we’ve improved file transfer quite a lot…by using Apache Httpclient 3.1 instead of the JRE URLConnection, by improving cancellation handling, by adding a better/more general exception/error reporting mechanism, adding more/better test cases, and a lot of other small/large fixes/changes.
A lot of this work has been done not by ECF committers, but rather contributors…and in my view (and Doug S’s) this is a good thing.
So…
Thanks Henrich Kraemer (and his employer IBM) for your terrific contributions. It’s much appreciated by me…and I’m sure by everyone.
Thanks Henrik Lindberg for creating Chester the data molester, and for your other work on the P2<->ECF filetransfer interaction. Also very much appreciated.
In the future, I hope to be able to do more than thank contributors like these, who add so much, but for the moment this will have to do.
In previous blog posts I’ve blogged about ECF’s upcoming implementation of RFC 119.
In this post, I would like to jump out of the description of RFC 119 and talk about how the implementation of RFC 119 and ECF remote services fit together…as our implementation of RFC119 is layered on top of the ECF remote services API.
I think of remote procedure call as a leaky abstraction. For those that haven’t read Joel Spolsky’s The Law of Leaky Abstractions, I highly recommend it. The reason I would say that it’s a leaky abstraction is that although transparent RPC *looks* like a local method call, it’s clearly not under some situations:
There are others, but I think these are the most compelling.
Note that all three of the above are runtime issues…i.e. they can happen at runtime for lots of reasons that have nothing to do with the semantics of the RPC itself. In the case of 1 and 2 they cannot be prevented. And they are likely to happen for non-trivial procedures.
Note also that since OSGi services are method calls on some service interface (a pojo), that remote services will also have the above issues. It doesn’t matter what your remoting implementation is, they will all be subject to such problems. Unfortunately, we (the distribution system implementers) can’t prevent it.
So, to me this means remote procedure call is a leaky abstraction…because even though it looks like normal/local/in memory method call, there are occasions where the ‘truth’ about networks leaks out.
So what to do? Well, I think there are several things to do, both from the service designer’s viewpoint (i.e. those defining the service to be remoted) and the distribution system implementer’s viewpoint (i.e. people that implement distribution infrastructures…like me).
From the service designer’s viewpoint you could design all of your services to prepared for 1, 2, 3 above…and/or document them as having these properties. This can/does definately help. But it is a major pain, and you can end up having services that are more complex (especially if they are used locally as well as remotely).
From the distribution system implementer’s viewpoint I feel one thing to do is what Joel describes in his paper as what TCP did for IP..layering.
That is the approach we’ve taken with implementing RFC 119…as the implementation of transparent remoting as specified in RFC119 is implemented on a non-transparent/explicit remoting API (ECF remote services). I think this is nice, because it allows/supports more use cases:
“I want to create a simple remote service, as easily as I can and have it work” (use RFC 119)
“I want to create a remote service that knows about or at least responds properly to the remoting leaks (1, 2, 3, etc above), and not simply crash/block/fail when the service is used” (use ECF remote services)
So, we’ve implemented RFC119 itself using ECF remote services…and layered transparent remoting on top of a non-transparent remoting runtime API. This gives choices to both service designers and service consumers about how much they want/need to care about the network leaking into remote OSGi services. That is, if they care about the leaks they can do something about them, but if they (or the service consumers) don’t care about such leaks they can have a standard way of publishing, discovering, and receiving remote OSGi services.
There’s a bug open for ECF to figure out a better/real name for it’s implementation of RFC 119. Soon, we’re going to have an online vote to decide the winning name…but I wanted to say that my favorite (given my frequent harping about reliability/failure detection in distributed systems) is comment #7. ..even though I don’t think it will/can win 
.
Please contribute any ideas for names to the bug…amusing or not.
In the last few days, there have been a number of provocative and interesting posts around the issue of how to maintain Eclipse’s innovation, as well as commercial growth, and project contributions
Doug’s – On the Future of Eclipse
and Michael’s – Eclipse Freeloader Award
There is some long-standing social science/economics work that is relevant. For example, see The Logic of Collective Action. Some thoughts from what I’ve read from Olson’s book:
1) Small groups differ from large groups in their incentives for participation
2) Selective incentives/disincentives for participation are sometimes necessary (i.e. different incentives for different group members)
3) Large groups are less able to act in their common interest than small ones
Some of the thinking that’s going on around how to get the EF membership to all act/contribute…in order that all can continue to benefit…and maintain a healthy community…could be informed by this work.
ECF is finishing up it’s implementation of RFC 119 (distributed OSGi) for release with Galileo (aka ECF 3.0).
Over beers at EclipseCon, Ian Skerrett suggested to me that we needed a name for this work…something other than ‘ECF RFC 119 Implementation’ 
.
So, we’ve opened a bug to identify a name for ECF’s distributed OSGi implementation. Please join the bug, and give us your ideas for a good, clear, understandable name.
As indicated in my last posting on RFC 119 and ECF, Equinox and the runtime project now have a full implementation of RFC 119.
I was remiss, however, in not saying previously that we were fortunate to get a contribution for SLP-based discovery from Siemens. This contribution is going through the Eclipse Foundation IP-process now, and will be part of the ECF 3.0/Galileo release.
Thanks specifically to Thomas Kiesslich, Philip Konradi, and Viktor Ransmayr for the contribution, as well as others at Siemens. We’re thrilled to have you on the team!
I’ve blogged previously about what we (ECF) are doing WRT to RFC 119 (Distributed OSGi Services): RFC 119 and ECF – part 2
The news for today…I’m rather excited to report…is that we now have a working implementation of RFC 119, with support for both discovery and distribution parts of the specification. This will be part of the ECF 3.0/Galileo release, and of course we will be talking about it in Markus’ and my tutorial, as well as some other talks.
What’s So Cool About Yet Another Spec Implementation?
In my view, what makes this implementation interesting and useful is that we’ve used the abstract ECF discovery and remote services APIs to implement this spec, meaning that without doing any other work, RFC compliant distributed OSGi services are supported using the following transports
Distribution
Discovery
And, of course any other transports that people are willing to create, as the ECF remote services API is completely open.
Our desire/hope is that others will implement ECF providers built from other protocols/transports of their choosing (both commercial and open source)…with the payoff to them being that they will then automagically have a RFC 119-compliant implementation…since all ECF providers are now RFC 119 compliant (and will remain so, as we will update the impl as the spec changes). All the existing implementations done by the ECF team (e.g. r-OSGi, ECF generic, etc) are open source and so may be used/reused to create new providers as desired.
Another thing that I think is cool is that even though RFC 119 specifies using transparent proxies for accessing remote OSGi services, it allows (and even encourages) implementations to support non-transparent/asynchronous remoting (i.e. using one-ways, asynchronous rpc, and futures). ECF’s remote services API already has support for futures, and asynchronous/non-blocking remote invocation via its IRemoteService interface.
With our current impl of RFC 119, clients can access the IRemoteService instance (in addition to or instead of the proxy), via a standard service property on the ServiceReference. This gives clients a runtime choice of whether to access a remoted service via a proxy…or via asynchronous/non-blocking techniques like futures and one-ways.
One final thing…the size cost for the ECF remote services API is fairly small (<60k of code) next to the size of the implementations…making it possible to use in smaller (as well as larger) execution environments.
Some time ago, I blogged about RFC 119 and ECF part 1, so now it’s time for part 2…
ECF now has a set of plugins that serve as a bridge between the RFC 119 spec (see part 1) and ECF’s discovery and remote services API (note: all the following plugins begin with ‘org.eclipse.ecf’ and can be found in /cvsroot/rt/org.eclipse.ecf/framework/bundles at dev.eclipse.org):
The osgi.service bundle only contains the classes that are included in the OSGi 4.2 compendium, i.e. interfaces such as DiscoveryProvider, DiscoveredServiceTracker, and DistributionProvider. These interfaces are defined by the OSGi Enterprise Experts group, and are part of the RFC 119 specification. The implementation is from the OSGi Alliance, as are other parts of OSGi that are implemented in Equinox.
RFC 119 is divided into two parts: 1) service discovery; 2) service distribution. Service discovery is optional, and service distribution is required.
The osgi.services.discovery bundle implements the RFC 119 discovery and uses ECF’s discovery API to implement protocol independent network service discovery. There are two existing provider implementations of the ECF discovery API right now: zeroconf/bonjour and SLP…so users will be able to publish their services automatically over both SLP and zeroconf discovery mechanisms with a single service publication.
Similarly the osgi.services.distribution bundle implements RFC 119 distribution using ECF’s remote services API. When this is complete, it will mean that without any additional coding, remote services will run immediately on the following transports:
Further, the ECF remote services API also allows non-transparent access to remote services, in order to support asynchronous remote method invocation, one-ways, futures, explicit failure handling, and other features not part of RFC 119 at this point.
Note also that other providers that implement the ECF remote services API will automatically be RFC 119 compliant…i.e. they have to do no work to be compliant with RFC 119 for transparent remote procedure call. Further, with ECF remote services, if clients wish to use non-transparent remoting…in combination with or in exclusion to transparent remoting…that is also available.
We’re closing in as rapidly as we can on having this all ready and working…and EclipseCon 2009 is coming up fast…so stay tuned.
There seems to be a lot of interest in doing distributed services with OSGi/Equinox, and so I wanted to begin a series of postings about what the ECF project is doing in this area (short story: a lot), and what we are planning for supporting standardization via RFC 119.
What is RFC 119?
First, here is the OSGi 4.2 early draft, which includes a draft of RFC 119 (toward end of PDF). RFC 119 is an effort to standardize some aspects of creating, publishing, discovering, and accessing out-of-process OSGi services. It is not yet complete, but is quickly moving toward standardization.
What is ECF Doing with Discovery and Remote OSGi Services?
For some time, ECF has had both a discovery API, and a remote services API.
These APIs have two interesting characteristics (relevant to this post, anyway):
Why is Transport Independence Useful?
Transport independence means that multiple implementations can and do exist, for both the discovery and remote services APIs. So, for example, for discovery ECF currently has two implementation…also known as ‘providers’: the Service Locator Protocol (RFC 2608), and Zeroconf/Bonjour.
Transport independence also exists for the remote services API, and we already have implementations based upon r-OSGi, XMPP, a simple tcp-based protocol, and JMS.
In general, transport independence is helpful because provider implementations can be replaced/changed without affecting clients (i.e. code that uses remote services).
Among other things, this is important for interoperability and cross-vendor openness, as it allows multiple servers/services to be used without rewriting the clients of those services.
Exposing both transparent and non-transparent remote method access
There is a long (and sometimes bitter) dialog in the distributed systems literature about whether transparency in networked services is a ‘good idea’. ECF has attempted to skirt this entire large and complicated philosophical issue by providing both transparent access to remote services (i.e. through the normal OSGi services registry), and non-transparent access (via IRemoteService interface…see javadocs here) and leaving the decision about which is appropriate to the service creator and service client.
For non-transparent access to remote service, ECF provides the IRemoteService interface. It exposes methods to remotely invoke methods via non-blocking techniques (e.g. with listeners for receiving callbacks or with a ‘future’). This gives clients the flexibility and the tools to use transparent invocation (via proxy) if desired/appropriate, or use the asynchronous, non-blocking mechanisms present on IRemoteService.
What about ECF and RFC 119?
The short story is that we have plans to implement RFC 119 with the ECF discovery and remote services APIs. Specifically, see bug 249240. Given the existing drafts of RFC 119, we believe that we can implement it by writing code above the ECF discovery and remote services APIs, without significant changes to the APIs themselves. This would mean that all ECF providers would then implement RFC 119, without having to do any additional work.
More on this and other issues in subsequent posts.
(18) (4) (120) (57) (8) (58) (127) (35) (9) (15) (3) (17) (12) (13) (12) (16) (3) (20) (8)