Good news, we’re even closer to having Eclipse 3.5 released!
Eclipse 3.5 RC3 is out for download (or update via p2).
There are no new features out in this release, as feature freeze has passed. There are only bug fixes.
From my count, there were 81 bugs fixed for the 3.5 RC3 release.
Enjoy!
Also, if you want to try out Galileo, RC1 is available (follow the release on Twitter)!
I have been working with the upcoming Equinox OSGi book authors quite a bit as of late. They are a demanding bunch, but they have helped me improve PDE’s OSGi tooling while they write and exercise parts of PDE that not many people know about. In particular, the book uses something in PDE called Automated Management of Dependencies (AMD). What’s that? Well, have you ever noticed this collapsed section in the PDE editor:
Let’s take a peak!
This section of the manifest editor was built for people who have different bundle development workflow. At Eclipse, we have traditionally done a manifest-first workflow where users are responsible for crafting their whole manifest. I personally believe this is the best workflow for advanced users. However, for new users, it’s very cumbersome to specify your dependencies. For example, if I want to use some JFace Databinding code, how do I know what bundles or packages I need to specify? Well, AMD was built to address this concern. The first step to using AMD, is to add the bundles that PDE will consider to when generating your dependencies (we call this the secondary classpath). The easiest thing you can do is add all the bundles as part of your target (although this isn’t always optimal):
Ok, now that we have added all the bundles that PDE will consider when generating the dependencies, we need to select what type of dependencies PDE should generate. In this case, we choose Import-Package, but you can as easily choose Require-Bundle if that was your preference. To test the AMD functionality out, we will remove all of our current dependency statements, which will result in a bunch of compile errors in our project:
So now, we can either click the analyze dependencies button in the AMD section of the editor… or we can simply launch our application and the dependencies will be automatically calculated. To verify, let’s look at our manifest:
Wow, magic! PDE analyzed the code and calculated the dependencies so you didn’t have to think about it.
This works great, however, not everything is perfect yet… there are a couple bits missing. If you noticed, the version ranges weren’t properly calculated on the dependencies. This is a hard problem and we are currently exploring it in the PDE API Tools project. It is also a bit cumbersome to add things to your secondary classpath… should the default be that we look at everything in your target? Should PDE look at using something like BND to do the dependency calculation? Should we allow people to somehow scope the bundles in advanced… while creating a new project? If you’re interested in this functionality, please let me know. The PDE team plans to explore and improve AMD in the next release cycle, so please comment if you have ideas. You can also file bugs and feature requests against PDE.
I’ve also created a quick video which shows AMD usage with the RCP mail example.
Enjoy.
Never judge a man until you have walked a mile in his shoes. This advice can be applied to the art of writing software. At a high-level software seems so easy, but when you start working on a project, truly understand the requirements and start writing code under a variety of constraints, the true nature of the beast reveals its ugly head.
Throughout the past 6 months I have had the good fortune of working with the p2 team. I’m not sure if I’ve walked a “mile in their shoes”, but I have read enough bug reports, suggested enough crazy ideas, and been humbled through a number of code reviews that I am starting to understand the complexities of writing and delivering a provisioning system.
I have been working on “The Publisher”, the part of p2 that is responsible for putting things into the repositories. One of the challenges faced with the publisher is that people are unlikely using the latest version. A number of bug reports have been filed with the following steps:
In many projects this would be a show stopper. In p2, this can be closed as a duplicate of a bug filed last year, or a regression, or a frustration, or poor job of handling incorrect metadata. Just because the user was running with the latest build doesn’t mean the repository they are installing from was generated with a recent tool. From the outside the answer is simple, update your tools! (Now walk a mile in the shoes of the release engineer).
With Galileo fast approaching it has become evident that everyone has a different build story. Some teams are using the metadata generator in their builds. Some teams are using an old version of the publisher, while others simply export from the UI and take whatever repository eclipse gives them. While it would be great if every team switched their builds to use the publisher RC3 (the build just finished), I know this isn’t going to happen. Build scripts are too hard to get right and there are too many other problems left to fix. This is especially true when things “appear” to work.
To help determine if there are any problems with the metadata generated by Galileo release train, we have written a Metadata Analyzer tool. The tool scans Galileo (or any repository you give it), and looks for known problems. In particular, it searchers the repositories for problems have have been recently fixed in p2. While there were a few problems in RC1, I’m happy to report that other than a few missing copyright notices, the Galileo repository is now stable. If you are interested in the metadata tool, it is currently attached to the following bug report. Once we ship RC4 I’ll try to give this tool a proper home.
The tool is very simple and written using Declarative Services. In fact, the crux of the tool can be summed up in 9 lines:
try { service.preAnalysis(repository); for (Iterator iter = ius.iterator(); iter.hasNext();) { IInstallableUnit iu = (IInstallableUnit) iter.next(); service.analyzeIU(iu); } service.postAnalysis(); // Yes, this should be done in a finally block. } catch (Exception e) { e.printStackTrace(); }
Each analysis step is simply another service, and that service is called for each InstallableUnit. There is also a pre and post method for setup / teardown of the service.
An example of a service is:
public void analyzeIU(IInstallableUnit iu) { if (iu.getId().contains("feature.jar")) { ITouchpointData[] touchpointData = iu.getTouchpointData(); if (touchpointData.length == 0) { System.out.println("[ERROR] No unzip touchpoint for: " + iu.getId()); } else { boolean found = false; for (int i = 0; i < touchpointData.length; i++) { ITouchpointInstruction instruction = touchpointData[i].getInstruction("zipped"); if (instruction.getBody().equals("true")) found = true; } if (!found) System.out.println("[ERROR] No unzip touchpoint for: " + iu.getId()); } } }
This service checks that each feature will be “unzipped” when installed.
Because everything is just a service you can take the tool as it is, or add your own analysis steps. If anyone has ideas for other tests we can / should run, please comment on the bug report.
I’m going to take a moment to thank some folks for some outstanding contributions. For those that weren’t aware, for over a year, ECF has contributed it’s filetransfer API and implementations to P2 and the Platform, to use for http/https/ftp/file downloading during installs/updates.
Over the past year we’ve improved file transfer quite a lot…by using Apache Httpclient 3.1 instead of the JRE URLConnection, by improving cancellation handling, by adding a better/more general exception/error reporting mechanism, adding more/better test cases, and a lot of other small/large fixes/changes.
A lot of this work has been done not by ECF committers, but rather contributors…and in my view (and Doug S’s) this is a good thing.
So…
Thanks Henrich Kraemer (and his employer IBM) for your terrific contributions. It’s much appreciated by me…and I’m sure by everyone.
Thanks Henrik Lindberg for creating Chester the data molester, and for your other work on the P2<->ECF filetransfer interaction. Also very much appreciated.
In the future, I hope to be able to do more than thank contributors like these, who add so much, but for the moment this will have to do.
The show is over …
With a crowd of almost 200 attendees, many cool demos and the right weather for beer and pretzels… this was not only one of the biggest DemoCamps ever, it also was a lot of fun!
First I’d thank all the other speakers for their great contributions:
During the evening, many photos were taken and some of the talks were even recorded on video.
The demo camp was introduced by Hervé Couturier and Ralph Mueller. They talked about SAP’s commitments to the Eclipse community and the growing level of participation.
After the introduction, Krum Tsvetkov gave a quick overview of the Eclipse Memory analyzer (MAT) and how you can use it to find memory leaks within 5 minutes. After the crowd understood what MAT is, I showed them how easy it is to have the same application running in the web by exchanging the runtime. Most of the people were quite impressed and it reminded me of what Chris once said: “Cool, one runtime to rule them all”. I’ll blog about the results of single sourcing MAT in the upcoming days so – in the meantime you can watch our talk.
Next Michael Scharf showed a small framework for handling tabular data on top of the existing JFace infrastructure that uses a columns-based approach instead of the existing row-based one. I hope they will consider contributing it to the community!
Achim Demelt gave us a fuzzy warm feelings about modeling with his demo “Modeling is fun!” He showed us some cool extensions to graphical editors like content assistance and compile/generate on save.
After a short break, Jochen Hiller showed us the power of dynamic OSGi systems. He did a demo on how to run some bundles and use several different bundle versions in parallel.
Elmar Eperiesi-Beck showed us a possible future Eclipse project – an Equinox based SOA Security Framework. He demonstrated how to secure a SOA application. With this approach he mentioned several times that Corisecio wants to contribute this project back to the Eclipse ecosystem (hope to see the proposal soon).
One new Eclipse project demonstrated was Gyrex. Gunnar gave us a quick introduction of the project. Gyrex provides a set of frameworks and tools around Equinox to allow seamless and pain-free operation of Equinox server clusters. Sounds pretty cool but we still need to “Stay tuned… they are working on an example for you to play with”.
Marcus Harringer of MicroDoc showed us an example of how to combine some of the cool kids of todays technology stack. They emulated real world devices like a turnstile with the help of Equinox and Flash for the client side. They used the emulation to write test scripts to create functional tests for their software implementions of the devices. Pretty cool demo on combining different technologies to solve real world problems.
The last but not least speaker was Guy Philipp Bollbach of itemis talking about Xtext. I think Xtext is one of the really cool technologies we have today. It helps you to write a grammar for your own DSL and let Xtext generate the whole stack of editors, outline views and other things like content assist for your DSL. Really impressive!
All in all it was a really nice evening – most of the people I talked to had a great evening and learned about many cool Eclipse technologies. Most of the attendees enjoyed the democamp as much as I did. I hope to see some of you at the next DemoCamp or at a Stammtisch.
And as always – the Twitter community was very active during the DemoCamp – see all the related tweets @eventtrack.
EclipseSource will be at JavaOne 2009 this year!
Elias Volanakis and Peter O’Blenis will be there talking up Eclipse, Equinox, RAP and OSGi:
In particular, Elias will be at the Eclipse booth (#216) presenting about the Rich Ajax Platform (RAP) during these times:
Also, if you stop by our booth, we’ll be holding a drawing for a copy of the upcoming Equinox OSGi book!
Feel free to email us if you want to setup a meeting in advance over a frosty beverage!
A webinar was just posted where Jochen Krause (EclipseSource), Kevin McGuire (IBM), Martin Oberhuber (Wind River), Steve Northover (IBM) and Ben Galbraith (Mozilla) discuss the “Eclipse 4.0″ (e4) project. If you’re interested in how the next generation Eclipse platform is shaping up, this webinar is a great place to start. If you want to dive deeper, check out the e4 wiki.
On a side note, if you’re an Eclipse committer, Kai Toedter is looking for vCards of Eclipse committers for his e4 contacts sample application. If you want to be famous, shoot him an email.
Good news, we are one step closer to having Eclipse 3.5 released!
Eclipse 3.5 RC1 is out for download (or update via p2).
There are no new features out in this release, as feature freeze has passed. There are only bug fixes.
From my count, there are 341 bugs fixed for the 3.5 RC1 release.
Enjoy!
A new project proposal came through my inbox this morning.
The Pave project is a proposed open source project under the Eclipse Web Tools Platform Project.
The Pave project provides a framework for composing sequences of frequently executed operations. These sequences are called patterns and are meant to provide error proof path for users in the execution of routine tasks. You are invited to comment on and/or join the project. Please send all feedback to the newsgroup.
While I’m still blurry on what exactly will be done in the project (maybe someone from the project can shed some light), it’s interesting to note that this project comes from a fairly large company known as SAP.
SAP’s involvement in Eclipse has grown significantly over the last couple of years. I believe it first started with the excellent Eclipse Memory Analyzer (MAT) project. Now it looks like SAP is involved in:
| modeling.m2t | modeling.tmf | technology.mat |
| webtools.ejbtools | webtools.jeetools | webtools.releng |
I think it’s a good sign when a large company starts investing into the technology they are basing their current and future product lines on. From a personal point of view, I would be curious to see the culture shift inside of SAP when it comes to open source. I’m fascinated with how companies change internally to work with open source communities. At my time with another large company, there was a certain set of people who understood open source and those who didn’t. I remember one time there were people that suggested working on a project internally and than just dumping the code over the “open source fence” every release would be good enough. Oh how far have we come 
In the end, I hope to hear more good things about SAP getting involved in Eclipse and open source.
Once is an oddity. Twice is a trend… Well, at least it starts looking like maybe there is a trend. In the last while the need to setup custom p2 profiles has come up a couple times for me. What do I mean by that? Well, Profiles are the data structure that p2 uses to track the set of software installed to make an executable application. Your Eclipse IDE is a profile for example. Profiles contain Installable Units (IUs). Each IU describes some thing that can be installed (bundle, feature, executable, command line argument, …) into a profile. The relationship between IUs is captured using provided and required capabilities and IUs themselves refer to artifacts (the actual bundles) and have touchpoint data used to configure the artifacts into a runtime.
OK, enough of the metadata lesson, what’s this all about? I had two problems.
Both of these (and more) can be addressed by pimping your profile using the pimpProfile() method below.
private void pimpProfile(IProfile profile) throws CoreException { IInstallableUnit iu = createIU(); Operand[] operands = new Operand[] {new InstallableUnitOperand(null, iu)}; IEngine engine = getEngine(); engine.perform(profile, new DefaultPhaseSet(), operands, null, null); }
The code dynamically creates an IU and the proceeds to install that IU using the engine. The engine is the p2 infrastructure that actually performs install/uninstall/update operations. The default engine is available as an OSGi service of type org.eclipse.equinox.internal.provisional.p2.engine.IEngine. You should implement getEngine() using your favorite services coding pattern (hint: the cool kids are using Declarative Services).
Normally you would use the p2 planner to make sure all the dependencies are satisfied and then modify the profile but here we know what we want to do — spoof up an IU and install it.
There are a few different kinds of operands but here we want to install an IU so the IU operand makes sense. IU operands are old/new pairs. That is, the first arg in the constructor is the IU currently in the profile, the second is what you want to be in the profile when the operation is finished. So install is {null, iu}, uninstall is {iu, null} and update is {iu.1, iu.2} — the pair expresses the desired transition.
Now for the IU… Depending on which problem above we are looking to solve we need a different IU. Of course you can install any number of IUs but here let’s keep it simple and do these separately. In scenario 1 we are going to add a special capability to our pimped profile and arrange to have all our special IUs require this capability. The net effect being that the special IUs will only ever install into pimped profiles. Here we are going to pimp a server profile to prevent server function from accidentally being installed on clients. See bug 27600 for another example usecase.
In createIU() below we first create an IU descriptor and then spoof up a serverside capability. The name and namespace of the capability don’t actually matter as long as they are unique. Having created the capability we add it to the IU descriptor and instantiate and return the IU. That’s it — the profile now says it is capable of supporting serverside function.
private IInstallableUnit createIU() { InstallableUnitDescription iu = new MetadataFactory.InstallableUnitDescription(); String id = "server-side"; Version version = Version.createOSGi(0, 0, 0); iu.setId(id); iu.setVersion(version); ArrayList capabilities = new ArrayList(); capabilities.add(MetadataFactory.createProvidedCapability(id, id, version)); iu.addProvidedCapabilities(capabilities); return MetadataFactory.createInstallableUnit(iu); }
Now to mark our server-side function. The key here is to make the server-side function require the server-side function. That is, make it require the server-side capability that we just added to the profile.
To do this, just create a p2.inf file as shown below and put it in the relevant server-side elements (root of features or META-INF dir of bundles). When the feature or bundle is published to a p2 repo, the indicated requirement is added to the corresponding IU. In practice you only need to annotate the top level IU that is being installed so that it fails to resolve on non-pimped profiles.
requires.1.namespace=server-side requires.1.name=server-side requires.1.range=[0.0.0,0.0.0]>
There is doc coming on p2.inf for the Galileo release but in the mean time see one of the original bugs.
Scenario 2 requires the setting of a dynamically determined property. This came up in a system admin tool that created profiles and allowed the users to set some values that had to be presented as command line args. Here we use VM args just for fun. You can tweak program args, configuration properties, …
The createIU() method below is largely the same as the previous one but now rather than creating a capability we create some touchpoint data — specifically some action calls to add VM args. You can find out more about action on the p2 wiki.
private IInstallableUnit createIU() { InstallableUnitDescription iu = new MetadataFactory.InstallableUnitDescription(); String time = Long.toString(System.currentTimeMillis()); iu.setId("property.setter"); iu.setVersion(Version.createOSGi(0, 0, 0, time)); Map touchpointData = new HashMap(); String id = getID(); // put your code here String data = "addJvmArg(jvmArg:-DmyProperty" + "=" + id + ");"; touchpointData.put("configure", data); data = "removeJvmArg(jvmArg:-DmyProperty" + "=" + id + ");"; touchpointData.put("unconfigure", data); iu.addTouchpointData(MetadataFactory.createTouchpointData(touchpointData)); ITouchpointType touchpoint = MetadataFactory.createTouchpointType( "org.eclipse.equinox.p2.osgi", new Version(1, 0, 0)); iu.setTouchpointType(touchpoint); return MetadataFactory.createInstallableUnit(iu); }
With this IU installed, the launched profile will have the given JVM arg set.
You can get similar effects by manually creating the relevant IUs, publishing them to a repo and then installing them like any other IU — but that’s boring and error prone. Using this approach you can create a custom installer that sets up all manner of things in the profiles it manages.
(18) (4) (120) (57) (8) (58) (127) (35) (9) (15) (3) (17) (12) (13) (12) (16) (3) (20) (8)