EclipseCon was quite a blast, lots of amazing talks, some very engaging keynotes and loads of interesting people to meet. One of the neat things about having everyone in one place is that the communication distances are so much shorter. While the internet has revolutionized the way we think about global communication, oftentimes it still doesn’t beat ad-hoc face-to-face conversations. So there was a lot to learn from just listening in on conversations (maybe that’s the secret of twitter) or throwing a question into a room full of highly intelligent people from various backgrounds.

One of these things I learned about was the Eclipse Project Dashboard. This neat little website shows tables and diagrams of all the stuff that is happening in the Eclipse repositories. I have to admit that I am a sucker for diagrams (a common affliction I assume), so this a really great way to get a high-level overview of the ongoing development activities, showing which projects or companies are “hot”, which committers are involved in what projects and a whole bunch of other stuff. And if you want to do some other analysis, there is even RESTful access to the raw data.

The eclipse webmasters have really put a lot of effort into this site, and it’s a shame that it isn’t advertised more. I am sure that it might be interesting for other people as well. So have a look at and see what juicy factoids you can mine.

Tags: commits, dash, eclipse, statistics

I’ve been busy this weekend preparing one of the EclipseCon talks I will be doing together with Jordi. The background story to this talk is that we wanted to make it easier for users of Yoxos (and us of course) to deploy new versions of software along with relevant updates. Traditional “.war” deployments are very monolithic and inflexible. The deployment model is simplistic as you only have to support a single version (which may be advantageous to some). But for our needs, we needed a more modular approach to the problem. Since we are not in the business of reinventing circular transportation devices, we started looking at some existing technologies to get as much mileage as possible – so we can concentrate on other neat features.

With Equinox p2, Eclipse already has a powerful provisioning component that performs updates and installation of components… on the client side. Our goal was to leverage the effort that has gone into p2 and adapt the technology to work on the server side. The results are quite promising – with a small amount of effort, we could create a very flexible, modular and robust deployment mechanism. Has anyone else worked with p2 on the server side yet?

Find out more about the technical details at our talk “Down with WAR. Server-side deployment with p2” at EclipseCon. Hope to see you there!

Tags: eclipsecon, p2, server, war

EclipseCon is coming up, and to my big suprise the Git BoF got accepted.

Initially, this BoF proposal was just a way to get the ball rolling on distributed version control systems at eclipse. In the recent weeks I have learned that the ball has been rolling for some time already and has gained quite a momentum, especially among the committers – just take a look at the comments on Bug 257706.

I’d like to get all stakeholders involved in the discussion. Denis Roy will be there to represent the technical and infrastructure side of things. Apart from downstream consumers, committers, contributors and potential contributors, it would also be really great to have some members of the legal team and the board present to get as many viewpoints as possible.

I know that git support was put on hold at the board meeting in December. The cited reason was the resource cost to deploy and support yet another VCS. While I can understand that concern, especially after the recent investment in subversion, I think there is another issue to consider: The opportunity cost of not deploying a DVCS in the near future. Contributions from developers are the life blood of any open source project, and as such, it should be as easy as possible to get people involved in the development process. I experienced first hand how cumbersome it can be to work with the current infrastructure. Centralized versioning may be fine for corporate software development, but for a project like Eclipse I fear it results in too many hurdles for developers. That’s why I’m hoping we can really get this off the ground sooner rather than later.

Regarding infrastructure costs, it has been my understanding that the effort to setup a repository is relatively low. Which I guess makes sense since each developer has to have his own repository. A possibility might even be to use a third party provider like github, at least during the transition period.

Another interesting issue is what work flow model to adopt if we decided to support a DVCS. Options include a “lieutenant” style process as practiced by the Linux kernel, or a more traditional approach with a central master repository that committers commit to.

These are some of the issues I’d like to discuss in this BoF. I am absolutely stoked to see this happening and am really looking forward to making some progress on this front. See you all there!

Tags: bof, eclipsecon, git

The other day I experienced an unexpected light bulb moment concerning unit testing. Maybe this one is obvious to most of you, but I wish someone would have told me earlier. So here goes.

My biggest gripes with unit testing has been that I couldn’t get any satisfactory answers to these two questions:

1. Why should I practice Test-First?
2. How do you test the tests?

Concerning the first issue, well we discussed some papers that where trying to answer that question when I attended a software quality course at university. The gist of the results were: There is no statistically significant difference in code quality between Test-First and and Test-Later. (Sorry can’t find links to the papers atm. Holler if you want me to find them and I’ll do some more digging.)

The second issue is discussed often as well: If tests are code and code should be tested, doesn’t that lead to more and more tests? This is sometimes referred to as the stack overflow of unit testing.

The revelation I had was this: These two questions answer each other! What’s the easiest way to test a test? A broken implementation. Where do you get a broken implementation? Just use an incomplete implementation. If you practice  Test-First all your implementations start off incomplete by definition. This means that each assertion in your test is guaranteed to fail at least once, giving you the confidence that the assertions actually perform significant work.

I am often surprised that assertions that I expect to fail actually go through just fine. This usually means one of two things: Either the functionality is already there by some fluke (cf. accidental correctness), or my test is incorrect. In either case I can then fine-tune and adapt the assertions to make sure that they fail and thus test some missing functionality.

Two conundrums solved.

Tags: first, test, testing, tips, unit

I have been pondering exceptions quite a lot lately. Especially how to break the bad news to the user. From a user’s point of view there are three categories of errors:

1. User or domain errors

These errors occur when the user enters invalid data, or tries to perform an action that the domain model does not allow. Typical examples of this kind of error are things like trying to enter a duplicate key or trying to delete an item that is still being referenced. Note that these kinds of errors are usually not even worth logging. These types of errors can usually be recovered by the user himself, either by choosing an alternative action, or by manipulating the model in such a way that the action becomes valid. While the exception usually describes what went wrong, the error message displayed to the user should also explain what can be done to correct the situation in such a way that the desired. Ideally the error dialog should also allow the user to directly apply those fixes in order to achieve his original goal. Prime example of such an implementation: Eclipse quick-fixes. While telling you what you did wrong, they also offer to repair your mistakes. In fact they are so convenient and helpful that I have come to rely on them when writing code: Write pseudo-code, then let quick-fix sort out the details.

2. Infrastructure errors

These kinds of errors are usually intermittent failures of one of the subsystems the application is running on. Usually these are network outages, a full disk or a server on fire. Here a detailed error message is helpful to enable the user to track down and rectify the problem. Depending on the type of application and business environment, it might also be a good idea to notify the responsible systems administrator that a problem has developed, so he can get a little head start before the hordes of users demand to know why they can’t access their TPS reports. In rare cases there might be backups or fail-overs in place which can be used in case of such infrastructure problems.

3. Programmer errors

While we all strive to avoid it, there are going to be programmer errors (internal errors aka bugs). These are basically all errors not in one of the above categories. The most common manifestation of these types of errors is the dreaded NullPointerExceptions. But we’ve all seen the code comments declaring: “This should never happen.” Sometimes assertions and assumptions we make when writing software turn out to be wrong. The best we can do in these situations is to fix the problems as soon as possible. Ideally the application in questions gathers as much information surrounding the error as possible, then allows the user to report this information, in addition to his notes about what he was doing at the time of the failure. The message to the user should be along the lines, “We are sorry for causing you trouble. Let’s make sure this particular problem does not show up in the future.” The idea is to show users that their feedback is not a pointless exercise, but an effective way to make their daily lives a little better by improving their tools.

Exception classes

The problem with many exception hierarchies in libraries and applications alike is that they distinguish exception classes at component, interface or service boundaries. If the StockTickerService returns a CannotReadStockSymbolDataException, the caller has oftentimes no way to distinguish between user error (Stock symbol does not exist), infrastructure error (server unreachable) or internal error (could not handle values above 255). This makes handling these exception at the user interface level quite challenging. Sure you can take the easy way out and display the message to the user, maybe log it and be done with it, but the application itself has no way to distinguish between the different levels of severity. A typo in the input field is nothing to write home about, while not being able to handle values above 255 is a systematic failure that might render the system unusable.

My point here is this: Being able distinguish the type of exception is often helpful when trying to recover from an error. But in my experience errors are not seldom recoverable. And in these cases it is more important to know who to inform about the failure. This makes the triage of errors much simpler, administrators and developers get notified about problems in their areas of responsibility, but are not flooded with reports about “trivial” user typos that drown out the real issues.

Tags: error, exception

I used what little free time I had over the holidays to catch up on the recent developments in source control management systems, which have been quite interesting to follow. Especially the arrival of Distributed Version Control Systems has caused quite a buzz in the software development industry.

Somehow Eclipse as a whole totally missed the SVN boat. Only in the last year has there been any uptake of Subversion within the Eclipse Foundation itself. And although there are two separate plugins (and by my count at least 3 different adapter libraries) it still doesn’t feel quite as solid as the CVS counterpart. But the writing is on the wall: CVS does not fulfill all the needs we expect from a contemporary source code management system. And while I have been a long-time fan of Subversion (atomic-commits and rename tracking, yay!) I still feel that maybe the time of centralized source control is over, especially in an open source environment. Doug Schaefer seems to agree. Distributed source control makes it much easier for potential contributors to get started. Experimental and “investigative” branches are easier to manage, and things like feature branches are more easily merged again. While it’s not all roses either (partial checkouts and IP-taints anyone?), but it does make the the development process quite a bit better. And isn’t that one of the main goals of the Eclipse Foundation?

So over the holiday I checked out (quite literally) the latest egit sources and played around with it a little. I must say that I came away quite impressed. Although there are still some features missing, the core looks very solid and usable.

The history of the egit project

Because the source for egit is openly available, it was very easy for me to “grok” the innards of the git system. My C is getting a little rusty these days, so a Java implementation made a much better reading experience for me. Debugging and Ctrl-clicking as well for that matter. With the core jgit library in hand, I even tried to implement a versioned POJO persistence store, but that’s a story for another time. What struck me most about git was the raw and brutal simplicity and the resulting elegance of its design. Believe it or not, but git only knows four different types of objects. Java’s “hello world” uses about that many! Another neat thing is the hash value that is calculated for virtually everything you do. When I first readabout this feature, I was quite skeptical to be honest: For starters, I don’t want to type 40 letter strings anytime I want to reference a revision and secondly, what about collisions? Well I worked out the math on that one, and what it comes down to is this: by the time I will create my first collision I will have been struck by lightning three times over and have survived the sun going supernova. And as for typing in those hashes, well you need it less than you would think and even then you usually only need the first few letters to make it unambiguous.

So kudos to the egit team for the great and continued work on this project. There even is an egit project proposal up. Although git support was postponed in the last board meeting, I still think this a step in the right direction and I’m looking forward to future developments.

Tags: eclipse, egit, git, jgit

The Eclipse Packaging Project is working on a wizard that allows you to build your own customizable download, combining plug-ins from predefined packages and Ganymede. The wizard uses Eclipse runtime technology – namely RAP for the web frontend and Equinox P2 for the installer. Before exposing the wizard to a wider audience, we had one burning question: Does it scale up to the volume of downloads being served by Eclipse.org? Read on to find out.

Load Test – Setup

We conducted a load test on the wizard and would like to share the results with you.

The tests were executed on a modern desktop computer with a Core2Duo 2.8 GHz CPU running Ubuntu Hardy. The JVM used a maximum of 512 MB of heap space and ran the concurrent marksweep garbage collector (-UseConcMarkSweepGC). The tests were run using JMeter 3.2 with 200 concurrent threads. Tomcat 6 was used for the server.

Load Test – Results

The results are promising: with a throughput of 100 requests per second we achieved an average response time of 14 miliseconds. Configuring a download takes up to 80 requests, which means we can serve approximately 100,000 downloads per day with a similar configuration.

Here’s a JMeter screenshot with more details on the results:

The fast average response time is paired with a low standard deviation of 60-80 milliseconds. The maximum response times were caused by full garbage collections, but only a few users should experience these response times (otherwise we would have higher average response times).

The allocation of heap is visualized in the diagram below, after multiple hours of load testing. A heap space of 250 MB was constantly needed for the approximately 1,000 concurrent user sessions that were simulated (250KB per session). Another 200 MB was built up and collected every one to two minutes.

Cloud Tests

To demonstrate performance we also ran tests against the same application running on the smallest available type of Amazon EC2 instance.
We ran the same load test as before – but using only 50 concurrent threads. After a period of 16 hours 16,000 sessions had been served, which means a throughput of about 24,000 sessions a day. This is almost equivalent to all the downloads served from Eclipse.org on a busy day. Average response time was approximately 100ms, with network latency accounting for most of the response time. CPU utilization on the server was around 15%.

Conclusion

We’re very positive about the results so far. Even though there is a memory overhead associated with RAP applications, in practice the performance is very solid. Random sampling of the application via a browser in parallel with the load tests showed that responsiveness was very good.

But don’t just take our word for it – try it out for yourself. The application is hosted on an EC2 instance reachable at
http://eppwizard.dyndns.org/eppwizard/rap.
The load test is still running with 50 concurrent sessions against that same instance.

We plan to beta test the wizard with the “Friends of Eclipse” after seeking initial feedback from the community on both usability and performance.
Take it for a spin and tell us what you think.

Tags: epp, load, performance, rap