Tagging Eclipse for the next integration build

Tagging Eclipse for the next integration build

For anyone who follows the Eclipse developer mailing lists, you might notice the weekly e-mails regarding the tagging process. The e-mails usually describe which bundles have been tagged and which changes are included in the build.  In this post, I’m going to describe a little more about this process and why it’s important to the Eclipse build.

Types of Builds:

Before we start, it’s important to understand the types of Eclipse builds. During normal development (not including the maintenance stream) there are two types of Builds. There are builds from head and integration builds.

The builds from head happen every night.  These can be considered the continuous builds and help ensure that our day-to-day work is not breaking the build. Since the build take several hours to complete, it’s impractical for Eclipse to run Builds on Check-In, so instead, Eclipse builds from head once a day. These builds are not really meant to be consumed by users.

The integration builds happen once a week (usually on Tuesday morning) and are built from tags.  The advantage of this approach is that you must explicitly tag your changes for them to be included in an integration build.

Bundle Versions:

If you have ever looked at set of plugins that make up your Eclipse install you will notice that the version numbers are all over the map.  It is well known that Eclipse uses a 4 part Version number scheme (Major.Minor.Service.Qualifier), and the first 3 parts evolve as the API changes.  The forth part (Qualifier) represents the CVS tags.

So this brings forth the obvious question: if Integration builds are built from tags, and the code is tagged once a week, why don’t all the bundles have the same qualifier/tag?

The answer is simple we only tag bundles that change.

Bringing it all together Releng Tools:

If you are doing Eclipse plug-in development with CVS as your SCM, you should be using the Eclipse Releng tools.  It doesn’t come included in the SDK, but you can install it from the Eclipse update site.

Using a map file, releng tools will automatically determine which bundles have changed since the last time they were tagged:

Show you what has changed:

Generate release notes:

And re-tag the bundle(s):

Eclipse tags are often (but not always) in the form: vYYYYMMDD-HHMM. I tend to use the Ottawa date and time (which is three hours later than my current time) — So no, I wasn’t up until midnight on valentines day tagging p2. The only real requirement is that tags / qualifier move forward in time.

Why go to all this trouble?

The biggest advantages of only tagging bundles that change is that unchanged bundles will have exactly the same version number. This means that when you update Eclipse (from week to week) you only need to download the modified bits Let me state that another way (because it’s important): if you update Eclipse and only 4 of the 500 bundles have changed — p2 will only download those 4 bundles.

This approach also enables re-creating an exact build. All you need is the map file, and you’ll have the full list of tags that constituted the build.

Finally, in CVS, tagging is considered a slow operation.  By having each team tag their builds, and by only tagging the bundles that changed, the workload is both reduced and distributed.

  • Posted at 8:36 am, February 15, 2011

    > it’s impractical for Eclipse to run Builds on Check-In

    There must be away to make it practical. One should not have to compile all dependencies – they could be fetched already compiled from nightly/integration/milestone repositories, right?

    It is a shame that a project that lives by componentization has such a monolithic approach to building….

  • Juan Alonso
    Posted at 11:50 am, February 15, 2011


    Does ‘automatically determine which bundles have changed since the last time they were tagged’ available in headless mode?


  • David Carver
    Posted at 8:53 pm, February 15, 2011

    Ian, you need to break the true CI builds down into compent based builds. Right now you have what you have is what I call a nightly integration since you are build the entire project and running all tests. Ideally, you have a set of components that gets build everytime there is a check in. The way the build process is setup right now for the platform it makes this difficult. (Now if it was a maven tycho build, you could do this easier).

    As for the other dependencies, that is what depending on P2 sites is for. And the other issue you have is that many of the items the platform calls unit tests are truely integration tests. There are very few true unit tests within the eclipse platform, but lots of Integration tests.

  • Mark
    Posted at 4:38 pm, February 16, 2011

    Good info. What i would also like to see is some info/article/etc on how to develop multiple plugins at one time. What i mean is how do you normally not get crossed up when using plugins that are being developed by other groups but you don’t want what is being changed, you want a certain release and only want to get releases when you want to. I would think you would use binaries of the plugins unless you actually need to make a change. Then you would “checkout” the source, which should somehow be used instead of the binary plugin. Note: I am building Eclipse RCP/RAP apps that share code amoungst them AND also with GWT and “web apps”.