Tagging Eclipse for the next integration build

For anyone who follows the Eclipse developer mailing lists, you might notice the weekly e-mails regarding the tagging process. The e-mails usually describe which bundles have been tagged and which changes are included in the build.  In this post, I’m going to describe a little more about this process and why it’s important to the Eclipse build.

Types of Builds:

Before we start, it’s important to understand the types of Eclipse builds. During normal development (not including the maintenance stream) there are two types of Builds. There are builds from head and integration builds.

The builds from head happen every night.  These can be considered the continuous builds and help ensure that our day-to-day work is not breaking the build. Since the build take several hours to complete, it’s impractical for Eclipse to run Builds on Check-In, so instead, Eclipse builds from head once a day. These builds are not really meant to be consumed by users.

The integration builds happen once a week (usually on Tuesday morning) and are built from tags.  The advantage of this approach is that you must explicitly tag your changes for them to be included in an integration build.

Bundle Versions:

If you have ever looked at set of plugins that make up your Eclipse install you will notice that the version numbers are all over the map.  It is well known that Eclipse uses a 4 part Version number scheme (Major.Minor.Service.Qualifier), and the first 3 parts evolve as the API changes.  The forth part (Qualifier) represents the CVS tags.

plugins Tagging Eclipse for the next integration build

So this brings forth the obvious question: if Integration builds are built from tags, and the code is tagged once a week, why don’t all the bundles have the same qualifier/tag?

The answer is simple we only tag bundles that change.

Bringing it all together Releng Tools:

If you are doing Eclipse plug-in development with CVS as your SCM, you should be using the Eclipse Releng tools.  It doesn’t come included in the SDK, but you can install it from the Eclipse update site.

Using a map file, releng tools will automatically determine which bundles have changed since the last time they were tagged:

changed Tagging Eclipse for the next integration build

Show you what has changed:

changes Tagging Eclipse for the next integration build

Generate release notes:

release notes1 Tagging Eclipse for the next integration buildAnd re-tag the bundle(s):

tag Tagging Eclipse for the next integration build

Eclipse tags are often (but not always) in the form: vYYYYMMDD-HHMM. I tend to use the Ottawa date and time (which is three hours later than my current time) — So no, I wasn’t up until midnight on valentines day tagging p2. The only real requirement is that tags / qualifier move forward in time.

Why go to all this trouble?

The biggest advantages of only tagging bundles that change is that unchanged bundles will have exactly the same version number. This means that when you update Eclipse (from week to week) you only need to download the modified bits Let me state that another way (because it’s important): if you update Eclipse and only 4 of the 500 bundles have changed — p2 will only download those 4 bundles.

This approach also enables re-creating an exact build. All you need is the map file, and you’ll have the full list of tags that constituted the build.

Finally, in CVS, tagging is considered a slow operation.  By having each team tag their builds, and by only tagging the bundles that changed, the workload is both reduced and distributed.

6 Responses to “Tagging Eclipse for the next integration build”

  1. > it’s impractical for Eclipse to run Builds on Check-In

    There must be away to make it practical. One should not have to compile all dependencies – they could be fetched already compiled from nightly/integration/milestone repositories, right?

    It is a shame that a project that lives by componentization has such a monolithic approach to building….

  2. Juan Alonso says:

    Ian,

    Does ‘automatically determine which bundles have changed since the last time they were tagged’ available in headless mode?

    Regards,
    Juan.

  3. Ian Bull says:

    @Juan,

    I’m not sure. I don’t know if releng-tools has a headless mode of operation. This seems doable. I’ll check my twitter followers and see if anyone knows.

  4. Ian Bull says:

    @Rafael, I think there is. It means having each platform team do their own builds. For example, the JDT might be able to do builds-on-checkin, but not if it has to builds PDE, Equinox, p2, platform, SWT, etc… But this would be a pretty big architectural change, and it’s hard to judge if it’s worth the effort. The ability to self host + nightly builds means that builds don’t break very often — well they don’t break in the way that CI building would help.

  5. David Carver says:

    Ian, you need to break the true CI builds down into compent based builds. Right now you have what you have is what I call a nightly integration since you are build the entire project and running all tests. Ideally, you have a set of components that gets build everytime there is a check in. The way the build process is setup right now for the platform it makes this difficult. (Now if it was a maven tycho build, you could do this easier).

    As for the other dependencies, that is what depending on P2 sites is for. And the other issue you have is that many of the items the platform calls unit tests are truely integration tests. There are very few true unit tests within the eclipse platform, but lots of Integration tests.

  6. Mark says:

    Good info. What i would also like to see is some info/article/etc on how to develop multiple plugins at one time. What i mean is how do you normally not get crossed up when using plugins that are being developed by other groups but you don’t want what is being changed, you want a certain release and only want to get releases when you want to. I would think you would use binaries of the plugins unless you actually need to make a change. Then you would “checkout” the source, which should somehow be used instead of the binary plugin. Note: I am building Eclipse RCP/RAP apps that share code amoungst them AND also with GWT and “web apps”.

6 responses so far

Written by . Published in Categories: EclipseSource News, Planet Eclipse