Over the past year I’ve developed a goal of making it easier to build and deploy the Equinox runtime as a web application server.  Not only does this fit my technical skills having previously worked on a commercial application server, it also fits my technical interests. It also seems that an ever increasing set of other people have been interested in running the Equinox OSGi runtime as a application server. Furthermore, many people may not be aware that there are now a number of really excellent  technologies available via the EclipseRT project that are relevant to creating modern application servers:

• ECF
• RAP
• P2
• PDE
• Equinox
• EclipseLink
• Riena
• Smila
• eRCP

There’s a lot more to come, for example, ECF’s work on supporting Google Wave server protocol as well as commercial technologies from EclipseSource and other technology companies devoted to the OSGi stackless stack.

However, in my opinion it’s far more difficult to build, deploy, configure and run an Equinox server than it should be.  There is existing documentation on doing so from scratch here, but it is a little dated.  I am contributing to the Equinox project to get the documentation updated with community help.

So, I would like to do a short set of blogs about building your own Equinox-based application server.  Hopefully this will be helpful to those who are interested. It’s possible to create two ‘kinds’ of Equinox-based application servers. Those that are actually run stand-alone (i.e., as a Equinox OSGi application) or those that are run within an existing servlet container such as Jetty or most commercial application servers.  For this series, I’m going to focus on the servlet container method as I think this will ultimately be the more common and popular method. Furthermore, servlet containers now make deployment, installation and configuration very easy (which goes a long way toward my goal).

Ok, so how to create an Equinox-based application server?  Well, with the Eclipse Galileo PDE build, p2, features, and products, it turns out it’s now very easy to do so (compliments to the particularly to the folks who worked on PDE, Equinox, and P2 in this area).  I’ve created a simple feature and product that does so and contributed them to this bug.  Here are the steps:

1) Download the zip attached to bug 245267

2) In Eclipse, open a new workspace and import the projects from the downloaded zip (File->Import…->Existing Projects in Workspace->Select Archive File).  This will put two very small projects into your workspace

3) There are two other projects (from Equinox CVS area) required before you can create your app server.  To add these projects right click on the equinox-servletbridge.psf file and select Import Project Set…  from the menu (use anonymous CVS login when prompted).  Now you should have four projects in the workspace

4) Now, open the PDE product editor by double clicking on equinox-milli.product file

5) Select Eclipse product export link in lower right of product editor

6) In the export wizard dialog set the following fields

a) The Root Directory field must be set to:  WEB-INF

b) The Synchronize before exporting check box can be unchecked

c) The Destination must be set to Archive file, and given a location/name of your choosing (e.g. myappserver.zip)

d) The Generate metadata repository should be checked (to include p2 metadata)

e) Click Finish

This will run PDE build and create a zip named (in the above case): myappserver.zip

6) Now comes the fun part.  Simply rename this zip file to a .war…i.e. myappserver.war

7) Start your servlet container (in this case I’m using tomcat)

Hot deploy the war file (in this case myappserver.war) in the manner appropriate for your servlet container (in tomcat’s case this is copying the myappserver.war file into the webapps directory).  After a few seconds, you should see something like the following output to the tomcat console…with the nice little osgi> prompt, indicating your Equinox server is deployed and running!

9) Give some OSGi console commands with the osgi> prompt just to convince yourself everything is working…e.g.

10) Go show your colleagues how easy it was to create, deploy and run your Equinox application server!

There’s a lot more that you can now do with your Equinox application server like:

• Adding servlets (via bundles)
• Using P2 to update your application server
• Creating your web applications in a modular fashion using bundles and features
• Reuse your favorite bundles from Eclipse or other projects
• Deploy and maintain your web application as bundles (and using PDE to do it)

In the future, I’ll describe how to do some of these things.

If people are interested in particular things to do, please comment and let me know.

Tags: ecf, equinox, OSGi, p2, rap

There has been a lot of talk lately about p2.  While many people know p2 as the update mechanism used by Eclipse, it is in fact a general purpose provisioning platform.  This means you can use the provisioning platform as a base to build a highly customized provisioning solution for your own needs… whether it is an Eclipse application, OSGi application or anything else.

On September 15, I will be presenting an introduction to p2. In particular, I will demonstrate how you can use p2 to build a provisioning solution for Eclipse RCP applications. The webinar will teach you how you can:

• Make use of the p2 provisioning platform for software management
• Integrate the creation of p2 repositories into your build system
• Reuse the p2 user interface and existing work-flows
• Customize the p2 user interface for your specific needs
• Debug common pitfalls

For more information, please see the registration page.

If you want anything else discussed about p2 in the webinar, please leave a comment and if there is time, I will try to get to it.

Tags: p2, rcp, webinar

If you’re using Eclipse for OSGi development, there’s a neat utility that you can use to help you run your OSGi application on a variety of frameworks. PAX Runner uses the PDE org.eclipse.pde.ui.osgiFrameworks extension point to supply frameworks.

To install PAX Runner, simply add their repository via software installation wizard.

There’s an important thing to note about PAX Runner. When you’re self-hosting with a PAX Runner based framework, your bundles will be deployed before executed. This may cause launching to take a bit longer than you’re used to depending on the size of your application. This is different from the traditional PDE workflow where we don’t have a deployment step when self hosting because Equinox provides us with some great hooks to skip that process. If you’re curious how this works under the covers, PDE instructs Equinox where to look for classes for the various bundles in your workspace (usually in the bin/ folder). This is known as the development time classpath and as far as I know, Equinox is the only framework to support this type of classpath (I filed a bug against Felix awhile ago).

On the whole, I recommend you give PAX Runner a try.

Also, if you have any ideas on how PDE can better support other frameworks, come talk to us.

Tags: OSGi, pde

Yesterday I discussed some of the issues with XML. Today I’ll be taking a look at three of the potential alternatives that may improve on the current situation.

YAML

YAML Ain’t Markup Language. To quote the spec, it is a “human-friendly, cross language, Unicode based data serialization language”. While mainly designed with so-called agile languages in mind, it can also be used with more traditional languages. The format is more data-centric than document-driven. Even so, one of its primary design goals is good readability. It was heavily influenced by RFC 2822 (Internet Message Format). That means it looks a lot like mail headers. It features built-in support for lists, hashes (i.e. dictionaries or associative array), and common data types. It also allows elements to have multiple parents, which also allows cross-references. There are some more minor features that make working with the format easier. Here’s a small example of a YAML document lifted from the spec:

invoice: 34843
date   : 2001-01-23
bill-to: &id001
    given  : Chris
    family : Dumars
    address:
        lines: |
            458 Walkman Dr.
            Suite #292
        city    : Royal Oak
        state   : MI
        postal  : 48046
ship-to: *id001
product:
    - sku         : BL394D
      quantity    : 4
      description : Basketball
      price       : 450.00
    - sku         : BL4438H
      quantity    : 1
      description : Super Hoop
      price       : 2392.00
tax  : 251.42
total: 4443.52
comments:
    Late afternoon is best.
    Backup contact is Nancy
    Billsmer @ 338-4338.

While YAML seems like a solid solution with implementations written for many languages, there are some issues, like the surprising lack of momentum in the software development community and the controversial use of significant whitespace.

JSON

The JavaScript Object Notation is basically a subset of JavaScript used to statically describe data. With the release of YAML 1.2 it is also a subset of YAML, which means every JSON document is a YAML file. Its focus is primarily simplicity and readability. While it is trivial to parse JSON in a JavaScript environment, its simplicity makes it also quite easy parse in other languages. Parsers for most popular modern development platforms exist. Being so easily accessible in browsers has earned it quite some support and momentum in modern web development, already often completely replacing XML in the AJAX stack. Here’s a small snippet lifted from the JSON Wikipedia page:

{
     "firstName": "John",
     "lastName": "Smith",
     "address": {
         "streetAddress": "21 2nd Street",
         "city": "New York",
         "state": "NY",
         "postalCode": 10021
     },
     "phoneNumbers": {
         "home": "212 555-1234",
         "fax": "646 555-4567"
     }
}

JSON is currently quite popular, though it may in certain cases be hampered by its simpleness. Maybe YAML forward compatibility can provide an convenient upgrade path, should a more sophisiticated format be necessary.

Groovy

Most of you may know Groovy as a JVM scripting language. I have also already blogged about using Groovy to replace especially painful parts of XML.

Groovy features a MarkupBuilder that let’s you create what is basically an XML DOM tree in memory using a slightly more fluent syntax. Have a look:

  car(name:'HSV Maloo', make:'Holden', year:2006) {
    country('Australia')
    record(type:'speed', 'Production Pickup Truck with speed of 271kph')
  }

Some might consider this just syntactic sugar but it really goes a long way.

But I think Groovy can also be used as a first class configuration language. There are already several projects out there that use groovy scripts for tasks that have traditionally been in the firm grip of XML. One such example is gant, which is basically just a thin wrapper to write ant files. Of course nowadays, everyone is using maven instead, but there’s also a neat tool for that: Gradle is build system configured using a Groovy DSL, while employing Apache ivy and maven under the hood. Take a look at this example from the official gradle documentation:

usePlugin 'java'
 
sourceCompatibility = 1.5
version = '1.0'
manifest.mainAttributes(
    'Implementation-Title': 'Gradle Quickstart',
    'Implementation-Version': version
)
 
repositories {
    mavenCentral()
}
 
dependencies {
    compile group: 'commons-collections', name: 'commons-collections', version: '3.2'
    testCompile group: 'junit', name: 'junit', version: '4.+'
}
 
test {
    options.systemProperties['property'] = 'value'
}
 
uploadArchives {
    repositories {
       flatDir(dirs: file('repos'))
    }
}

A lot easier on the eyes, while still providing interoperability with “legacy” systems like maven.

The power and expressiveness of groovy make it really easy to create domain-specific languages like these. Such a special purpose language might not always be desirable, especially when interoperability is a key concern. But for certain applications these DSLs might be a better solution than any general purpose format.

On the whole, these are interesting times we live in and I’m curious to see what the future holds in store. As I said before, XML is probably gonna be here for quite a while, but there are some compelling alternatives out there. What are your thoughts on the legacy of XML?

Tags: gant, gradle, groovy, json, syndicate, xml, yaml

The XML format has done a lot in the last decade to reduce some of the pains of legacy formats and to encourage application interoperability. Having a standardized syntax and object model makes the development process a lot easier. But I still feel that there are some severe shortcomings when it comes to the general format itself and and the concrete implementation of XML dialects that I want to discuss in this blog post.

XML is a markup language

As the name Extensible Markup Language implies, the language is first and foremost a markup language. That means that the language annotates a body of text. So if you were to strip out all the markup from an XML document, it should still end up with legible and comprehensible. This is certainly true for (X)HTML and DocBook. It might be slightly harder to read and understand, but all the crucial information is right there in plain text. The markup just adds semantic or presentational meta-information, e.g. which bits of text are headers or quotes.

In many cases XML is misused as a general data format. This often means that there is no actual text (character data) whatsoever in the resulting files. Then why use a markup-language in the first place? The Eclipse plugin.xml format for example is a markup only format (save some extension points schemas).

Attribute and Element Dichotomy

Another gripe with XML is when to use attributes vs. elements. There are at least three options that at first glance seem equally plausible. For example:

Name as PCDATA

<customer>
ACME Inc.
</customer>

Name as extra element, PCDATA

<customer>
<name>ACME Inc.</name>
</customer>

Name as attribute

<customer name="ACME Inc." />

I have seen these three styles mixed within the same document for almost exactly the same fields. Should a general purpose data interchange format really be that hard to get right, or at least consistent?

Awkward mapping of common constructs

Because xml implicitly is a tree structure, all structures involving cyclic references or multiple cross-references in general are not easily mapped to an XML compatible form. Object-oriented models can often contain back-references, or reference a single object from different places in the object graph. Although there is some support for such constructs in the form of ID/IDREF attributes, this is neither supported by all parsers nor even publicly widespread information.

Another common data type that is painful to describe in XML is the associative map. An example from the Eclipse plugin.xml:

<property name="aboutImage" value="eclipse_lg.gif" />

Compare that to a simple properties file format:

aboutImage=eclipse_lg.gif

The third type of commonly found data that is hard to put into XML is relational data. That kind of data is traditionally found in RDBMS, CSV files and unfortunately spreadsheets. While it is fairly trivial to create a corresponding xml format, the result is often needlessly verbose and repetitive, since all elements usually have the same attributes.

Readability

By far the biggest gripe I have with xml on a day to day basis is that it is really hard to parse for me as a human. Between the angle brackets bunched up against the element names, the endless repetition and escaped entities it seems like this format was not really designed with a legibility in mind. Another reason could be related to my first point: In regular markup languages the tags only contribute a relatively small percentage of the overall content. The majority is plain text, so the tags are few and far between. In markup-only languages however, there’s a much higher density of markup elements. In my opinion this redundant repetition lowers the signal to noise ratio significantly, making these documents much harder to read.

These are the main problems that I currently see with XML. I concede that the common, extensible meta format was a huge step forward, and that for some problem domains it is a pretty good fit. I also realize that XML is gonna be here for quite a while, but I think it’s time to stop resting on these laurels and see how we can address these problems in the future.

Tomorrow I’ll be looking at the some of the alternatives that might be potential successors to XML.

Tags: xml

Good news, Eclipse 3.6 M1 is out and available for download (new and noteworthy)!

Why is this good news? Well, this marks the beginning of the regular heart beat of the Eclipse release cycle. The next release train is called Helios and is due out June 23rd, 2010. This week the Eclipse Planning Council finalized the dates for the Helios milestones:

• Helios M1 8/7 – 8/21
• Helios M2 9/18 – 10/2
• Helios M3 10/30 – 11/13
• Helios M4 12/11 – 12/18
• Helios M5 1/29 – 2/5
• Helios M6 3/12 – 3/19
• Helios M7 4/30 – 5/7
• Helios Release: 6/23/2010

Onwards to the Eclipse Helios release!

Tags: helios, milestone

There is a user interface design issue that has been bugging me for quite some time, but I haven’t been able to put my finger on it until recently. How do you best represent disabled interaction elements (most often in the form of grayed out buttons and menu entries)? The visual cue here is that the action is not available at the moment. While that information is quite convenient, it raises the obvious question of:

Why is that action not available?

Often the answer might be obvious to the experienced user but this isn’t good for novices.

What about new or inexperienced users? Can’t we ease the learning curve a little and help users get to grips with the desired functionality? This might help avoid statements like…

“Oh wow, so that’s what that button does. I never used it because it was always disabled.”

In many cases there might only be a single reason why the action is unavailable. That means if the button is gray it is fairly simple to figure out what is amiss. But in non-trivial applications there may be a whole host of reasons why an action can not be performed. Typical reasons include:

• technical reasons: e.g. a “delete” action can not be performed if nothing is selected
• infrastructure reasons: e.g. a required service is unavailable
• operational reasons: e.g. user has insufficient privileges, or  the action can only be performed during business hours
• business domain reasons: e.g. certain preconditions have not been met, or the action is incompatible with another action that has already been performed

That last reason can really be a big can of worms… because at least in my experience “businessy” applications have a propensity for growing complex rule sets that have to be modeled in software. That may even leave domain experts guessing why the button “re-evaluate taco consumption rates” is grayed out.

So is there way to deal with that problem in a more sophisticated and helpful manner?

Here is an idea how the information could be presented in a natural and obvious way.

Tooltip for disabled elements - Answering the question

We’re doing nothing mind-boggling here but it gets the message across. On the implementation side of things, I could imagine something analogous to JFace style validators to attach to the button so that the appropriate message can be created on the fly. This might make for a cleaner technical design because different concerns of when to disable the button can be decoupled. One big drawback of this approach is that disabled (Windows) buttons don’t seem to show their tool tips. It may be possible to work around this problem with some SWT-fu though.

A more radical approach to the whole dilemma is the standpoint that disabled user interface elements are design smells that should be avoided completely. Applications should be designed in a way to support the necessary work flows and guide the user along logical steps… instead of doing trial and error and guesswork.

After all, trial and error and guesswork should be a domain reserved for point-and-click adventures.

So have you run into this issue as well? What were your experiences or solutions?

Tags: button, disabled, swt, syndicate, tooltip, ui

As a reminder, there’s two weeks left to submit talks to Eclipse Summit Europe 2009.

I hope to see everyone in Ludwigsburg!

There is a very cool new tutorial by Bryan Hunt for using ECF‘s implementation of the OSGi Distributed OSGi spec (RFC 119) and OSGi Declarative Services (DS) together to do remote declarative services.

In addition to demonstrating the power of combining DS with ECF 3.0′s support for distributed OSGi services, the tutorial has two other very cool aspects:

• It was created by a community member after using these technologies to build their own system
• The tutorial will soon be contributed to the ECF documentation section available in the ECF wiki area

Enjoy!

Tags: ecf, OSGi, rfc119

Last week, the Eclipse e4 project had its alpha release.

What’s e4?

It’s simply the next generation of the Eclipse platform. The Eclipse platform team has learned a lot of lessons since the Eclipse project started and it’s time to experiment and innovate. There’s also a desire to involve more people at the platform level.

To get a better idea of where e4 is going, here’s some resources:

• e4 whitepaper
• e4 New and Noteworthy
• Mike Wilson’s blog post on the future of Eclipse
• Register for e4 webinars
• e4 wiki

If you’re interested in the project or the future of Eclipse, the e4 team would love to hear from you.

The first general release of e4 is expected in 2010.

Tags: e4