Exhibit A: git repositories

Let’s start with the first image. This one is taken from Scott Chacon’s talk “Getting Git”. (Actually I had to mix slides 138 and 142 together to better fit the blog format.) I’ll try not to go to much into the details of git here, listen to the talk instead of you want to know more. The thing I do want to talk about though is git’s tree structures. These tree structures describe the project contents (both directory and file) for one specific commit. These trees are completely immutable (this is ensured by SHA-1 hashes). A new commit creates what amounts to a completely new tree. But usually the changes in each commit are only a fraction of the whole tree. Since a lot of the sub-trees of the original tree have not changed (and are immutable), these can be safely recycled. Only files and folders that have changed need to be added. Note that this means parent nodes of changed nodes have to be created as well, recursively up to the tree root. But even though the tree itself is new (as evidenced by the new root) the additional space requirements are quite low, on the order of approximately O(log n). This makes for extremely compact repository format, and an astonishingly simple one.
Another thing I want to point out that the only element that needs to be mutated is the reference to the root of the tree, everything else is immutable.

Exhibit B: Clojure’s persistent data structures

Next up is a picture lifted from Rich Hickey’s “Clojure Concurrency” talk. If you are interested in figuring out ways to deal with the impending multi-core age, I highly recommend you take a look at this talk. The concepts and techniques are not exclusive to Clojure, so even if you are not into Lisp’s you might want to watch it.

The way clojure deals with concurrency (in a nutshell) is that every core data structure (list, trees, hash maps, etc) is immutable. This has one big perceived drawback: Immutable data structures incur a huge overhead because of all the copying involved in creating new instances. But this is only a problem if the immutability is implemented naively. Analysis shows that usually only a part of a data structure is actually modified at any given time. This again allows recycling large parts of the the “old” data structure and only creating part of it new, and just pointing to the bits of the old structure that have not changed. As these are immutable as well, it is completely safe to do so. In the picture the two green boxes are the root nodes of two data structures sharing large parts of their trees.

Since a program that only deals with static, immutable data is pretty boring, there are mechanisms for introducing changes in a program. These mechanisms are called references, and these do mutate. But the runtime ensures that these changes happen in a controlled and well behaved manner, e.g. in the form of Software Transactional Memory (STM) or agents. But everything else is essentially immutable. Note that this has the neat side effect that reading process are inherently thread-safe since nothing can suddenly change while you are looking at it, i.e. no more ConcurrentModificationExceptions.

Exhibit C: CouchDB’s append-only file format

The third picture is from a blog post by Ricky Ho titled “NoSQL patterns”. CouchDB has made quite a few waves recently, so I wanted to see what makes it tick under the hood. In keeping with the Erlang philosophy of robustness and failure tolerance, couchdb uses an append-only file format. This means that all data written is not modified any further and there is thus little chance for corruption. CouchDB uses a B+ Tree to index its entries, so again we have a tree structure. And because the tree entries live inside an append only file, these trees are immutable as well. When making a change to the database, the new data is written along with all changes to nodes in the B+ Tree. Since the B+ Tree is very flat, that means even for databases with millions of entries the number of updated tree nodes is fairly small. The most recent root of the index tree can always be found at the very end of the file, and thus the database file is read from the end backwards. If any kind of failure occurs while writing changes, the software can see that the last entry is corrupt and can seek further back until it finds a “good” entry and proceed from there. Another interesting benefit is that reading processes don’t block writing processes and vice versa. A reading process can just find the latest root and then work from there. Because all references go backward in the file, and everything is immutable there is no contention. Meanwhile a writing process can happily append at the end of the file without disrupting any readers.

Further exhibits

There are probably more examples of this pattern to be found. The oldest reference I could find is “The Design and Implementation of a Log-Structured File System” written by Mendel Rosenblum and John K. Ousterhout in 1991. The most recent file system effort seems to be NILFS. The flash-based SSDs that are currently entering the market are also rumored to use something along these lines internally. I also suspect that a few of the traditional DBMS might use similar data structures under the hood

Shared characteristics

Although all the applications do very different things and come from completely different backgrounds, they share a common data structure underneath. Unfortunately I have not been able to find a well published moniker for this pattern. If anyone does, please let me know.

Update: As FuncFan pointed out in the comments the moniker I was looking for was “Purely functional trees“. Thanks for the quick reply!

Immutable, recycling trees

All three systems have these immutable trees in some way way or form that allow sharing structure. In git these are tree objects, in Clojure they are called persistent data structures, and in CouchDB they are part of the internal index tree.

Mutable references

To allow for changes in an otherwise immutable world, all systems allow for mutable reference constructs. All change is encapsulated in these references, which are called exactly that in both git and Clojure. In CouchDB the story is a little different. Here the “reference” to the latest tree is simply the end of the database file

More common ground

Apart from these “primary” characteristics there are other shared features among these three systems, that are a direct result of the underlying data structure.

Garbage collection

Because data may be shared between data structures, it is often not safe to delete all children when a root node is no longer needed. Instead a garbage collection mechanism is needed to free unused structures. Git does have a special command that does exactly that, Clojure uses the Garbage Collection in the JVM implicitly, and CouchDB offers a compact operation, which removes old versions and tree indices.

Versioning

Because no structure is changed, it is trivially easy to keep old versions around, and since a lot of data is shared it is usually fairly cheap in terms of memory to do so. Versioning is the primary application of git, so no big surprises on that end. In Clojure it is also fairly easy to add versioning for things like an “undo” buffer: Just keep a list of the old objects around. CouchDB also offers some lightweight versioning out-of-box, but it is mostly used for replication. But it should be fairly simple to add more sophisticated versioning features.

Concurrency Safety

The immutability properties of all three system make reasoning about concurrent changes and processes a lot simpler. Reading is trivial because all that is read is set in stone, so to speak. Writing is made easier by the fact that there is only a single point – the reference – that is modified to effect a specific change. This critical point can then be guarded with traditional synchronization primitives, without having to worry about the rest of the data structure.

Conclusion

I have to admit I found it a bit eerie seeing a single pattern pop up in so many different places. It may be just an idea whose time has come (similar to Garbage Collection a few years back). Or it may be just that now we simply have enough memory and computing resources at our disposal, which allows us to never have to delete something from the record, but instead only add incrementally. And oftentimes the history of data is just as interesting as the data itself. The benefits of immutability may also be a good way to tame the concurrency beast in the years to come.

I hope you enjoyed this comparative trip into the internals of these software systems. Again, check out the two talks and the blog post, they are well worth the time.

My triggering problem was that I was writing some unit-tests which required OSGi to be started up. All my other tests were plain (i.e. non-OSGi) tests. Since I didn’t want to suffer from the startup delay every time I ran the unit tests, I thought about “ignoring” these OSGi dependent tests based on whether I was running plain tests or PDE tests. A bit of googling turned up an interesting feature of JUnit (since 4.4): Assumptions. Assumptions are defined in the Assume class and are basically the exact opposite of JUnit assertions: If an assumption fails, the test automatically passes. This keeps tests from failing when certain preconditions have not been met. Ideally these would be marked as “ignored” or “skipped” but still better than nothing (maybe someone with commit rights to the TestRunner could make this happen).

This neat feature allows us to do the following:

  @Before
  public void setUp() {
    Assume.assumeTrue( Activator.isOsgiRunning() );
  }

Now the tests will only be executed if they are running in an OSGi environment.

If you put this in an abstract base class that all PDE Tests inherit, these will be ignored when running in “plain” mode, for even quicker test runs. In that case be sure to name your @Before method in a unique way so it does not get overridden (and thus nullified) by subclasses.

These assumptions might also be useful for tests known to fail on particular platforms, JREs, runtime environments etc. Another neat addition to my toolbox. The bit of hair in this soup is that JUnit 4 support in Eclipse is a bit lacking, especially when it comes to the PDE build. Some of this is detailed in Bug #153429 which hopefully will be fixed in 3.6. (Hint: Vote!)

What I wanted to with jetty was to create an executable, standalone and self-contained WAR. I first encountered this concept in Hudson. The hudson.war contains an embedded Winstone servlet container, which makes it possible to run the application by executing

java -jar hudson.war

This makes test driving the application really simple. The idea was to do the same with Jetty. Embedding the Jetty runtime in the war proved to be the easy part, as it was just a matter of declaring the jetty dependencies in the maven pom.xml.

The tricky part was telling jetty where to find the war-file to serve. My first try was to hardcode the filename, but that left a foul aftertaste. Finding a solution took quite some time, which is why I am posting this for future reference. This is the Main-Class used to bootstrap Jetty (adapted from the Wicket quickstart archetype):

import org.mortbay.jetty.Connector;
import org.mortbay.jetty.Server;
import org.mortbay.jetty.bio.SocketConnector;
import org.mortbay.jetty.webapp.WebAppContext;
 
public class Start {
 
  public static void main(String[] args) throws Exception {
    Server server = new Server();
    SocketConnector connector = new SocketConnector();
 
    // Set some timeout options to make debugging easier.
    connector.setMaxIdleTime(1000 * 60 * 60);
    connector.setSoLingerTime(-1);
    connector.setPort(8080);
    server.setConnectors(new Connector[] { connector });
 
    WebAppContext context = new WebAppContext();
    context.setServer(server);
    context.setContextPath("/");
 
    ProtectionDomain protectionDomain = Start.class.getProtectionDomain();
    URL location = protectionDomain.getCodeSource().getLocation();
    context.setWar(location.toExternalForm());
 
    server.addHandler(context);
    try {
      server.start();
      System.in.read();
      server.stop();
      server.join();
    } catch (Exception e) {
      e.printStackTrace();
      System.exit(100);
    }
  }
}

The interesting bit is the getProtectionDomain()/getCodeSource() part, which tells us the location of the war-file. That’s all there is to it. Presto, executable web-application powered by Jetty in jar.

Edit: Added the import statements as per Tim’s suggestion.

b3

The b3 project is currently in incubation at eclipse.org. Its goal is basically to be to the PDE build what p2 was to the update manager: A revised, more powerful version of one of the main components of the Eclipse Platform. But while p2 was more or less a complete rewrite, the key with b3 idea is to build upon the exisiting technologies like the PDE build core, Buckminster, EMF and P2. Important key aspects are extensibility and customizability. as well as interoperability with other build tools such as ant and maven.

It is refreshing to see the build process getting more attention as of late, and that the Eclipse community itself is stepping up to the task of making sure that Eclipse based applications are built in a simple, repeatable and reproducible manner.

Tycho

While Eclipse is certainly one of the defacto industry standards, another one is maven. Just as Eclipse is more than an IDE, maven is more than a build tool. The Tycho project is the maven implementation for building Eclipse plugins. While there are currently maven plugins out there capable of building “plain” OSGi bundles, Eclipse plugins are quite a lot more involved. Since both the maven pom.xml and the OSGi manifest include dependency information, there is a bit of redundancy here. The idea is here to have one of these be the “master” version, and then synthesize the other one from that. This means that projects can either be “pom-first” or “manifest-first” depending on the sepcific requirements and project setup.

There a certainly some roadblocks in the way. For example, both OSGi and maven have a concept of versions that is important for resolving dependencies. And while their format is superficially the same, their notion of version ordering is subtly but definitely incompatible. This can lead to all kinds of havoc.

Hopefully issues like these will be resolved sooner rather than later. Eclipse is not an island. Many applications based on Eclipse RCP are actual rich clients, which means they are connected to a server. These server components are usually more traditional “read: non-OSGI” Java applications, and thus tend to be built by a maven setup. In cases like these it would be desirable to have client and server share the same build infrastructure. This would be a huge boon to many real-life projects out there.

Better builds

Both these projects show an encouraging amount of progress, and I am convinced that these efforts will go a long way in making the Eclipse and the Java Platform an even better combination. While I think friendly competition is a good thing, competitive cooperation can be even better. Maybe these two projects have slightly different goals and will fill slightly different niches in the future. But I believe there is a lot of common ground, and I hope there is some healthy cross-pollination between these two projects, if not more. Both teams have a quite a background in their respective areas, and thus could bring a lot to the table.

I for one am looking forward to see these two project bear fruit, making the Eclipse build process a lot less painful and a lot more powerful.

And lo and behold, there’s not one but two eclipse projects for integrating maven. One is the IAM project (formerly known as Q4E) and the second is m2eclipse. In that regard it’s a bit like subversive/subclipse but hopefully without all that licensing nonsense. But it is usually  good to have some choice – and competition of course. To get a clearer picture I decided to give both plugins a try.

IAM

I started with IAM. Setting up my sample project was a snap, and the maven integration immediately started downloading dependencies and adding them to the project classpath. Maven repositories can also hold source jars, making development and debugging much easier. The pom editor seems to cover all options and is looking quite solid. I was more interested in the core feature set, so I didn’t check out any advanced options.

m2eclipse

Next up I tested m2eclipse, which offers basically the same core feature set. Dependencies are automatically downloaded and added to the project classpath. The pom editor covered the same functionality as IAM’s, but I personally liked the the looks and layout better in m2eclipse. One really nice feature is code completion for dependencies.

Code completion - rocks!

I know I am spoiled eclipse developer, and I expect my IDE to finish my thoughts for me. But it gets even better: There is also a quickfix (Ctrl+1) for unresolved imports. Talk about convenience!

Quickfix to the rescue!

It’s good to see that there at least two very viable options when it comes to developing maven  projects in eclipse. Kudos to both the IAM and the m2eclipse team for the fantastic work.

The Price of Modularity

To me, one of the greatest strengths of the Java Platform has been its rich ecosystem. There are so many java libraries and frameworks out there, that when developing for the Java platform you almost never have to start from scratch. Most of the time it’s finding the right libraries, writing a little adapter code and the core business logic. No need for reinventing wheels. This truly is modular and reusable software development, and one of the main reasons why the Java platform is so competitive.

But this modularity does come at a price known as dependency hell. Any non-trivial project has a dozens of dependencies. Even your basic run-of-the mill webapp requires a web framework, logging, OR mapper, JDBC drivers, etc. Add in all the indirect dependencies and you are looking at quite a lot of libraries. This is why strong dependency management is such a compelling argument for build systems like maven.

But there is another issue apart from painfully assembled builds: Jumpstarting new developers. Especially for open source projects it is quite a turnoff for a potential contributor to look at the long list of requirements and dependencies needed to get to the point where the code even compiles cleanly. This is where strong dependency management comes to the rescue. Sure, maven may download two and a half internets on the first compile, but when its done you have everything ready to start working.

“apt-get for Java”

Due to popularity and pervasiveness of maven, there are plugins for integrating almost all imaginable build tools. For most of projects maven provides everything needed right out of the box: unni testing, coverage, javadoc, pmd, you name it. Combined with hudson this makes it ridiculously easy to get a continuous integration server running literally within minutes.

It is good to see that automatic dependency management is making such inroads in different areas of computing. Apt-get and maven, and even p2 have helped a lot to make the dependency hell a little more bearable.

By definition, dynamic languages have no concept of variable types. This means there is no need for a type declaration… so for example, in Python or Ruby you can just write:

randomNumber=4

Some statically typed languages also allow the type declaration of variables to be omitted. But since the language is statically typed, where the does the variable type come from? A reasonably smart compiler can infer the type of a variable by looking at how the variable is used. Haskell and Scala are two examples of such languages that allow implicit typing and then determine the types through compile time type inference. This allows you to write the following scala code:

var randomNumber=new BigDecimal(4)

The type of randomNumber is java.math.BigDecimal. This avoids having to type BigDecimal twice… once for the variable type declaration and once for the constructor. Since Eclipse has a full blown Java compiler under the hood (a fairly smart one), it allows you to do almost the same. You can basically omit the type declaration altogether, and when Eclipse complains just, hit Ctrl+1 and select the appropriate option. The official name for that shortcut is Quickfix, but I like to think of as: “Dear Eclipse, I’m just too lazy, why don’t you fix it for me?”

I don’t know whether this quickfix was created with this usage pattern in mind, but I find myself using this quite often. A close relative is the Extract variable refactoring which also does design time type inference. Another refactoring where type inference comes into play is Generalize declared type, which lets you reduce the specificness of a variable type to its lowest common denominator.

I am sure there are a number of similar refactorings and quick fixes. These powerful features demonstrate how a smart IDE can make working with a statically typed language much easier, while still reaping the benefits of static type checks.

What are your favorite type inference shortcuts?

YAML

YAML Ain’t Markup Language. To quote the spec, it is a “human-friendly, cross language, Unicode based data serialization language”. While mainly designed with so-called agile languages in mind, it can also be used with more traditional languages. The format is more data-centric than document-driven. Even so, one of its primary design goals is good readability. It was heavily influenced by RFC 2822 (Internet Message Format). That means it looks a lot like mail headers. It features built-in support for lists, hashes (i.e. dictionaries or associative array), and common data types. It also allows elements to have multiple parents, which also allows cross-references. There are some more minor features that make working with the format easier. Here’s a small example of a YAML document lifted from the spec:

invoice: 34843
date   : 2001-01-23
bill-to: &id001
    given  : Chris
    family : Dumars
    address:
        lines: |
            458 Walkman Dr.
            Suite #292
        city    : Royal Oak
        state   : MI
        postal  : 48046
ship-to: *id001
product:
    - sku         : BL394D
      quantity    : 4
      description : Basketball
      price       : 450.00
    - sku         : BL4438H
      quantity    : 1
      description : Super Hoop
      price       : 2392.00
tax  : 251.42
total: 4443.52
comments:
    Late afternoon is best.
    Backup contact is Nancy
    Billsmer @ 338-4338.

While YAML seems like a solid solution with implementations written for many languages, there are some issues, like the surprising lack of momentum in the software development community and the controversial use of significant whitespace.

JSON

The JavaScript Object Notation is basically a subset of JavaScript used to statically describe data. With the release of YAML 1.2 it is also a subset of YAML, which means every JSON document is a YAML file. Its focus is primarily simplicity and readability. While it is trivial to parse JSON in a JavaScript environment, its simplicity makes it also quite easy parse in other languages. Parsers for most popular modern development platforms exist. Being so easily accessible in browsers has earned it quite some support and momentum in modern web development, already often completely replacing XML in the AJAX stack. Here’s a small snippet lifted from the JSON Wikipedia page:

{
     "firstName": "John",
     "lastName": "Smith",
     "address": {
         "streetAddress": "21 2nd Street",
         "city": "New York",
         "state": "NY",
         "postalCode": 10021
     },
     "phoneNumbers": {
         "home": "212 555-1234",
         "fax": "646 555-4567"
     }
}

JSON is currently quite popular, though it may in certain cases be hampered by its simpleness. Maybe YAML forward compatibility can provide an convenient upgrade path, should a more sophisiticated format be necessary.

Groovy

Most of you may know Groovy as a JVM scripting language. I have also already blogged about using Groovy to replace especially painful parts of XML.

Groovy features a MarkupBuilder that let’s you create what is basically an XML DOM tree in memory using a slightly more fluent syntax. Have a look:

  car(name:'HSV Maloo', make:'Holden', year:2006) {
    country('Australia')
    record(type:'speed', 'Production Pickup Truck with speed of 271kph')
  }

Some might consider this just syntactic sugar but it really goes a long way.

But I think Groovy can also be used as a first class configuration language. There are already several projects out there that use groovy scripts for tasks that have traditionally been in the firm grip of XML. One such example is gant, which is basically just a thin wrapper to write ant files. Of course nowadays, everyone is using maven instead, but there’s also a neat tool for that: Gradle is build system configured using a Groovy DSL, while employing Apache ivy and maven under the hood. Take a look at this example from the official gradle documentation:

usePlugin 'java'
 
sourceCompatibility = 1.5
version = '1.0'
manifest.mainAttributes(
    'Implementation-Title': 'Gradle Quickstart',
    'Implementation-Version': version
)
 
repositories {
    mavenCentral()
}
 
dependencies {
    compile group: 'commons-collections', name: 'commons-collections', version: '3.2'
    testCompile group: 'junit', name: 'junit', version: '4.+'
}
 
test {
    options.systemProperties['property'] = 'value'
}
 
uploadArchives {
    repositories {
       flatDir(dirs: file('repos'))
    }
}

A lot easier on the eyes, while still providing interoperability with “legacy” systems like maven.

The power and expressiveness of groovy make it really easy to create domain-specific languages like these. Such a special purpose language might not always be desirable, especially when interoperability is a key concern. But for certain applications these DSLs might be a better solution than any general purpose format.

On the whole, these are interesting times we live in and I’m curious to see what the future holds in store. As I said before, XML is probably gonna be here for quite a while, but there are some compelling alternatives out there. What are your thoughts on the legacy of XML?

XML is a markup language

As the name Extensible Markup Language implies, the language is first and foremost a markup language. That means that the language annotates a body of text. So if you were to strip out all the markup from an XML document, it should still end up with legible and comprehensible. This is certainly true for (X)HTML and DocBook. It might be slightly harder to read and understand, but all the crucial information is right there in plain text. The markup just adds semantic or presentational meta-information, e.g. which bits of text are headers or quotes.

In many cases XML is misused as a general data format. This often means that there is no actual text (character data) whatsoever in the resulting files. Then why use a markup-language in the first place? The Eclipse plugin.xml format for example is a markup only format (save some extension points schemas).

Attribute and Element Dichotomy

Another gripe with XML is when to use attributes vs. elements. There are at least three options that at first glance seem equally plausible. For example:

Name as PCDATA

<customer>
ACME Inc.
</customer>

Name as extra element, PCDATA

<customer>
<name>ACME Inc.</name>
</customer>

Name as attribute

<customer name="ACME Inc." />

I have seen these three styles mixed within the same document for almost exactly the same fields. Should a general purpose data interchange format really be that hard to get right, or at least consistent?

Awkward mapping of common constructs

Because xml implicitly is a tree structure, all structures involving cyclic references or multiple cross-references in general are not easily mapped to an XML compatible form. Object-oriented models can often contain back-references, or reference a single object from different places in the object graph. Although there is some support for such constructs in the form of ID/IDREF attributes, this is neither supported by all parsers nor even publicly widespread information.

Another common data type that is painful to describe in XML is the associative map. An example from the Eclipse plugin.xml:

<property name="aboutImage" value="eclipse_lg.gif" />

Compare that to a simple properties file format:

aboutImage=eclipse_lg.gif

The third type of commonly found data that is hard to put into XML is relational data. That kind of data is traditionally found in RDBMS, CSV files and unfortunately spreadsheets. While it is fairly trivial to create a corresponding xml format, the result is often needlessly verbose and repetitive, since all elements usually have the same attributes.

Readability

By far the biggest gripe I have with xml on a day to day basis is that it is really hard to parse for me as a human. Between the angle brackets bunched up against the element names, the endless repetition and escaped entities it seems like this format was not really designed with a legibility in mind. Another reason could be related to my first point: In regular markup languages the tags only contribute a relatively small percentage of the overall content. The majority is plain text, so the tags are few and far between. In markup-only languages however, there’s a much higher density of markup elements. In my opinion this redundant repetition lowers the signal to noise ratio significantly, making these documents much harder to read.

These are the main problems that I currently see with XML. I concede that the common, extensible meta format was a huge step forward, and that for some problem domains it is a pretty good fit. I also realize that XML is gonna be here for quite a while, but I think it’s time to stop resting on these laurels and see how we can address these problems in the future.

Tomorrow I’ll be looking at the some of the alternatives that might be potential successors to XML.

Why is that action not available?

Often the answer might be obvious to the experienced user but this isn’t good for novices.

What about new or inexperienced users? Can’t we ease the learning curve a little and help users get to grips with the desired functionality? This might help avoid statements like…

“Oh wow, so that’s what that button does. I never used it because it was always disabled.”

In many cases there might only be a single reason why the action is unavailable. That means if the button is gray it is fairly simple to figure out what is amiss. But in non-trivial applications there may be a whole host of reasons why an action can not be performed. Typical reasons include:

• technical reasons: e.g. a “delete” action can not be performed if nothing is selected
• infrastructure reasons: e.g. a required service is unavailable
• operational reasons: e.g. user has insufficient privileges, or  the action can only be performed during business hours
• business domain reasons: e.g. certain preconditions have not been met, or the action is incompatible with another action that has already been performed

That last reason can really be a big can of worms… because at least in my experience “businessy” applications have a propensity for growing complex rule sets that have to be modeled in software. That may even leave domain experts guessing why the button “re-evaluate taco consumption rates” is grayed out.

So is there way to deal with that problem in a more sophisticated and helpful manner?

Here is an idea how the information could be presented in a natural and obvious way.

Tooltip for disabled elements - Answering the question

We’re doing nothing mind-boggling here but it gets the message across. On the implementation side of things, I could imagine something analogous to JFace style validators to attach to the button so that the appropriate message can be created on the fly. This might make for a cleaner technical design because different concerns of when to disable the button can be decoupled. One big drawback of this approach is that disabled (Windows) buttons don’t seem to show their tool tips. It may be possible to work around this problem with some SWT-fu though.

A more radical approach to the whole dilemma is the standpoint that disabled user interface elements are design smells that should be avoided completely. Applications should be designed in a way to support the necessary work flows and guide the user along logical steps… instead of doing trial and error and guesswork.

After all, trial and error and guesswork should be a domain reserved for point-and-click adventures.

So have you run into this issue as well? What were your experiences or solutions?

I’m not a big fan of XML in general and the idea of writing code in XML gives me the shivers. I have written enough XSLT to know pain… in copious amounts. We already have a long-established notation for formal expressions: programming languages. There is a very good reason Java’s syntax (or that of any other popular programming language) looks nothing like XML.

One argument that proponents of XML often make is that you usually shouldn’t have to look at XML at all. Well, I think they are wrong. The XML leaks through the cracks way too often. As a developer I am probably more exposed to that than the average user but nevertheless I still have to put up with it. Sometimes there may be a layer of GUI eye candy in between, but for something as specific as an expression editor… the standard extension editor just doesn’t cut it for me.

With a quick look at the following image: Tell me when the expression is true. Oh right, you can’t – the expression is distributed over several tree nodes.

Classic expressions

Now let’s look at the XML. Only slightly better, but at least you can see everything at once. Still clocking in at 6 elements, the signal to noise ratio is abysmally low.

<handler
	class="groovy.expressions.handlers.SampleClassicHandler"
	commandId="groovy.expressions.commands.sampleCommand">
 <enabledWhen>
	<or>
	   <with variable="activePartId">
		  <not>
			  <equals value="groovy.expressions.navigationView" />
		  </not>
	   </with>
	   <with variable="selection">
		  <count value="2" />
	   </with>
	</or>
 </enabledWhen>
</handler>

To put my money code where my mouth is… so I tried to implement a more elegant solution to the problem. Note though that this is just a proof of concept hacked together in my spare time. I wanted to integrate a scripting language with the command framework so that expressions could be written as small snippets of code. The ecosystem of scripting languages is quite prolific at the moment, so there is a lot of choice. But choice is good right? I settled for Groovy since I had done some embedding experiments with it before. But I am convinced that other scripting languages would work just as well.

Hooking up the groovy runtime was a snap, adding another expression type (creatively named “groovy”) was easy as well. I found out before long that I had to specify which command variables to use. The first reason for that was that I had to bind these command variables to groovy variables – although this could also have been achieved with some propertyMissing trickery. The more compelling reason was that the command framework had to known when to re-evaluate the expression, i.e. when relevant variables had changed. The resulting expression now looks like this:

<handler
	class="groovy.expressions.handlers.SampleGroovyHandler"
	commandId="groovy.expressions.commands.sampleCommandGroovy">
 <enabledWhen>
	<groovy using="selection,activePartId">
	activePartId != "groovy.expressions.navigationView" || selection.size() == 2
	</groovy>
 </enabledWhen>
</handler>

Quite a bit more readable, eh? Notice the “using” attribute specifying the used variables mentioned above. It might even be possible to compute these variables by traversing the syntax tree of the expression. This feature is offered by some scripting engines and would mean we could completely drop the “using” attribute. The rest is just plain groovy code, and I’m sure most of you will have no problem understanding an expression like that.

Next I want to discuss some of the caveats and advantages of this approach.

Caveats:

Needs a scripting engine
As far as I know, Java 1.6 ships with a Rhino interpreter so that could be an option for the future. Other small scripting engines (e.g. BeanShell) could be integrated as well. That is unless the big bad licensing monster rears its ugly head.

Might need standardization
As mentioned above there is a whole plethora of scripting languages available right now on the JVM, so choosing “the one” might be a problem. Another option of course would be to delegate to JSR 223 and let the user decide.

Yet another language?
There might some developer resistance concerning yet another language to learn. While this is certainly a concern, some scripting options are very close Java (Groovy and particularly BeanShell come to mind). I would argue that the current XML expression language is “yet another language” to learn as well – and a cumbersome and ill-specified one at that.

Advantages:

Legible expressions
As mentioned above, the expression is much easier to parse for developers. The machine doesn’t care either way, but software should not only be list of machine-operable instructions but also as a means of documentation and communication. The current implementation falls way short in that regard.

Debuggable
Ever tried to debug a highly nested XML expression? Not fun, let me tell you. With “scripted” expressions in the worst case you can pepper your code with println statements. In the best case you can set breakpoints on your expression code editor. The latter would probably be quite a bit of work but I don’t think it is totally infeasible.

More powerful
I haven’t looked into it too deeply but the current expression engine is probably a turing tarpit eager for prey. A “proper” scripting language would gives us a well understood, turing-complete way to describe complex expressions. It would also make it easier to factor common functionality into shared functions in order to reduce coupling and repetition.

Improved Testability
A scripting environment might also make it easier to write and execute unit tests on the expression to verify its correctness. While this also possible with the current expression engine, it is quite a bit of work to get up and running. With scripted expressions one could fairly easily set up the variables and run a minimal execution wrapper around the expression code to test its behaviour in a classical JUnit test.

I put up the code on the github here. Basically the application is your vanilla RCP Mail example. The meat of the code is in org.eclipse.core.internal.expressions.GroovyExpression. Apart from the mucking in the expression engine, the only thing I did was add two commands and handlers with the same activeWhen semantics, but once described as classic XML and once expressed as Groovy code. Have a look, I’m looking forward to your comments and critique.