Accessing a huge data set with the web browser

The Enron Corporation was the American energy company that was involved in accounting fraud which led to the Enron scandal in 2001. During the investigation, large parts of the company’s email conversations were published. The result is that a huge, real-life data set including more than half a million emails from 150 Enron executives came into the public domain.

I thought that this data would be a good example to show the ability of the new Tree widget in RAP to display huge datasets.

It’s clear that you cannot create half a million UI elements in a browser without running out of memory. Neither can you load the entire dataset (> 2GB on disk) into memory on the server. You need a mechanism to load and display data on demand.

As of this week, the RWT Tree widget has full SWT.VIRTUAL support, i.e. it is capable of creating Tree items only at the moment they become visible. This feature is used by the JFace TreeViewer to request data from a lazy content provider on demand.

So, in order to make a complex data set accessible with a web UI, you just need to write an ILazyTreeContentProvider implementation. That’s about one screen of Java code, mostly copied from a JFace snippet. That’s all.

Check out the example on our online demo (navigate to the tab called Complex Data). The source code of this demo page is available on github.

Full SWT.VIRTUAL support for the Tree widget is available in CVS and will be part of RAP 1.5M1.