Accessing a huge data set with the web browser

August 15, 2011 | 2 min Read

The Enron Corporation was the American energy company that was involved in accounting fraud which led to the Enron scandal in 2001. During the investigation, large parts of the company’s email conversations were published. The result is that a huge, real-life data set including more than half a million emails from 150 Enron executives came into the public domain.

I thought that this data would be a good example to show the ability of the new Tree widget in RAP to display huge datasets.

It’s clear that you cannot create half a million UI elements in a browser without running out of memory. Neither can you load the entire dataset (> 2GB on disk) into memory on the server. You need a mechanism to load and display data on demand.

As of this week, the RWT Tree widget has full SWT.VIRTUAL support, i.e. it is capable of creating Tree items only at the moment they become visible. This feature is used by the JFace TreeViewer to request data from a lazy content provider on demand.

So, in order to make a complex data set accessible with a web UI, you just need to write an ILazyTreeContentProvider implementation. That’s about one screen of Java code, mostly copied from a JFace snippet. That’s all.

Check out the example on our online demo (navigate to the tab called Complex Data). The source code of this demo page is available on github.

Full SWT.VIRTUAL support for the Tree widget is available in CVS and will be part of RAP 1.5M1.

Ralf Sternberg

Ralf Sternberg

Ralf is a software engineer with a history as Eclipse committer and project lead.

In recent years, he devoted himself to JavaScript technologies and helped pulling off …