There are many better JSON libraries out there, but most do much more than we need. We really only need a bare-bones parser that can read JSON into a simple Java representation and generate JSON from Java. As we like to keep the core library self-contained, we don’t want a dependency to an external JSON library.

One winter Sunday, I started to write a JSON parser just for the fun of it, and was quickly surprised how simple it is to parse JSON. That’s because the first character of every token uniquely defines its type ('[' for an array, '"' for a string, 't' or 'f' for a boolean, and so forth). There’s no backtracking involved. It went so well that I decided to continue and create a JSON parser tailored to our needs. Which are:

• Fast – we read and create so much JSON that the parser directly affects the server performance
• Lightweight – it should deal with memory sparingly as we deal with lots of messages
• Minimal – the less code the better, as we have to maintain it
• Simple to use – we’ll expose the API for custom component developers, so it should be simple and clear
• No dependencies – only Java 5

The result is called minimal-json and it’s already included in RAP. It is fast, lightweight, consists only of 10 classes and I hope it’s simple to use:

Usage

You can read a JSON object or array from a Reader or from a String:

JsonObject jsonObject = JsonObject.readFrom( reader );
JsonArray jsonArray = JsonArray.readFrom( string );

Once you have a JsonObject, you can access its contents using the get() method:

String name = jsonObject.get( "name" ).asString();
int age = jsonObject.get( "age" ).asInt(); // asLong(), asDouble(), ...

The elements of a JSON array can be accessed in a similar way:

String name = jsonArray.get( 0 ).asString();
int age = jsonArray.get( 1 ).asInt(); // asLong(), asDouble(), ...

As you can see, the get() method always returns an instance of JsonValue, which can then be transformed to the target type using asString(), asInt(), asDouble(), etc. There’s no automatic conversion to Java types, no instanceof needed. If you’re not sure about the type of a value you can check it using isString(), isNumber(), etc.

Nested arrays and objects can be accessed using asArray() and asObject():

JsonArray nestedArray = jsonObject.get( "items" ).asArray();

You can also iterate over the elements of an JsonArray and the names of a JsonObject, e.g.:

for( String name : jsonObject.names() ) {
  JsonValue value = jsonObject.get( name );
  ...
}

Writing JSON

A JsonObject or JsonArray can output JSON to a Writer or as a string using the toString() method. The JSON is currently not pretty-printed, formatting support might be added later.

jsonObject.writeTo( writer );
String json = jsonArray.toString();

To create a JsonObject or a JsonArray, use the add() methods that exist for the relevant types. These methods return the object instance to allow method chaining:

jsonObject = new JsonObject().add( "name", "John" ).add( "age", 23 );
jsonArray = new JsonArray().add( "John" ).add( 23 );

You may have noticed that also the object has an add() method instead of a put() or set(). That’s because the JsonObject stores and writes its members in the order they are added. It allows you to define the output order, it even allows you to add the same key twice, which is discouraged but not forbidden by the JSON RFC.

To replace an element in an array of object, you first have to remove() the old value and then add() the new one. A replace() method may be added later. However, JsonArray and JsonObject are designed to be containers for reading and writing JSON, not general purpose data structures.

Performance

I’ve compared the time required to read and write a typical RAP message with other popular parser implementations, namely org.json (20091211), Gson (2.2.2), Jackson (1.9.12), and JSON.simple (1.1). Disclaimer: This benchmark is restricted to our use case and my limited knowledge on the other libraries. It may be unfair as it ignores other use cases and perhaps better ways to use these libraries. However, I think these results show that minimal-json can take comparison with state-of-the-art parsers.

I also ran a number of micro-benchmarks using Google caliper while optimizing the implementation. One interesting detail was the choice of the data structure for the JsonObject. Since JSON objects are key-value maps, a HashMap seems like an obvious choice. However, after some experiments, I ended up with two separate ArrayLists for names and values.

Of course, looking up a key in a list requires a linear search while a hash lookup is much quicker. However, creating a HashMap and adding elements has a considerable overhead. It turns out that this overhead squashes the benefits of a HashMap for very small numbers of items (< 10). Since JSON messages in RAP typically contain many objects with only a few items, a HashMap would even impair the overall performance. Moreover, the two ArrayLists require less than a third of the footprint of a HashMap.

I was able to improve the lookup performance for small item counts by adding a very small hash structure to the names list. This structure consists of a 32-elements byte array. It does not handle collisions but resorts to indexOf() in case of a miss. This version (shown in the middle of the chart above) seems to be a good compromise between HashMap and plain ArrayList. Optimizations for bigger JSON objects are possible, but not needed at the moment.

How to Use it

If you’re looking for a bare-bones JSON parser with zero dependencies, you are welcome to use minimal-json. It goes without saying that it’s developed test-driven and complies with the RFC. The code lives at github and is EPL-licensed. It is also included in RAP.

I didn’t setup a build. If you would like to use the code, I suggest that you simply copy these 10 Java files to your project.

Let me know if you find this useful, fork it on github, and feel free to open an issue if you find a problem.

The Eclipse RAP Project has just launched its long-awaited 2.0 version.

With this release, the project has changed its name to “Remote Application Platform” to indicate the transition from a browser platform to a universal platform for remote applications. RAP 2.0 supports alternative clients and has new APIs to distinguish between clients, access client-specific features, and write custom components that use the new protocol. During the past few days, I’ve written a series of blog posts to explain the most important changes and features in RAP 2.0:

1. Remote – The New “R” in RAP
2. The Pillars of the RAP 2.0 API
3. Know Your Client
4. What is Server Push and Why Should I Care?
5. Custom Widgets with RAP 2.0

A detailed list of changes in this release can be found on the project’s New & Noteworthy page. We are excited about the possibilities opened up by the new RAP and hope that you will enjoy working with this release.

The release is available as p2 update sites and as .zip files on the RAP download page.

Custom Widgets with RAP 2.0

There are several alternatives to writing real custom widgets in RAP: you can compose widgets out of existing ones, draw on a Canvas, or embed mashups in a Browser widget. However, for advanced widgets you don’t get around writing a “real” custom widget with a server part and a client part that are connected by the protocol.

This has become a lot easier with RAP 2.0, since you don’t have to understand the internals of RWT anymore. The new Remote API allows both parts of an object to communicate with each other over the JSON protocol. In fact, its methods are direct representations of the protocol operations such as set, call, etc. Just like the protocol itself, this API is not restricted to widgets, it can be used to synchronize any kind of objects.

The Remote API at a Glance

Let’s have a quick look at this API. The central interface is called RemoteObject. As the name says, it represents the corresponding object on the client. When an instance of the server-side object is created, we first need to create a client-side object as well and synchronize with this object. We do so by creating a RemoteObject. The constructor of our custom widget would be a suitable place to do so:

Connection connection = RWT.getUISession().getConnection();
remoteObject = connection.createRemoteObject( "charts.PieChart" );

This will render a create message that instructs the client to create an object of the type charts.PieChart. When a property is changed on the custom widget, we need to pass the new value to the remote object:

remoteObject.set( "radius", 200 );

When the custom widget is being disposed of, the remote object has to be destroyed as well:

remoteObject.destroy();

There are also methods for the other protocol operations and a handler to receive updates from the client. I don’t want to go through all details here, just give you a rough idea how the Remote API looks like.

JavaScript API

We also need a client implementation to display your custom widget and allow users to interact. For the web client, this has become a lot easier. For the first time ever, RAP 2.0 provides a minimal JavaScript API. It’s minimal because it just contains the methods needed to hook your JavaScript code into the RAP client. You don’t have to care about RAP’s client library anymore. Use whatever JavaScript library you like.

To look at an example, let’s see how the client notifies the server of an event:

var remoteObject = rap.getRemoteObject( this );
remoteObject.notify( "Selection", { "index": index } );

Of course, it uses a remote object to do so, and its API is quite similar to the Java RemoteObject.

Let’s Create Widgets!

With this new Remote API, custom widgets are no longer reserved for experts. Writing them is easier, requires less code, and it’s definitely more fun. That’s what we found when we employed the Remote API for existing components. As an example, switching the clientscripting component to the Remote API, allowed us to get rid of more than 30% of the Java code in this component!

Tim just updated his CKEditor widget for RAP to use the new API. You can try it out in the online demo. If you’re looking for a rich text editor for RAP, here’s a 2.0-compatible version.

During the last days, I’ve created a prototype for a charting widget (the one seen above) based on the fascinating d3 toolkit. You can play with it already in the online demo. It’s just a spike, but I’m going to publish a stable version soon and announce it in this blog.

The new Remote API is still provisional and might change slightly over time. We’ve already published it anyway to allow others to start experimenting. I’m looking forward to more custom widgets becoming available for RAP. Let us know about your custom widgets! Contributions are welcome in the RAP Incubator project.

Thank you for following me on this tour through the new RAP. RAP 2.0 is now available for download. I hope you enjoy working with it.

What is Server Push and Why Should I Care?

An HTTP server cannot simply “push” information onto the client. It has to wait for the client to ask for changes. Server push techniques allow clients to be notified immediately in the event of a server-side change. In RAP, server push comes into play whenever you access the UI from a thread that is not the UI thread, that is, outside of the normal event handler code.

Normal Operation (The Blue Part)

Let’s start with the simple case. I’d like to ask you to focus only on the blue part of the diagram below and try to cut out everything else for now.

Whenever an event occurs (e.g. a button press), the client sends a POST request to the server with a JSON message that contains the event. All event handler code is executed in the UIThread on the server. Any changes to the UI made in this thread are immediately sent to the client.

Background Processing (The Red Part)

Now let’s add the dark red line to the picture (green is still invisible). Another thread is forked from the UI thread to execute some background task. We use to call those non-UI threads background threads. In SWT, the UI may only be accessed from the UI thread. When you want to update the UI from a background thread, you have to wrap the respective code in a runnable and hand it over to Display.asyncExec() or syncExec(). The runnable will be put into a queue to be executed in the UI thread.

The runnables in this queue can be processed in the UI thread only when the client sends another POST request. The updates will not appear on the user’s screen until there is some user interaction.

Server Push (Green)

Now for the green part. When server push is enabled, the client will send a GET request everytime a POST returns. This GET request is blocked on the server, so the client keeps waiting for a response. When Display.asyncExec() is called and a runnable put into the queue, the server releases the blocked request and sends an empty response. This notifies the client to send another POST request to fetch the updates from the server.

As you can see, this process results in a continuous serpentine line that keeps the client up to date.

The New Server Push API

In RAP 1.x, this feature was called UICallBack. Since “server push” has become an established term, we decided to use this for the new API. However, there’s no ServerPush class, but only a ServerPushSession. Instead of activating the server push directly, you create and start a push session. There can be multiple overlapping push sessions throughout your code, and as long as at least one of them is active, the framework will keep the green part enabled.

To request server push, create a new ServerPushSession and call start on it. It’s crucial to do that in the UIThread before you fork a background thread. When your code doesn’t require the server push anymore, call stop() on the push session. That’s all. In contrast to the old API, you can stop the session from any thread you like.

final ServerPushSession pushSession = new ServerPushSession();
Runnable runnable = new Runnable() {
  public void run() {
 
    // do some background work ...
 
    // update the UI
    display.asyncExec( new Runnable() {
      public void run() {
        if( !label.isDisposed() ) {
          label.setText( "updated" );
        }
      }
    } );
 
    // close push session when no longer needed
    pushSession.stop();
  };
};
 
// start push session before starting the thread
pushSession.start();
Thread thread = new Thread( runnable );
thread.setDaemon( true );
thread.start();

The server push in RAP has gone through many iterations. It has been completely reworked in RAP 1.5 and again revised in 2.0. For example, one improvement in 2.0 is that the client is now immediately notified in case of a session timeout when server push is enabled. So I’m pleased to say that the server push in RAP 2.0 is now very robust and mature.

RAP 2.0 will be published on Monday around noon CET.

Know Your Client

When developing a RAP application, you normally don’t have to know which client is connected. SWT screens written for a native client can be accessed with the default web client just as well. You can check the Display for the client’s display resolution and adapt your SWT layout to the available screen estate (the client area).

However,  in some cases you may want to make certain adjustments to your application, depending on the client. In this case, you can find out what client is connected to the current UI session by calling UISession.getClient() (you can also use the shortcut RWT.getClient()). The result is an instance of the interface Client. When accessed with the web client, it will be an instance of WebClient, for Tabris, it will be a TabrisClient.

Accessing Client Features

Most clients can provide information about their timezone and locale, some can also provide their geo position. Accessing this information can help to supply the user with more appropriate information. To make this information available, we could have added methods like getTimeZone() or getGeoPosition() to the Client interface. However, mobile clients can offer a lot more services, such as scanning bar codes, taking photos, or making phone calls. Moreover, some services are not specific to one client but can be provided by different clients. Adding methods for every possible feature to the Client interface was clearly not an option but we also wanted to avoid code that looks like this:

if( client instance of ThisClient ) {
  ((ThisClient) client).doThis();
} else if( client instance of ThatClient ) {
  ((ThatClient) client).doThat();
} else if( client instance of SomeOtherClient ) {
  ((SomeOtherClient) client).doSomethingElse();
  …

The Service Locator Pattern

Using the Service Locator pattern allowed us to come up with an extensible API. Every client implements the method getService( ClientService ):

GeoLocation location = client.getService( GeoLocation.class );
if( location != null )  {
  location.getLatitude();
}

Using this method is like going to a hardware store to buy a tool that you need. If you’re lucky, the item is in stock, otherwise you walk out with empty hands. If the client cannot provide the requested service, there’s nothing we can do. Whether a service is available or not depends on the client implementation, and it may even depend on the client device (does it have a camera?). The RAP API contains only the hardware store (the Client) and those tools that the default web client can provide. Alternative clients may implement these services, but they can also provide their own specific service interfaces.

Employing a service locator started as an experiment. There is some controversy about this pattern and I’m sure there are plenty of ways to misuse it. However, since we introduced the client services, we found much more use cases than we initially expected. The WebClient already supports six different services (ClientInfo, JavaScriptExecutor, JavaScriptLoader, just to name a few), and more will follow. In Tabris, there are plenty of client services, such as AppLauncher, GeoLocation, Camera, etc. (some of them are not implemented as client services yet as they predate this API).

So it feels like a good choice. I can’t wait to see more stores opening along that road!

The Pillars of the RAP 2.0 API

In a multi-user and multi-application environment, you have to distinguish between different scopes to avoid mixing up your user’s data. There has been some confusion around the session scope in RAP 1.x. In the new API, the scopes are more explicit as they’re represented by interfaces. The two most important interfaces that you should be familiar with are UISession and ApplicationContext. However, let’s start with the basics, the servlet container.

ServletContext and ApplicationContext

RAP runs on a standard servlet container. A servlet container hosts a number of webapps, each containing servlets and static resources. All servlets of a webapp share the same ServletContext. That is, the ServletContext represents a web application.

A RAP application is represented by an ApplicationContext. Also because it’s a web application in itself, this ApplicationContext is bound to a ServletContext. You can run multiple RAP applications in parallel, e.g. in different web applications, or even different servlet containers in the same VM (e.g. different Jetty instances on different ports). ServletContext and the RAP ApplicationContext have a 1 to 1 relationship. The actual instance of ApplicationContext can be obtained from RWT.getApplicationContext().

The ApplicationContext provides access to application-scoped framework services such as the ResourceManager (actually, RWT.getResourceManager() is now just a shortcut for RWT.getApplicationContext().getResourceManager()). It can also be used to store arbitrary objects as attributes. Whenever you need to share information application-wide, you should store them in the ApplicationContext, rather than keeping them statically.

HttpSession and UISession

When a user connects to a web application, the servlet container creates an HttpSession. To recognize the user in subsequent requests, the session ID is usually stored in a session cookie. The HttpSession is valid until either the browser is closed or it is left unattended for a configurable time (the session timeout).

For every instance of a RAP UI, a separate UISession is created. That means that when a user accesses an entry point, that’s a new UISession. When the user reloads the page in the browser, that’s also a new UISession – but the HTTPSession remains the same. There is a 1 to many relationship between HTTPSesssion and the RAP UISession.

The current instance of a UISession is available from RWT.getUISession(). Just like the ApplicationContext, you can use the UISession to store attributes (actually, UISession replaces the old SessionStore). If you need to share information that is related to the EntryPoint instance (e.g. a selection that the user has made), keep it in the UISession. User-related information should rather be stored in the HttpSession, because it survives a single UISession.

Better Support for Testing

One advantage of the new API structure is that it facilitates test-driven development. In contrast to the RWT class, the new interfaces UISession and ApplicationContext provide non-static methods to access framework services. Hence, these instances can be replaced with mocks in a unit test. We also plan to add methods to the test fixture that allow framework instances to be replaced with mocks.

Better Multi-Tab Support

I particularly like the new structure because it will simplify adding better Multi-Tab support in RAP. When a user opens a RAP application in multiple browser tabs, there are of course different UI sessions but they share the same HttpSession because cookies are shared across all browser tabs. At the moment, RAP does not handle this particular case and you need a workaround. RAP 2.1 (June 2013) will include multi-tab support out-of-the-box. With this change, a single HttpSession can have multiple UISessions at the same time.

You can find more details about the new API in the RAP developer’s guide. If you update an existing application to RAP 2.0, also check out the migration guide.

Remote – The New “R” in RAP

At the very core of RAP, there is an implementation of the “Half Object plus Protocol” (HOPP) pattern. The widgets in RAP are cut in half, one half is located on the server, the other on the client. Both halves are connected by a protocol. One big advantage of this design is that the application doesn’t have to take care of the client/server communication at all, the platform covers this.

RAP always had half objects but the protocol part was missing. The communication was based on undocumented JavaScript commands and HTTP parameters. Starting with RAP 1.5, we migrated the entire platform to a new, open JSON protocol. This protocol opens the platform for alternative clients, and those clients already exist. With Tabris we provide native clients for mobile devices with iOS and Android. For me the best proof that the protocol works is that although the Tabris team works next door, there’s very little need for technical discussions. In fact, the protocol is the interface between us.

Writing a RAP client isn’t rocket science, all you need is an HTTP client, a JSON parser, and part of a graphics library for displaying widgets. You don’t have to implement the full widget set, only those widgets that your applications require. We’ve already seen some PoC clients and we can’t wait to see RAP on more devices.

The RAP protocol has been under development for quite a while and has been simplified a couple of times. Let’s have a quick look at the structure of client and server exchange JSON messages. A message consists of a header part and a list of operations:

{
 head: { … },
 operations: [ … ]
}

The operations are executed by the receiver one by one. There are six types of operations: create – to create a remote object, set – to set one or more properties on the remote object, and destroy – to well, you guessed it. Besides set there is also call to call a method on a remote object. In addition, finally listen and notify to exchange events. Every remote object is identified by an id and every operation addresses exactly one target object. That is basically it. To give an example, here’s a message that creates a Button with a Selection listener:

{
  "head": {
    "requestCounter": 1
  },
  "operations": [
    [ "create", "w4", "rwt.widgets.Button", {
        "parent": "w2",
        "style": [ "PUSH" ],
        "bounds": [ 5, 68, 131, 32 ],
        "text": "I am a button" } ],
    [ "listen", "w4", { "Selection": true } ]
  ]
}

You can see that the create operation already contains the initial properties for the Button. With the listen operation, the client is informed that there are Selection listeners attached. When the button is pressed, the client will send this message to the server:

{
  "head": {
    "requestCounter": 1
  },
  "operations": [
    [ "notify", "w4", "Selection", {
        "shiftKey": false,
        "ctrlKey": false,
        "altKey": false
      } ],
    [ "set", "w1", {
        "cursorLocation": [ 121, 104 ],
        "focusControl": "w4"
      } ]
  ]
}

A more detailed description of the protocol syntax can be found in our wiki.

It’s important to point out that although most half objects in RAP are widgets, there are no UI specifics in the protocol. It is also used to synchronize service objects like the script executor or the geo location service in Tabris. The protocol no longer depends on HTTP. Even though RAP is still bound to HTTP, a future version could additionally work with other protocols.

With the completion of the protocol and the new RAP clients we decided to rename the project to “Remote Application Platform”. So the “R” in RAP now stands for “remote” and that’s how we see RAP: as a versatile platform for modular remote applications of all scales. These range from full-blown business applications with forms and tables in a web browser to minimal apps on a handheld device. All with the same programming model.

In my next post, I will introduce you to the interfaces that constitute the pillars of the RAP 2.0 API.

Grails is a ready-to-use web framework with some great featues such as a persistence layer with ORM, dependency injection, unit testing, and advanced web APIs. It comes with a command line that helps you create and run web applications and can also export them as .war files. Grails is based on Groovy, and since Groovy runs on the JVM, it allows to leverage existing Java libraries–such as the RAP Widget Toolkit.

Benjamin says that he was “always missing the nice API and widgets that RAP provides” when working with Grails. So he tried to integrate RWT into a Grails application. With the new decoupled design introduced in RAP 1.5, this was an easy task for someone who knows both RAP and Grails.

After this worked out well, he decided to make his integration available for others as a Grails plug-in. Now you can start writing RWT applications in Grails right away by simply including a plug-in. An RWT application is set up the Grails way, using a configuration:

rwt {
  entrypoints {
 
    // Configuration of an entry point named 'hello', the relative URL path will be '/hello'
    hello {
      bean = 'helloEntryPointService' // The name of the entry point bean (required)
      pageTitle = 'Hello RWT!' // The browser title that will be displayed (optional)
    }
 
  }
}

You can find a simple example RWT application using some Grails features on the GitHub page of the RWT plug-in. As you see, the plug-in not only provides plain RWT, but also JFace, making it easy to bind complex data to your UI. And Grails’ O/R mapping and dependency injection make it even simpler.

This is just the first release of the plug-in. Benjamin is planning some improvements and is interested in any feedback. At the moment, the plug-in incorporates RWT in the latest version 2.0M3. But when RAP 2.0 is released in February 2013, there will be a update of this plug-in available that includes the RAP artifacts directly from Maven.

We are always happy to see RAP being used in new contexts, and in particular it’s great to see that our decoupling in RAP 1.5 proves helpful for others.

RAP 2.0 M3 contains a brand new event system implementation that allowed us to get rid of some old bugs and to attach listeners more precisely in the protocol. Many RAP developers will like the new Client API that provides access to client features through a service locator interface. With this new interface, RAP finally offers a public API to execute JavaScript on the client, using the new JavaScriptExecutor service (check Ivan’s post for details):

JavaScriptExecutor executor = RWT.getClient().getService( JavaScriptExecutor.class );
if( executor != null ) {
  executor.execute( "alert( \"Hello World!\" );" );
}

Some public classes and interfaces, most notably the resource manager, have been cleaned up in M3 and made ready for RAP 2.0. After 6+ years of development, RAP’s API simply needs some cleanup. But don’t worry, we will keep the number of breaking changes low and let deprecated classes in place where possible to ease the migration. And we’re maintaining a RAP 2.0 migration guide to assist you.

For a more detailed list of changes and bugfixes in this milestone, have a look at our New and Noteworthy page. The build is available for download on the project’s download page and is also included in the Eclipse Kepler repository: http://download.eclipse.org/releases/kepler/.

RAP 2.0 is going to be released in February 8, 2013. For next year’s simultaneous release, Eclipse Kepler, we’ll plan a follow-up release. We hope you are as excited about RAP 2.0 as we are and the migration goes smooth for you. Let us know what you think!

In RAP 1.5, we replaced the communication between server and client with a new, documented JSON-based format. Now, the client uses the same format to report changes to the server.

See the new and noteworthy for a complete list of changes and bug fixes in 2.0 M2. The milestone build is available on our download site and in the Kepler repository. If you’re migrating from RAP 1.x, please also check out our RAP 2.0 migration guide.

For the next milestone, we plan some new features like public API to access client information such as the timezone and to execute Javascript on the client. We’re also started to rework the event system in RWT to get rid of some nasty bugs related to events and listeners. M3 will be available on November 16.