How to write UI tests for RAP with Selenium 2.0

Tim Buschtoens
April 29, 2014 | 10 min Read

We’re occasionally asked how RAP supports UI tests. And while we were aware that there are developers creating UI tests for RAP, we never gathered much experience ourselves. However, in recent months I had the opportunity to delve deeper into this topic myself, specifically researching if Selenium 2.0 works with RAP 2.x. Spoiler: It does.

NOTE: This video has been edited for presentation.

I should mention that this article is about manually writing UI tests, not recording them. While recording Selenium tests for RAP is feasible, it’s a whole other topic for research.

Requirements

To run my example tests you will need:

The bundle org.eclipse.rap.selenium contains the “RapBot”, which helps dispatching DOM events in the manner best suited for RAP, and an XPath builder, which simplifies creating the XPath-strings used by Selenium to find HTML elements. Both of these are experimental and entirely optional when writing RAP Selenium tests.

Loading the RAP application to test

Assuming you have Selenium and your RAP application (here: the Controls Demo) up and running, how do you load the application? With the RapBot it’s simple:

[enlighter lang=java
driver = new ChromeDriver();
selenium = new WebDriverBackedSelenium( driver, URL );
driver.manage().window().setSize( new Dimension( 1024, 768 ) );
rap = new RapBot( driver, selenium );
rap.loadApplication( URL );
[/enlighter]

The loadApplication method waits for the initial UI to be fully rendered, runs some additional JavaScript (needed by the bot), and then returns. Put all of this in your setUp method, and in the tearDown just the line “selenium.stop();". Now you can write the tests.

Controlling the RAP application

The RapBot provides you with some basic methods to dispatch fake events on RAP widgets and retrieve information about their state. It has methods to click, press keys, enter text and more. A few methods are specific to RAP, but most would work with any web application. You can also always use the WebDriver API directly. The difficult question is how to identify the right HTML elements for the bot to work with. All methods of the bot take an XPath as the first parameter, either as a string or as an instance of org.eclipse.rap.selenium.xpath.XPath. I created this fluent XPath API because I found that writing XPath syntax directly is very error-prone, but you can use whatever suits you more. In any case, the bot will always check that there is exactly one match for the given XPath, or it will throw an exception. I experimented with three basic (non-exclusive) approaches to identifying a specific widgets in the RAP UI: Using just the text content of an HTML element, using Test-IDs, or using ARIA-attributes. Under no circumstances should you use absolute paths (e.g. “/div[2]/div[1]/div[4]”) or the widget IDs ("//*[@id='w4']”) to find widgets - even minuscule changes in your application or in RAP would break your test.

Good: Identification by text content

See UnmodifiedControlsDemo_Test.java. A test that just presses a few buttons on the initial page of the Controls Demo could look like this:

[enlighter lang=java
rap.click( any().textElement( “Push\n Button” ) );
rap.click( any().textElement( “Check” ) );
rap.click( any().textElement( “Check with image” ) );
[/enlighter]

Note that any() is a static import from the XPath class. A more complex example would be to navigate to the Text tab of the demo, then insert some text there that is changed by a Verify listener.

[enlighter lang=java XPath someNavItem = any().textElement( “Button” ).firstMatch(); XPath navigation = byId( rap.getId( someNavItem.parent().parent().parent() ) ); XPath navItemText = navigation.descendants().textElement( “Text” ); // [1] while( !rap.isElementAvailable( navItemText ) ) { rap.scrollWheel( navigation.firstChild(), -1 ); } rap.click( navItemText ); rap.waitForAppear( any().textElement( “Text:” ) ); // [2] rap.click( any().textElement( “VerifyListener (numbers only)” ) ); // [3] rap.input( any().element( “input” ).firstMatch(), “hello123world” ); // [4] rap.click( any().textElement( “getText” ) ); // [5] rap.waitForAppear( any().textElement( “123” ) ); // [6] : numbers only! [/enlighter]

There are a couple of hurdles here. In the first line we look for an entry in the navigation bar called Button, but since there are multiple elements with this text, we have to specify that it’s the first. Not ideal, especially if a later change in your application adds another element with “Button” in it or switches the order of the elements. The navigation bar is actually a Tree widget, which is especially tricky to control due to its complexity. To get to the bar itself we need to navigate three levels up from any of its items and identify it by its HTML ID. (This requires the widget IDs to be rendered in the DOM, which is enabled by setting the environment variable org.eclipse.rap.rwt.enableUITests=true. We need to use the ID because the item disappears later.) Then we scroll the tree until the Text item becomes visible and click that. Since there is an ajax request before the UI is updated we have to use the waitForAppear method to wait for that label to exist. There is also a waitForServer method (using some JavaScript magic) that would work, but the former is more precise. The rest of the test is pretty straight-forward: enable the verify listener, enter text, press “getText”. The last line is a simple way to verify that the verify-listener did what it was supposed to do.

Better: Identification by Test-ID

See PatchedControlsDemo_Test.java. As of RAP 2.3(M3), there is public API for setting HTML-attributes on the HTML element of an widget. We can use this to assign “Test-IDs”, i.e. an attribute (“test-id”) that is ignored by the browser but can be targeted by Selenium. Using a small helper class you can assign any widget (that is included in RWT) such an ID. For example:

pushButton = new Button( parent, style | SWT.PUSH );
pushButton.setText( "Push\n Button" );
setTestId( pushButton, "pushButton" );

If we do this in a couple of places, the above tests can be written as:

rap.click( byTestId( "pushButton" ) );
rap.click( byTestId( "checkButton1" ) );
rap.click( byTestId( "checkButton2" ) );

And:

XPath textEl // [1]
  = byTestId( "demoNavigation" ).descendants().textElement( "Text" ); 
while( !rap.isElementAvailable( textEl ) ) {
  rap.scrollWheel( byTestId( "demoNavigation" ).firstChild(), -1 );
}
rap.click( textEl );
rap.waitForAppear( byTestId( "textWidget" ) );  // [4]
rap.click( byTestId( "btnNumbersOnlyVerifyListener" ) ); // [3]
rap.input( byTestId( "textWidget" ), "hello123world" ); // [4]
rap.click( byTestId( "btnGetText" ) ); // [5]
rap.waitForServer();
assertEquals( "123", rap.getText( byTestId( "textLabel" ) ) ); // [6]

A good deal shorter and more readable, and much more reliable. No more need to traverse backwards or choose from multiple matches, and we can check the content of a specific label - not just that there is ANY label like this. The drawbacks of this method are that you need to modify your application code, and that there is a slight increase in the size of the server-to-client messages. (Though you could just render Test-IDs depending on a constant or environmental variable.) Also, most widgets consist of multiple elements, and currently you can modify (in most cases) only one of them.

Best: Additional Identification by ARIA-Attributes

See AriaControls_Demo_Test.java. The W3C WAI-ARIA standard defines a number of HTML-attributes that can be evaluated by screen-reader software to aid users that are visually impaired. Each HTML element can be assigned a role, and depending on the role, a number of state and relationship attributes. While RAP itself does not render these attributes, there is an (commercial) Add-On available from EclipseSource that does. Using the Add-On does not automatically make your application WCAG-compliant (there is more to it than that), but it does add ARIA-attributes to most widgets, reflecting their classes and states (though not [yet] their relationships). There is no one-to-one match of the roles defined by ARIA and the widget classes of SWT/RWT, but it’s pretty close. Using these attributes we can write much more meaningful Selenium tests, relying not only on text content and explicit IDs, but on semantic information about the elements. The XPath API used in our examples has a couple of methods specifically targeting ARIA attributes:

rap.click( any().widget( BUTTON, "Push\n Button" ) );
rap.click( any().widget( CHECK_BOX, "Check" ) );
rap.click( any().widget( CHECK_BOX, "Check with image" ) );

You can also look for widgets within shells or groups with specific names. This code is addressing an “OK” PUSH button inside a Group labeled “Information”:

XPath OK =  any().widget( DIALOG, "label", "Information" )
                 .descendants()
                 .widget( BUTTON, "OK" );

You can also verify that a specific checkbox is selected/checked, or check if an item of a VIRTUAL table is still being loaded, etc. There is also a GridBot class in the git repository that uses ARIA attributes to handle Table (and, within limits, Tree/Nebula-Grid) widgets, even if they have VIRTUAL flags. Specifically, it takes over the task of finding a specific item, as demonstrated by this ARIA-only variant of our previous test:

AriaGridBot grid = new AriaGridBot( rap, any().widget( TREE_GRID ) );
rap.click( grid.scrollCellIntoView( "Text" ) ); // [1] : Now just one line!
rap.waitForAppear( any().textElement( "Text:" ) ); // <-[2], [3]:
rap.click( any().widget( CHECK_BOX, "VerifyListener (numbers only)" ) ); 
rap.input( any().widget( TEXT_BOX ).firstMatch(), "hello123world" ); // [4]
rap.click( any().widget( BUTTON, "getText" ) ); // [5]
rap.waitForAppear( any().textElement( "123" ) ); // [6]

Pitfalls and Debugging

  • Again: Do not rely on the traditional widget HTML IDs to be stable. They have their purpose, but they WILL change quite easily.
  • Hidden elements: It is possible for an XPath to match an element that isn’t visible to the user. This does not just include widgets that are set to be not visible, but also any element that may have been created by RAP internally. This problem doesn’t exist when using ARIA (as there is an “aria-hidden” attribute) or Test-IDs, but a hard-to-find issue in all other cases. It’s also a very likely explanation for unstable tests.
  • Limitations of faking events: Faked events do not always have the same effect as real ones, for example key events do not insert text into a text-field.
  • Browser/WebDriver issues: The different WebDrivers for Selenium all can have quirks of their own. I recommend to choose one and stick to it unless you want to implicitly also test the driver, browser and the RAP framework.
  • Escaped and invisible characters: Not always is the text value of a widget identical with what ends up in the HTML element. For example, white spaces may be replaced with " “, etc. Use the HTML inspector of your browser to see what the actual text is.

Finally, if a test doesn’t work because there is no (or more than one) match for an XPath, there is an easy way to find out why: Temporarily remove selenium.stop(); from the tearDown method and execute the failing test. The browser window will stay open and allow you to open the browser’s developer tools. You can inspect the HTML or execute different XPaths to see what works and what doesn’t. You need to manually close the browser window and kill the WebDriver process after you’re done.

Conclusion

Though I certainly have not explored every possible scenario, it seems to me that Selenium 2.0 (with some utils on top of it) is an excellent match for RAP, especially with ARIA and/or Test-ID’s in the mix. I don’t know if there is a need to be able to record tests (instead of writing them). If you have an opinion on that, let me know. Please note that the code written for this article is considered experimental and not part of the Eclipse RAP project. However, pull requests are certainly welcome.

Tim Buschtoens