How to write UI tests for RAP with Selenium 2.0
We’re occasionally asked how RAP supports UI tests. And while we were aware that there are developers creating UI tests for RAP, we never gathered much experience ourselves. However, in recent months I had the opportunity to delve deeper into this topic myself, specifically researching if Selenium 2.0 works with RAP 2.x. Spoiler: It does.
<iframe src="//player.vimeo.com/video/92944636" width="700" height="395" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p> <p><i style="color: #555; font-size: smaller;">NOTE: This video has been edited for presentation.</i></p> <p>I should mention that this article is about manually writing UI tests, not recording them. While recording Selenium tests for RAP is feasible, it’s a whole other topic for research.
To run my example tests you will need:
- An Eclipse Installation with an active RAP Target. For some examples you need a RAP 2.3 target (or – as of the time of this writing – the latest nightly Build), but in principle all RAP 2.x versions work.
- The RAP Controls Demo. This is the application we will test.
- The contents of this git repository, specifically the bundle org.eclipse.rap.selenium and org.eclipse.rap.selenium.examples.
- Maven, to resolve the dependencies of org.eclipse.rap.selenium.examples
- An installed WebDriver for the browser of your choice. For my tests I used ChromeDriver.
The bundle org.eclipse.rap.selenium contains the “RapBot“, which helps dispatching DOM events in the manner best suited for RAP, and an XPath builder, which simplifies creating the XPath-strings used by Selenium to find HTML elements. Both of these are experimental and entirely optional when writing RAP Selenium tests.
Loading the RAP application to test
Assuming you have Selenium and your RAP application (here: the Controls Demo) up and running, how do you load the application? With the RapBot it’s simple:
setUp method, and in the
tearDown just the line “
selenium.stop();". Now you can write the tests.
Controlling the RAP application
The RapBot provides you with some basic methods to dispatch fake events on RAP widgets and retrieve information about their state. It has methods to click, press keys, enter text and more. A few methods are specific to RAP, but most would work with any web application. You can also always use the WebDriver API directly. The difficult question is how to identify the right HTML elements for the bot to work with. All methods of the bot take an XPath as the first parameter, either as a string or as an instance of
org.eclipse.rap.selenium.xpath.XPath. I created this fluent XPath API because I found that writing XPath syntax directly is very error-prone, but you can use whatever suits you more. In any case, the bot will always check that there is exactly one match for the given XPath, or it will throw an exception. I experimented with three basic (non-exclusive) approaches to identifying a specific widgets in the RAP UI: Using just the text content of an HTML element, using Test-IDs, or using ARIA-attributes. Under no circumstances should you use absolute paths (e.g. “
/div/div/div“) or the widget IDs (“
//*[@id='w4']“) to find widgets – even minuscule changes in your application or in RAP would break your test.
Good: Identification by text content
See UnmodifiedControlsDemo_Test.java. A test that just presses a few buttons on the initial page of the Controls Demo could look like this:
any() is a static import from the XPath class. A more complex example would be to navigate to the Text tab of the demo, then insert some text there that is changed by a
There are a couple of hurdles here. In the first line we look for an entry in the navigation bar called Button, but since there are multiple elements with this text, we have to specify that it’s the first. Not ideal, especially if a later change in your application adds another element with “Button” in it or switches the order of the elements. The navigation bar is actually a
Tree widget, which is especially tricky to control due to its complexity. To get to the bar itself we need to navigate three levels up from any of its items and identify it by its HTML ID. (This requires the widget IDs to be rendered in the DOM, which is enabled by setting the environment variable
org.eclipse.rap.rwt.enableUITests=true. We need to use the ID because the item disappears later.) Then we scroll the tree until the Text item becomes visible and click that. Since there is an ajax request before the UI is updated we have to use the
waitForAppear method to wait for that label to exist. There is also a
Better: Identification by Test-ID
See PatchedControlsDemo_Test.java. As of RAP 2.3(M3), there is public API for setting HTML-attributes on the HTML element of an widget. We can use this to assign “Test-IDs”, i.e. an attribute (“test-id”) that is ignored by the browser but can be targeted by Selenium. Using a small helper class you can assign any widget (that is included in RWT) such an ID. For example:
If we do this in a couple of places, the above tests can be written as:
A good deal shorter and more readable, and much more reliable. No more need to traverse backwards or choose from multiple matches, and we can check the content of a specific label – not just that there is ANY label like this. The drawbacks of this method are that you need to modify your application code, and that there is a slight increase in the size of the server-to-client messages. (Though you could just render Test-IDs depending on a constant or environmental variable.) Also, most widgets consist of multiple elements, and currently you can modify (in most cases) only one of them.
Best: Additional Identification by ARIA-Attributes
See AriaControls_Demo_Test.java. The W3C WAI-ARIA standard defines a number of HTML-attributes that can be evaluated by screen-reader software to aid users that are visually impaired. Each HTML element can be assigned a role, and depending on the role, a number of state and relationship attributes. While RAP itself does not render these attributes, there is an (commercial) Add-On available from EclipseSource that does. Using the Add-On does not automatically make your application WCAG-compliant (there is more to it than that), but it does add ARIA-attributes to most widgets, reflecting their classes and states (though not [yet] their relationships). There is no one-to-one match of the roles defined by ARIA and the widget classes of SWT/RWT, but it’s pretty close. Using these attributes we can write much more meaningful Selenium tests, relying not only on text content and explicit IDs, but on semantic information about the elements. The XPath API used in our examples has a couple of methods specifically targeting ARIA attributes:
You can also look for widgets within shells or groups with specific names. This code is addressing an “OK”
PUSH button inside a
Group labeled “Information”:
You can also verify that a specific checkbox is selected/checked, or check if an item of a
VIRTUAL table is still being loaded, etc. There is also a
GridBot class in the git repository that uses ARIA attributes to handle
Table (and, within limits,
Grid) widgets, even if they have
VIRTUAL flags. Specifically, it takes over the task of finding a specific item, as demonstrated by this ARIA-only variant of our previous test:
Pitfalls and Debugging
- Again: Do not rely on the traditional widget HTML IDs to be stable. They have their purpose, but they WILL change quite easily.
- Hidden elements: It is possible for an XPath to match an element that isn’t visible to the user. This does not just include widgets that are set to be not visible, but also any element that may have been created by RAP internally. This problem doesn’t exist when using ARIA (as there is an “aria-hidden” attribute) or Test-IDs, but a hard-to-find issue in all other cases. It’s also a very likely explanation for unstable tests.
- Limitations of faking events: Faked events do not always have the same effect as real ones, for example key events do not insert text into a text-field.
- Browser/WebDriver issues: The different WebDrivers for Selenium all can have quirks of their own. I recommend to choose one and stick to it unless you want to implicitly also test the driver, browser and the RAP framework.
- Escaped and invisible characters: Not always is the text value of a widget identical with what ends up in the HTML element. For example, white spaces may be replaced with “
Finally, if a test doesn’t work because there is no (or more than one) match for an XPath, there is an easy way to find out why: Temporarily remove
selenium.stop(); from the
tearDown method and execute the failing test. The browser window will stay open and allow you to open the browser’s developer tools. You can inspect the HTML or execute different XPaths to see what works and what doesn’t. You need to manually close the browser window and kill the WebDriver process after you’re done.
Though I certainly have not explored every possible scenario, it seems to me that Selenium 2.0 (with some utils on top of it) is an excellent match for RAP, especially with ARIA and/or Test-ID’s in the mix. I don’t know if there is a need to be able to record tests (instead of writing them). If you have an opinion on that, let me know. Please note that the code written for this article is considered experimental and not part of the Eclipse RAP project. However, pull requests are certainly welcome.