Theia Playwright - End-to-end testing Theia applications

Are you developing a custom tool with Eclipse Theia and are in need of a comprehensive test strategy? A test strategy that covers not only unit and integration testing but also verifies the expected behavior for the most important user flows? Then Theia 🎭 Playwright – a recent contribution to Theia – has you covered!

Otherwise, continue reading to find out more!

If you would like to watch a video on Theia 🎭 Playwright, please see this video.

Otherwise, continue reading to find out more!

Unit & integration tests vs. end-to-end tests

No doubt, unit and integration testing is the solid foundation of every test strategy and this is also true for Theia applications. Implementing such tests for Theia-based tools is straightforward and doesn’t require much explanation. The Javascript ecosystem offers plenty of great choices for test runners, assertion libraries, mocking frameworks, etc. Probably the most prominent options are mocha and jest, as well as chai for assertions and sinon for mocking external services etc.

The flexible architecture of Theia is built with dependency injection in its heart, which not only makes Theia highly customizable, but also facilitates unit and integration testing. Dependency injection provides full control over the implementations being used in a certain setup, which simplifies isolating classes for particular test scenarios, such as testing single classes in isolation for unit tests or in combination for integration tests.

However, eventually you will also need to verify whether the Theia application, as provided to the user, also works when consumed just like a normal user would use it. Therefore, your battery of tests should be complemented with a reasonable set of automated end-to-end tests covering the most important flows. End-to-end testing Theia applications entails interacting with the user interface and observing the results via the user interface (and not necessarily the internal state), just like a user.

Even though there are many great browser automation frameworks available, developing stable and lean end-to-end tests for complex applications from scratch is a time-consuming process. These tests have to automate complex UI interactions, deal with variations in response time and wait for the right conditions before the next steps can be performed or assertions about the application state can be verified.

This is precisely where Theia 🎭 Playwright is now ready to support you!

Welcome Theia 🎭 Playwright

Theia 🎭 Playwright is a recent addition to the Theia universe and provides a page object framework simplifying the development of end-to-end tests for Theia-based applications. It is written in Typescript and built on top of Playwright, a very popular and powerful browser-automation and testing library.

The main benefit of Theia 🎭 Playwright is its comprehensive set of Theia-specific page objects, which introduce an abstraction layer over Theia’s user interface components, encapsulating the complex details of the user interface interactions, wait conditions, and flows. Thus, your end-to-end tests don’t have to work on DOM elements directly, such as an HTML element that has certain CSS classes and contains some other element that represents a file and so on. Instead, there are dedicated page objects, such as TheiaTextEditor, TheiaExplorerView on which you perform actions on a higher level of abstraction, such as typing some content into a specific line in a text editor or renaming folders in the explorer, double-clicking a file matched by its name to open an editor, and many more.

This not only helps to keep your tests more concise and stable, but also significantly increases reuse across multiple tests and even multiple products. As Theia 🎭 Playwright is open source now and part of Theia, the development and maintenance effort of providing these reusable page objects can be shared with the community. For instance, if the DOM structure of Theia changes in a new Theia version, this will only have to be adapted once in the respective page object, but your test suites won’t have to change at all.

Time for an example

Let’s have a look at a small end-to-end test, which verifies the behavior of the Theia text editor. In particular, let’s perform the following steps:

1. Initialize a prepared workspace containing a file sampleFolder/sample.txt and open the workspace with the Theia application under test
2. Open the Theia text editor
3. Replace the contents of line 1 with the text “change”, check the line contents and the dirty state, which now should indicate that the editor is dirty.
4. Perform undo and verify whether the line contents is again what it was before the change. The dirty state should be clean again too.
5. …

As you’ll see, the test code below doesn’t need to deal with the HTML elements of the Theia application, doesn’t fiddle with complex wait conditions, but instead really focuses on the high-level steps that the test needs to perform. In addition, it can query the state as reflected by the user interface to verify assertions.

But let’s stop talking and let the code speak for itself:

test("should undo and redo text changes and correctly update the dirty state", async () => {
    // 1. set up workspace contents and open Theia app
    const ws = new TheiaWorkspace(["tests/resources/sample-files1"]);
    const app = await TheiaApp.load(page, ws);
    // 2. open Theia text editor
    const sampleTextEditor = await app.openEditor(
        "sample.txt",
        TheiaTextEditor
    );
    // 3. make a change and verify contents and dirty
    await sampleTextEditor.replaceLineWithLineNumber("change", 1);
    expect(await sampleTextEditor.textContentOfLineByLineNumber(1)).toBe(
        "change"
    );
    expect(await sampleTextEditor.isDirty()).toBe(true);
    // 4. undo and verify contents and dirty state
    await sampleTextEditor.undo(2);
    expect(await sampleTextEditor.textContentOfLineByLineNumber(1)).toBe(
        "this is just a sample file"
    );
    expect(await sampleTextEditor.isDirty()).toBe(false);
    await sampleTextEditor.close();
    …
});

Below you can see this example test in action by stepping through the code with the VSCode debug tools.

Extensibility

Theia 🎭 Playwright not only covers Theia’s generic capabilities, such as handling views, the quick command palette, file explorer, etc. Like Theia itself, it is built to be extensible, so you can add dedicated page objects for custom Theia components, such as custom menus, views, editors, etc. Typically these custom page objects extend the generic page objects for Theia, such as TheiaView, TheiaEditor, etc. With that, you can immediately inherit the generic behavior of views or editors, such as activating or closing them, querying the title or checking whether editors are dirty, etc.

Let’s take a custom view as an example. This custom view has a button that we want to be able to click.

export class MyView extends TheiaView {
  constructor(public app: TheiaApp) {
    super(
      {
        tabSelector: "#shell-tab-my-view", // the id of the tab
        viewSelector: "#my-view-container", // the id of the view container
        viewName: "My View", // the user visible view name
      },
      app
    );
  }
  async clickMyButton(): Promise<void> {
    await this.activate();
    const viewElement = await this.viewElement();
    const button = await viewElement?.waitForSelector("#idOfMyButton");
    await button?.click();
  }
}

As our custom view inherits all basic functionality of Theia views, it extends `TheiaView`. Therefore, we have to specify the selectors for the view tab and for the view container element, as specified in the implementation of the custom view. If needed, we can additionally provide a view name corresponding to the label in Theia’s view menu. This is already enough to open, close, find and interact with our custom view from a test case.

Now let’s add a custom method clickMyButton() for the view-specific action that we want to be able to invoke from our test cases. To actually implement this functionality, you can either call other page objects, or directly interact with the DOM of the view. For the latter, you have the full power of the excellent Playwright library at your fingertips, which provides methods such as waitForSelector(selector), click(), hover(), waitForElementState(selector, state), and much more. For more information, please have a look at the comprehensive documentation of Playwright.

Once we have implemented the actual behavior of the action, e.g., to click a button within a view, we can use our custom page object in a test. Therefore, we open it by passing our custom page object as a parameter to app.openView(viewClass) and then interact with the page object by invoking its methods.

const app = await TheiaApp.load(page, ws);
const myView = await app.openView(MyView);
await myView.clickMyButton();

Conclusion

Theia 🎭 Playwright is another exciting addition to the thriving Theia ecosystem. With its comprehensive page object library for Theia workbench components, Theia 🎭 Playwright greatly simplifies end-to-end testing of Theia-based applications. By encapsulating the complexity and typical challenges of writing reliable end-to-end tests, it facilitates making your tests more concise, more maintainable, and stable.

Its extensibility is an especially important feature. Theia itself is an extensible platform for domain-specific tools, consequently also a testing framework for Theia requires first-class support for custom user interface components with custom page objects. When it comes to implementing such custom page objects, the power of Playwright provides a modern and excellent feature set for automating user flows in a browser-based application.

If you are developing a Theia-based tool and require a solid strategy for quality assurance, future-proof tool architectures, and technical expertise, then contact us. EclipseSource is happy to support you with our profound experience in building customized domain-specific tools based on modern web technologies. Please have a look at our consulting offering for building tools based on Eclipse Theia, support for web-based tools, consulting for building modeling tools or tools in general. We are looking forward to talking to you!