• Open the “Problems” view menu by clicking on the little triangle in the upper right corner of the view. Select “New Problems View”.

• Choose a suitable name. This view only shows the errors in my current editor, so I enter “Error Outline”.

• Place the “Error Outline” under the Outline.
• In the “Error Outline”, open the view menu. Select “Configure Contents”.

• The configuration dialog is fairly straight-forward. Configure it to show Errors/Warnings on Selection and it’s children as shown here.

Now you have a fully configured Error Outline that shows a list of the current problems in your Editor.

By the way, you can also move through the problems in your editor by keeping your CTRL key pressed and hitting ‘.’ and ‘,’.

I find it useful to have 2 Problem views configured, one that focuses on the editor and one that shows me the problems in my project. The workbench layout is then shown in the following picture.

• Allow functional testing of RAP/RWT applications.
• Allow stress testing of RAP/RWT applications, working together with a performance test library.

A secondary requirement is to:

• Be similar enough to the SWTBot API so that the effort for testing single-sourced applications is minimal

which gives this project the name RWTBot.

The following example test connects to the RWT Controls Demo and tests the toggle button functionality:

@Test
public void selectToggleButton() throws Exception {
  String rwtAppUrl = "http://rap.eclipsesource.com/controlsdemo/controls";
  RWTBot bot = new RWTBot(new RWTBotClientParameters(rwtAppUrl));
  RWTBotDisplay session = bot.startSession(); // 1. handshake
  RWTBotShell shell = session.shell(); // 2. identify the main window
  RWTBotButton observed = shell.button("Toggle"); // 3. Find a button named "Toggle"
  assertFalse(observed.getSelection()); // 4. check precondition
 
  shell.button("Toggle Button").click(); // 5. trigger UI change
 
  assertTrue(observed.getSelection()); // 6. check post condition
}

During the test, this happens inside RWTBot:

1. The initial handshake is executed with the RWT Demo. This opens a RAP session and a display is created. All widgets on the main screen are received. RWTBot control handlers for roughly 100 controls are created and configured.
2. Initially there is just the main window. This is identified and the RWTBot handle is received.
3. The main window has a button named “Toggle”. A handle for the button is received and remembered as a local variable. It will reflect the state change in the UI.
4. Assert that the precondition is as expected, ie. that the toggle button is deselected.
5. The “Toggle Button” is clicked. This is sent to the server which responds with changes for the UI. These changes include the activation of the observed button.
6. Assert that the postcondition is as expected, ie. that the toggle button is activated.

Did I miss any design criteria for RWTBot? Are you interested in a working RWTBot? Leave a comment or email me to let us know.

Of course every motor vehicle is supposed to have working brakes. In software things are different.

In any application many thoughts are given to the when and how things start up. Not as often are thoughts given to application shutdown. Of course, there are always these stop() methods, i.e. in the IApplication in Eclipse RCP or RAP and in every OSGi bundles activator. But what are you supposed to do with it?  You’re supposed to shut down your connections, release your resources and stop your threads. Wait until this has happened, then exit stop(). Otherwise you risk ugly log entries in the best case, but also memory, UI handle or file handle leakages.

Cleaning up your resources is especially important in environments where the application lifecycle is not bound to a user interaction, i.e. on the server side or in RCP applications that just keep running.

In the following example, I present a small OSGi bundle with an activator and a class TimeIndicator. The Activator has the default start() and stop() methods, and at this moment the stop() method is implemented somewhat carelessly.

public void start(BundleContext bundleContext) throws Exception {
   Activator.context = bundleContext;
   clock = new TimeIndicator();
   clock.start();
}
public void stop(BundleContext bundleContext) throws Exception {
   // TODO free up resources
   Activator.context = null;
}

The TimeIndicator is a Thread that gets the OSGi LogService and logs a message “Tick” every second until it is shut down. For the purposes of this example the TimeIndicator assumes that it will always get the LogService via the activators bundle context.

@Override
public void run() {
  running = true;
  while (running) {
    BundleContext context = Activator.getContext();
    ServiceReference reference = context.getServiceReference(LogService.class);
    LogService logService = context.getService(reference);
    logService.log(LogService.LOG_ERROR, "Tick");
    waitASecond();
  }
}

Lets start this bundle. I’m starting this bundle as OSGi application.
In the OSGi console, I stop my bundle, and then inspect the status:

osgi> stop com.eclipsesource.examples.cleanup
osgi> ss

Framework is launched.

id State       Bundle
0 ACTIVE      org.eclipse.osgi_3.7.2.v20120110-1415
1 ACTIVE      org.eclipse.core.contenttype_3.4.100.v20110423-0524
2 RESOLVED com.eclipsesource.examples.cleanup_1.0.0.qualifier
3 ACTIVE      org.eclipse.equinox.app_1.3.100.v20110321
4 ACTIVE      org.eclipse.equinox.common_3.6.0.v20110523

!ENTRY com.eclipsesource.examples.cleanup 4 0 2012-06-05 09:25:52.036
!MESSAGE Tick

This is a dangerous state. The bundle stopped, but the thread is still running. It will soon fail because the context will be null in the next iteration, but in some bigger jobs it is not even sure that required classes would be loaded.

But the solution is straightforward. Here is what should be done in the stop() method:

public void stop(BundleContext bundleContext) throws Exception {
if( clock != null ) {
      clock.stopTicTac();
      clock.join();
    }
    Activator.context = null;
}

Checking your brakes is not hard, but just as a vehicle needs an inspection from time to time you should have a look now and then at cleaning up your resources properly.

Credit: Motorcycle photo by Richard Taylor via CC BY 2.0

The general theme for this year’s testing day is “Testing and Beyond”. Topics of interest include:

1. Testing Eclipse applications
2. Testing within the Eclipse Ecosystem
3. Testing on Eclipse Projects
4. Design for testability in Eclipse
5. Case studies of testing projects
6. Eclipse tooling and technology for the test process
7. Testing as a part of the application lifecycle
8. Continuous integration and testing for Eclipse applications

The call for papers is open. If you would like give a talk on one of the given topics, or maybe if you have a visionary ‘beyond’, don’t hesitate to submit a talk.  The deadline for submissions is May 31st.

You’ll find more information on the wiki at http://wiki.eclipse.org/Eclipse_Testing_Day_2012

In the morning we heard several talks about various testing strategies in different commercial products. Alexander Klein from BeOne held an inspiring talk about testing the users experience for a product. Among other ideas he recommended to watch users while they are confronted with the product to get ideas for the next iteration.

In the afternoon things became more hands-on. Michaela Greiler presented her Eclipse Plug-in Test Suite. This is a test runner that allows static and dynamic analysis to report on executed extension points and OSGi services. By itself it is merely interesting, but imho not interesting enough to be a standalone project. I think this technology should go as additional metrics in something like EclEmma. Maybe JaCoCo will provide a home? Hear me, Marc?

Jubula was a major point of talk. Felix Ziesel from BREDEX showed how he uses the Eclipse for Testers download to define an automated smoke test suite to test the completeness of the Eclipse for RCP and RAP developers. I hope he will repeat this talk at an EclipseCon where other packagers are present. This just looked as it ought to be, but obviously it wasn’t feasible before Jubula.

The last talk was mine. I presented an example of a full-featured PDE build from a git repository with test automation. Even though it was the last one of a full day of talks the discussion afterwards was interesting and showed I had hit a sore spot in many projects.

At the side where a few booths from various companies. I was excited to see Xored advertising Q7 there. This is a UI testing tool that only supports Eclipse. Because of this exclusiveness the scripting language gets right to the point. Capture and Replay functionality is also available. Why was I excited about that? – Coming as complete outsiders, they did a talk at last years Eclipse Testing Day about this side product they developed to make testing their application easier. It was extremely well perceived and for a while they where the focus of attention with this thing that’s now Q7. I’d really like to see them more involved with the Eclipse community. Specifically I think their tool might be in a good position to allow UI testing for a RAP workbench.

As a conclusion I have mixed feelings about the Eclipse Testing Day. It shows that things are moving and keeps a handful of people talking. I think it could be a real benefit for the community, if it would show up. But while last year all talks where held in english, now only 3 out of 10 did so, making the Testing Day too much of a german and too less an international event. Many key players where missing. Nobody mentioned SWTBot or JaCoCo. WindowTester didn’t even seem to exist. Is FrogLogic still in business (Their website says, yes)?
It seemed to me that compared to last year there was almost an entirely new audience. I recognized only a handful people and 2 of them where speakers.
It think that the talks and the audience are a bit too diverse. The name “Testing Day” brings together developers people who drive test processes in larger companies, but did’t lead them to talk to one another. Instead of accepting just mixing them in a single branch of talks, the Testing Day could improve by cutting down the program for the whole audience to maybe half a day. Fill the other half day with special interest topics, i.e. bring together the developers to talk about TDD, JUnit, issues about test automation with their test runners and hard times with AspectJ. Bring the people together who want to discuss test processes in big projects, but separate these two groups for a while at least.

The organizers are aware of this, too, so I’m looking forward for a strong and interesting Eclipse Testing Day 2012.

I’m planning a series of blog entries with and around that example. Some features are:

• Works system independent
• Executes tests
• Builds from map files
• Separates compile step and assemble step (Improves build time)
• Mostly self-contained (Required: An Eclipse SDK and Git)

How to get it running in short:

• Fetch from git://github.com/mkempka/hyperbola-pde-build-demo.git
• Adjust values in org.eclipsercp.hyperbola.releng/custom.properties
• Execute org.eclipsercp.hyperbola.releng/build.xml with ant

Here are my slides from my talk at the Eclipse Testing Day 2011 that include some documentation. I’ll do more documentation in separate blog entries the next weeks.

I was astonished to find nothing in the SWTBot code or documentation that suggests how to automatically capture a screenshot when a SWTBot test fails. This is why I came up with a JUnit4 Rule that does this.

In your SWTBot test class include the field

public class MyTest {
  @Rule
  public CaptureScreenshotOnFailure screenshot = new CaptureScreenshotOnFailure();
  
  @Test public void mytestmethod() {
    SWTBot bot = new SWTBot(...);
    // swtbot clicks and types
  }

So, if now mytestmethod fails, a screenshot will be put in the current directory with the name mypackage.MyTest.mytestmethod.png

This is the rule, put it somewhere where your test classes can access it:

import org.eclipse.swtbot.eclipse.finder.SWTWorkbenchBot;
import org.junit.rules.MethodRule;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.Statement;

public class CaptureScreenshotOnFailure implements MethodRule {
	public final Statement apply(final Statement base,
			final FrameworkMethod method, Object target) {
		return new Statement() {
			@Override
			public void evaluate() throws Throwable {
				try {
					base.evaluate();
				} catch (Throwable onHold) {
					String fileName = constructFilename(method);
					new SWTWorkbenchBot().captureScreenshot(fileName);
					throw onHold;
				}
			}

			private String constructFilename(final FrameworkMethod method) {
				return "./"
						+ method.getMethod().getDeclaringClass()
								.getCanonicalName() + "."
						+ method.getName() + ".png";
			}
		};
	}
}

I think this makes writing tests with SWTBot more comfortable. Do you see ways to improve this rule?

Background

Good test suites

• are easy to set up
• pick up new tests as they are included in the projects (no need to include them manually)
• can differentiate between long running tests and fast unit tests
• Can be used in both the IDE and the continuous integration
• Don’t add performance

One way to create dynamic test suites is the ClassPathSuite. It goes through the classpath and adds all tests it can find to a test suite which is then executed. This approach has two problems that both come from loading all classes before inspecting them:

• The time to dynamically create the test suite goes up
• Static initializers may have wrong assumptions

For OSGi applications there is another way to create a test suite dynamically, thats the BundleTestCollector from Patrick Paulin. It is a similar approach in that it goes through specified bundles and adds classes to a test suite, but has neither of the above drawbacks

• No class is loaded before a test is actually executed
• There is no need to scan dependencies: You can narrow down the plug-ins that are scanned for tests

Sounds almost perfect, doesn’t it? Hold your breath, it has some problems, too, but they are small against everything else I stumbled across.

First, Patricks version only works with JUnit 3. I used a modified version for a while now that works with JUnit 4 and has some more assertions than Patricks original version. The remaining problem is, when looking at the test results in the JUnit view, the stack trace is not linked to the workspace. A double-click fails, and I need to open the class and navigate to the line by hand.

The BundleTestCollector requires OSGi Bundles. You can’t just run it as JUnit test. In your IDE it must be started as PDE Test (or SWTBotTest), and in the continuous integration, you should use the Eclipse Testing Framework or the SWTBot test runners.

The BundleTestCollector

Patrick published his version of the BundleTestCollector under the EPL, so my modifications are EPL, too. Here is a zip containing the BundleTestCollector and it’s single dependency, an Activator. You’ll have to copy them into an actual bundle and adjust the namespace. JUnit4 must be on the bundle dependency path.

The BundleTestCollector picks up the tests by naming conventions. If your test doesn’t adhere to a naming convention, it won’t be picked up.

Here is how to define a TestSuite. It’s still a JUnit 3 test suite, but the actual tests may be both, JUnit 3 or JUnit 4.

public class MyTestSuite extends TestSuite {

  public static Test suite() {
    BundleTestCollector testCollector = new BundleTestCollector();
    TestSuite suite = new TestSuite( "All Tests" );
    testCollector.collectTests(suite, "my.bundle.namespace", "my.package.namespace", "*Test");
    // add more lines collectTests(...) calls if necessary
    return suite;
  }
}

Hope that eases some of your test suite maintenance pains.

With some of my workspaces I want to have a similar IDE as with others, but some IDEs require special plug-ins. A while ago, as I still unzipped Eclipse-downloads, this was a huge pain. Every time I wanted to work in a specific workspace I had to remember which IDE I used for what, then find the workspace location on the disk, before I could do anything.

Permanent workspaces

Now I double-click a .yoxos file on my Desktop, then start working. Related .yoxos files hang out together on different areas on my desktop. This is possible with the Yoxos 5 Launcher which I explained in my last blog post.

I associate the workspace I want to start in terms of “upper right” or “vaguely in the middle, left” on the screen. No need to remember long directory names.
Remember, a .yoxos file is a definition of both the workspace and the IDE that works on the workspace. The actual workspace is somewhere in my home directory. Since every .yoxos file defines a separate IDE, I always click “Use this as default” after defining the first time where the workspace is located.

Temporary workspaces

Throwaway workspaces go to /tmp. Along with everything else in /tmp, they will be deleted the next time I reboot. The .yoxos file that defines a throwaway workspace should be deleted with it, so this belongs to /tmp as well. The IDE definition works this way:

• Start the Yoxos Launcher
• Add “Project SDK” and all the other desired plug-ins
• Save the .yoxos file to “/tmp/throwaway-workspace” (a new empty directory)
• Hit “Launch”

With the bundle pool I don’t worry about the plug-ins that compose the throwaway IDE. Only rarely something new gets downloaded anyway.

.yoxos files and workspaces

A special handling of .yoxos files in otherwise empty directories supports this workflow. If a .yoxos file is is started while residing in an empty directory, the IDE uses this directory as workspace. This provides an easy answer to the question about the “where”, and I use this feature at a regular basis.

On Mac OS X, I can append the extension “.yoxosws” to a directory that contains a .yoxos file. This defines a workspace that I can start directly with a double-click, without bothering about opening a folder to access the .yoxos file.

Conclusion

The Yoxos 5 Launcher makes it simple to handle a multitude of workspaces. The best thing is that you can stop wondering about the IDE contents and start thinking about workspaces. The Launcher provides a consistent UI to define new IDEs, including a huge number of 3rd-party plug-ins that are not shipped with the default Eclipse downloads.

But the number of useful plug-ins increased, and many are not included in the downloads from eclipse.org. Developers use different plug-ins in different workspace. For some developers, this leads to as many Eclipse installations as workspaces. Others capitulated and just don’t use many plug-ins even though they see their value; but managing the installations is just too hard. Others again have one huge installation that includes about everything for all the workspaces, and they too have pain with plug-in dependencies. They all suffer from plug-in dependencies.

Imagine you had a system where each plug-in you use is downloaded just once and reused whenever you need it for a new Eclipse IDE.

Yoxos 5 provides that.

Imagine you could just start your workspace and your IDE starts up including all plug-ins you want to work with in that workspace. If it is a new workspace you’d have automatically adjusted predefined settings, import projects etc.

Yoxos 5 excels at that.

Yoxos 5 unifies the workspace settings and its IDE description in a Yoxos Profile. A Yoxos Profile can be defined in a .yoxos file. The Yoxos Launcher creates Yoxos Profiles and starts them, for example when double-clicking the .yoxos file. Plug-ins are downloaded to the bundle pool and started only if the profile includes them.

Yoxos 5 is now in beta phase. You can try it out now:

• Download and install the Yoxos Launcher
• Download and start one of the sample profiles (further down at the download page)

Get more information at http://eclipsesource.com/yoxos5