jmhofer's insignificant insights

After my little blog series about Jubula, I thought I’d try out something similar using SWTBot. However, there’s no use in writing the same tutorial again for the RCP mail application because it’s already there…

Nevertheless, I attempted to implement the same few tests I wrote for Jubula, and you can view the results in the piece of code below:

package de.johoop;

import static org.junit.Assert.*;

import org.eclipse.jface.bindings.keys.KeyStroke;
import org.eclipse.swtbot.eclipse.finder.SWTWorkbenchBot;
import org.eclipse.swtbot.swt.finder.keyboard.Keystrokes;
import org.eclipse.swtbot.swt.finder.utils.SWTBotPreferences;
import org.eclipse.swtbot.swt.finder.widgets.SWTBotShell;
import org.junit.Before;
import org.junit.Test;

public class FirstSwtBotTest {

  private SWTWorkbenchBot bot;

  @Before
  public void setUp() throws Exception {
    bot = new SWTWorkbenchBot();
  }

  @Test
  public void mailToolbarButtonShouldOpenMessageDialog()
      throws Exception {

    bot.toolbarButton(1).click();
    assertThatCorrectDialogIsDisplayed();
  }

  @Test
  public void openMessageMenuEntryShouldOpenMessageDialog()
      throws Exception {

    bot.menu("Open Message").click();
    assertThatCorrectDialogIsDisplayed();
  }

  @Test
  public void keyboardShortcutShouldOpenMessageDialog()
      throws Exception {

    SWTBotPreferences.KEYBOARD_LAYOUT = "EN_US";
    bot.activeShell().pressShortcut(
        Keystrokes.CTRL, KeyStroke.getInstance("3"));
    assertThatCorrectDialogIsDisplayed();
  }

  private void assertThatCorrectDialogIsDisplayed() {
    final SWTBotShell openDialog = bot.activeShell();
    assertEquals("Open", openDialog.getText());
    assertTrue(bot.label("Open Message Dialog!").isVisible());

    bot.button("OK").click();
    assertFalse(openDialog.isOpen());
  }
}

It’s a set of three tests opening the “Open Message” dialog via toolbar button, keyboard shortcut or menu entry. Simple.

At first glance, this looks neat, especially if you’re a developer: few lines of code, all directly in Java, using the already familiar JUnit framework, well supported from within the Eclipse IDE.

Nevertheless, here’s my list of things I don’t like about this:

1. Let’s begin with a minor nitpick: I’m using a German keyboard layout, and SWTBot complains that it can’t find my layout. This means that I either have to somehow create one (which might be easy, I haven’t checked), or I have to set my keyboard layout to the US one (it doesn’t matter for my simple Ctrl-3 test).
2. Next, I can’t write these tests in Scala for some reason (I’d really like to use Scala for my tests, so very much more concise): In Scala, the following line:

   new SWTWorkbenchBot().toolbarButton(1).click()

   fails with a compile error in Eclipse: “Access to protected method click not permitted because enclosing class object ScalaUser is not a subclass of class AbstractSWTBot in package widgets where target is defined”. – What’s that?

   If you ask me, it’s somehow caused by the SWTBotToolbarButton having an abstract public method click() returning an SWTBotToolbarButton, and inheriting from AbstractSWTBot<T>, which has an abstract protected method click() returning an AbstractSWTBot<T>.

3. Also, I don’t want to use JUnit for system tests. JUnit is a great framework if all you do is unit tests. If you’re testing on the level of integration or system tests, however, TestNG is a way better fit if you ask me. The execution model of TestNG is a lot more flexible than that of JUnit (even if you use decorators and stuff there). For example, have a look at the data providers of TestNG. Unfortunately, there’s no TestNG integration for SWTBot.
4. Finally, here’s what I’d call a showstopper. Have you noticed how I say toolbarButton(1) in my code instead of the toolbarButtonWithTooltip("Open Message") you’d probably expect? That’s because using the latter method will require another dependency in your plug-in project. That dependency will have to get injected into your test object, too (the RCP mail application in my case). And unfortunately, it will add toolbar buttons and keyboard shortcuts to your test object!

   In my case, this leads to Ctrl-3 suddenly not working as expected anymore: The tests have changed the behaviour of the test object! I absolutely can’t have that happening.

For me, these four reasons unfortunately mean that SWTBot is not a viable tool for automating UI tests. I would really like to use it, especially combined with Scala, for quick and easy developer UI testing; I’d be quite happy if you could refute any of my four reasons above.

I’ve still got hope that maybe I’m just too stupid to use SWTBot correctly.