Testing the same behaviour every time

Testing the same behaviour every time

e2e_repeat

Tests give us an assurance which application works fine. We usually want to execute them on some Continuous Integration servers. It is very convenient because we can configure them to run after each repository commit or once a day.

Continuous integration flow usually works as follows. The application is built on CI server and then it executes all tests. Because all tests run on the same database, it is hard to make all them all independent. However, test independence is a very important principle. The result of one test may affect the correctness of another test. It is especially problematic if we want to run tests in parallel. We can’t be sure what will be the order of execution. It is not only the problem of the database. The browser can also store a state of the application.

Test configuration

Fortunately, Selenium WebDriver tests have an ability to manage all testing environment. This is not exactly the WebDriver library functionality, but NUnit – testing framework. We can set up an initialization and deactivation methods around testing code.

NUnit provides following attributes to mark configuration methods:

  • [OneTimeSetUp] – initialization executed once at the beginning of TestFixture execution
  • [OneTimeTearDown] – deactivation executed once at the end of TestFixture execution
  • [SetUp] – initialization method executed before each test
  • [TearDown] – deactivation method executed after each test

Independent database tests

We can use SQLServer Snapshot functionality to save database state between every test.

public abstract class BaseTestCaseWithDatabase
{
    protected IWebDriver Driver { get; private set; }

    [SetUp]
    public void Initalize()
    {
        Driver = new ChromeDriver();
    }

    [OneTimeSetUp]
    public void SetUpFixture()
    {
        DbManager.Instance.CreateSnapshot();
    }

    [OneTimeTearDown]
    public void Cleanup()
    {
        DbManager.Instance.DropSnapshot();
    }

    [TearDown]
    public void TearDown()
    {
        Driver.Quit();
        Driver = null;
        DbManager.Instance.RestoreSnapshot();
    }
}

You can see above that this functionality consists of just three actions.

  • create a snapshot at the beginning
  • restore it after each test
  • remove it when all tests finished

Independent browser tests

We can also clear the browser state after each test. It is important if our application stores cookies or data in localStorage. If we use any other type of client-side storage like sessionStorage or WebSQL, we need to manage them too.

public abstract class BaseTestCasePersistent
{
    protected IWebDriver Driver { get; }

    [OneTimeSetUp]
    public void Initalize()
    {
        Driver = new ChromeDriver();
    }

    [TearDown]
    public void TearDown()
    {
        driver.ExecuteScript("window.localStorage.clear();");
        _chromeDriver.Manage().Cookies
            .DeleteAllCookies();
    }
}

This example is very simple. After each test, we clear the local storage by executing JS code. Then we delete all cookies.

We can still combine this two mechanisms and extend it freely. The only thing that we should remember is to remove all application data between tests.