Browsed by
Author: suvroc

Testing the same behaviour every time

Testing the same behaviour every time

e2e_repeat

Tests give us an assurance which application works fine. We usually want to execute them on some Continuous Integration servers. It is very convenient because we can configure them to run after each repository commit or once a day.

Continuous integration flow usually works as follows. The application is built on CI server and then it executes all tests. Because all tests run on the same database, it is hard to make all them all independent. However, test independence is a very important principle. The result of one test may affect the correctness of another test. It is especially problematic if we want to run tests in parallel. We can’t be sure what will be the order of execution. It is not only the problem of the database. The browser can also store a state of the application.

Test configuration

Fortunately, Selenium WebDriver tests have an ability to manage all testing environment. This is not exactly the WebDriver library functionality, but NUnit – testing framework. We can set up an initialization and deactivation methods around testing code.

NUnit provides following attributes to mark configuration methods:

  • [OneTimeSetUp] – initialization executed once at the beginning of TestFixture execution
  • [OneTimeTearDown] – deactivation executed once at the end of TestFixture execution
  • [SetUp] – initialization method executed before each test
  • [TearDown] – deactivation method executed after each test

Independent database tests

We can use SQLServer Snapshot functionality to save database state between every test.

public abstract class BaseTestCaseWithDatabase
{
    protected IWebDriver Driver { get; private set; }

    [SetUp]
    public void Initalize()
    {
        Driver = new ChromeDriver();
    }

    [OneTimeSetUp]
    public void SetUpFixture()
    {
        DbManager.Instance.CreateSnapshot();
    }

    [OneTimeTearDown]
    public void Cleanup()
    {
        DbManager.Instance.DropSnapshot();
    }

    [TearDown]
    public void TearDown()
    {
        Driver.Quit();
        Driver = null;
        DbManager.Instance.RestoreSnapshot();
    }
}

You can see above that this functionality consists of just three actions.

  • create a snapshot at the beginning
  • restore it after each test
  • remove it when all tests finished

Independent browser tests

We can also clear the browser state after each test. It is important if our application stores cookies or data in localStorage. If we use any other type of client-side storage like sessionStorage or WebSQL, we need to manage them too.

public abstract class BaseTestCasePersistent
{
    protected IWebDriver Driver { get; }

    [OneTimeSetUp]
    public void Initalize()
    {
        Driver = new ChromeDriver();
    }

    [TearDown]
    public void TearDown()
    {
        driver.ExecuteScript("window.localStorage.clear();");
        _chromeDriver.Manage().Cookies
            .DeleteAllCookies();
    }
}

This example is very simple. After each test, we clear the local storage by executing JS code. Then we delete all cookies.

We can still combine this two mechanisms and extend it freely. The only thing that we should remember is to remove all application data between tests.

QA interview questions

QA interview questions

e2e_interview

This post is a bit different that other post on this blog. I decided to present a set of questions which may be asked during the interview to a QA job. Of course, it is only a small subset of all possible questions, but it gives you an overview what you can expect from that kind of meeting. Each question has an answer.

If you would like to be sure that you can answer this and many other questions during QA interview, you can sign up for a weekly mail course http://mailcourse.seleniumbook.com/

Sign up for free end-to-end testing training

Learn how to create end-to-end tests for your applications from the beginning to mobile testing
Name
Email address

What is a Test Case? What does it include?

A Test Case is a step by step description of the testing process and a set of conditions under which a tester can determine if the application works as expected.

It includes:

  • test case description
  • test steps
  • requirements
  • expected result

What are different types of software testing?

It is four main categories, but there are much more test types in each category.

  • Unit Testing – test smallest parts of application, single methods or classes
  • Integration Testing – test single modules combination, treated as a group
  • System Testing – test without knowledge of inner design or logic. Evaluate the complete system
    • Functional – related to functionality
    • Smoke testing – preliminary tests, that checks application condition
    • Regression testing – verifies that software previously developed and tested still performs correctly
    • Non-functional – test the way how system operates, rather than specific behaviors of that system
    • Performance testing – determine how a system performs regarding responsiveness and stability under a particular workload
    • Security testing – reveal flaws in the security mechanisms of an information system that protect data and maintain functionality as intended
  • UAT – formal testing on user needs and requirements to enable the user or customers to determine whether or not to accept the system

Source

When should testing start in a project? Why?

For Unit and Integration testing you should start as soon as possible. When the requirement is ready and you will start to write code.

For System and End to end testing, you should start before the first presentation for the customer.

What is end-to-end testing?

It is testing a complete application. It simulates real-world use, such as interacting with a database, using network communication, or interacting with other hardware, application, or system.

What is acceptance testing?

Acceptance testing is done by the user or customer although other stakeholders may be involved as well.
The goal of acceptance testing is to establish confidence in the system.

Source

How will you find an element using Selenium?

In Selenium every object or control in a web page is referred as an elements, there are different ways to find an element in a web page:

  • Id
  • Name
  • TagName
  • CssSelector
  • LinkText
  • PartialLinkText
  • XPath
  • ClassName

Explain what is assertion in Selenium

The assertion is used to verify a point in a test case. It verifies that the state of the application conforms to expectations.

What are the types of assertion?

Assertion types in NUnit:

  • Equals
  • Pass
  • Fail
  • Ignore
  • Inconclusive
  • That
  • Throws
  • DowsNotThrow
  • IsTrue
  • IsFalse
  • IsNotNull
  • IsNull
  • AreEqual
  • AreNotEqual
  • AreSame
  • AreNotSame
  • IsNaN
  • IsEmpty
  • IsNotEmpty
  • IsAssignableFrom
  • IsInstanceOf
  • Greater
  • GreaterOrEqual
  • Less
  • LEssOrEqual
  • Contains

Mention what is the use of X-path?

It is used to find elements on web page. It is another for of CSS selectors. It is a language to address parts of XML documents.

List out the technical challenges with Selenium?

  • only for web applications
  • have to depend on third party tools for reporting related capabilities
  • hard to maintain element selectors during development

What is the difference between Implicit wait and Explicit wait?

Implicit Wait

Sets a timeout for all Web Element searches. It will try looking for element presence again and again before throwing a NoSuchElementException for the specified amount of time.

Explicit Wait

It is a timeout for a particular operation.

Explain what is the difference between FindElement() and FindElements()?

FindElement()

It finds the first element within the current page using the given selector. It returns a single WebElement.

FindElements()

It finds all elements within the current page using the given selector. It returns a list of web elements as IEnumerable<IWebElement>.

How you can fill the text box using WebDriver?

You can find proper element and send text for it:

driver.FindElement(By.id(“textBoxId”)).sendKeys(“value”);

How you can perform double click using WebDriver?

You can perform double click using:

var action = new Actions(driver);
action.DoubleClick(driver.FindElement(By.Id(“elementId”)))
.Perform();

Explain how you can debug the tests in Selenium WebDriver?

  • Place the breakpoint in Visual Studio into the test case code
  • Run test in debug mode
  • Wait for test to stop at breakpoint
  • Enjoy debugging features in Visual Studio

Explain what can cause a Selenium IDE test to fail?

  • selector to element changed and Selenium cannot locate the element
  • selenium waiting to access an element which did not appear on the web page and the operation timed out
  • when an assertion fails

What are the different types of Drivers available in WebDriver?

The drivers available in WebDriver are:

How to assert the title of the web page?

You can assert page title using Title property of driver object.

Assert.AreEqual(driver.Title, “Page Title”);

How to retrieve CSS properties of an element?

The values of the CSS property can be retrieved using a GetCssValue method:

driver.FindElement(By.Id(“elementId”)).GetCssValue(“font-size”);

Make sure that your application works on multiple browsers

Make sure that your application works on multiple browsers

e2e_every

Sometimes we need to create a system as a public web application. We may even need to create it for the broad group of customers. In some cases, we need to develop it working on many different types of browser versions. In usual development, we would be forced to open the website in each browser ant test in manually. It would be very beneficial if we can automate it.

We can perform testing on multiple browsers very simple using Selenium WebDriver. We will just use its ability to works with different drivers and configure them.

Simple test

In the beginning, we will write some basic test for our application. We will test Google Translate page for the simplicity of this example. This test checks if translation works well for some simple word.

[Test]
public void ShouldTranslateWord()
{
    var driver = new ChromeDriver();
    driver.Navigate().GoToUrl("https://translate.google.com/#en/pl");

    var sourceTextBox = driver.FindElement(By.Id("source"));
    sourceTextBox.SendKeys("cat");

    driver.FindElement(By.Id("gt-submit")).Click();

    var resultTextBox = driver.FindElement(By.Id("result_box"));

    Assert.AreEqual(resultTextBox.Text, "kot");
}

Test support of different browsers

To be sure that application works fine on many browsers we can define a list of them and use it in a loop. It gives us a flexible method to configure these environments.

[Test]
public void ShouldTranslateWord_MultipleBrowsers()
{
    IWebDriver[] drivers = new IWebDriver[]
    {
        new ChromeDriver(),
        new FirefoxDriver(),
        new InternetExplorerDriver(),
        new SafariDriver(),
        new OperaDriver()
    };

    foreach (var driver in drivers)
    {
        driver.Navigate().GoToUrl("https://translate.google.com/#en/pl");

        var sourceTextBox = driver.FindElement(By.Id("source"));
        sourceTextBox.SendKeys("cat");

        driver.FindElement(By.Id("gt-submit")).Click();

        var resultTextBox = driver.FindElement(By.Id("result_box"));

        Assert.AreEqual(resultTextBox.Text, "kot");
    }
}

It is very easy and straightforward approach to this problem. However, it causes that we have to modify each test in a repeated way.

Test for different browser settings

We may make this test strategy more adjustable using TestCaseSource attribute of NUnit library. Create a list of preconfigured web drivers as a source of our contexts. It will be the only place to put logic related drivers creation and selection.

public static IEnumerable<IWebDriver> TestingWebDrivers
{
    get {
        ChromeOptions options = new ChromeOptions();
        options.AddExtensions("/path/to/extension.crx");

        Proxy proxy = new Proxy();
        proxy.HttpProxy = "myhttpproxy:3337";

        var options2 = new ChromeOptions();
        options2.AddAdditionalCapability("proxy", proxy);

        return new IWebDriver[]
        {
            // regular Chrome
            new ChromeDriver(),
            // Chrome with extension
            new ChromeDriver(options),
            // Chrome behind proxy
            new ChromeDriver(options2),

            new FirefoxDriver(),
            new OperaDriver(),
            new SafariDriver()
        };
    }
} 

We defined driver for four different browsers and three drivers for Chrome with different capabilities. You can extend this list as you want, but you should be aware that the more drivers are present on the list, the more time consuming will be test execution.

We can modify our test a bit, using this data source:

[Test, TestCaseSource("TestingWebDrivers")]
public void ShouldTranslateWord(IWebDriver driver)
{
    driver.Navigate().GoToUrl("https://translate.google.com/#en/pl");

    var sourceTextBox = driver.FindElement(By.Id("source"));
    sourceTextBox.SendKeys("cat");

    driver.FindElement(By.Id("gt-submit")).Click();

    var resultTextBox = driver.FindElement(By.Id("result_box"));

    Thread.Sleep(500);

    Assert.AreEqual(resultTextBox.Text, "kot");
}

[Test, TestCaseSource("TestingWebDrivers")]
public void ShouldDoOther(IWebDriver driver)
{
    // ....
}

We just need to add a TestCaseSource attribute to each test method and pass a parameter with IWebDriver implementation. We will be able to use a driver from method parameter and NUnit will take care of the rest.

End-to-end testing mail course

End-to-end testing mail course

Some time ago I noticed some interesting fact. Developing application in the real world doesn’t mean just writing code. Definitely not in modern development. Before we start coding, we have to know what we want to create and how it should look like. Moreover, after the writing code, we ensure that it works fine. It doesn’t have bugs, it can handle the incoming traffic, and so on … After creating the project we have plenty of work to do. When we realize it, we can discover that there exist some factors which may decide on your project long term success. Some of the are listed below:

  • testing (to ensure quality)
  • logging (to monitor bugs easily)
  • automatic deployment (to fix bugs quickly)

That’s why I though that it is important to know how to apply these factors to your project. The first subject is testing. Generally, testing is a very wide topic and we could discuss it for a long time. However, it exists one test type, which is quite easy and fast to create but it can give you many benefits.

This is an end-to-end testing. We can create is using the Selenium WebDriver tool in .NET environment. This is very easy to learn and it has many capabilities. That’s why I decide to create an email course to learn, how to utilize their potential.

Sign up for free end-to-end testing training

Learn how to create end-to-end tests for your applications from the beginning to mobile testing
Name
Email address

This is a 7 weeks course. It starts 15th February. If you are interested in writing end-to-end tests, you can sing into it on above form. You will get one lesson every week on Thursday. Every lesson is designed to teach you a particular skill. It also provides an exercise to train your new gained knowledge.

It will consist of following subjects:

  1. Testing static pages
  2. Making actions on page
  3. Checking advanced page elements properties
  4. Page Objects pattern
  5. Making advanced actions
  6. Testing mobile friendly pages
  7. Waiting for actions
PageObject use – is it rewarding?

PageObject use – is it rewarding?

PageObject use

In this post, I want to show you the principles of the most useful pattern for end-to-end testing. I will use Selenium for all examples. I assume that you know this tool. It would be easier to explain some things.

The biggest problem with Selenium

Selenium WebDriver gives us powerful tools for writing end-to-end tests. We can imitate the user behavior on every aspect. However, it doesn’t come without some pain. It has several problems that testers must deal:

  • code repeats between different tests
  • it is hard to read long test scenarios
  • tests are high sensitive to changes in the page structure

The more tests you write, the more painful would be above problems. We can reduce them in many mans, but it exists one recommended method, which can help you with most of them.

Page Objects pattern

It is a design pattern considered as a best practice during test creation process. It consists of one fundamental concept. Actions and data available on a web page can be represented as class members. Therefore we can manipulate the page just by using class and its properties.

Let’s take a Google Translate Page as an example.

translate_page_object

For this page, we should create following Page Object class.

public class TranslatePage
{
    private IWebDriver _driver;

    public TranslatePage(IWebDriver driver)
    {
        this._driver = driver;
    }

    public IWebElement SourceText {
        get {
            return _driver.FindElement(By.Id("sourceTextId"));
        }
    }

    public IWebElement TargetText {
        get {
            return _driver.FindElement(By.Id("targetTextId"));
        }
    }

    public IWebElement TranslateButton {
        get {
            return _driver.FindElement(By.Id("translateButtonId"));
        }
    }
}

As you can see, it consists of 3 properties. Each one corresponds to one functional element on the page. If we want to use these elements in our test we can use suitable properties.

var translatePage = new TranslatePage(driver);

translatePage.SourceText.SendKeys("cat");

translatePage.TranslateButton.Click();

Assert.AreEqual(translatePage.TargetText.Text, "kot");

As you see if we have page objects created earlier and we want to write next test scenarios, it will be a piece of cake.

Light and dark sides of Page Objects

This pattern looks excellent at first sight. After working a bit longer with it, we can notice additional advantages.

  • easy to maintenance – we have one place to define selector for the particular element,
  • low level of redundancy – we don’t have to repeat the FindElement code,
  • scalable solution – when we create a Page Object, we can use it without additional time effort,
  • a natural approach to programmers – it is very easy for developers to set up and use programming structures,
  • encapsulate functionality – it separates application modules. If we modify some module, there is a limited space when we need to check selectors.

However, it also comes with some disadvantages. There are mainly related with a time cost.

  • the higher cost of Page Objects creation – initial cost of creation necessary Page Objects is higher than creating a regular test. It pays back in future.
  • requires programming skills – due to use class structures

When it pays back?

It is the most important question for this pattern. It all depends on different things.

If we want to create just a few test cases, probably it won’t be many profits from using Page Objects.

In the other case, if we plan to test multiple test cases for a simple page it worth to use it. In that case, we should start creating them at the beginning of end-to-end testing and use this approach entirely.

The situation changes when we decide to use Page Objects generators. But this is a topic for next post.

Sign up for free end-to-end testing training

Learn how to create end-to-end tests for your applications from the beginning to mobile testing
Name
Email address
cover6_small End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.
This book is a complete guide to Selenium tools. Both for beginners and experienced developers.
It just makes sure that system works fine, saving your time after all.

See more

Simple method to improve application quality

Simple method to improve application quality

e2e_improve_quality

Some times ago I faced the problem with the low quality of the customer’s demonstration. When we try to present our application for the wider audience, we have problems with doing some basic scenarios again and again. We thought that we well tested this functionality. We were using it every day, and everybody in our team was checking it after each change. However, the application functionality still evolving. Moreover, any modification of the code may result in changing final functionality. Because of many changes before demo we can’t test it enough to be 100% sure that everything works fine.

We had a set of unit tests, but even this doesn’t secure us from failures. Fortunately, we discovered a very simple tool that helps us.

What saves our application?

After one presentation we decide to start looking for the solution for us. We found it fast. It was end-to-en testing. When we read more about it, we concluded that it is a perfect tool for us. We create a set of tests that covers most scenarios used in demonstrations. It gives us confidence that everything works. Our primary problem was solved.

Other end-to-end testing advantages

Beside of the help during the presentation with customers, end-to-end testing provides a lot of possibilities.

  • increase trust to application development team – shows managers, that application works on every time,
  • can test the behavior of different browsers and environments fast – we can configure these tests to run against different machine configurations. It can significantly reduce the amount of time needed to test before release,
  • can examine if the web application is fully accessible – accessibility is quite underestimated part of our applications, but it is not hard to achieve it. With a tool that can point us accessibility errors, we can do this much faster

Of course, it is high-level tests, and it shouldn’t replace unit tests for our application. They should only be a supplement to them for ensuring that system works as a whole.

Sign up for free end-to-end testing training

Learn how to create end-to-end tests for your applications from the beginning to mobile testing
Name
Email address
cover6_small End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.
This book is a complete guide to Selenium tools. Both for beginners and experienced developers.
It just makes sure that system works fine, saving your time after all.

See more

ProtractorJS + TypeScript – tutorial

ProtractorJS + TypeScript – tutorial

e2e_protractor

ProtractorJs is an end-to-end framework written for AngularJS applications. It uses Selenium WebDriver features internally. The main advantage of the regular WebDriver test is an ability to cooperate well with SPA application. The main problem with this type of systems is asynchronization. Single Page Applications change their look without reloading the whole page. So it is very hard to determine when the application is ready for action. Fortunately, Protractor knows the Angular internal mechanisms, so he is aware when application finish all operations.

The biggest problem with this framework is writing them in JavaScript. It is very powerful language, but not efficient enough for writing fast and without errors. Moreover, because it is not typed language and we don’t rely on IntelliSense in the editor, starting to work in the new library is hard. The best solution for this would be TypeScript. It is actively developed superset of JavaScript that gives us a power of static typing and hints in code. That’s why I decided to combine each other.

In this example, we will use a combination of TypeScript, Gulp and Jasmine to run Protractor tests.

Installing dependencies

Before start ensure that we have NodeJs installed. Then we can configure our build system. We will use gulp to do this. If we already have an existing project, we will need to modify our package.json and gulpfile.js files.

However, let’s start with an empty project.

At first create package.json file using

npm init

Ensure that you install following packages globally:

npm install -g gulp typings typescript

Then we can install all dependencies used in this tutorial.

npm install browser-sync gulp gulp-protractor gulp-typescript typescript --save-dev

To use TypeScript features correctly, we will also need definitions for our library

typings install dt~angular-protractor --global --save

Additionally, we would need to add one extra typing. Most of the basic TypeScript definitions will be loaded automatically from npm packages, but this one is an exception because we want to choose the specific version.

npm install @types/selenium-webdriver@2.44.* --save-dev

Our package.json the file should contains following dependencies.

"devDependencies": {
    "@types/selenium-webdriver": "^2.44.29",
    "browser-sync": "^2.17.3",
    "gulp": "^3.9.1",
    "gulp-protractor": "^3.0.0",
    "gulp-typescript": "^3.0.2",
    "typescript": "^2.0.3"
}

Simple test

Out development environment is configured now. We can create a simple end-to-end test. It is a sample test from Protractor website.

e2e/test.ts file

describe('angularjs homepage todo list', function() {
    it('should add a todo', function() {
        browser.get('https://angularjs.org');

        element(by.model('todoList.todoText')).sendKeys('write first protractor test');
        element(by.css('[value="add"]')).click();

        var todoList = element.all(by.repeater('todo in todoList.todos'));
        expect(todoList.count()).toEqual(3);
        expect(todoList.get(2).getText()).toEqual('write first protractor test');

        // You wrote your first test, cross it off the list
        todoList.get(2).element(by.css('input')).click();
        var completedAmount = element.all(by.css('.done-true'));
        expect(completedAmount.count()).toEqual(2);
    });
});

As you can see, it enters the AngularJs website and tries to add a task to the sample page. It is a perfect example for our configuration case.

To execute this test, we will need to configure a method how Protractor should search for them.

protractor.conf.js file

'use strict';

exports.config = {

    // Capabilities to be passed to the WebDriver instance.
    capabilities: {
        'browserName': 'chrome'
    },

    baseUrl: 'http://localhost:3000',

    specs: ['tmp/e2e/**/*.js'],

    jasmineNodeOpts: {
        showColors: true,
        defaultTimeoutInterval: 10000
    }
};

With these two files, we can try to run Protractor manually. However, it will not be a success, because we don’t have JavaScript files in a proper place yet.

Configure build system

As the last step, we will need to setup a Gulp task. We will define it in a separate file and include into the main gulpfile.js file.

This task compile TypeScript tests put them in specified directory and execute Protractor tests using given configuration file. It sounds quite easy. Below I show you complete code of this solution.

e2e.js file

'use strict';

var path = require('path');
var gulp = require('gulp');
var conf = require('./conf');

//var browserSync = require('browser-sync');

var protractor = require('gulp-protractor');
var ts = require('gulp-typescript');

// Downloads the selenium webdriver
gulp.task('webdriver-update', protractor.webdriver_update);

var tsProject = ts.createProject('./tsconfig.json');

gulp.task('webdriver-standalone', protractor.webdriver_standalone);

function runProtractor(done) {
    var params = process.argv;
    var args = params.length > 3 ? [params[3], params[4]] : [];

    gulp.src([conf.paths.typings, path.join(conf.paths.e2e, '/**/*.ts')])
        .pipe(ts({
            out: 'output.js'
        }))
        //protractor needs files on disk, cannot get them from stream
        .pipe(gulp.dest(conf.paths.e2eOutput)) 
        .pipe(protractor.protractor({
            configFile: 'protractor.conf.js',
            args: args
        }))
        .on('error', function (err) {
            // Make sure failed tests cause gulp to exit non-zero
            throw err;
        })
        .on('end', function () {
            // Close browser sync server
            //browserSync.exit();
            done();
        });
}

gulp.task('e2e', ['protractor:src']);
gulp.task('protractor:src', ['webdriver-update'], runProtractor);

You can add it to the main gulp configuration file using one line.

gulpfile.js file

...
require('./gulp_tasks/e2e.js');
...

After all these configurations we are ready to execute our test. We should execute above task to see how tests click on the page.

gulp protractor

That’s it. This configuration is very easy, but you can find it hard to manage type definitions for these libraries. I attach some additional resources to help you with it.

Source code for this post:

https://github.com/suvroc/protractor-typescript-template

Other resources:

https://github.com/angular/protractor-cookbook/tree/master/protractor-typescript
https://github.com/skovmand/angular-protractor-typescript

You can find more information about end-to-end testing in my book and newsletter

Sign up for free end-to-end testing training

Learn how to create end-to-end tests for your applications from the beginning to mobile testing
Name
Email address
cover6_small End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.
This book is a complete guide to Selenium tools. Both for beginners and experienced developers.
It just makes sure that system works fine, saving your time after all.

See more

What developers should know about acceptance tests details

What developers should know about acceptance tests details

e2e_developers

Many people think that testing is a subject that only testers should worry about them. I completely disagree with that statement. I can event say that developers can write better tests technically and faster. They know the application. They have technical knowledge about creating reusable code and good architecture. Nevertheless, they shouldn’t write an end to end acceptance tests. They are too focused on functionalists that write. So it is possible that they repeat the testing patterns from the development time, not on overall application image.

That’s why it is important that test should be written by other people that write the functionality. It’s the time when separate test team comes into the scene. They are focused only on testing and don’t have a deeper look into the technical details. That’s very beneficial.

How developers and testers can cooperate

Testers use the result of developers work. Usually, they can’t modify the application code, so this is very important to write it in a proper way.

When we asked testers, which part of the test writing process takes the most of the time? It works on defining and testing page element selectors. That’s the reason why developers should pay particular attention to design HTML correctly.

They should remember about few simple rules:

  • always use name attribute for input elements
  • use id if an element is unique on the whole page and its subpages
  • use class always if the element contains any data relevant to the user or it can make an action
  • use proper HTML semantic elements to mark page areas

As you can see this is only four rules. All of them is very simple and helps to organize HTML document. Try to think about it when you write HTML next time. Maybe you have some other methods to organize HTML structure?

You can find more information about end-to-end testing in my book and newsletter

[mc4wp_form id=”510″]

Selenium accessibility testing

Selenium accessibility testing

accessibility testing

Beside of the normal usage, Selenium WebDriver has a lot of extra capabilities. The one of the most clever and not usual use of end-to-end testing is checking if web page fulfills accessibility testing rules.

Accessibility is a very broad topic, but there are some guidelines how to create the web application, that people with some disabilities (visual, hearing or manual) can use it. If we learn how to do this and on what elements we should focus to provide the accessible website, it turns out to be very simple. There is few number of rules keep and that’s it. Screen readers or other software will take care of the rest.

Checking compliance with Accessibility rules

However, it would be very beneficial to have a tool to automatic check compliance with accessibility standards. Of course, we shouldn’t implement all rules by ourselves. The best idea would be to use some library for this purpose. In our example, we will use a WAVE Evaluator. It is a Chrome extension, but we can use it directly from WebDriver test code.

We can install this extension to Chrome and check how it works.

wave_translate

It marks elements that do not comply with the rules. It highlights errors and warnings, but only the firs one will be important for us.

Accessibility testing automation

Our test will use this WAVE Evaluator extension. It will have simple logic.

  1. Open web page to check
  2. Start WAVE Evaluator
  3. Check if it highlights some errors

Looks very simple. The hardest part would be running this extension directly from test code. At first, we should download the extension in crx format. Place it in the main project directory.

Then we can use following code:

[Test]
public void AccessibilityTest()
{
    // 1
    ChromeOptions options = new ChromeOptions();
    options.AddExtension(@"WAVE-Evaluation-Tool_v1.0.1.crx");
    var driver = new ChromeDriver(options);

    // 2 - setup key shourtcut for extension
    driver.Navigate().GoToUrl("chrome://extensions-frame/");
    driver.FindElement(By.XPath("//a[@class='extension-commands-config']"))
        .Click();
    driver.FindElement(By.XPath("//span[@class='command-shortcut-text']"))
        .SendKeys(Keys.Control + "m");
    driver.FindElement(By.Id("extension-commands-dismiss"))
        .Click();

    // 3
    driver.Navigate().GoToUrl("http://www.google.pl");
    // 4 - open WAVE extension 
    new Actions(driver).KeyDown(Keys.Control).SendKeys("m").Build().Perform();

    // 5
    var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
    wait.Until(ExpectedConditions.ElementExists(By.ClassName("wave5icon")));

    // 6
    var waveTips = driver.FindElements(By.ClassName("wave5icon"));
    if (waveTips.Count == 0) Assert.Fail(
        "Could not locate any WAVE validations - " +
        "please ensure that WAVE is installed correctly");
    foreach (var waveTip in waveTips)
    {
        if (!waveTip.GetAttribute("alt").StartsWith("ERROR")) continue;

        var fileName = String.Format("{0}{1}{2}", 
            "WAVE", DateTime.Now.ToString("HHmmss"), ".png");
        var screenShot = ((ITakesScreenshot)driver).GetScreenshot();
        screenShot.SaveAsFile(
            Path.Combine(System.IO.Path.GetTempPath(), fileName), ImageFormat.Png);
        driver.Close();
        Assert.Fail(
            "WAVE errors were found on the page. Please see screenshot for details");
    }
    driver.Close();
}

It consists of 6 parts:

  1. Open ChromeDriver with appropriate extension enabled
  2. Setup keyboard shortcut to run WAVE Evaluator
  3. Open testing website
  4. Press keyboard shortcut to run evaluator
  5. Wait for WAVE to finish analysis
  6. Check if exists any highlight with category – error

You can see also a short example how we can use extensions in our tests above. The idea to do this is taken from this blog post. I improve it and make it work for current WebDriver versions.

You can find more information about end-to-end testing in my book

The power of end-to-end testing

cover6_small
See more
End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.

This book is a complete guide to Selenium tools. Both for beginners and experienced developers.

It just makes sure that system works fine, saving your time after all.

3 questions to ask before you start testing

3 questions to ask before you start testing

start testing

Before we start testing, we should ask yourself some very important questions.

  • Why do we want to test? – is helps us to answer the next question
  • When do we need to start testing?
  • What do we want to test?

We will try to address them here.

Why we want to test?

In most of the cases, we will start end-to-end testing for one of the following reason:

  • to increase application quality
  • to reduce the risk of failure for customer
  • to check functionalists after changes
  • simulate real scenarios

You should define your own reasons to test before the start. It helps you to choose the most suitable scope of testing and effort that you want to spend on them. Your reasons may be different, but based on my experience, these are the most common.

When we need to start testing?

In the case of unit testing, it is very easy. They should be created from the start of development. The other situation is for end-to-end and acceptance testing. We can’t write them right at the beginning because we don’t have anything to test. The best moment is when some functionality is fully implemented but before the first presentation for a customer. This is the time when we have a stable version of at least one application functionality and we want to be sure that it works. It is good to do the same for every functionality.

Why shouldn’t we start earlier? Because if we still working on the best solution, the HTML code may change. Each change of web page code may result in test fixing. This is the additional work that we can leave for later.

What we want to test?

This is the most interesting question because it depends on the number of people dedicated to writing tests and their engagement. It will list here the possibilities depending on the size of the dedicated testing team. I will give you an example based on shopping application.

  • none – (if you don’t have the separate testing team) – you should focus only on few the most crucial happy path in applications. For example – adding item to cart and making an order
  • 1-2 – you should test only important functionality. For example – changing password or checking order details
  • 3 or more – you should have tests for each user story in the project. For example sorting orders in the admin panel.

This is only my point of view. Do you agree with it or maybe you would make different decisions?

You can find more information about end-to-end testing in my book

The power of end-to-end testing

cover6_small
See more
End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.

This book is a complete guide to Selenium tools. Both for beginners and experienced developers.

It just makes sure that system works fine, saving your time after all.