Browsed by
Month: November 2016

Simple method to improve application quality

Simple method to improve application quality

e2e_improve_quality

Some times ago I faced the problem with the low quality of the customer’s demonstration. When we try to present our application for the wider audience, we have problems with doing some basic scenarios again and again. We thought that we well tested this functionality. We were using it every day, and everybody in our team was checking it after each change. However, the application functionality still evolving. Moreover, any modification of the code may result in changing final functionality. Because of many changes before demo we can’t test it enough to be 100% sure that everything works fine.

We had a set of unit tests, but even this doesn’t secure us from failures. Fortunately, we discovered a very simple tool that helps us.

What saves our application?

After one presentation we decide to start looking for the solution for us. We found it fast. It was end-to-en testing. When we read more about it, we concluded that it is a perfect tool for us. We create a set of tests that covers most scenarios used in demonstrations. It gives us confidence that everything works. Our primary problem was solved.

Other end-to-end testing advantages

Beside of the help during the presentation with customers, end-to-end testing provides a lot of possibilities.

  • increase trust to application development team – shows managers, that application works on every time,
  • can test the behavior of different browsers and environments fast – we can configure these tests to run against different machine configurations. It can significantly reduce the amount of time needed to test before release,
  • can examine if the web application is fully accessible – accessibility is quite underestimated part of our applications, but it is not hard to achieve it. With a tool that can point us accessibility errors, we can do this much faster

Of course, it is high-level tests, and it shouldn’t replace unit tests for our application. They should only be a supplement to them for ensuring that system works as a whole.

Sign up for free end-to-end testing training

Learn how to create end-to-end tests for your applications from the beginning to mobile testing
Name
Email address
cover6_small End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.
This book is a complete guide to Selenium tools. Both for beginners and experienced developers.
It just makes sure that system works fine, saving your time after all.

See more

ProtractorJS + TypeScript – tutorial

ProtractorJS + TypeScript – tutorial

e2e_protractor

ProtractorJs is an end-to-end framework written for AngularJS applications. It uses Selenium WebDriver features internally. The main advantage of the regular WebDriver test is an ability to cooperate well with SPA application. The main problem with this type of systems is asynchronization. Single Page Applications change their look without reloading the whole page. So it is very hard to determine when the application is ready for action. Fortunately, Protractor knows the Angular internal mechanisms, so he is aware when application finish all operations.

The biggest problem with this framework is writing them in JavaScript. It is very powerful language, but not efficient enough for writing fast and without errors. Moreover, because it is not typed language and we don’t rely on IntelliSense in the editor, starting to work in the new library is hard. The best solution for this would be TypeScript. It is actively developed superset of JavaScript that gives us a power of static typing and hints in code. That’s why I decided to combine each other.

In this example, we will use a combination of TypeScript, Gulp and Jasmine to run Protractor tests.

Installing dependencies

Before start ensure that we have NodeJs installed. Then we can configure our build system. We will use gulp to do this. If we already have an existing project, we will need to modify our package.json and gulpfile.js files.

However, let’s start with an empty project.

At first create package.json file using

npm init

Ensure that you install following packages globally:

npm install -g gulp typings typescript

Then we can install all dependencies used in this tutorial.

npm install browser-sync gulp gulp-protractor gulp-typescript typescript --save-dev

To use TypeScript features correctly, we will also need definitions for our library

typings install dt~angular-protractor --global --save

Additionally, we would need to add one extra typing. Most of the basic TypeScript definitions will be loaded automatically from npm packages, but this one is an exception because we want to choose the specific version.

npm install @types/selenium-webdriver@2.44.* --save-dev

Our package.json the file should contains following dependencies.

"devDependencies": {
    "@types/selenium-webdriver": "^2.44.29",
    "browser-sync": "^2.17.3",
    "gulp": "^3.9.1",
    "gulp-protractor": "^3.0.0",
    "gulp-typescript": "^3.0.2",
    "typescript": "^2.0.3"
}

Simple test

Out development environment is configured now. We can create a simple end-to-end test. It is a sample test from Protractor website.

e2e/test.ts file

describe('angularjs homepage todo list', function() {
    it('should add a todo', function() {
        browser.get('https://angularjs.org');

        element(by.model('todoList.todoText')).sendKeys('write first protractor test');
        element(by.css('[value="add"]')).click();

        var todoList = element.all(by.repeater('todo in todoList.todos'));
        expect(todoList.count()).toEqual(3);
        expect(todoList.get(2).getText()).toEqual('write first protractor test');

        // You wrote your first test, cross it off the list
        todoList.get(2).element(by.css('input')).click();
        var completedAmount = element.all(by.css('.done-true'));
        expect(completedAmount.count()).toEqual(2);
    });
});

As you can see, it enters the AngularJs website and tries to add a task to the sample page. It is a perfect example for our configuration case.

To execute this test, we will need to configure a method how Protractor should search for them.

protractor.conf.js file

'use strict';

exports.config = {

    // Capabilities to be passed to the WebDriver instance.
    capabilities: {
        'browserName': 'chrome'
    },

    baseUrl: 'http://localhost:3000',

    specs: ['tmp/e2e/**/*.js'],

    jasmineNodeOpts: {
        showColors: true,
        defaultTimeoutInterval: 10000
    }
};

With these two files, we can try to run Protractor manually. However, it will not be a success, because we don’t have JavaScript files in a proper place yet.

Configure build system

As the last step, we will need to setup a Gulp task. We will define it in a separate file and include into the main gulpfile.js file.

This task compile TypeScript tests put them in specified directory and execute Protractor tests using given configuration file. It sounds quite easy. Below I show you complete code of this solution.

e2e.js file

'use strict';

var path = require('path');
var gulp = require('gulp');
var conf = require('./conf');

//var browserSync = require('browser-sync');

var protractor = require('gulp-protractor');
var ts = require('gulp-typescript');

// Downloads the selenium webdriver
gulp.task('webdriver-update', protractor.webdriver_update);

var tsProject = ts.createProject('./tsconfig.json');

gulp.task('webdriver-standalone', protractor.webdriver_standalone);

function runProtractor(done) {
    var params = process.argv;
    var args = params.length > 3 ? [params[3], params[4]] : [];

    gulp.src([conf.paths.typings, path.join(conf.paths.e2e, '/**/*.ts')])
        .pipe(ts({
            out: 'output.js'
        }))
        //protractor needs files on disk, cannot get them from stream
        .pipe(gulp.dest(conf.paths.e2eOutput)) 
        .pipe(protractor.protractor({
            configFile: 'protractor.conf.js',
            args: args
        }))
        .on('error', function (err) {
            // Make sure failed tests cause gulp to exit non-zero
            throw err;
        })
        .on('end', function () {
            // Close browser sync server
            //browserSync.exit();
            done();
        });
}

gulp.task('e2e', ['protractor:src']);
gulp.task('protractor:src', ['webdriver-update'], runProtractor);

You can add it to the main gulp configuration file using one line.

gulpfile.js file

...
require('./gulp_tasks/e2e.js');
...

After all these configurations we are ready to execute our test. We should execute above task to see how tests click on the page.

gulp protractor

That’s it. This configuration is very easy, but you can find it hard to manage type definitions for these libraries. I attach some additional resources to help you with it.

Source code for this post:

https://github.com/suvroc/protractor-typescript-template

Other resources:

https://github.com/angular/protractor-cookbook/tree/master/protractor-typescript
https://github.com/skovmand/angular-protractor-typescript

You can find more information about end-to-end testing in my book and newsletter

Sign up for free end-to-end testing training

Learn how to create end-to-end tests for your applications from the beginning to mobile testing
Name
Email address
cover6_small End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.
This book is a complete guide to Selenium tools. Both for beginners and experienced developers.
It just makes sure that system works fine, saving your time after all.

See more

What developers should know about acceptance tests details

What developers should know about acceptance tests details

e2e_developers

Many people think that testing is a subject that only testers should worry about them. I completely disagree with that statement. I can event say that developers can write better tests technically and faster. They know the application. They have technical knowledge about creating reusable code and good architecture. Nevertheless, they shouldn’t write an end to end acceptance tests. They are too focused on functionalists that write. So it is possible that they repeat the testing patterns from the development time, not on overall application image.

That’s why it is important that test should be written by other people that write the functionality. It’s the time when separate test team comes into the scene. They are focused only on testing and don’t have a deeper look into the technical details. That’s very beneficial.

How developers and testers can cooperate

Testers use the result of developers work. Usually, they can’t modify the application code, so this is very important to write it in a proper way.

When we asked testers, which part of the test writing process takes the most of the time? It works on defining and testing page element selectors. That’s the reason why developers should pay particular attention to design HTML correctly.

They should remember about few simple rules:

  • always use name attribute for input elements
  • use id if an element is unique on the whole page and its subpages
  • use class always if the element contains any data relevant to the user or it can make an action
  • use proper HTML semantic elements to mark page areas

As you can see this is only four rules. All of them is very simple and helps to organize HTML document. Try to think about it when you write HTML next time. Maybe you have some other methods to organize HTML structure?

You can find more information about end-to-end testing in my book and newsletter

[mc4wp_form id=”510″]

Selenium accessibility testing

Selenium accessibility testing

accessibility testing

Beside of the normal usage, Selenium WebDriver has a lot of extra capabilities. The one of the most clever and not usual use of end-to-end testing is checking if web page fulfills accessibility testing rules.

Accessibility is a very broad topic, but there are some guidelines how to create the web application, that people with some disabilities (visual, hearing or manual) can use it. If we learn how to do this and on what elements we should focus to provide the accessible website, it turns out to be very simple. There is few number of rules keep and that’s it. Screen readers or other software will take care of the rest.

Checking compliance with Accessibility rules

However, it would be very beneficial to have a tool to automatic check compliance with accessibility standards. Of course, we shouldn’t implement all rules by ourselves. The best idea would be to use some library for this purpose. In our example, we will use a WAVE Evaluator. It is a Chrome extension, but we can use it directly from WebDriver test code.

We can install this extension to Chrome and check how it works.

wave_translate

It marks elements that do not comply with the rules. It highlights errors and warnings, but only the firs one will be important for us.

Accessibility testing automation

Our test will use this WAVE Evaluator extension. It will have simple logic.

  1. Open web page to check
  2. Start WAVE Evaluator
  3. Check if it highlights some errors

Looks very simple. The hardest part would be running this extension directly from test code. At first, we should download the extension in crx format. Place it in the main project directory.

Then we can use following code:

[Test]
public void AccessibilityTest()
{
    // 1
    ChromeOptions options = new ChromeOptions();
    options.AddExtension(@"WAVE-Evaluation-Tool_v1.0.1.crx");
    var driver = new ChromeDriver(options);

    // 2 - setup key shourtcut for extension
    driver.Navigate().GoToUrl("chrome://extensions-frame/");
    driver.FindElement(By.XPath("//a[@class='extension-commands-config']"))
        .Click();
    driver.FindElement(By.XPath("//span[@class='command-shortcut-text']"))
        .SendKeys(Keys.Control + "m");
    driver.FindElement(By.Id("extension-commands-dismiss"))
        .Click();

    // 3
    driver.Navigate().GoToUrl("http://www.google.pl");
    // 4 - open WAVE extension 
    new Actions(driver).KeyDown(Keys.Control).SendKeys("m").Build().Perform();

    // 5
    var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
    wait.Until(ExpectedConditions.ElementExists(By.ClassName("wave5icon")));

    // 6
    var waveTips = driver.FindElements(By.ClassName("wave5icon"));
    if (waveTips.Count == 0) Assert.Fail(
        "Could not locate any WAVE validations - " +
        "please ensure that WAVE is installed correctly");
    foreach (var waveTip in waveTips)
    {
        if (!waveTip.GetAttribute("alt").StartsWith("ERROR")) continue;

        var fileName = String.Format("{0}{1}{2}", 
            "WAVE", DateTime.Now.ToString("HHmmss"), ".png");
        var screenShot = ((ITakesScreenshot)driver).GetScreenshot();
        screenShot.SaveAsFile(
            Path.Combine(System.IO.Path.GetTempPath(), fileName), ImageFormat.Png);
        driver.Close();
        Assert.Fail(
            "WAVE errors were found on the page. Please see screenshot for details");
    }
    driver.Close();
}

It consists of 6 parts:

  1. Open ChromeDriver with appropriate extension enabled
  2. Setup keyboard shortcut to run WAVE Evaluator
  3. Open testing website
  4. Press keyboard shortcut to run evaluator
  5. Wait for WAVE to finish analysis
  6. Check if exists any highlight with category – error

You can see also a short example how we can use extensions in our tests above. The idea to do this is taken from this blog post. I improve it and make it work for current WebDriver versions.

You can find more information about end-to-end testing in my book

The power of end-to-end testing

cover6_small
See more
End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.

This book is a complete guide to Selenium tools. Both for beginners and experienced developers.

It just makes sure that system works fine, saving your time after all.

3 questions to ask before you start testing

3 questions to ask before you start testing

start testing

Before we start testing, we should ask yourself some very important questions.

  • Why do we want to test? – is helps us to answer the next question
  • When do we need to start testing?
  • What do we want to test?

We will try to address them here.

Why we want to test?

In most of the cases, we will start end-to-end testing for one of the following reason:

  • to increase application quality
  • to reduce the risk of failure for customer
  • to check functionalists after changes
  • simulate real scenarios

You should define your own reasons to test before the start. It helps you to choose the most suitable scope of testing and effort that you want to spend on them. Your reasons may be different, but based on my experience, these are the most common.

When we need to start testing?

In the case of unit testing, it is very easy. They should be created from the start of development. The other situation is for end-to-end and acceptance testing. We can’t write them right at the beginning because we don’t have anything to test. The best moment is when some functionality is fully implemented but before the first presentation for a customer. This is the time when we have a stable version of at least one application functionality and we want to be sure that it works. It is good to do the same for every functionality.

Why shouldn’t we start earlier? Because if we still working on the best solution, the HTML code may change. Each change of web page code may result in test fixing. This is the additional work that we can leave for later.

What we want to test?

This is the most interesting question because it depends on the number of people dedicated to writing tests and their engagement. It will list here the possibilities depending on the size of the dedicated testing team. I will give you an example based on shopping application.

  • none – (if you don’t have the separate testing team) – you should focus only on few the most crucial happy path in applications. For example – adding item to cart and making an order
  • 1-2 – you should test only important functionality. For example – changing password or checking order details
  • 3 or more – you should have tests for each user story in the project. For example sorting orders in the admin panel.

This is only my point of view. Do you agree with it or maybe you would make different decisions?

You can find more information about end-to-end testing in my book

The power of end-to-end testing

cover6_small
See more
End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.

This book is a complete guide to Selenium tools. Both for beginners and experienced developers.

It just makes sure that system works fine, saving your time after all.

How to choose e2e test language?

How to choose e2e test language?

e2e_language

Some time ago I faced a big dilemma. In our company, we have two teams: back-end and front-end. When we start thinking about end-to-end testing in our current project, the discussion begins. We were thinking about the best technology to do it. We are using AngularJs in this project, so we want to take advantage of Protractor library. It is the library for testing Angular Single Page Applications without waiting after each action.

We have two possibilities.

  • write in TypeScript in JS environment,
  • write in C# and .NET environment

Each one of this approach has its advantages, but I want to present you my opinion on this subject.

.NET and C#

This combination is very easy for beginners. We don’t need too many configurations to start testing. It is a very extensible solution. C# has an important attribute – strong typing. It reduces errors and helps to write tests faster because every error is visible right after it is typed.

Moreover, we can prepare right architecture of test cases and PageObjects using inheritance and object creation.

There are many third party libraries to help us with everyday tasks. For example WebDriver.Support for manipulating select elements. We can also integrate .NET Selenium with many external services like [BrowserStack] for testing on many devices.

There is also good [Protractor.NET] library. It is a port o JS version, but it works pretty well.

It also has a very useful mechanism of debugging. You can stop the test execution at any point and check its state or test for selectors. Debugging is may be invaluable help during the development.

TypeScript

TypeScript is a strongly typed language as well, but in fact, it is much more liberal than C#. It is worse for newcomers, but if you already know what you do, it is very powerful.

Initial configuration of compiling TypeScript and running tests may be problematic even for experienced developers. It differs between versions and front-end environment changes fast. If you have people skilled in this area, you will manage to do this.

TypeScript has a weak mechanism of debugging. Holding on breakpoints is almost impossible. You can do this with debugger command, but you have to use it explicitly in your code. It is annoying if you need to stop on many lines in several executions.

Conclusion

If you have a separate team of testers, you should choose C# as a language for tests. It gives you a powerful object oriented and strongly typed mechanisms, that can speed up your test development and increase their quality.

However, if you don’t have a separate team and front-end team is responsible for end-to-end testing, you should do it in TypeScript. They already know this language and can easily integrate tests into the building or CI workflow.

You can find more information about end-to-end testing in my book

The power of end-to-end testing

cover6_small
See more
End-to-end testing is a great improvement in quality testing automation. It reflects the user experience the most of all tests.

This book is a complete guide to Selenium tools. Both for beginners and experienced developers.

It just makes sure that system works fine, saving your time after all.