You want to be productive when building web applications. That means iterating fast and delivering continuously. As your application grows in complexity, how can you assure its quality - on each build?

Say goodbye to manual regression testing and full mocked unit tests - get started with end-to-end automated UI-testing!

In this blog post I will argue why automated UI-tests should be a part of modern, complex web applications.

End-to-end with Selenium:

This blog post will be the starting point/intro for a series about Selenium tests where speed will be the focal point.

What is automated UI-testing?

UI-testing is automating the web browser to perform actions and asserting the result of these actions.

Basically, these test can simulate any action a user would perform, and assert any feedback the user would get.

Automated UI-testing is not something new, still it seems to me that UI-tests are not commonly used. Why is that? "They take too much time to maintain", "The feedback loop is too long", "We have other types of tests, we don't need more", "They're boring". Heard it before?

If your goal is continuous delivery, can you afford to disregard UI-tests?
Maybe stability and confidence isn't that boring after all?

Real life experience

Once, I was part of a big project where the standard procedure for deploying to production meant taking the web portal offline for a whole weekend. In the weeks before, the test team spent hours testing the solution in several test environments, and during the "offline-weekend" everything was re-tested in the production environment.

Countless hours spent following written, simple procedural test scripts. The same routine for each environment. A tedious and expensive routine which also prevented us from frequent releases because of the protracted release cycle, and that we couldn't really afford the downtime.

How would UI-tests help this project? Obviously the manual regression tests could largley have been replaced with UI-tests. Even if UI-tests are considered slow they would be much faster than manual testing. Also, the test team would free up time they could use for more specific and more advanced testing.

Hopefully we would retain our confidence, spend less time performing the test loop and ultimately achieve more frequent deploys without deteriorating the test quality.

Pros and cons

Mike Cohn has described the concept of the test pyramid in his book "Succeeding with Agile". He describes UI-tests as brittle, expensive and time consuming, with high redundancy. His conclusion is to use as few UI-tests as possible, therefore placing them on top of the test pyramid.

And he's right:

  • They are slow (This will be a topic in a follow up blog post)
  • They are brittle (a small change in the CSS can break several tests, therefore the HTML must be structured in a testable way)
  • They are time consuming to implement and maintain
  • They might cover the same logic several times (although I would argue that this is also dependant on how the tests are written)

The big advantage is that you get full end-to-end testing. A feature not provided by any other type of tests.

In modern web applications logic resides on all levels: Navigation and routing, updating the page without reloading, asynchronous changes, live updating the site using signalR events etc. All of these can be tested using UI testing.

It's important to understand that UI-testing is not limited to the UI. APIs called from the backend, databases with sprocs/triggers etc. The UI-tests can cover the whole application - and that's why it offers a lot of value.

Why should you write UI-tests

If these conditions are met, I highly recommend UI-tests for your application:

  • You deploy to production regularly (or you want to)
  • You spend a lot of time on manual regression testing
  • Your application supports multiple user scenarios for web interaction

UI testing done correctly will give you massive value over time. It will become your safety net for assuring quality - for each build. It will enable you to do frequent deploys, because even if you introduce small bugs (that might pass through manual regression testing anyway) you will have confidence that the core functionality is still working.

Respect the test pyramid, though. You shouldn't write UI-tests for everything.
Instead, think of UI-tests as a sanity test for your solution - a health check for your core functionality.

Involving the test team

Projects of a certain size should have dedicated testers (test team). The testers will usually create the test scripts for manual regression testing, while the developer usually writes his own unit tests and integration tests and what not. Often there are no coherence between the two, and they don't know what the other is testing.

UI testing has the possibility of changing this paradigm. Using tools like SpecFlow, tests can be written in human readable form, enabling the the tester to write the UI-test:

* Given I navigate to "www.novanet.no"
* When I click on menu item "Blog"
* Then I will see 5 blog posts on the first page

In an ideal world the tester can describe the tests like this and have full control over what's being tested. The developer will write the code to tie the test script to actual operations, without really needing to think about the test coverage. If provided a sufficient toolbox of test statements, the testers could even write tests without the developers involvement. This will give the testers ownership for the automated tests.

I have tried this approach in a project and my experience is that this can be very valuable and the test team really enjoys the level of involvement. But it also requires a big effort from the test team which must be accounted for. Writing and maintaining tests are time consuming, and the developer might be faster accomplishing this task.

There are pros and cons to both approaches, so it really depends on what the project wants to achieve. The point is that more UI-tests does not equal less involvement from the test team, instead the test team can focus on other tasks - like creating good test scenarios.

Getting started

Now you know why you should UI-test, but how do you proceed?
Check out this blog post on how to get started writing your own UI-test:
End-to-end with Selenium: Writing your first UI-test