Sep 08 2008
What’s wrong with current web testing approaches
Of late I have been taking a very keen interest in the area of web-testing (I wrote previously on automating testing with selenium ). It is not an easy task to tackle efficiently.
Of course you can get a team of QA’s to write up tests in your favourite web testing tool. How much time/effort/money does this cost? What about when your web application starts to change? What if I change some pages dramatically - how will my tests change?
From my experience the issue isn’t getting tests written up and working. It is more so the maintainability of the tests which determine the success of your web-testing strategy. I have looked at various products and frameworks which are available today in this space and there is a common approach to testing that 99% of them take or suggest to take (via their IDEs and docs).
The approach is quite simple and leverages aspects of the web ‘ecosystem’ -
- html identifiers via ids and/or xpath expressions
- wait times for page loads and ajax
- user actions such as clicks, hover and typing
You write tests using a combination of the above to simulate what a user would do on your web application. To give an example (taken from my previous post) of how this would like look in Selenium -
public class AuthenticationTests extends AbstractSeleniunTest {
@Test
public void testUserLogin() throws InterruptedException {
selenium.open("/");
selenium.click("link=Login");
waitForElement("login-form_j_username", "Login not shown");
selenium.type("login-form_j_username", "xxxx@xxxx.com");
selenium.type("login-form_j_password", "xxxx");
selenium.click("login-form_0");
waitForElement("user-home-link", "User home link not shown");
assertTrue(selenium.isElementPresent("user-home-link"));
}
}
- Line 5: direct references are made to pages to open
- Line 6: things to click by name
- Line 7: wait for something
- Line 12: check if some field is present identified by its id
There is, I think, a very big problem with writing tests in this way. The test cases are very tightly coupled to the html code of the web application. If this code changes - your tests will start to break. This may or may not be a problem for most people however if you are working on a dynamic web application - this is most likely a very big problem. As I alluded to earlier, this problem is not specific to Selenium which I have used to illustrate the example.
Page Object Pattern
A design pattern which is emerging that addresses this issue is the Page Object pattern. I first read about it here and have seen mentions of it else where. As with all design patterns, people are almost certainly already using it (that is where patterns come from) and now has been given a name. Also something to note is the items raised in this post and the pattern itself are not confined to web applications - it covers any GUI based testing.
The ‘Page Object’ pattern is akin to the Facade design pattern from the GoF. The PageObject is a layer of abstraction called which hides away the details of the page implementation. This object then exposes only actions/methods relevant to the tests (e.g. user actions, specific test methods).
The pattern very much follows the principle of seperation of concerns. The tests are not concerned about the ‘id’ of a particular ‘div’ or the name of a button, nor should they be. Instead they should only be concerned with actions a user may do on a page and additional operations required for the test cases (there are exceptions). In this way, the test cases should begin to develop an immunity to changes in the GUI.
Below is an example of the previous test rewritten using the Page Object pattern. I have left out the actual PageObject’s to keep this post to a sane length. Inside the PageObject, the actions and methods are driven by the same web constructs discussed in the beginning of the post (ids, clicks etc). In later posts I will show how to implement the actual PageObject’s and test cases using Selenium.
public class AuthenticationTests extends AbstractSeleniunTest {
@Test
public void testUserLogin() {
IndexPage index = new IndexPage(); // Open home page
LoginPage loginPage = index.loginPage(); // Go to login page
index = loginPage.login("user", "password"); // Login entering user and password
assertTrue(index.isLoggedIn()); // Ensure I am logged in
}
}
- Line 5: IndexPage PageObject opened (i.e. open “/”)
- Line 6: Go to login page (LoginPage PageObject returned)
- Line 7: Login by passing username and password to the login method
- Line 8: check if I am logged in
Looking at the above, it is quite clear that even if the html of my page changes the test case will not break (not a single html related entity is referenced). Test cases won’t break in this case - the PageObject(s) will. This is a very important distinction because the majority of the effort and work will be in the test cases. The PageObject’s will be reused by many tests and should be relatively small classes with simple methods.
Another very important characteristic of the above test case is the language/constructs used in the above test case are ‘natural’ and intuitive. This allows the non-technical to define test cases in a manner very similar to the actual tests themselves. Given the right tools, it would be quite easy for testers and analysts to write test cases using this pattern which are maintainable and more resistant to lower level changes.
As mentioned earlier, I will follow up this post in the next week or so with an actual test case and implementation using Selenium.
Hopefully someone in the field is reading..
