Has anyone told you that software testing can become a nightmare? If someone made such a confession, they must have experienced a situation where coping with complex business logic and emerging technologies was not handled properly. Fortunately, we have a tried-and-tested recipe for preventing that feeling of helplessness and turning the battle with code changes and bugs into a victory of Quality.
Our smart and strong “Code-Accelerator”
After going through a similar (or even identical) process of setting up the test automation framework on a few different projects, we have decided to combine all the building blocks of this process together into a so-called “Code-Accelerator” framework. These components include (but are not limited to):
- Page-Object model
- Rich reporting tool
- API requests and mocking
- UI testing and visual validation
- Data Management
- Parallel test execution
- Cross-browser testing
- Retry logic on failures
The main purpose of having such a codebase is to boost QA productivity from the very beginning of the project. The main tool for achieving this is a platform for adding new automated tests by just pulling the repository from our GitLab domain and executing a few commands, as shown on the screenshot below.
Since I started working as a QA 6 years ago, discussing test automation for web apps always involved bringing up Selenium or one of its flavors. After many challenging but successful test automation ventures with Selenium-based tools, I discovered another platform that brought my automated tests to a whole new level. Although Selenium was a de-facto standard for web test automation for more than a decade, Cypress has already managed to go beyond that in some areas even though it is a relatively new framework (it was launched in October of 2017). One of its main advantages, from my point of view, is the ability to have both UI and API tests written in the same tool without much hassle, all as an out-of-the-box feature.
The key features of our code-accelerator framework
In order to introduce our code-accelerator, I will share the basic structure of the framework where I have implemented some guidelines and patterns that are considered best practices in the field.
Having tests organized into a meaningful structure is one of the main steps in building a robust test automation framework. Although the page-object pattern is not officially recommended by the Cypress founders, I believe that most of the QA engineers would agree that it does make our lives easier. The main purpose of having that in place is to separate the business logic from the implementation details. All that we need and want to see in our tests are the steps that look almost like a plain English sentence in order to focus on test coverage with different scenarios.
Having said that, we implement the methods and corresponding elements in the page class that is later instantiated and called from the test in form of a test step.
This practice significantly increases our productivity as we have:
- reusable element selectors in the page itself
- reusable components in a base page that is being inherited by all the other pages
Once we build the structure like that, tracking a test automation coverage becomes much easier as we have neat and clean test steps, with no coding logic exposed there at all.
Rich reporting tool with screenshots, videos, and console logs
Once we write and execute automated tests, we want to have a comprehensive overview of the test results and an ability to easily debug the test failures. In our code accelerator, we use a combination of the built-in reporting features from Cypress (screenshots and videos) as well as a custom logger and a few plugins, which all together make our Reports an excellent starting-point for test debugging and test maintenance.
API requests leveraged for test precondition or mocking the API response
On almost all projects, having some complex and time-consuming UI flows (that can even act as preconditions for many other test scenarios) is almost inevitable. In that case, the recommended approach for test automation is the combination of API and UI tests, where API calls are made just as a test precondition in order to decrease total execution time and thus improve test debugging and maintenance. The same set of steps should also be covered through the UI in a separate test; however, having them wrapped in API calls allows us to avoid the long flows being repeated over and over again, because they ve become a precondition for the other test. One more powerful usage of API calls is mocking the response from the API and thus validating the frontend components in the scenarios that would otherwise be unreachable (e.g. impossible to trigger Server Error 500 by an automated test).
Parallel test execution
When the test framework becomes bigger and test executions last long, the QA engineers usually tend to introduce some mechanism for parallel test execution. Cypress has good documentation about setting up the parallelization on the CI environment, but there are also some plugins that enable this on the local machine. To leverage that option for faster test execution locally, we added a separate command that accepts a specific number of threads as an argument.
Running automated tests on different browsers has never been easier, as there is no additional setup needed in Cypress configuration, other than specifying the browser name as an argument like this:
The other option, when running tests on a local machine, would be to select any browser from the list of available ones on the Cypress Test Runner. This list gets automatically populated as Cypress automatically detects the installed browsers on the machine.
Retry logic on failures
The automated tests that are not stable enough and do not yield the same results after being executed a 100 times in a row on the same app version are almost useless, as any test flakiness and instability requires a lot of effort for investigation of issues as the framework gets bigger.
In order to prevent any random issues leading to false-negative results, most QA engineers tend to use some sort of retry mechanism. If the framework itself does not provide such a mechanism, it can be a cumbersome task to implement one, but fortunately, Cypress has that as a built-in feature. The only thing we need to specify is the number of attempts we want to have for all tests or override that number on the level of a specific test or test suite.
Considering the importance of the efficient test automation process, we have developed the framework that helps us achieve much better results in a shorter period of time by writing the new scripts on the top of the existing code accelerator.
The benefits of having such a template are enormous as we have everything prepared in advance and we can start writing new automated tests as soon as we get familiar with the app and the business logic. This way, we are becoming better in what we do, all for the sake of making better software.
The idea for Maestral Code Accelerator was born out of a conversation between members of our Tech Engagement Enablement (EE) * team. Seemingly an “aha moment”, this sudden strike of inspiration was actually far from it. Rather, it was a culmination of years of working together, many products under the belt, and a reflex-like need to look for points of improvement in the software development process.
The group eagerly began working hard to bring this idea to life. Today, the code accelerator is greatly appreciated by Maestral clients as a unique concept in the Product-Development-as-a-Service domain.
* Engagement Enablement is a group of seasoned Maestral professionals in software architecture, development, and product management in charge of Discover, project intake, and continuous support to product teams during the development process.