Common Automation Tips

Problems we are striving to solve

• Define an effective approach to cover requirements

• Define an approach to process NOT PASSED tests

• Optimize work with the test stand

The tests should take a minimum amount of time, but you should have many of them, as many as functional requirements (acceptance criteria).

The test can be FAILED only if you have assertionError.

Any errors other than assertionError test must mark the test as Skipped or Broken. These errors should be caught as Failed in another test that checks the corresponding requirements. This approach contributes to the pragmatic results of automation tests. Only PASSED and FAILED tests have the real value, other test do not carry information about the quality. This approach speeds up the analysis of results and can provide quick information about the real issues, ignoring Skipped results.

Test data can not be created through UI.

• Test data cannot be created through UI - it puts an end to efficient automation because of long and unstable tests

• Test data must be configured in advance, or created using quick scripts, through API (using any fast way)

The test stand must be productive and controllable.

• To run tests you should have a separate independent stand, which no one else uses

• Each member of Automation Team should work with an independent stand

• The stand should have enough resources for adequate application speed - nothing should slow down

It is necessary to define the parallel and non-parallel groups of tests to run.

• Tests that can be run independently are grouped for parallel run in parallel_suite.xml. One virtual machine works well for 3 flows

• Tests that use the global configuration, and may affect other tests need to be grouped for the consecutive start in non_parallel_suite.xml. They will be run separately and will affect nothing

You need to clean the generated data regularly in order not to slow the stand work.

Tests regularly clean their generated data in order not to slow down the application. As an option, it is convenient to do on @BeforeSuite level and divide tests into separate tests suite.xml

If you need to scale up unbiased QA testing on real devices throughout the development cycle, pay attention to crowdtesting platform Test IO. Test setup takes minutes, and results arrive is as little as an hour.