I’ve asked lots of folks over the years to offer suggestions to testers on how to design their automated tests to run in parallel execution.
When starting off, running your tests sequentially might work okay, but as your test suite grows, so will your tests’ run execution time.
This is not good.
It’s critical as your test suite develops that you continue to give your developers feedback as quickly as possible to let them know if any of the new code they’ve checked has broken the build.
At some point, you will need to scale your tests to run in parallel against a Selenium Grid, a Cloud-based provider like Sauce Labs, or a Master Slave Jenkins, CI-type configuration.
One of the biggest issues I’ve seen teams deal with is when, regardless of using the Cloud or a Selenium grid, they can't run their tests in parallel because of the way they’ve structured their automation framework.
Here are a few common themes that seem to pop up over and over again, and some tips for with testers that are struggling with achieving parallel execution when running your automated tests.
Make Your Test Atomic
The first one is ensuring that your tests are atomic, in that they are independent and can run on their own. In other words, they need to be self-contained.
Each time I kick off a test, I think of that test as independent; needing to be atomic and complete in itself. Each test should be able to run independently of any other tests.
You should be able to run any test at any time without having to worry about the order in which you run your test suite.
You don’t want to have to deal with a need for certain tests to run before others in order for them to succeed. Having dependencies between tests will not scale, and will become a maintenance nightmare.
Making them independent will also make them faster, and simpler to debug and maintain.
Mike Millgate shares how he was able to do this in our Test Talks Testing @ the Speed of Concurrency episode:
Don’t Hard Code Values
A primary reason for having to run your tests in a certain order due to hard-coded data. Your automation testing should not have any data dependencies between them.
A common approach is to have each test take care of itself by creating its own data, and then tear itself down and delete any data that it has created afterward. This will ensure the tests don’t impact any of the others’ data that might be required for their scenarios to pass.
Some automation engineers use a data-driven testing approach in their frameworks. This avoids hard coding any value and it increases test maintainable.
For example, Anshil Sharma uses CSV files for her data-driven testing and leverages the build tool Maven. Using a POM file, she can pretty much do all her test configurations dynamically when the tests are running in parallel.
If your planning to migrate your automation test suite to a cloud service check out Anshil's TestTalks episode Moving from Selenium Grid to the Cloud:
Think of Your Parallel Execution Test Management Strategy
Of course, there are many approaches to test data management when running your automation tests concurrently. In episode 105 of my Test Talks podcast, Paul Merrill shares four test data strategies that can help you.
Build Tests to Run in Parallel
An easy way to avoid most of these issues is to develop your automated test cases with parallelization in mind right from the beginning.
You don’t want to start creating a bunch of tests only to realize months or years later that you can’t run them in parallel.
If you don’t plan correctly up front, you’ll end up with a bunch of tech debt, and insufficient time to fix the underlying issues.
Common sense, yes. Common practice, no.
Use Your Framework’s Built-In Functionality
Automation engineers can sometimes be their own worst enemies. We can be quick to develop a solution before taking the time to determine whether a solution already exists.
Virtually all testing frameworks have pre-baked annotations like running tests in parallel—and if you can run the test in parallel, you can run the methods of that test class in parallel.
Concurrent Test Success
This should help you to begin running your test in parallel. Remember to start this sooner rather than later to ensure your concurrent testing success.
