On the latest Test Talks podcast I interviewed Richard Bradshaw, AKA Friendly Tester about can automation do actual testing? I wanted to share a few of our main talking points with you and hopefully help you to discover a mind shift you can use to improve your own testing efforts.
The first point that Richard made was that he doesn’t like using the word test automation at all, but if he was forced to give it a definition he would follow the same lines as Rapid Software Testing, which defines it as software or hardware that supports testing.
Testing Mind-Shift
Instead of calling it test automation, he believes it’s better to use the phrase automation in testing instead. Simply changing the words around creates a powerful mind shift, because typically when someone hears “test automation” they automatically think “automated tests”. Shifting it to “automation in testing”, Richard feels, correctly places the emphasis back on testing and how automation can be used in the space of testing. The difference is that this shift forces you to reconsider other ways that automation can be used. A change in the works can make you start thinking a little bit differently, — the greatest benefit being that people who are new to testing will not immediately jump to automated tests or automated testing – rather, they are going to think in terms of how automation works within testing instead.
Can automation do testing?
So, can automation do testing? Richard doesn’t think so because automation can’t learn, and learning is a significant part of testing. For example, when you perform a test, what you learn from it is going to influence your next test. Automation can’t do that. Instead, automation should be considered more as checking than testing. The definition of a check is an evaluation that is made based on an algorithm. Checking is a big part of testing, but it’s not the only testing that should be done.
Once again — if we think about automation in testing we can break out of this narrow automation “checking” mentality and find other opportunities for automation — like creating tools that simplify what the tester is doing or might do. It will represent only a small part of the testing as opposed to the whole end-to-end scenario. An example of this would be using automation to automate, say, the population of a database that we will then use for our testing efforts.
Testing is misunderstood by many people. Some see it as a process that anyone can perform because if all one is doing is checking to see whether something matches a specification it’s considered to be a checking test which can be automated. There is, however, so much more testing that needs to be done.
Exploring an application and finding new things, then using the information you just uncovered to perform new testing is the type of testing activity that usually uncovers bugs and issues that a deterministic automated test would not find. Testing is the thinking piece that can’t be automated. Automation is not a replacement for testers.
Test Automation and Google Cars
All this being said, I’m currently reading the book The Glass Cage – Automation and Us, and the author talks about how in another book, The Division of Labor, some economist were talking in 2004 about how there were practical limits to the ability of software programs to actually replace human talents. They really focused on the task that involves sensory perception, pattern recognition, and conceptual knowledge. One of the examples they used was driving a car on the open road, and how it required all kinds of instantaneous interpretations of the weather, visual signals, and the ability to adapt seamlessly to shifting as well as unanticipated situations. The conclusion drawn from this was that driving would never be replaced by software because it involves too many factors, and it’s difficult to imagine a set of rules that can replicate a driver’s behavior. In the decade since, most of us have heard of a little something called Google Cars, and based on all the news I’ve heard about it, the self-driving car has been a great success.
So I was thinking … can this principle also be applied to software testing? If it doesn’t apply to testing yet, will it eventually? Can automation do actual testing? As of right now I think the answer is no, BUT — never say never.
