In my last Tech Beacon article, I wrote an in-depth review of the Gartner Magic Quadrant for Software Test Automation Report, in which I dissected the full report piece by piece. But one thing I didn’t mention was that the Gartner was actually wrong…with a few small caveats.
The Strategic Planning Assumption section of the Gartner Report states that by the year 2020 Selenium WebDriver will become the standard for functional test execution and that this will marginalize vendors that can't provide strong, higher-level test functionality.
The Time is Now
I believe Gartner's date is incorrect; in fact, some might say that Selenium is already the de facto standard for web-based test automation. It would follow, then, that the time for vendors to embrace open source tools like Selenium is not 2020; it’s now.
Another reason why I feel the 2020 date is way off is that I mentioned in the original article that there is currently a W3C WebDriver standard in draft form; once this is approved and becomes official, standard browser vendors will be responsible for creating their own browser-based implementation of Selenium WebDriver. I believe the WebDriver W3C working draft will become official way before 2020 and once it does, vendors will be better able to support it.
Once WebDriver becomes standard, I’m hoping it will make automating the browser much easier (even for IE), since each browser vendor will own their respective implementations and have a vested interest in them.
If Microsoft Can Embrace Open Source, So Should You
In fact, Microsoft has already announced (in July 2015) that they are supporting the Microsoft Edge browser through the W3C WebDriver standard. They have also explained their rationale, saying that they did it because it’s a best practice and that all web content, browsers, and specifications should align to the same well-defined behavior. This is incredible — and a significant win for open-source proponents.
If you’re a bit older you may remember how militant Microsoft’s stance was against all things open source-related, but this move – along with what I’ve actually heard from several Microsoft employees — is an excellent example of a complete culture change to the embracement of open source technologies. Open source is the future, and that future is now.
In a further show of support of the growing trend in open source containerization, Microsoft will be adding support for Docker in Windows Server 2016. Testing tools and technology have always seemed to lag behind developer technology, but test tool vendors need to adapt to this changing open source world like Microsoft has if they want to survive and compete in the coming years.
Where’s Selenium?
I also feel it was a mistake for Gartner to leave Selenium out of his report as a viable option for Software Test Automation. I understand that it didn’t meet the criteria he clearly listed, but to my (and many other’s) way of thinking, it was a glaring omission. I also think it’s clear to anyone who has been involved in automation for any length of time that many companies — even larger ones — are moving closer to utilizing open source solutions in their development efforts. To leave Selenium and other open source options out is, as far as I’m concerned, backwards thinking. It’s definitely not in line with what I’ve been seeing in the testing field over the last few years.
Combine Technologies to Create a Full Solution
There are also many ways in which you can combine multiple, open source projects to create an automated testing framework that could meet Gartner’s key use cases for tools needing to support mobile applications, responsive design and packaged applications like ERP and SAP.
For example, many SAP applications are web-based, and the majority of those applications could be automated using Selenium Webdriver. If you were to encounter controls that are not supported or recognized by Selenium, you could always use another open source, image recognition-based tool like SikuliX. SikuliX is powered by OpenCV, can automate anything that can be seen on the screen, and it works on Windows Mac and Linux.
For mobile there are a ton of open source options for testing both android and iOS apps like Appium, Selendroid, Robotium and Frank just to name a few.
Test management software and requirements can be achieved using Cucumber, jBehave and Specflow, which are all open source, behavior driven development frameworks. There is also an open source solution called the Galen Framework that was created with responsive design in mind.
Open source is a reality, but it’s not as free and easy as some folks would make it seem. While cobbling together a bunch of open source solutions to create a framework that can meet many companies’ agile software testing needs is a challenge, but it can be done.
This is where I feel that vendors can step in to fill the void; their support of open source technology would make it easier for people to get their automation efforts up and running.
Proprietary is the Past
After publishing my original post, I also heard from a few people who mentioned that this year’s Gartner Report was too focused on automation from a software development perspective as opposed to a business perspective.
They pointed to advances in “code-less” automation as being the future. Sorry, but I don’t see it. While it might have its place in simple test workflows, test automation is part of development and needs to be treated just like any other development efforts — using the same rigor that would be expended on production code. Furthermore, any solution – even a code-less one –should be compatible with a company’s open source development ecosystem.
The Time is Right for Open Source
Many companies have already begun embracing open source transformation, and I believe that’s critical because the non-inclusive vendor doomsday clock is rapidly approaching the point of no return. I feel that in next year’s Test Automation Magic Quadrant, in order to be seen a credible and relevant, Gartner should begin taking open source technologies more seriously in their analysis and include them as viable options for test automation.
