For the past couple of years, I've heard more and more folks talk about using UI browser-based automation tools for performance testing as an alternative to more traditional load testing tools like JMeter and Gatling at the protocol level.
Back in the day, I used to do something similar—running a WinRunner GUI Vuser in LoadRunner to try to get performance measurements from a “user’s perspective.”
The approach worked, but it was cumbersome.
I recently heard about an open-source tool called Flood Element that allows users to do something similar to the old-time GUI Vuser but in a more modern, scalable way.
Read on to learn about:
- What Flood Element is and how it works
- Why should you consider using Flood Element
- What is the best web performance testing approach
What is Flood Element?
Element is an open-source Puppeteer that uses a browser-based load testing tool. Scripts are created using Typescript.
You can use it against your Web app in the same way that your customers do, opening a browser and interacting with page elements.
It's written like a functional test script, but with the help of Flood, you can quickly run a load test on as many nodes as you want in the Cloud.
Their goal was to make the “black art” of performance testing more accessible to folks who are more familiar with functional testing script development.
What is Load Testing?
Before we take a more in-depth look at Flood Element, let’s be sure we’re on the same page when talking about “performance” testing. Flood.io’s Tim Koopman’s definition is: “Load testing is the art of stimulating demand on the system and then measuring it for performance.”
In his PerfGuild 2019 session, Making Load Testing with Browser a Reality, Tim shared a mnemonic he likes to use— PEARS—to remember the other dimensions of performance testing that you need to focus on:
- Performance
- Elasticity
- Availability
- Reliability
- Scalability
Here's the deal.
Why Use This Browser Performance Testing Approach?
Some folks might be wondering why they shouldn’t just use a non-UI-based protocol.
While I agree that’s often the optimal approach, there’s something to be said about not having to figure out how to write scripts in, say, Winsock, since it's not the easiest thing to do.
Remember: load testing is simply a means to an end. The main goal is to observe the above different PEARS dimensions under load. It's what you do about those dimensions that make all the difference in your load test efforts.
To do something about them, you have to be able to simulate load in the first place. And that’s where I see many people, both new and old to performance testing, spend too much time.
In my experience, a disproportionate amount of time is typically spent building and maintaining load test scripts at the protocol level to simulate that demand. And this is all at the cost of execution and analysis, which is the pointy end of load testing (and where we should be spending the majority of our time), but unfortunately, we kind of get sucked into this enormous time sink of load test creation.
That’s really what this post is about: the reality of using load tests with browsers. When I was first beginning my career as a performance test engineer, I always defaulted to protocol-level scripting when getting involved with site performance testing.
But over the past couple of years, most of the sites I’ve had to test have become increasingly complicated when it comes to the protocol level underneath.
For example, it's not uncommon to see more than 300 requests on the first visit to a site like Amazon.com. And not only are there tons of requests going back and forth between the browser and server, but the browsers themselves are doing a lot more work for us these days than they did 20 years ago.
So everything from passing and executing JavaScript frameworks, through processing client-side business logic, all the way to caching responses and even handling security falls under the domain of the modern browser.
Hybrid Browser and Protocol Approach
So, should you forget about protocol-level performance testing completely and do everything at the browser?
No!
I believe a hybrid approach is best for most of us.
Browser-based driven performance is not a silver bullet. It comes with its own set of issues.
For example, this style of testing is a resource consumption hog. It takes many more resources to spin up a concurrent load with a browser-level approach compared to a traditional, protocol-level approach. So you have to be considerate of the environment.
Ultimately I think it's always good to consider both browser and protocol load-testing approaches.
Protocol and Browser Performance in Harmony
Tim mentioned that a common approach his firm’s customers take is that they will often use protocol-level stuff for more well-defined targets.
A good example is a Restful HTTP-based JSON endpoint that might be testing a mobile site that has well-articulated requirements of specs around how to use the API.
In this instance, they may use a protocol-level approach for that—and offset some browsers as well—to see what it looks like from more of a user perspective as opposed to just underneath the hood at the protocol level.
They also have customers that only test at the browser level because some of the implied complexity, particularly with a single-page application with more complicated businesses or users, flows throughout the application.
It just becomes a whole lot easier to build and maintain the scripts at the browser level.
For example, being able to easily create scripts for an application that has functionality for things like ticketing, gaming, or any type of transactional actions in the browser— even video streaming—are some of the things customers find easier to do at the browser level.
In the end, it's about choosing your performance testing tool wisely and making sure that you're aware of all the options that are available to you in the open-source space.
How does Flood Element Work?
Flood started off using Selenium for Element (they still support the Java Selenium WebDriver) but later switched to Puppeteer.
The problem they found using the Selenium approach was that it was even more resource hungry than the Puppeteer approach.
Using Selenium, you get 5 to 10 browsers per call, versus about 40 (depending on the script) with an Element Puppeteer approach.
Flood still supports Selenium because many customers prefer to write their scripts with, but the preferred method is the Element Puppeteer approach.
Other Benefits of Puppeteer for Performance
Puppeteer answered a lot of requirements for Flood when they were deciding on what approach to take with Element.
For one, it has an excellent way of interacting programmatically with Chrome without a lot of bloat. Based on that fact, they came up with the idea of borrowing the DSL from something like SeleniumJS.
Since a lot of customers were already familiar with that approach, that's pretty much how Element was born.
Want to Learn More?
To learn more, you can download and check out Element for yourself.
And if you’d like to view Tim’s PerfGuild session on Element, you can still register for the event and get instant access to all the event video content.
