Find Out Why Your Automated Tests Are Failing Using ReportPortal

Looking for an easy-to-install-and-use dashboard? Need to triage your automation test results as well as create awesome graphs?

Even better, what if it were free?

Well, I’ve got something you should check out if you’ve been looking for an automation test results dashboard solution: ReportPortal.io.

Here’s a quick video of me learning about it.

Index

Easy To Get Started Automation Testing Focused Dashboard
Real-Time Reporting
Test Automation Triage Center
Classify Automation Test Failures
AI Machine Learning
Save Crazy Time
Report Portal is Easy to Customize
How Extensible is Report Portal?
What Automation Framework Integrations are Supported
Love it So Far

Get Automation Testing Tips

Easy To Get Started Automation Testing Focused Dashboard

I know some of you have created dashboarding solutions with Elastic Search and Grafana, but those tools have a pretty big learning curve.

With Report Portal, you get all the benefits of those types of dashboards in an out-of-the-box, almost one-click solution.

Real-Time Reporting

One of the critical benefits of Report Portal is its real-time reporting.

It’s an excellent time-saving feature. Sometimes teams are waiting around to begin viewing their test results, or they make the excuse that they didn’t have time to debug.

With this solution, they can start seeing results in the dashboard a few seconds after the execution has begun, which allows for real-time debugging.

That means if a test case begins failing because of environmental issues or bad build, etc., rather than wait until the next day to find out, issues can be checked out in a matter of seconds. This allows you to act quickly to stop a bad run and restart without wasting hours or days.

Test Automation Triage Center

Report Portal was also created to act as a single point of entry for all test automation and results for your teams’ projects, making it an excellent center for both managers and test engineers.

Classify Automation Test Failures

I also like the feature that allows for the classification of test failures, which means I can go in and set a failure as a defect, or mark it as a system issue.

Now anyone that logs in can scan and see which tests still need to be investigated.

In the Lunches section, test engineers can do their analysis as well as group and tag tests based on failure type. So all future runs and builds can tell you if in case you assigned or linked an issue in Jira or any other bug tracking integration.

You’ll see that the bug already exists.

It can also show, for example, whether it is open or closed, and you can also view all the comments.

AI Machine Learning

With the latest version of Report Portal, you can use machine-learning algorithms to help you to analyze your results automatically.

The machine learning algorithms use all the historical data that is already in the dashboard database for your project. That means it can analyze your latest execution, and you can be confident about the status of your test cases.

Get FREE Automation Testing Courses

Save Crazy Time

This is a huge time saver.

Dzmitry mentioned that with some projects, this feature reduced the analysis of failures up to 90% of the time—time the team previously spent on reporting and result analysis.

From my brief experience with it, I can already tell it’s going to save my teams a bunch of time. You can see everything about your test: history, logs, screenshots–
all in one place.

For managers, the dashboard contains all the statistics and pretty eye candy graphs you could want. You can set up an executive panel based on widgets that nicely shows any KPI metrics your company uses.

This all means that automation engineers can triage their tests quickly and easily, and managers can see the high-level KPIs that they crave–all in one place.

Report Portal is Easy to Customize

Making the tool customizable was a crucial goal for the team that developed Report Portal. Since their vision was to make it open-sourced, they designed it with this in mind from day one.

To achieve this, they built it using microservices architecture, which makes it easy to customize since you can put your own microservice in place using their APIs. Your microservice will work like any of the other built-in microservices.

To see all of the API possibilities in Report Portal, you can click your profile menu option and API from the drop-down to view the Swagger. That will describe the array of APIs available as well as all the methods, which are behind the portal.

Being able to make a request to these APIs also makes Report Portal able to integrate into your CI/CD pipeline and create triggers for different actions within tools like your CI systems.

FREE TESTGUILD COURSES

How Extensible is Report Portal?

Want proof that Report Portal can handle almost any integration?

They actually have a working Silk Test integration!

Remember Silk Test from the 90s or early 2000s?

They have integrated it with the Silk Test agent, which allows Silk to send the old-school results inside the report portal. And since it’s open source, you can see all the code at the same time, which means you can change the code and contribute to the project.

Between the APIs and the fact that everything works through the HTTP channel, you can integrate any solution that triggers particular events. It's pretty easy, and in most cases, a lead engineer will only need to spend a few hours implementing the adapters from scratch.

What Automation Framework Integrations are Supported

However, it appears as though Report Portal already has a bunch of integrations available so chances are you won’t have to even worry about this. If they don’t have an integration with the framework I use (Serenity), it is easy to find on one Github.

Here are the supported out-of-the-box integrations:

• TestNG
• JUnit
• Cucumber-JVM (Java)
• ScalaTest
• Cucumber
• Node.js client
• NUnit
• Specflow
• SoapUI

Love it So Far

I’ll let you know more as I continue to implement Report Portal into my company’s enterprise environment. But from what I’ve seen so far, if you’re looking for an open-sourced dashboarding solution explicitly made for automation testing, you should consider giving this a try.

Check out my full TestTalks interview with Dzmitry Humianiuk all about AI-powered Test Automation Dashboard with ReportPortal