Naeem Malik
Hi, everyone. Welcome to the show, welcome to the talk/seminar, whatever you want to call it. I am Naeem Malik and I am a senior test automation engineer. There is a lot of interest in Playwright these days and all of us know in the past that we used to have separate libraries for testing APIs and for testing UI. For example, Selenium was a totally different thing and we used to REST Assured. There was another library for C#. I'm forgetting the name, these used to be very like silos. So today, I'm going to speak about how those silos can be somewhat, I would say, how the gap between the silos can be bridged for some projects, not for all projects, because Playwright is mainly focused on UI testing. But it also has some functionalities, some methods and assertions which can be used for API testing and verification. First of all, let us proceed. In the past, we still have the testing test automation pyramid where we have like tests on the bottom and then the integration tests and then the UI tests. And as I said, that the UI tests and the API tests live in separate silos. But with tools like Playwright, there is an attempt going on to basically reduce the distance between the API tests and the UI tests. Playwright provides us basically a means for reducing this fragmentation. And here you can see we used to have like multiple things. Like, for example, there is Postman and if you create your API tests in Postman, you can run those through Newman in your CI/CD pipelines. But again, that's a separate code base to manage. The check-ins are separate and everything else, is separate. Postman makes API testing very easy, I know, and running tests using Newman is also not a big deal. But again, you have to maintain a separate pipeline and you have to maintain a separate code base and so on and so forth. And when the tests, like tests suite in Postman grows large, it can be a challenge to handle. When we are using multiple tools, of course, the complexity of the project is increased. This is where Playwright's unique architecture comes into play. First of all, we know that other than the UI things, like, for example, we had the out of process execution model and then like low level browser interaction. These are, I know, Cypress also has those and Selenium was also pretty good in this respect. But direct HTTP requests were something which were not present in Selenium. Those are, I know that there are libraries available in JavaScript and Node.js, which can be used for calling APIs directly. For example, I would say, like, there is Axios library and there is nothing unique that Playwright is doing here. First of all, let me clarify, Playwright is not doing anything unique here. But the only thing is that it provides us an opportunity for simpler API tests and for creating test data using the same code base. I'm not saying that Playwright can basically be used for like larger API testing projects, but for smaller API testing projects we have direct HTTP requests built in and we can use those without the page object and through the page object as well in Playwright. This is basically the basic and the benefit is that, of course, that we get to have like a single library and we can have a single code base. Things are slightly simple. So simplification is very important. Easy maintenance of a code base is also, very important, so in that regard, Playwright is helpful for us. And moving forward here, we have a small quick comparison. Basically, we know that Postman is great for manual API testing, but the tests you write in Playwright, you create in Postman, if you create some tests in Postman. You cannot include any UI testing code in there basically, not like the first class support. You cannot mix your Postman API calls with the UI, UI tests or UI operations. And then there is REST Assured, we nowadays basically the undisputed champion in the API testing, but we know that you will have to have a separate framework for UI and a separate framework for APIs. This is the challenge which basically causes fragmentation in your test code base. And then we have basically a Playwright which provides us opportunities to unify the UI testing and API testing. Using TypeScript or JavaScript in or even C# or Python under one basically code base. And yes, I believe Java support is also available for Playwright. So we have a single language and we have a consistent API and reporting as well. We also have this detailed comparison, which we can ponder on basically. For example, as I said that we can combine API testing and UI testing in a single code base through a single library using Playwright. But in case of Postman, we have to keep them basically API tests stay separate and UI tests stay separate because we cannot bring them together using Playwright, using Postman. And then again, there is rest assured, which is a completely different paradigm lives separately. It's a separate thing. And the key differentiator here is basically that Playwright allows us to seamlessly integrate UI tests and API tests. While Postman is user friendly, it has got a GUI, which we can use for like developing APIs and like testing as we go. Playwright does not provide any such functionality related to APIs. The REST assured also does not provide any such functionality. So we don't have a REST Assured UI, which could basically use to generate tests. Postman is better in this regard, at least. And then there is language support. Playwright supports multiple languages like there is JavaScript, TypeScript, there's also Python, Java and .NET. Playwright has got language support for multiple languages, which means you can basically use these functionalities in multiple languages. You're not limited to only one language which is not the case for basically REST Assured where we get Java and Groovy. And for Postman, we have only JavaScript. Moving forward, CI/CD integration for basically Playwright is good. You got templates for GitHub actions and Jenkins and GitLab and so on and so forth. In fact, when we are creating a new project in Playwright, it asks us whether if we want to basically create a GitHub actions workflow or YAML file, a GitHub actions file, and we can create a GitHub Actions file right there and then. And it is very easy to run your automation Playwright based automation script using GitHub actions. And then for Postman, if you want to run your tests on CI/CD, for PostMan, you'll need to have Newman installed, which is not a very big deal, of course, you can include it in the pipeline and it will run seamlessly. And then there is Java and Groovy, and then we have REST Assured, of course, we have like very strong CI/CD integration with the Java ecosystem, of course. There's Jenkins and so on and so forth. There is support for, of course JUnit is separate, but a lot of people. People generally use TestNG or JUnit with their Java based unit tests, which are written in REST Assured. So all of the reporting, standard reporting of JUnits, for example, is present in case of Java. And then, well, there is a small overhead when it comes to Playwright. And I am going to speak about it later on. There is a slide about pros and cons of Playwright as well. And I will not skip the cons, of course. And then we have the learning curve. You see, in case of Playwright, there is the learning curve for non-developers can be slightly, curvy, slightly steep, I would say, because the syntax is like peculiar. I would see the syntax is not like very beautiful. For example, here, if you see this could be a lot of a lot a jargon for a new person for a non-developer, like there is a semicolon, there are commas and there are double quotes and so on and so on. And there is a lambda function always. Moving forward, basically, there is a learning curve for people who don't have any coding experience when it comes to Playwright. For Postman, of course, it is mostly like Postman has got a very nice, a very excellent user interface, which we can use for creating API tests. And then basically those tests can be run using Newman on CI/CD. So that is not at all a problem. And for REST Assured, again, there is a learning curve. You need to know Java if you want to use REST Assured. If you want to do your API testing with REST Assured, you will need to now Java. And in the end, the slide says that Playwright is ideal for teams who are seeking a single scalable solution to test the entire application from back-end logic to front end user experience. And also, I would like to add here if you want to create your data quickly using the APIs. If you want to create some test data quickly using APIs, you can use Playwright. In case of Postman, you should use it. If you have a team that needs mostly visual collaboration tool for API development, it is not for like very hardcore coders. And again, REST Assured has got its own basically world where we have basically it's the Java centric thing. And we need to have basically it is again for Developers. This was the crux of this slide. And then there are some bunch of pros. Pros say that, as I have said so many times in this talk that we can use Playwright for unifying the UI testing and API testing. And it needs eliminates the need for multiple tools and there is seamless integration with CI/CD, although it is heavy because of the browser binaries which are included with Playwright and data flow and business logic can be validated very easily using Playwright. And here are the cons. For example, we already spoke that I already said that there is no UI support. If you want to create your API tests, if you want to like play with the APIs using a user interface, no UI is available in case of Postman, in case of Playwright for Postman. We have a UI for REST Assured, we don't have UI again. So Playwright does not have a UI. For like early adopters and you will have to install the browser binaries no matter what you do. And those are slightly heavy. If we are like running our tests in CI/CD, we are creating new machines for our runs or we are setting up new agents. We'll have to basically download 100MB or 150MB of data every time, like it or not. And that it also makes it unsuitable for just API tests. Because API tests not need the browser binaries. Browser binaries are important for UI tests. And yeah, the API testing, tooling and reporting is not very helpful in case of Playwright. If you are making simple assertions, you will be able to basically see what's going wrong. But for complex scenarios, the error messages shown by the Playwright reporting are not very helpful. And then there is basically the last point that which I said earlier that Playwright API testing functionality is useful for simple projects, but it is not very useful or it is not great for large scale API only testing projects. Because of course, we have to download the browser binaries so if we're running them every time will be slightly costly and so on and so on. And then the reporting is not that great in case of the Playwright tests. The Playwright test framework does not provide very good reporting for API testing. The conclusion, so far conclusion is that simple API testing in smaller projects can be done with Playwright. But if you have deep and API only testing, we should use specialized API testing tools like Axios, Jest, Chai, or REST Assured with Java, XUnit or NUnit, depends on your preference. And it is possible to have basically a single config file where we can save the base URL. We can save the storage State as I talked earlier, page storage state, which can be basically reused for logins later on. And Playwright test also has got support for parallel testing and Playwright library provides us with this object, API request context. We can use API request context to make all the basic calls related to API, all the basically HTTP calls like the get call, the post call, the put call, the patch and the delete. All of these are exposed as methods on basically Playwright page object and Playwright request object.