Why Service Virtualization is needed in the Internet of Things Era
As the dawn of the Internet Of Things Era begins testers and developers need a better way to test the functionality of their growing API, cloud based software applications. One of the new tools/techniques developed for this purpose is the concept of Service Virtualization.
Find out why Service Virtualization is so important and why you need it to have confidence that your modern application will work as expected in the wild.
About Paul Bruce
Paul Bruce is a web performance engineer, seasoned software developer, technology consultant, permaculturalist, and family enthusiast.
He has worked as a solutions provider in non-profit higher education and a web applications architect in the health care industry. He has also consulted with small and medium size start-ups on technology related-issues, and currently works at Fully-Verified, specializing in both back-end and front-end web performance optimization.
Quotes & Insights from this Test Talk
- Service Virtualization is essentially an approached emulating behaviors or components in a larger system.
- You use a virtual API, stand that up, and then basically mock out a little bit of the data behind it so that the responses coming back are valid enough for your front-end so you can start testing it.
- API testing itself is really just a subset of quality, but increasingly critical!
- Service Virtualization simplifies the technology constraints.
- Data is still King
- With virtual API, you can either change the behavior, make it slower, maybe cause it to produce bad data, or even just simply turn it off and see what it's like for a front-end app to live without its precious API.
- Typically, around the performance track, they want to see what it's like to deal without one of those calls. They call them black holes
- A virtual API is only as good as you make it
- Much, much more!
Resources
- Service Virtualization: Reality Is Overrated – by John Michelsen
- What is Service Virtualization
Connect with Paul
- Twitter: @paulsbruce & @ready_api
- Company:SmartBear
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.
Special offer for TestTalks listeners, get 20 hours of automated testing for free when you sign-up with promo code testtalks14 (more info).
I recently spoke with Paul Bruce about API testing and Service Virtualization and why it's so important with the growth of the Internet of Things. Here is the full transcript from my TestTalks podcast episode:
Joe: Hey Paul, welcome to Test Talks.
Paul: Hi. How are you doing, Joe?
Joe: Great, thanks. Actually, I should have said welcome back to Test Talks, Paul. It's been a while since we last spoke. You were on Episode 14 where we touched on performance testing using LoadUI. Anything new going on at SmartBear since last time we spoke?
Paul: Yeah, so our new digs in Boston are really great. It allows us to host things like meetups and a bunch of other community stuff. I just want to mention, we are heavily investing in the open source community through a number of different channels. We do play host lovingly to an independent podcast called APIs Uncensored. And that's featuring Ole Lensmar and Lorinda Brandon and they bring guests that are very fundamental to how people all think about APIs on.
We're at a whole bunch of conferences this year. So for instance, APIDays and APIStrat happen in Berlin next month; Gluecon, Velocity, App World. We've also launched a new thing, Community Portal and even the developer network.
There's a whole bunch of partners in Europe that we're growing our customers' capabilities through and we are even increasing our technology alliances daily with folks from IBM, Intel, Mashery, WSO2, Apiary, and MuleSoft. As always, we love feedback. But this year, we're even more focused on earlier user feedback to help us drive our decisions here so that eventually people can benefit from those and then we benefit from their feedback and so on and so forth.
Service visualization is essentially an approached emulating behaviors or components in a larger system
Joe: So today, I'd like to talk about service visualization. So at a high level, what is service visualization?
Paul: Well, Joe, that's a great question. It covers a lot of area here. So service visualization is essentially an approached emulating behaviors or components in a larger system. Now, often we're talking about distributive systems, cloud-based apps and services, but also even down at the very desktop software level. It's to reduce dependencies in teams.
So for instance, maybe I'm building a service over here or I take a server down over here, and what does my test team do? Right? So that's not just reduce dependencies between teams, but specifically to overcome resource conflicts and ultimately be able to take a little more control over the simulation that you're running. Now if you're a tester, that simulation is for your tests. If it's for Devs or integration teams, it's for integration purposes.
Joe: So as a tester, why should I care?
Paul: Well, testers really care about availability of resources. They also care about time to life when it comes to how quickly they can start building all the meta stuff around their testing strategy.
In a typical situation, what we find, we actually have a product to help with the situation. We find that people get a charter; a project charter ready. They get an implementation plan and they start in on the implementation plan and so the Devs start cooking code.
It's based on a specific design, but the testers basically have to sit there on their hands and wait in order to start building something out or the designers for instance, user experience designers.
They need to wait for the Devs to come out with the Alpha version 0.01 of the service that their webpages are connected to. And that is a huge waste of time, especially when you get into highly agile situations and very quick turnaround, two-week sprints and stuff.
So what we find is that service virtualization, and specifically for our goals here at SmartBear, API specific virtualization – we call those Sandboxes – allow testing teams to be able to start building their test suites out, start getting their test data all ready, cook this into a CI system much earlier on.
And then eventually, when it comes a time when the Devs come out with that implementation of that first service, they can press the play button on a whole set of test suites that they already got working on, in some times, months in advance. So it saves them a lot of time and management is really happy because they've got cross-functional and parallel development over both the code and the testing of that code.
API testing itself is really just a subset of quality, but increasingly critical
Joe: Am I understanding it correctly if I say it's kind of an agile approach where you can easily say design a virtual API and then you could think of ways that you would need to test that before a line of code is written?
Paul: Yes, exactly. So in one example, there's a designer and a tester, and those test and design resources are connected to typically development systems. So the idea is that you can start building from the idea that the developers are working on at the same time; that idea being a specification between systems like a WSDL, in SOAP terms or a swagger definition. There are plenty of other definitions. But that Dev Team can quickly bake out what let's say a back-end API is going to look like and then hand that definition right off to a testing team and let them build out their tests cases on it and also virtualize those APIs.
So they can even set up their, for instance, load tests against the virtual API, and that actually has implications far beyond just getting your tests built earlier. In load testing specifically, well pretty much all the time now, you run into the situation where a third party, maybe you can't load test that. For instance, trying to go get a Salesforce token. Or in some cases, these are other groups who are connected to financial systems that you can't necessarily load test.
But you do want to make sure that your code's performance is up to date. So you use a virtual API, stand that up, and then basically mock out a little bit of the data behind it so that the responses coming back are valid enough for your front-end
This is where we bump in to not just API teams or API testers. Really, testers are testers. Whether we're testing APIs or whether we're testing front-end or mobile or desktop applications, this is a situation whereby I'm finding a lot of customers; they don't have API-specific testing teams
So what we want to do is increase quality in both the end-user experience, and then oftentimes, ensure that the APIs themselves satisfy their own SLAs, their service-level agreements, so that we know where the problem is if we're running a load test or performance test and the final experience is slow. Well, that could be because of a back-end service or because of a front-end rendering problem. So we want to be able to test each of these systems, both independently and as a whole.
API testing itself is really just a subset of quality, but increasingly critical. It's just as APIs are in the connected world. We're seeing growth there beyond our wildest dreams. So the more these APIs proliferate, really the less we can afford to ignore aspects; things of APIs.
Not just functionally testing them but also doing performance testing over them and even more security testing, which is actually I will drop a hint that SmartBear is really invested in this approach to APIs; this whole API readiness strategy. So for a front-end tester to be able to stand up a virtual API and remove certain dependencies on their Devs or even on some of their operations team to wait for a server, to be spun up, well they can just virtualize that API and get running
The next step is to figure out how does this fit into a team strategy. So the team sometimes needs to use the same APIs. Maybe we've got the same front-end. Let's say you've got test complete and you are doing testing against an Android device and your application connects up to the APIs that are hosted on a development server. What if the developers decide to roll a new code, because they've got to do what they've got to do? And now, the API version doesn't match what's on the front-end. Well now, you're in a pickle because there's no rolling it back. We've got to do these things in very quick, agile sprints now. But you need to be able to have the front-end testable to prove that the front-end itself doesn't have any bugs and obvious flaws in it.
So in this case, you spin up the virtual API, you have the front-end connect to that virtual API. And even better, this empowers testers to be able to mess with the data that's coming back from their APIs. Typically, you build a test, you want to make sure it succeeds, press the button and there's a green checkmark, and it's good. But now, all of the sudden, what if you build a test where it doesn't necessarily prove that bad data will fail the test? How do you know that that's happening?
So you want to build tests to not only prove that the thing is successful when there's the right data in it, but also fail when there's a wrong data in it. With virtual APIs, now you can actually do that. You can change what's coming back from the “API,” literally fire in there and change the response into bad data so that then you can go back in to your front-end test and prove that the front-end test fails when bad data comes through. So this is that aspect of control over your testing that I mentioned earlier.
Joe: So it's almost a way where you can seed your virtual service with data, and then you could almost check different edge cases if you wanted to. You could check not only does it respond to a valid request to a service, but also what happens if that API that you're doing will start sending you back some really funky data. You can actually test that without actually invoking the real API.
Paul: Exactly. So that's one situation where I've heard customers really need a solution and we provide that through a tool at our API Readiness Platform called Service V. So Service V allows you to design these virtual APIs, pretty much take in a definition, and then spin them up as virtual APIs, and then go in. And if you want to do a little data driving to what the response comes back with, then you can get rolling. Now, this is really great for specifically API tests, but it's really great for front-end to back end API calls.
Joe: So I'm just going to ask. Can you actually virtualize an API that calls another virtual API and string them together?
Paul: Absolutely. In fact, that is one of the things that I heard last week. It's that one of our customers was looking to simulate the response that came back but pipes some of their traffic to the real service, sort of like an A/B test around their API calls. So their specific condition was that they want a virtual API that will act as a front to what the API looks like, and then 90% of those calls will go to another fake API; another virtual API in the background like almost load balancing in the performance world. But that 10%, they want to go to the staging system to prove what happens when they send a different data.
So that was another case where routing comes into play here. And in essence, when you have the control through a virtual API to do this kind of stuff, now it's not just these edge cases. You're now empowered to be able to focus on the worst problems in your services and then also allow teams to function cross-department as well as be able to do things in parallel. There was another situation where they used the double virtual API to do versioning to prove if their “rest version” of their traditional services process faster than their old SOAP based systems.
So they needed a virtual API in-between to transform the data that was coming in from their clients and route to the new restful version of their services. So it gives you complete control over that. We also can control behaviors like latency, and then like for instance, performance specific stuff; the number of threads that it's going to use and so on and so forth.
We also have a component called VRTServer that takes this a step further, So instead of just being able to design these virtual APIs and then use them for your own tests, you can design them and then throw them up on the VRTServer and other people can pull them down and use them locally as well. You can either host it on that server, so now you're all using a shared server as a VRTServer. Or you could drag them down, because maybe I want to continue working on my test suites but I'm hopping on a plane or my VPN connection is flaky and it doesn't go down or I want to go work at a Starbucks or something. Those are cases where you want to be able to drag down the VRT itself to your local environment and still be able to function. It's very easy to do that in SoapUI.
So let me just describe real quick what I mean by API Readiness. So ready API is a platform of API Readiness tools that allows you to go from one part in the development life cycle all the way to the other. And what we do is we go from functional testing all the way through security, then performance through load UI. And then into, for instance, these virtualization situations, and then all the way into API Monitoring. So we also allow you to monitor those APIs that you build the tests on in SoapUI all the way at the beginning.
So that platform starts with SoapUI NG. That's our functional API testing tool, and that covers things like of course SOAP and REST services. And we even have support for things like JMS test steps. So if you've got a message in your background, then you can test those interactions as well. So that's functional testing, then there's security, then there's LoadUI NG which allows you to basically take those test cases you built and spin them up as large scale load tests. You can monitor servers at the same time.
Basically, the stuff that used to be in our previous product LoadUI Pro, well that's now in the platform as another tool. Then there's security. And I mentioned security specifically because next month we're actually coming out with a formal security testing framework.
Joe: Awesome.
Paul: And I can't say too much before we release it, but I can say that the people with REST services are gonna be pretty happy.
Joe: Cool. Is this possible? Can you share your virtual service outside the organization and get feedback before you actually build the real API? Almost like put it on Git and say, “This is what we're thinking how our API is going to look like,” and try to get some input that way? Does that make sense?
Paul: That's a really great idea, honestly. You definitely can do that with your service descriptions. I think one of the common misunderstandings for non-API geeks is that when you're sharing your definition with somebody, it oftentimes really doesn't expose much other than what they call the schema or the way that the API is going to look from the outside in. But it doesn't actually give them your code. It's just the way that it looks; the skin on the outside or the wrapping on the outside.
So that's one way to allow other people to prototype, play around with ideas like maybe if you switch to this process looking this way or have this work flow go this way instead of that one. Or have you seen that new third party that is doing the authentication differently? Those kinds of things you could do with just the service descriptors. When it comes to the VRTs, we've got a couple of options. Obviously, you can package it up as a word file and send that over to any kind of Apache system that you've got out there. You could also, in theory, open up access to your VRT server. You'd already deployed to that VRT server and then be able to allow certain people access, but that really more of a network permissions thing.
Service virtualization simplifies the technology constraints
Joe: It almost sounds to me also, with service virtualization, almost like its test-driven development approach where you can show your developers the types of patterns of usage you think your users will have when using your API before even developing it. Is that also something you see as an advantage of using the service virtualization?
Paul: Well, service virtualization simplifies the technology constraints. For instance, I mentioned third parties earlier. So availability isn't the only problem with third parties. Sometimes, they make themselves widely available and then you get a big fat overage bill afterwards, and then your CTO comes back at you with a “What on earth did we just do?” So your test process itself is service virtualization, or specifically in our case, virtual APIs and API virtualization. That just helps alleviate these resource constraints and these common points of problems between teams.
Your question about TDD is great because it's way high up at pattern and practice level. And whether it's BDD or TDD, it really depends on your team's choices around what you can feasibly pull off within a sprint or some kind of a development life cycle. Really, that's up at the high level and we don't want to get in the way of that. We don't want to specifically say, “No, we're only TDD-centered.” There's a lot of people who are still very waterfall, and we want to provide tools that are agnostic to the process by which you decide what tests you're going to build and what resources you are either going to constrain or to free up to save time.
Joe: So Paul, how difficult will you say it is to create virtual APIs and simulate API traffic?
Paul: To be able to simulate APIs, it's very lightweight. It's very easy to be able to take a description or take a test case and build a simulated or sort of, I like to use the terms “cardboard cutout”, of that API. Stand it up and then use it and data drive it only as necessary to just the reasons that you need to simulate that and not have to stand up a whole stack. I mean, we're not talking simulating things like LDAP for authentication. We're not talking about simulating things like DCOM and stuff. These are not our goal. Our goal is to do APIs, help testers do them well, build those tests quickly.
So to be able to simulate API traffic over HTTP using SOAP and REST primarily, and then JMS; to be able to virtualize JMS, that's actually really important to our customers so we're planning that kind of stuff too. This should not be complicated. Because for one thing, even if I am somebody who can write some code. If I'm a developer who just writes code all day and loves code and scripts and everything, that's great. But how does it hand off to testing teams?
The testing teams can have senior testers and mid-level testers and even entry-level testers in there. In our experience, there's not a lot of call to just write scripts all day long and that's your testing approach. If you want to do that, fine. In Ready API, we have group support everywhere. But most people just want to be able to pull in the description, right-click a couple times and have a relatively simplified approach to testing so that they know that things like, for instance, if the server starts coming back with malformed XML or if the REST request comes back with a 401 Not Authorized. They want to hear about that stuff right upfront. It's very easy to do.
In virtualization, we've taken that ease of approach and pulled that in as well. So you can very quickly just right click and say New VRT based on a Soap or Rest interface that I've already got in my project. Then the default responses come back and you can fill those out with the data, either static data directly from the test case results or you can fire in there and just put in some data. You could data drive that from spread sheets like Excel. How easy is that? Or even connect in to a back-end database to pull that stuff in.
Joe: How about OAuth? Do you support OAuth? The reason why I asked is because I worked with another vendor and they asked their tech support for help with OAuth and they said, “Oh, it's not in the documentation, so I can't help you.” Just curious.
Paul: Yes, we do support OAuth, both of course on the test side. So when you're generating tests creating requests off to your APIs, we of course can handle OAuth there. Then also through VRTServer and Service V, we can do OAuth from there as well.
Data is still somehow king
Joe: I know we touched on this a little bit already, but can you tell us a little bit more about the data driving capabilities of service virtualization?
Paul: Data is still somehow king. In the world we live in, awash with data, yet somehow it really defines and shapes the way we do things from moment to moment especially in automated testing. Now, specific test cases need to be able to have varied input into them. So maybe you're testing a payroll authorization system and you cannot send the same data twice. That's a constraint of the applicationary API itself. So you've got to be able to pull data in and use it as input, right? But also with responses, with virtualization, you need to be able to have that same flexibility to be able to respond with dynamic data as well and to also be able to route it in different ways and so on and so forth.
So we've also got that in there and same kind of point and click navigation to be able to get data from a previous request. It's a lot to do with testing teams that don't' want to spend time scripting. They want to just build their tests pretty quickly and then move on.
Joe: Cool. So Paul, how distributable are these virtualization tests or components. For example, in my company, we have multiple environments that we need to run tests against; QA environments, CI environments, Dev environments. How hard is this to do? Is it even possible?
Paul: So one of those things is we want to be able to collaborate on the same resources and mix and match and go, “Well, let's change that VRT up, because now, maybe the developers have changed the code. So let's ship that back up the server so that everybody can benefit from that.” Then maybe, when we pull it down, pull a copy of it down when we're out in the road or something or when we're disconnected from that environment, I would actually frame that as more portability.
And I'm sure you've bumped into the terminology around Docker and containers at this point, but that solves a problem of deployment, that whole containerizing stuff amongst other things. It really impacts Devs. It really impacts operations to be able to create these immutable containers that allow you to ship the code throughout multiple environments.
At first, the tester might not think that that's impacting them. But in reality, anything that the Devs are doing and anything that the operations are doing at the same time are gonna impact testers. So where this comes into play is the same reasons why they thought portability was good from a deployment perspective are the same reasons why we think portability from a virtual API is just as necessary. It's all about the people. If there's more than a couple people on the team, you're probably gonna have to be able to share resources, specifically the virtual APIs. So we've got that capability in VRTServer and with Service V.
Joe: So how would you respond to someone who says, “This sounds great. But isn't this almost double work to put all your effort into virtualization, and then you have to actually create the API after?”
Paul: If we're talking about developers who are writing the APIs, it's not duplicate work to do that if they haven't built it yet, so to be able to get testing really quickly. And yeah, the point of this is to be able to throw it away once you have a real simulation except for when your real simulation in, let's say a staging environment, does not have the proper data. And maybe we're talking about testers here that don't have access or don't have the capability to be able to change the data either all the way down in the database or change the code.
When was the last time a tester was allowed to change any code? Unless they were the Dev. And testers and Devs, they probably have something to say about that. So this gives you control as a tester that you otherwise don't have both before development has put out its initial implementation as well as after.
What happens if on your mobile app or on your desktop app or on your web app, you have to go without one of those APIs?
Joe: Sounds good. So also, doesn't it help to actually be able to allow you to automate certain things that may not be automatable? Say you need to mimic that a service is down, so someone actually needs to physically go put the service down. It's like an activity that can only be done manually. Would service visualization help us with activities like this?
Paul: Yeah. With virtual API, you can either change the behavior, make it slower, maybe cause it to produce bad data, or even just simply turn it off and see what it's like for a front-end app to live without its precious API. And maybe that's the whole kit and caboodle. Maybe your mobile app is connected to one API, but not likely. More likely, it's connected to some static resources in a CDN, like images and stuff, and then it's also connected to a couple of APIs at the same time; maybe for analytics. So there's some analytics maybe even to the store that you purchased it from or the marketplace you purchased it from, and then also to your own code and then maybe somebody else's code hosted in a different place.
So what happens if on your mobile app or on your desktop app or on your web app, you have to go without one of those APIs? Did you have any way to test that without talking to the operations team and trying to put them in a firewall or put them in some kind of exclusion set? Or do you have another way for your developers to purposely turn off that test server or that development server? In those cases, you definitely don't have control. But with virtual APIs, all you have to do is virtualize that out and then you've got control to do like for instance a single point of failure, the SPOF test.
Typically, around the performance track, they want to see what it's like to deal without one of those calls. They call them black holes. So what happens to the traffic when the domain is not available? That domain from a third party is not available. Those things, the performance people know how valuable it is to test that, but the functional testers could also benefit from that wisdom as well in the sense that the function of the application itself is sometimes impeded and sometimes entirely ruined by not being able to access simply just one of those API calls in the chain.
Joe: So we talked about all the positive things of using virtual services. Are there any drawbacks or things people need to be aware of when their using virtual services to make sure they don't have a false sense of security?
Paul: Yes. Well first, a virtual API is only as good as you make it. So it really comes down to the types of situations you know you're gonna need more control over how you're testing. That's usually if your developers of course don't have the code posted somewhere, or for whatever reason, you don't have access to those development severs. Maybe you're not supposed to have access to those development servers. So that's great and all, but when you spin up a virtual API, just keep in mind, it's a virtual API.
So in load testing, for instance, you want some virtual APIs like Google Analytics. Let's say that's part of your API stock or calls. Or in your mobile app, the calls out to the APIs are calling to something like keen eye or something. Maybe if you tried to do a load test on them, you're going to cause a denial of service. They're going to consider you the denial of service. So it's gonna stop you from being able to run your test, and then you're gonna run into Timebox Windows where you can't actually get the job done on time and your boss is gonna be mad at you.
So you would virtualize that one resource. But then keep in mind that the total picture of performance throughout whether it's a webpage on the website or whether it's a workflow in APIs, that the total picture of performance needs to come from more than one location. So if you're doing load testing, that is fantastic. If you use virtual APIs to cut through the red tape around load testing, that's great. But just keep in mind that when you go to production, you will want to monitor the entire picture of performance over that API or that webpage, and that comes from being able to monitor that with tools that help like AlertSite.
So that's one of those catches. Virtual APIs can help you alleviate team interdependency that's not good. But keep in mind, you also want to make sure that that goes live and that you're monitoring it afterwards. Then coming back to the team thing, you do want to make sure that people are talking to each other. Because if the Devs are revisioning their servers and the testing teams are, “oh, great. We have our virtual API,” you also want to make sure that that virtual API version is relatively close to the Devs so you get that freedom where you don't have to version just because the Devs wrote some new code.
But then keep in mind that in your agile sprints, you want the Devs and the testers to come back and say, “Okay. This week, we are going to refresh the virtual API for this thing. And yes, that might break some of your tests, so let's stage that out. Let's figure out what's a good time for you to do that,” so that the team is really still collaborating. Because the whole point of this is to remove unnecessary dependencies and allow teams to collaborate in healthy ways.
Joe: Okay, Paul. Before we go, is there one piece of actual advice you would give someone to improve or start our API virtualization efforts, and let us know the best way to find out more about you or source virtualization.
Paul: Well, start with a tool that's going to fit your needs. So a lot of the times, people will step back and go, “Wait. If we're gonna do virtualization, let's do it right and let's do it in a big bureaucratic way.” And I'm all for making sure that we have a unified strategy within an organization to accomplish something. But keep in mind, APIs themselves are pretty straightforward and they are an increasing abundance in your organization. Whether you know it or not, there's probably a bunch of APIs already dealing with critical work flows in either your business process or your revenue stream.
So those APIs themselves, you really want them to be high quality. And one of the ways to do that of course is through API testing tools like SmartBear has a Ready API. But in that timeframe, you don't necessarily have the luxury of providing high quality services by respecting the aspects of performance, security and function and all that kind of stuff while waiting for the big, big, big, big decisions to be made. So keep in mind that when it comes to virtualization, sometimes you just need the tool for the job that's right for that job. So in our case with APIs, it's all about API virtualization, that's why this is not about all those other protocols. It's really just about the APIs.
And the best way to contact us is of course through, for instance, our website. There's a bunch of trials you can download of both Service V itself, SoapUI NG. These are independent tools within a common platform, so you can trial them independently or you can trial them all at the same time. You can connect up through Twitter as well.
We've got the SmartBear handle; @ready_api is our Ready API Twitter handle, and my personal handle is @paulsbruce. So if you have any questions, you can drop any one of us a line and we'll get back to you.
Then also, if you're at one of these conferences like STPCon, feel free to stop by our booth. We'll be there. App World. A whole bunch of other conferences. We're typically at IRC, and then of course the API conferences throughout the year.
Listen to the full interview now:==>http://www.testtalks.com/52
