Do you struggle to decide when to use a browser emulation (in case of hybrid apps), mobile simulators/emulators and real devices? In this episode, Kwo Ding will review his mobile test pyramid approach and discuss best practices about when to use what. Kwo will also reveal his tips for how to best structure and execute mobile test and more!
About Kwo Ding
Kwo Ding is a hands-on test automation architect/consultant with 10 years of experience in software testing. His focus is implementing test automation (strategies) and designing the test infrastructure at organizations. He is specialized in web, mobile and API test automation.
Quotes & Insights from this Test Talk
- So I think that most people who are doing software testing.Know the regular test pyramid from Mike Cohen. He wrote the book succeeding with Agile in 2009 with the bottom layer unit testing and on the top layer UI testing. So basically what it says is the broader the layer in the pyramid. So in that pyramid unit testing has the broadest layer. So yes the most complex test and the more tests you have there so you don't want to do super extended of testing on the UI. Because it's it's brittle and especially if automation you want to get the fast feedback in your SDLC.
- Kind of based on that concept since I was doing mobile testing for some time and I came up with my own pyramid because there are many many aspects to cover when you talk about mobile testing. So it's not only your UI or your web application but it's also the device itself the operating system and maybe some connectivity issues from the device to the network. So there are many more aspects but I thought I needed a more detailed level on this strategy for mobile.
- So looking at the pyramid I came up with general three layers. On the bottom layer, there are the desktop browsers for example Chrome, Firefox, Safari just runs a laptop to simulate some mobile behavior like responsive design. And you can even inject user agent strings so you can really pretend that you're a mobile device. It's the bottom layer because you can get fast feedback. It is quite fast to boot up and you can get really create some cool stuff. So next layer is the simulator/emulator. I think there this is the most underestimated layer of the pyramid because at the beginning in the early stages of mobile simulators/emulators it wasn't that stable. In the last year or so it's become much more mature and stable. So you can really validate some touch interactions especially visual testing. Then on real devices. It's more about usability and real-life conditions.
- For testing in general including mobile testing I like to follow some generic test guidelines which applies to any situation. One of them is test in isolation. What I mean is really isolates your tests unit as much as possible. Do not do full integration with live systems or life mainframe systems at the backend. Try to stop and mock as much as possible so you can really test what you want to test. Focus on automation as much as possible. Of course, not everything is automatable but who most of it in most cases are. Really focus on automating them so that you can run regression testing. And you can focus on the more edge cases or stuff like visual testing that really requires more like a human intervention. Set a zero tolerance on flaky tests. Trie to make it as stable as possible and reproducible. Focus on fast feedback.
- My actionable piece of advice is when you dive into mobile testing use the mobile test pyramid approach where you utilize each layer of the pyramid: desktop browsers with mobile simulation for extensive functional testing in response to design testing. The mobile simulator emulators use it for functional end user flows and visuals. Real devices focus on usability testing and real life conditions.
Resources
Connect with Kwo Ding
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.
Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!
