Welcome to Episode 86 of TestTalks. In this episode, we'll discuss mobile testing with the creator of Appium, Dan Cuellar. Discover the reason Dan created Appium as well as how it can help you with your mobile testing automation efforts.
Appium is a cross-platform, open-source, mobile UI automation framework. It allows you to write UI tests for your mobile apps, Android iOS and some other OSs. Appium is close to becoming the defacto tool for mobile test automation. Based on some analysis Dan saw a year or so ago, Appium passed Robotium and then Calabash as the tool most people choose for mobile testing. Appium is used by companies like Amazon, Zameron, Perfecto and Sauce Labs.
So if you haven’t yet learned Appium, now is the time. Discover how to get started with Appium in this week’s TestTalks episode.
Listen to the Audio
In this episode, you'll discover:
- The Appium philosophy
- Why Appium was created and made open-source.
- Is XCode7 and Appium killer?
- Tips to improve your mobile app Appium automation efforts
- The Appium Roadmap for 1.5 and beyond.
Join the Conversation
My favorite part of doing these podcasts is participating in the conversations they provoke. Each week, I pull out one question that I like to get your thoughts on.
This week, it is this:
Question: Is there anything you see in the future, any new mobile testing trends you think coming up that most testers need to be aware of?? Share your answer in the comments below.
Want to Test Talk?
If you have a question, comment, thought or concern, you can do so by clicking here. I'd love to hear from you.
How to Get Promoted on the Show and Increase your Kama
Subscribe to the show in iTunes and give us a rating and review. Make sure you put your real name and website in the text of the review itself. We will definitely mention you on this show.
We are also on Stitcher.com so if you prefer Stitcher, please subscribe there.
Read the Full Transcript
Joe: Hi Dan, welcome to Test Talks.
Dan: Thanks for having me, Joe.
Joe: It's great to have you on the show. Today I'd like to talk about Appium, and since you're the creator of Appium, what better person to ask? So, in a nutshell, people that don't know, what is Appium?
Dan: Appium is a cross-platform mobile UI automation framework. It allows you to write UI tests for your mobile apps, for Android iOS, and some other OSs. Maybe Windows phone, soon, we can talk about that later. And, it lets you write it using the Selenium JSON writer you know and love from web automation.
Joe: I'm always fascinated by people that actually develop software and then make it open source. Why did you feel the need to create Appium?
Dan: Okay, yeah, interesting. Necessity is the mother of invention, I guess is sort of, you know, pretty typical. I took a job, my second job ever. I was in charge of the test organization at a dating site called Zoosk dot com in San Francisco. When I got to the job, a mobile app for still kind of, they were popular but it wasn't mobile first quite yet at the time. We had decided as a company to go to sort of really dive in to mobile, which none of the other dating sites had done really at the time. They had all done HTML5 apps, but we went full native right away.
The problem we were having was with the iOS app in particular, it was that when we would make a mistake, which, at a start-up, especially one run by people who are used to the web, where if you make a mistake, you can just deploy new code to the servers and everyone gets the fix instantly. That wasn't the case on iOS because of the app store and the app store review process.
For those of you not familiar with it, which many of you may be, the way Apple's app store worked, especially back then, they've made an effort in the last few months to get better about this, but, you would submit your application to them and they would run some static analysis on it. Then, they'd review the app, and they'd try out some scenarios and all-in-all, it would take about usually, four to five days from when you submitted the app for you to be able to actually put it in the store so your customers could download it.
What was happening with our app is that we would move fast and break stuff, as was the culture at the time, was the culture in web start-up era, anyway. It cost us a lot of money, because we were pulling a good chunk of change in on mobile, but when we broke stuff, it would take us five days to get a patch out to customers. Plus, the development time, and the investigation time, and all that.
So, if we broke something like chat on the dating site, we'd have to live with it for five days, and none of our mobile people could chat. Sometimes even longer, because back then there wasn't the auto update functionality on iOS or Android like there is now.
Dan: So you'd actually have to have customers opt-in to the update. So because of that, I thought as the head of testing, “how do we tackle this?” Then I think “well, at [inaudible 00:03:04] website, we just use Selenium for everything.” And then, you know, that was sort of how we caught this kind of stuff. I looked around at the tool set. It all really sucked. I won't name names, but back then this is 2010 or whatever it was at the time, and there was Robotium, and there was some other things out there. They just really were very primitive and you couldn't write stable, well-architected tests with them.
Joe: What's your place now with the project?
Dan: Fair enough. I would say largely mascot at the moment. I go to a dozen conferences a year, or more. I think the next three months, I have Tel Aviv, Kiev, Istanbul, Siberia, Latvia. I don't know why, that's just the next three months, I have to look at my calendar, it's really bad. But, I'll go anywhere that'll have me. So if anyone here is listening, I'm looking at you, South America. I haven't been yet. Invite me.
But yeah, I do contribute a little bit here and there. But like, largely Sauce Labs has so many full time employees working on it, along with a few other companies that have contributed some people that I'm not needed to build the stuff anymore. And, at this point, the problems we're solving, I guess, it's always that kind of thing that everyone likes to do. “Oh, I want to do the fun part, then everyone else can sort out the details.” Until we're to the point where like, “oh, well there's a new Android SDK. We need to make it work with Appium.” I guess that doesn't really interest me, per say, right? It's kind of nitty-gritty.
I'm always interested in the high level, like, “oh, Windows Phone doesn't work with Appium. That's what I want to work on,” or “oh, iWatch doesn't work, or Apple watch doesn't work with it.” So yes, I do work on the GUIs, as I think I'm the only one on the project, I think there's one other person that knows how to write Coco Apps. Other than that, and I'm actually in the process of making it work for 1.5 at the moment. I don't contribute so much.
I more just probably spend thirty or forty-five nights a year in a hotel room somewhere, wherever will have me, whatever weird corner of the world with whatever weird, terrible Siberian winter that they're having. And, just go and talk for forty-five minutes and meet people, and all that kind of stuff, and just try and promote the product. Most of the code is written by a few guys at Sauce Labs these days. I haven't been like a heavy contributor in two or three years at this point. There's a lot of people, it's not just me. I made the thing, but without the guys at Sauce Labs, it never takes off. Without people like Johnathan Lipps and some other people like, it doesn't get as big as it has.
Joe: Awesome. Seems like it's almost like a Selenium was a few years ago. More and more people are talking about it and in all sense, Selenium took off as the tool to use for automating web browsers. So, do you see that as a reflection point now with Appium now that Appium has reached that point where the amount of adoption now is someone's starting mobile app development testing, that they probably would use Appium?
Dan: Yeah. I've been sent some charts and things by different people that sort of show that at some point, I think about a year ago, Appium passed out Robotium and then Calabash was also a big player for a while as well. So it seems like finally only about a year ago, that started to happen. Then there was a big month we had where Amazon came on board, and Zameron came on board, and Perfecto, and it was just like one week where five or six big players all decided to put Appium in their clouds. That was like a huge win.
So when you see big companies like that, and then there's some people at Microsoft actually working right now on getting Appium support built into the mobile operating system at the moment. So you're seeing like all these big companies start to support it now, and that's…yeah, I definitely think we've hit that point where it's tipped a bit.
Joe: Awesome. I think that goes back to the Appium philosophy. Did you create the Appium philosophy or is that something that someone else came up with? I guess it's some main core principles that Appium, the project, believes in.
Dan: So yeah. I'm not exactly sure who you can attribute to. I think, I'd have to go back and check the tape, Jason Huggins, who originally created Selenium and used to work at Sauce Labs. When we had this conference called the Mobile Test Summit, and basically what Sauce Labs had done unbeknownst to all of the speakers at the conference, myself included, was they had gathered all the people that made all the tools. The Calabash people were there, [Sucooley 00:08:10] people were there, all the big tools at the time. This is in 2011, I want to say.
And, they sort of wanted to pick one to put in their cloud and Jason had designed this rubric sort of for grading them. He presented it as part of the keynote of that conference. The Appium philosophy you see is largely based on that rubric he put out.
Dan: For like what makes a good automation tool. So you'll see that like, you don't have to modify the application under tests. That's like a huge one. It runs in whatever language I want to run it in. It's open source. It's free. All those kind of things that other tools like Calabash, they do have to modify your app. Frank you have to modify your app. There are other tools that aren't free, Soasta and what not. So that was the rubric that Sauce was using to sort of pick the tool that they wanted to put in their cloud. Eventually, they fell on board with Appium because it's met that rubric that Jason put out.
I think the Appium philosophy, it was definitely written by the guys at Sauce Labs, not me. I definitely thought about it, though. Like I definitely, when I was making Appium, like in my head, I'm like, those were the things that I thought were important.
Joe: How does something like Appium work with all these different language bindings, all these different technologies? Can you just explain a little bit more? I guess you mentioned the JSON protocol? Is that what does the translation behind the scenes through an API that makes it possible to just kind of interpret whatever you're sending it and just switch it over to what Selenium needs, or the application test needs?
Dan: Yeah, so the way you can think about Appium is it's just a server that runs, and it listens to our HTTP requests, just like any other server. And all the language binding is in C#, the Python, the Ruby, the whatever. All they do is just send web requests to this server, and so because we use the same protocol as Selenium, we were able to steal all their code. Not steal it, you know it's re-purposed and re-use, and share all their language bindings that people had built up over the years for PHP and all these Pearl and all these weird languages.
Joe: As I mentioned earlier, I actually got an email recently about someone that asked me a question about Appium. I'm not sure if you're aware of this, but he said the his team doesn't want to use it yet because they read in the Xcode 7 release notes that the existing UI automation support and instruments is deprecated.
Joe: Awesome. Yeah, that's great news, because that was the first time I heard of it and I panicked and noticed that he was right, so. Just so happens that I was talking to you today, so, awesome. Great answer.
Dan: Yeah, don't worry. We thought of it. It's covered.
Joe: So along the same lines. Can you talk a little bit about the road map? Where do you see Appium 1.5 or 2? I don't know if you can even talk about it, or, not make commitments, but is there anything you can elude to? What you plan on? Or, what your vision is for Appium in the future?
Dan: Yeah, so 1.5, which is going to release really soon was a total rewrite of the whole thing to make it stable and sane. Appium has sort of has been…it had been monkey patched for three years. They just add things left and right. And so, the guys at Sauce, I don't remember how long ago, maybe it was May, they just sat down and said, “we need to make this maintainable.” So it's been re-written in [ES7, 00:12:20] I think, and now it's no longer call-back pyramids forty levels deep. Things like that. It's [saying a sync? 00:12:33] away structure. It's very modularized now, to where if you wanted to add support for a new operating system, you can inherit most of the implementations from the base class, and then just implement a few methods here and there to do it.
But, coming in it, there's a lot of cool things. For one, there's the two new iOS support systems. So we have the old UI automation [JS 00:12:54] implementation that I wrote back in the day that is going to be deprecated in about five months or so, or so I've been told. But there's a new one that uses the newer [frameware(firmware?) XE UI 00:13:05] test, and there's also one that uses the Facebook web driver agent framework. There's some cool advantages you get with each.
With the Facebook, the web driver agent framework, you can do things like multiple simulators. You can automate apps that you don't have the source code for. You can automate UI iOS dialogues, things like that. With the [XE UI 00:13:27] test, but I guess the Facebook one won't run on a real device. And then, with the [XE UI 00:13:32] test, you get the one that Apple provides, and you can run on real devices. And, it basically does everything the old one did before.
But the Facebook one is I think multiple simulators at the same time, which is why we hadn't been able to do iOS without some crazy hacks. So did a lot of cool stuff you're going to be able to do on a simulator in 1.5 that you couldn't do before. So, things like logging with Facebook, or adding contacts in the contacts app, changing system preferences, all that, you're going to be able to automate with the new one, I believe. With that, for simulators.
And then, with real devices, you'll get support going forward. And, the nice thing is, if you do use a legacy device, an iOS six, seven, eight device, Appium is that translation there in the middle that can fork it over to the right framework to automate that device. So, we'll continue to support the older implementations for the legacy devices, much like we do for Android where we use [Selendroid 00:14:20] for anything API fifteen or lower, I believe. Which, at this point, there aren't a lot of Android devices out there with that API, but in theory, if you were in a market where that was important, you could do it.
Then I guess the other thing is [Nod 00:14:32] and 1.5 are coming soon. We're re-writing the Android support to use the new UI automator two framework that Google released a little while ago. That will make it much more stable, much more reliable as well.
Then I also mentioned that Microsoft, as I've been told, there's some three guys there I've been emailing forever with. They're working on getting Appium to work out of the box with the new Windows OS system, desktop and mobile.
Joe: Awesome, and like I said, just how Selenium is taken the lead, it sounds like Appium has also taken the lead for mobile automation. So, I get a lot of emails. I would say that people should not worry about investing in it, because it sounds like you have a very active community. You have some big companies like Sauce Labs, and sounds like Microsoft, that actually know about it and are supporting it, so that's a good sign for any project.
Dan: Yeah, there's a lot of big companies out there that are using it now. I don't know if I can name names, but you know, a lot of the bigger companies are out there using it. Some of them don't, but yeah. There are bigger ones like Amazon that support it in their cloud. Then there's places like Sales Force and LinkedIn that use it on their stuff. So, you know, it's had some bigger adoption now, which is good. But a lot of the small guys will, too. I know that with the 1.0, I think we hit, was it a hundred and eighty thousand downloads of it? So there's quite a lot.
Dan: Yeah, and that was two years ago, so I don't know what it is now. But, yeah it could be a lot more than that.
Joe: Are the things you see over and over again, that you think most people using Appium are doing wrong, or you scratch your head and like, “why are they using it like this?” Or, that's probably not a best practice or a good idea?
Dan: Yeah, yeah. Some of it's my own fault. Yeah. The recorder. Oh, the recorder. It hasn't been updated in three years. I don't know if it generates valid code anymore. A lot of people I see using that, and I just…it's sort of there just kind of for fun, and then it's never been maintained.
Dan: What's some other stuff I see… The crazy X path, you know that you always see where it's like, people will copy the X path from the inspector-
Dan: -where it has like, you know. It basically just lists every element in the dom and gives it an index for what order it's in. So you make these really brittle locators for all your tests? See, that kind of thing? I think other things that shake my head at…
A lot of times people don't know about the how to automate hybrid apps, but that's sort of more our fault for not…I mean it's sort of not obvious, but they don't know to switch the context and then they'll get regular Selenium tests inside the web view object. Yeah, stuff like that, I guess.
But, in general, like there are a lot of companies out there putting out and training people better. So, it's not nearly as bad as it used to be. When the project first started, we would look at the message board every day and we'd see the same damn questions. And there'd be some bugs where, ‘we should just fix this stuff, because I'm tired of answering it over and over again.' But in general, I think people pick it out pretty quickly and there's enough…the Google returns pretty good results for most common problems nowadays, which is nice.
Joe: Is there anything you see in the future, any new mobile testing trends you think coming up that most testers need to be aware of? Or, anything you see on the horizon that you thinks just about to explode? That people need to get their heads around? Including mobile, for mobile testing or anything like that?
Dan: There is this whole idea of like, it was just the phone, and now it's the phone and the watch, or the phone and the car, and the phone and the house. We're working on some of that by getting Apple watch support into Appium and things like that. That's something that's a bit new and a bit different. I mean, Apple has gone sort of resolution crazy now, and Android's sort of always been there where it can always be any shape and any size. But, we're sort of focusing on Apple now, which is a whole new pile of bugs for people.
Yeah, I guess mobile is, it's kind of it a rut right now, right? There's not a whole lot. Like it's not like when the iPhone came out like eight years ago or whatever it was, and everything's changing every week. So yeah, it's staying pretty stable I think at the moment. People are kind of running out of ideas. Like, “what else can I put on my phone?”
Joe: Just curious, how did you end up in London after working in the valley for a while?
Dan: Oh all right, this is a interesting question. In general, I guess I found myself working seven to seven every day in the Valley. You know, on call on the weekends, or when disasters would happen and things like that. I just thought to myself, there's gotta be something better, right? I always heard over in Europe that people don't work as much as we do in America. And, I also never lived abroad, and I kind of wanted to do it as like a mind bending kind of thing. To just try it out. And, you know there's some other stuff in there where I kind of wanted to slow down a bit. So yeah, I thought maybe give it a go.
And I found that Shazam needed someone to manage their testing and, like I'm a music slash computer science student, right? Degree.
Dan: And so, Shazam seemed like a good fit. And you know, I've been in the dating site for about three years and I kind of like, I had done whatever it was there to do out there at the time. So yeah, I thought it was a good time and that was in London, so I took it. And yeah, I really enjoy that I get twenty five holidays a year. Vacation days, I should say. I'm in America, right now, on this radio thing. I'm imagining most of the audience is not British. But yeah, I get twenty five vacation days now. I generally work eight or nine hours a day. It's not twelve. Totally a good experience from like a personal growth thing.
I've learned, I look at the world very differently now. There's things I never questioned before, you know? But now I see that people don't think the same way over here. Just silly stuff, like in America, newer's always better, right? In Britain, that's not the case at all. Almost like, older is better. You know and like America, it's sort of designed around working long hours.
Dan: Like the food is fast. Everything's open late. Everything's open weekends, you know. And here, like on Sunday, things are open twelve to five. If I need to go to a pharmacy, I can only go between eight in the morning and six. So it's kind of silly stuff like that, like “oh yeah. I remember when we were like that.”
You know, and then I guess I also manage people out here. It's been like a good growth from a manager perspective to say, “how do you motivate people that aren't American?” I feel like in America, we're very ambitious. We're very…the guy that makes your Slurpee at Seven Eleven thinks he's going to own the place. Everyone on American Idol thinks they're famous, everyone just wants to hear them sing. It's definitely not the case here at all. I don't know why it is, I'm not, whatever. It's silly things like that.
It's been a challenge, like how do you get through to a different culture. Like when people come to America, they sort of buy-in to the whole American dream, work hard and I'll be rewarded kind of thing. In a different country, you have to buy-in to the way they think. It's not that way here. It's been an interesting challenge, and just little other things like that. And, I've gone to see a part of the world. I get to travel to conferences every year, all the time. So I've probably gotten to see two dozen countries in the last year and a half. All I have to do is go talk for half an hour, and they'll give me a flight and a hotel for the evening.
So yeah, there is a bunch of other reasons why, but mainly I wanted to…I realized all I was doing was working, and there was more to life than that. And, I want to give it a try, and try something different, and I've really enjoyed it. I don't think I'm coming back.
Dan: Yeah, I like America, and I don't want it to change. I want it to keep doing what it does.
Dan: But I enjoy other stuff too, and I like having more balance out here. That's sort of part of the story, I guess. In the end of the day, the guy who came up with the thing everyone uses now, this wasn't developed by Elon Musk, or some of his brainiacs over in a secret bunker in New Mexico, or whatever. I was just a guy that was tired of losing money and getting yelled at by my boss, because shit kept breaking. So I was out there, I built this thing, and I came to a conference in London. I applied to talk about something entirely different, and it was at the Savoy here in London. It was the Selenium Conference of 2012, I think.
As part of my talk, I showed off that I was using the Selenium protocol to control iOS apps, and people just went nuts. They were like, “what? You're doing that? How are you doing it?” So, I gave a lightening talk at the end of the conference, and people just told me, “you have to give a lightening conference, show us how it works.” And, I showed them how it works and it was so funny. Everyone clapped and cheered, and there was fanfare and I thought, “oh wow, people are really going to like this.” Then, I heard nothing for like, three months. No one emailed me. No one called me. It wasn't…It didn't even happen.
Then, over time, eventually Jason Huggins from Sauce Labs invited me to a bar one day, three months later in San Francisco, to show it to some guys. Eventually, the whole thing takes off, and whatever. But, I guess the whole moral of the story from my perspective is people should share their crazy ideas with people and open source more stuff, and just be more transparent about some of the cool stuff they come up with, because it might be quite useful to people. I mean, it probably won't be, and I've done other stuff lately that no one seems to give a crap about. Not everything is like this. But, it might also be something like Appium, and ends up being really, really important to people.
I always leave it like, you may be some random Selenium whatever, Selenium developer at some company, but you probably, you might have some good ideas, too. You should go to conferences, you should blog, and you should tell people about them because you can do this, too. I'm no smarter than you guys, right? I'm one of you. I'm not…I don't work at a fancy company, or whatever. I don't do it. I worked at a dating site and I just needed to make this thing, and I made it. So I always try to encourage people to do the same sort of thing and think outside the box. Not what is everyone else doing, but what is everyone else not doing? What else can be done? Most people will just go out and, “I'll read a few blog posts and see what the experts are saying is the tool to use.” I guess, that's not what I did. I did that first, and then I said, “well this is all terrible.” Then, Appium was born, right?
Joe: I'm sure a lot of people are going to be inspired by this. So how can someone contribute to Appium? Can they contribute? Is there a process in place? Do you encourage that?
Dan: Yeah, yeah. We have probably around a hundred contributors to Appium, like all time? A hundred and something. Probably like twenty regulars, maybe. Maybe a little bit more. Actually, I could get the number, but I won't. It could be more like two hundred, I haven't looked at it in a while. But, yeah.
Johnathan just did a presentation in San Francisco this week and he posted his slides up, and it explains the whole layout of the new Appium 1.5 architecture. It's quite modularized, and it's a lot friendlier now that it's not called [back 00:25:04] hell. It's a lot easier to contribute.
But even though, if not that, stack overflow is people who are experts can post answers there. There's also documentation that needs to be written. There's a website that is four years old that could probably use a face lift. I mean, bootstrap has sort of gone out of fashion, I think. The website doesn't really convey the modern idea of what Appium is anymore, anyway. But there's just tons of stuff people can do that could be quite useful. So, I always encourage it.
Joe: I think more people should do like you said, and just go out and try something and see. Just help people, and give back.
Dan: Yeah, there's a Steve Jobs quote there, and I think it's like, “once you learn that the people came before you are no smarter than you were, and the things that they made were just their best idea at the time, and that you can change and influence that, you'll never be the same” is the quote, right? I think that's totally true. Like, question everything. Just because the guys at Facebook use the thing doesn't mean it's the best solution. I think Appium is a shining example of something like that.
Joe: Okay Dan, before we go, is there one piece of actual advice you can give someone to improve their Appium automation efforts? Let us know the best way to find or contact you?
Dan: I would say, read the release notes, because we're always tweaking stuff and making it better. A lot of times, that knowledge doesn't get out any other way. We do releases almost every two weeks, I feel like at this point. Yeah, I would say, read the release notes. There's good stuff in there. It will also prevent, we get the same bugs logged over, and over, and over again. I think we have eight hundred bugs right now in the Appium repository.
Dan: And I think there's probably only fifty actual bugs, but they all appear like forty times each. Then, best way to contact me. I'm on Twitter. My last name is hard to spell. My Twitter handle has my last name in it, but it's the as in like, the word the, and then Dan, as in the first three letters of Daniel, and then my last name is spelled C as in California, U as in umbrella, E as in elephant, L as in library, L as in library, A as in apple, R as in radio (@thedancuellar). That's my Twitter handle. The website is Appium.io.
If anything could explain to me why all the things end in IO now, I never found that out. But, we have a dot IO domain, and it seems to be hip and cool. Those are probably the best places. Twitter, I like twitter. It's my favorite, and now we're going to be able to send, what is it, ten thousand character tweets? So you can send me actual questions on Twitter.