Anton Malinski
Shift left, like, I guess, like the whole phenomenon here, like is shift left can be described from production to staging environments, from staging environments to CI. And I guess we're focusing more on this, like CI to local development loop. When building mobile infrastructure or a mobile platform, ideally, you were like, whenever you say you run your test cases in CI and something failed. You don't really want people to have to run it on very different devices. You want to build this thing as a proper service where something failed on a device with a configuration, let's say it was a five-inch screen, this specific DPI, this setting, like this specific version of the operating system, and you have exactly the same API on your development machine, you can use exactly the same CLI tool, for example, or a web UI and request exactly the same device. If it's a real device, you can get exactly the same device where it failed. And this way you can kind of shift left the signal from CI to your local execution. I don't really like the pattern of trying to push for a signal. Like if you can run something from your local and there is no real difference between CI and your local executions at the end of the day. So like, if I want to check something, if it's working end to end, I'm just like my dual user experience, I have a button. It just runs everything as fast as possible without any flakiness, without any kind of something unstable, and just gives me a signal. Does it still work or not? And in my opinion, it's definitely possible, but for 99.9% of cases. So there are those things where you really need a specific hardware. What I mean by this is we obviously have to talk about emulated and simulated devices versus real ones. For simulated emulated ones, you can spin them as much as possible. You can scale them. You can buy more servers for spinning up Android emulators, buy Apple Mac Minis for running your simulators. That's very cheap. But if you want like rare devices, let's say you want some Chinese device without Google Play services, you're not going to buy hundreds of them. And you're not going to buy them for like a scaled execution so that everyone can run all of those tests in like 15 minutes. And what I mean by this, so in our case, for example, in Agoda, we're running around 24-25 hours of tests in 15 minutes and just simple math like how many devices do you need at minimum to even achieve this? It's around 50, right? So like it's around 50 devices, the minimum you need, and that's if you have multiple people doing this, like concurrently, 150 devices of the same model, it's unpractical and not from the perspective of it's a once-off investment. It's the problem of your users are changing devices. Whatever was popular one year ago, you bought those, like, let's say you bought 150 devices. You check it now, like your top 15 devices that your users are using, they're different. You need to buy new ones. And what are you going to do with the old ones? Like, are you gonna recycle them? Like, what's gonna end in a year? You're gonna redo it again? It's unpractical to use this at scale, in my opinion. It's like when the proper process here, I think, is when you see an issue, and usually you see issues on like real devices. First you try to really replicate it on a simulator and emulator. There are lots of APIs available now. There is like anything around network shaping, packet delays, throttling, you have all the access to the camera including you know faking video streams, you have you can mock GPS. There's so many things you can do with sensors even like for an accelerometer and try to reproduce it on an emulator and simulator. If it's at all not possible, and there are cases like this, of course, shift it to the real devices. And that's where kind of balancing between those like real devices and virtual devices comes in. How many things can you realistically reproduce on emulated devices? If most of them you can do, it really depends on your application. Some applications are just simple crud things around API. They don't have really use any specific sensors. They don't need like low level access to the hardware. Most of them can be realistically done on an emulator, but if you're doing something very, very interesting, I remember I was working on one app. It was called BookReader, called BookMate, and we were partnering with this company which was building this very interesting phone with an E Ink screen on the back of it. So kind of front face was LCD, on the back you had E Ink device screen, and you can't emulate this. Like, whatever you do, it's just not possible. You're going to have to test on an actual device, and there's going to be some proprietary API. So there's only one way to develop this. You can try to abstract everything behind interfaces, but there are those cases where this really is just a requirement to use real devices. But again, you want to try to strive for using emulated devices because those ones are easier to maintain, they're cheaper to support, they are updatable like whenever the new version of the operating system comes in, you can just update the package and deploy the new version of the simulator and emulator. But with real devices, whenever someone even says like Samsung S25 or something, which specific version of this device does it mean? Because they are varied between different markets even, they might have a very different silicon chip inside. They might have very different like modem chip, they may have a different firmware version. So which specific device are we talking about? And even in that case of updating firmware that I mentioned before, like, do you want to update your firmware? Because some of your users will update the firmware and some of your users are going to say no to this firmware. Whenever you have a crash from a specific device, like do you want to store all the different combinations of firmware that you just want to have at hand to reproduce the issues? So it's a very hard challenge.