Projects can fail for many reasons. Some fail because they test the wrong things. Others fail because they test too much. In this episode Todd H. Gardner, an enterprise consultant and founder of TrackJs, shares his many years of development experience in testing atrocities (AKA terrible testing) and what we can learn from them. You’ll come away questioning your own testing. So let’s forget about our long-held testing dogmas and start doing a better job of testing the right things in our software.
[thrive_megabutton mt=”Read Full Transcript” st=”” color=”blue” link=”https://testguild.com/2015/11/25/episode-79-terrible-testing-case-studies-podcast/” target=”_self” align=””]
Todd H Gardner is the president and co-founder of TrackJS (trackjs.com) and an independent software consultant putting software and businesses on the web. With over a decade of experience building software systems, Todd has built large enterprise systems, complex software products, and launched companies.
Quotes & Insights from this Test Talk
- The question of, “Are you testing the right thing?” I don't think is asked often enough, because it's really hard. It's probably one of the hardest problems in software development to answer, because the fundamental question inside of it is, “What is the risk of this thing that we're doing?” and that's a really contextual question about what your project is. There's all different kinds of risk that comes in with software development, because this is a incredibly creative and complex activity that we're doing, but we're also working in an incredibly complex market, or niche, that you're operating in, so there's all different things that can go work at so many levels. You have to think really critically about that.
- The Testing Pyramid is misleading. I think that that is a flawed model for two reasons. The first is that it misses this whole thing that we've been talking about: market risk. About whether or not the project itself is a good idea, and how do you test that? Which is really operating at a level above system tests. The second problem that I have with the model is that it's implying volume. It's saying you should have more unit tests than integration tests, because unit tests are cheap to write, and I think that that's missing the point. It's not how cheap or expensive it is to write and maintain. It's what kind of risks are being addressed by those sort of tests.
- You can never test everything. It doesn't make financial sense to test everything. There's always some parts of your system that you can't adequately test for, nor should you, because it's just not financially worth it. That's where monitoring comes in.
- The overarching point of my Talk and my message is that it depends. I'm not saying, “Hey, Todd said you don't need to test! Go ahead and push it into production – it's fine!” That's totally not what I'm saying. What I'm saying is, that you need to think critically about your application and your tolerance for risk, and build a testing structure and a monitoring structure that accommodates what you're willing to do as cheaply as possible.
[tweet_box design=”box_3″]Any sufficiently advanced, automated testing system is indistinguishable from monitoring~@toddhgardner[/tweet_box]
- Case Studies in Terrible Testing Todd Garder 2015 OreDev video presentation
- TEST AUTOMATION TRENDS FOR 2016 AND BEYOND: STAYING EMPLOYABLE IN CHANGING TIMES
- THE TEST AUTOMATION PLAYBOOK : SUCCEEDING WITH AUTOMATION AWESOMENESS
Connect with Todd
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.
Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!