Automated testing for Web applications is a crucial piece of the development puzzle, ensuring that everything works when the customer starts using it. However, mobile app testing brings along new challenges that are different from desktop Web testing.
Jason Huggins, co-founder and CTO of Sauce Labs, a software-testing cloud provider, shares tips on automated testing and how it applies to mobile.
Click through for 10 tips that can help you successfully test mobile apps, as identified by Jason Huggins, co-founder and CTO of Sauce Labs, a software-testing cloud provider.
Pick one language and one machine to test Web applications. Find the right combo that resonates with developers and stick with it. For example, Selenium was pretty good at testing JavaScript-heavy Web applications. But it really excelled at driving Firefox on either OS X or Linux with Java client libraries. This was the “killer combo” that resonated with developers. Firefox was their favorite browser and MacBooks were their favorite development machine. Java probably wasn’t their favorite programming language, but it’s what most large-scale Web apps were written in.
Like the folk story where a village turned a pot and a simple stone into delicious soup, testing programs need lots of help from lots of people to get the job done. In the beginning the programs may fail, but over time and with lots of help they improve. You don’t need a detailed road map, but you do need to tell people what mountain you’re climbing. If you start small enough, yet dream big enough, you just might be able to convince strangers to help you on your journey to testing success.
Don’t require developers to modify their apps in any way just to start using your test library. Developers should be able to test their app “as-is.” There are several reasons this is important — some technical, others cultural. Developers want to keep the surface area of their apps as small as possible. Having extra code lying around in the app leads to an unnecessary increase in file size and another potential vector for bugs or security holes. And some teams charged with test automation are separated from the developers who compiled the app. They have to write tests, but they might not have any (or at least, an easy) way to modify the source just to make it automatable. Yes, that’s sad and wrong, but a reality for many organizations. The least you can do is make their lives a little more comfortable. Make it easy to test an app as-is.
Newbies want some help. Don’t leave them hanging. Focus on the “first 15 minutes” experience. Although it’s controversial in the testing world, you should consider having a record and playback tool for helping new users write their first test. And like a good mentor, encourage them not to abuse the tool.
Pay attention to how developers prefer to work. Let developers use their favorite text editors or IDEs. They also like to write their tests in the same language as the app’s source code, and check those tests into the same code repository. And don’t invent a new programming language.
Instead of inventing a new protocol to talk in a new way to a browser, reuse one that already works, such as HTTP for message transport. This allows testing programs to drive tests remotely on different machines with different OS and browser configurations from anywhere on the network. Using HTTP also lets testing scale up gracefully. Functional testing can be slow, but throwing more computers at the problem is a great way to speed things back up again. Through the magic of HTTP proxying, you can distribute tests to huge farms of machines without having to push that complexity into the tests themselves. So, even though worrying about scaling is usually considered a premature optimization, it’s not that premature when it comes to testing. Sometimes, you are going to need it.
Be welcoming of ideas from other projects. And occasionally, merge and join forces. It’s important to be humble and know your project’s weaknesses. When someone comes along and fixes all your problems, and they inconveniently didn’t solve them in your own codebase, don’t get mad. Instead, convince them to take over your project.
It’s not enough to simply create. You also have to promote your work. Speak at a meet-up. Speak at conferences. Write blog posts. Create screencasts. But you don’t have to do it alone. If you solve an important problem for the people around you, they’ll be more than happy to tell the world about it on your behalf.
Everyone has their own story for why they believe in the power of open source. Having access to the source code is especially important in testing — as SDKs and standards evolve, testing tools are on a treadmill racing to keep up. It doesn’t scale to expect just one project or one team to know how to automate every crazy cool new HTML5 or iOS feature at all times. With open source, when your users run into a feature that your testing tool can’t handle, they’re not stuck. They have the power to fix things and move on.
Do the things that others think are crazy. Within reason, of course. If you believe passionately in a particular idea, do it anyway, despite what people say. In 2003, using JavaScript was considered a very unprofessional thing. It was buggy and inconsistent in every browser. But using JavaScript let us create some really cool and fast features that we couldn’t do any other way. Then six months later Gmail and Google Maps came along and using JavaScript was no longer crazy. Foolish today, genius tomorrow.
