It's December, folks, and you know what that means: holiday cheer!
The Firebase Test Lab team is fully invested in making this season the greatest of seasons. Remember Halloween? What a hoot! Also, this happened:
(Note: these are actual Test Lab engineers, in costume, actually beating each other up with foam sticks at a Halloween party. Both get lumps of coal this year.)
We're getting ready for the holidays! So, sit back, pour yourself some eggnog, and read about what's new for your Android testing enjoyment.
Many of you are using Robo to automatically test your apps in Test Lab. Since you don't have to write any code to make it work, it's the gift that keeps on giving. Even better, you can have it fill in specific form fields and push buttons with some easy configuration.
We've found that some apps require more of a nudge to navigate into the parts that need the most testing. (Hey, even Santa needs help from a few elves!) Now, with Robo scripts, you can record a series of actions to take in your app, and play that back before Robo takes over. It works a lot like Espresso Test Recorder, except the output is a JSON file that you upload along with your APK when running a Robo test. With these extra instructions, you can guide your app past introductions or login screens.
Of course, your best bet to maximize the test coverage is writing Espresso tests that drive your app. I heard that it's easier than driving a team of reindeer!
Do you use the screenshots in Test Lab results to check if your app displays correctly? It's a great way to see if you app renders "naughty or nice" on many different types of screens. But if you test with lots of devices in a single test matrix, it can be kind of a pain to sort through all the results to compare the same screen among multiple devices. Now, Test Lab will cluster them together in your test results, so you can see all the different screen sizes, densities, and orientations from your test in a single place.
The Test Lab team is always busy at the North Pole (located at a data center in Atlanta) bringing you new devices to test with. The latest additions are the Sony Xperia XZ Premium, the Moto G4 Play, and the Huawei P8lite, delivered straight to your digital stocking. However, sometimes old toys break and need to be recycled. At the Test Lab workshop, we call that "device deprecation", which means we take old devices out of commission as they become unreliable. To see a (twice-checked) list of devices that are currently available, in addition to those being deprecated, click through to this page. Once a device is marked as "deprecated", it'll remain available for a month, then removed.
Deprecated devices look like this in the Firebase console:
And like this in the gcloud command line (note the "deprecated" tag in red):
You better not pout, you better not cry ‐ these devices served longer than their expected lifetime!
Or, just join us on the Firebase Slack in the #test-lab channel. We're all friendly there, so be good, for goodness sake!
At Google, we understand how important the quality of your application is in order to grow your user base, increase customer satisfaction and boost your revenues. When we took a closer look at data from 1-star reviews on Google Play, we observed that more than 50% of these reviews are related to bugs and crashes in the app.
For many developers, finding and resolving these problems prior to release can be very challenging. As the Android ecosystem continues to grow rapidly, it can be a daunting task to maintain a high quality, performant app that scales well for your users. As your user base grows, you may only have access to a handful of devices to use for testing. Accessing devices that are unavailable in your country is also difficult. Moreover, setting up a proper testing infrastructure that facilitates pre-release testing is a very expensive and time-consuming process that requires continual maintenance.
To help streamline the testing of mobile applications, we’re introducing Firebase Test Lab for Android, a platform where you can automate testing with the same tools that Google uses to test its own products. With a few easy steps, you can launch your tests on our physical devices to help ensure the best possible quality for your applications.
Testing Android applications normally involves writing instrumentation tests to script the interactions with the app. If you’ve already written tests using Espresso, UI Automator 2.0, or Robotium, you can begin running those tests today on devices hosted by Firebase Test Lab.
Authoring instrumentation tests is easier now with Android Studio 2.2 (or newer) using the new Espresso Test Recorder tool. All you have to do is launch your app in recording mode, and the test recorder will observe and remember all your interactions with the app, then generate test code in Espresso that duplicates those interactions. You can then turn around and run these tests in Firebase Test Lab.
Even if you’re not writing your own tests, you can still use Firebase Test Lab. We have a fully automated, intelligent test called a Robo test that will crawl your app by performing interactions with your app’s user interface. You don’t have to write a single line of code to gain the benefits of this automated coverage.
With each test you run, you will select from a variety of device manufacturers and models, Android versions, and configurations (Virtual devices are also now available as a beta offering). Firebase Test Lab will then run your tests on multiple devices simultaneously to satisfy your selections as quickly as possible. When the tests are complete, the results from the test will be stored in your Firebase project.
Testing works best when it happens throughout the development process, not just before you publish your app on Google Play. To simplify this process, Firebase Test Lab gives you the ability to invoke tests directly from within the tools you’re already using, such as Android Studio and the Firebase Console. Our command-line interface allows you to run your tests through your continuous integration setup. Additionally, after you opt-in to receive the pre-launch report in the Google Play Developer Console, every new version of an app you publish to an Alpha or Beta channel will receive a free, five minute Robo test. These test results will become visible in the Play Store Developer Console.
The Play pre-launch report generated from running the Robo tests is currently available at no cost! For more customized testing using Firebase Test Lab, customers on the Blaze Billing Plan will be charged $5 per physical device hour, and after October 1st, 2016, $1 per virtual device hour. Prior to that, virtual devices are available at no charge. See our pricing page for details.
Running your first test on Firebase Test Lab is easy. You can follow our step-by-step codelab which walks you through various scenarios for running your first test. You can refer to our full documentation here. If you have questions or encounter any issues, please ask in the Firebase Google group.
Happy testing!
