API Acceptance Tests with Cucumber and Rest-assured
On projects where multiple systems undergo development at the same time, it’s crucial to maintain a clear picture of how they should interact. We commonly have a backend system providing a REST API to multiple frontends (browsers, mobile apps, chatbots, IoT, etc.). Because it will likely change over time, keeping the API picture clear and up to date can be a significant challenge. How can we efficiently describe the currently expected behavior and know if it’s working as expected so teams don’t spin their wheels due to miscommunication?
Cucumber
Cucumber helps us write readable requirements upfront that can be tied directly to executable tests. Here’s an example for a guestbook REST API:
Scenario: Read a list of guestbook entries Given I'm using the staging API environment And the guestbook has at least "2" entries When I make a GET request to "/guestbook/entries" Then I get a response code of "200" And I get a response with at least "2" entries And each entry has a "name" And each entry has a "date" formatted as a Unix timestamp
Each line in the scenario above represents a discrete Cucumber step. A developer can now write a short block of code to fulfill each step.
There are options in various languages for fulfilling Cucumber step definitions (e.g. Ruby, Javascript, Python, .NET, Java). I chose the Java implementation, Cucumber-JVM, for these reasons:
- Works with many out-of-the-box reporting and automation tools - because it’s JUnit-based
- Intuitive IDE support for code assist, breakpoints, debugging, output formatting, etc. (Intellij and Eclipse)
- Easy-to-build HTTP request and response assertions using the Rest-assured library
Cucumber on Java
Using Cucumber-JVM and the Intellij IDE, I get automatically generated step definitions like this:
public GuestbookStepDefinitions() { Given("^I'm using the staging API environment$", () -> { //short block of code goes here }); When("^I make a GET request to \"([^\"]*)\"$", (String path) -> { //another block here }); Then("^I get a response code of \"([^\"]*)\"$", (Integer code) -> { //and another }); }
Next we fill in the implementations using Rest-assured…
Rest-assured
public GuestbookStepDefinitions() { private RequestSpecification request; private ValidatableResponse response; @Before public void before(Scenario scenario) { request = RestAssured.with(); } Given("^I'm using the staging API environment$", () -> { request.given() .contentType(ContentType.JSON) .baseUri("https://staging.mycompany.com"); }); When("^I make a GET request to \"([^\"]*)\"$", (String path) -> { response = request.get(path + ".json").then(); }); Then("^I get a response code of \"([^\"]*)\"$", (Integer code) -> { response.statusCode(code); }); }
The
Given
andWhen
steps are building a request with details for our REST API. TheThen
step callsresponse.statusCode(...)
which is an assertion of the status code returned by the REST API. If any step fails we get targeted feedback like this:java.lang.AssertionError: 1 expectation failed. Expected status code but was . ... io.restassured.internal.ValidatableResponseOptionsImpl.statusCode(ValidatableResponseOptionsImpl.java:117) at GuestbookStepDefinitions.lambda$new$8(GuestbookStepDefinitions.java:66) at ✽.Then I get a response code of "100"(guestbook-entries-read.feature:12) Failed scenarios: guestbook-entries-read.feature:9 # Scenario: Read a list of guestbook entries 1 Scenarios (1 failed) 3 Steps (1 failed, 2 passed) 0m1.370s
This output is a bit verbose (we’ll worry about report formatting later), but contains important information about the failure.
The step: we see that the “Then I get a response code of 100” step of our “Read a list of guestbook entries” scenario is where we’re failing. That means the previous two steps passed successfully.
The expectation: we see that we got a response code but expected a response code. If we change the expected status code back to 200, we should get a passing test:
Scenario: Read all guestbook entries # guestbook-entries-read.feature:9 Given I'm using the staging API environment # GuestbookStepDefinitions.java:89 When I make a GET request to "/guestbook" # GuestbookStepDefinitions.java:59 Then I get a response code of "200" # GuestbookStepDefinitions.java:65 1 Scenarios (1 passed) 3 Steps (3 passed) 0m1.937s
Since we’re making HTTP calls it’d be nice to see the request and response details too. We can tell Rest-assured to print those along with our test results:
Request method: GET Request URI: https://staging.mycompany.com/guestbook/entries.json Proxy: <none> Request params: <none> Query params: <none> Form params: <none> Path params: <none> Multiparts: <none> Headers: Accept=*/* Content-Type=application/json; charset=UTF-8 Cookies: <none> Body: <none> HTTP/1.1 200 OK Server: nginx Date: Mon, 10 Apr 2017 19:09:16 GMT Content-Type: application/json; charset=utf-8 Content-Length: 4830 Connection: keep-alive Access-Control-Allow-Origin: * Cache-Control: no-cache Strict-Transport-Security: max-age=31556926; includeSubDomains; preload { "-KgBbHUcv2NWn2M6tzGp": { "comment": "Hello Guestbook", "name": "Test User", "timestamp": 1490565277672 }, "-KgBbzZE2WtRD9wz1t-D": { "comment": "Hello Guestbook", "name": "Test User", "timestamp": 1490565462287 } }
Reporting
Now when we run this test we immediately know three things: 1. What we expect to happen (the Given-When-Then statement) 2. How to make it happen (the printed request and response) 3. Is it currently working as expected (Pass or Fail)
Having this feedback continuously throughout development mitigates communication issues early before teams waste time heading in different directions. The easy-to-read Cucumber steps that everyone can read tie directly to the gritty HTTP definitions that developers need and we can drop it on a CI server to generate formatted reports visible to the whole team.
Here’s an example of formatted results from the Intellij IDE:
On the left we have a collapsible, colored outline of our features, scenarios, and steps. We can select anything in the tree and see corresponding details on the right.
And here’s a standalone HTML report:
Again we have a collapsible, colored outline that documents the expected behavior and HTTP details.
Process
This approach is designed drive collaboration early in the process so it’s a great chance to work in pairs. Pairing a frontend developer and a backend developer can help start the conversation about how systems should interact. Getting other roles like analysts, designers, and testers involved can level-set everyone’s understanding of how the product is supposed to work. As soon as we have requirements for our first feature, we can start writing tests. The code required to fulfill step definitions should be easy enough for any developer to pick up quickly regardless of language choice. I prefer to put API acceptance tests in a separate repository apart from any other production code. This limits external dependencies from affecting our ability to write and run the tests.
Serverless apps with Firebase
Firebase is a set of backend platform services (owned by Google and closely integrated with the Google Cloud Platform) for building web and mobile apps. They have SDKs for Android, iOS, web, C++, Unity, Node.js, and Java. Their generous free tier makes it easy to launch fully functional apps to a modest user base without cost.
Free and unlimited features include: Authentication, Analytics, App Indexing, Cloud Messaging, Crash Reporting, Dynamic Links, Notifications, Remote Config
Free features with usage limits: Realtime Database, Cloud Functions, Hosting, Storage, Test Lab
There is way too much to cover here. If you want all the details, their website docs are some of the best I’ve encountered. You can also demo many features right from the web console. Here are a few of the highlights that I think can considerably reduce effort and improve quality for app development.
Sign in/up simply
Firebase Authentication provides email, social, anonymous, and custom sign in methods out of the box. Accounts can be managed and each method enabled or disabled from the Firebase web console. Access tokens are based on the JWT feature of OpenID Connect which encrypts portable authorization data in each token. This means multi-system architectures can share tokens and without the expense of server-to-server callbacks on client requests.
Realtime apps are no longer a luxury
The Realtime Database is a NoSQL cloud database with REST and SDK (websocket) support. Data is synced across all clients in realtime, and remains available even when offline. Network calls, cache updates, device resources, and intermittent connectivity are managed automatically by the SDK. Clients simply listen for data changes and react with UI updates.
Free backend - if needed
For the most part Firebase requires no server-side code, but if you want something to be handled in a trusted backend environment (e.g. push notification logic), there are two relatively simple and free-tier options: Cloud Functions and App Engine. Cloud Functions is a hosted, private, and scalable Node.js environment where you can run JavaScript code and interact with Firebase. App Engine is a hosted, private, and scalable environment that supports Java, Python, PHP, and Go. If you’re building an Android app, App Engine is a convenient option because your Java code and IDE tools can be shared between the two.
Deep links that survive installs
A Dynamic Link is a deep link that can survive the optional app installation step (on Android and iOS) or fall back to a web link if the user is on a desktop machine. Firebase will generate short links that contain all the details required. This works great for letting users invite their friends to an app or tracking referral codes.
Final thoughts
Serverless isn’t the right approach for every situation, but the potential for reduced effort, early feedback, and tighter operational management is compelling - especially in the early life stages of an app. Firebase is one of the most complete Backend-as-a-Service platforms, but the field is still fairly new. The competition will evolve and so will your app. Design principles like dependency inversion can help minimize these future risks.
Further Reading
Reuse Android code without remote dependencies
You’re starting to write similar code project after project and think it would be useful to establish some base components that can be reused across multiple projects. First you create a library module to isolate your code. Then you need to figure out how you’re going to include it in each project.
If it’s open source, hosting it in a public maven repository like JCenter/Maven Central is a good solution. If the code can’t be shared publicly, the simplest way I’ve found is this:
Observable cache policies with Shelf
Mobile apps often depend on network data that needs to be stored locally, restructured, and periodically refreshed. You’ll find plenty of implementation options for http calls, serialization, disk storage, encryption, etc. You’ll also find plenty of changing requirements that make you want to replace the choices you made a month ago.
- Wouldn’t it be nice to minimize the dependencies your application code has on these persistence details?
- Wouldn’t it be extra nice to defer complex and highly optimized implementations you may never need?
3 ways to reach Android users beyond your app
If you have an app in the Google Play store and are looking for better ways to connect with users, there are some powerful Android utilities you should know about. Users don’t need to have your app open or even installed to reap the benefits. Here are 3 tools for reaching Android users beyond your app.
Android Material Transitions
This Android project samples some Material Design-ish transitions for list items and floating action buttons. It uses the the shared element concept introduced in Android 5.0. I tried to pull it off with pure fragment transitions and ran into a few stags (see below) so my current solution uses an activity transition for each step.
Mocking backend features in an Android app
Many of the apps we build rely on backend web services. During development these systems are often still in flux. Test environments go on and offline. Web services change. Data from the servier is inconsistent or incomplete. This article outlines a few techniques to mock external network dependencies so development and user testing can keep moving.
Dev settings
Android’s Setting/Preference API is a handy way for developers and testers to configure mock features directly in your app.
Automatically boot and unlock an Android emulator
For all you DRY-conscious developers out there who want to eliminate some repetitive manual steps when testing against an Android emulator, here’s a simple bash script. It boots the emulator if it’s not already running, waits for the full boot, and then unlocks the lock screen. It’s pretty handy automated tests against an emulator with something like Calabash.
#!/bin/bash # init variables DEVICE_ID="@nexus5" EMULATOR_CHECK_STATUS="adb shell getprop init.svc.bootanim" EMULATOR_IS_BOOTED="stopped" # see if emulator needs to be booted OUT=$($EMULATOR_CHECK_STATUS) if [[ ${OUT:0:7} == $EMULATOR_IS_BOOTED ]] then # do nothing if already booted echo "Emulator already booted." else # Start the emulator in the background emulator $DEVICE_ID & # for Android SDK emulator # /Applications/Genymotion.app/Contents/MacOS/player --vm-name "$DEVICE_ID" & # for genymotion emulator # wait for full emulator boot OUT=$($EMULATOR_CHECK_STATUS) while [[ ${OUT:0:7} != $EMULATOR_IS_BOOTED ]]; do OUT=$($EMULATOR_CHECK_STATUS) echo "Waiting for emulator to fully boot..." sleep 3 done echo "Emulator booted!" adb shell input keyevent 82 # unlock lock screen fi
Browse some of the best Android library demos with DevAppsDirect
DevAppsDirect is a great tool for Android designers and developers. Install it on any Android device and get quick access to demo UI patterns, custom views, widgets, game engines, and other libraries.
LinkEasy local stack management with Vagrant
I’m using Vagrant to hand-off complete copies of my local development environment to other members of my team. This is a great way to lower setup time, isolate dependencies and eliminate inconsistencies. Frontend developers are able to work against a full local environment without wasting time on backend configuration. The following describes how Vagrant can make this possible without any additional provisioning tools. Those tools are powerful and offer even more efficiencies, but we’re leaving them out in the interest of simplicity.