Open Data Kit » Briefcase

Briefcase

Use ODK Briefcase to easily gather and aggregate data from ODK Collect when you are without internet connectivity or do not want to set up an ODK Aggregate server (or other ODK-compatible server).

Additionally, you can use ODK Briefcase to incrementally pull data from ODK Aggregate (or push data to ODK Aggregate); the resulting local copy of your data can then archived or transformed by your own tools and transmitted to a third-party application.

ODK Briefcase can:

pull blank forms and finalized forms (submissions) from ODK Collect or ODK Aggregate into a local ODK Briefcase Storage location.
push blank forms and finalized forms (submissions) from the ODK Briefcase Storage location up to an ODK Aggregate 1.0 instance.
export the finalized forms (submissions) to a CSV file for processing by other applications.
Beginning with ODK Briefcase 1.4.4, there is a command line interface.

By pulling data into the local ODK Briefcase Storage location and then pushing data up to an ODK Aggregate instance, ODK Briefcase provides a mechanism to transfer data across incompatible upgrades of ODK Aggregate (e.g., between ODK Aggregate 0.9.8.1 and ODK Aggregate 1.0.x).

If a form is encrypted, ODK Briefcase is also the only mechanism to decrypt the finalized forms and generate a CSV of their unencrypted contents.

Installation

Make sure Java 7 or higher is installed on the computer you plan to use. If it is not, download and install it.
Navigate to downloads/download-category/briefcase/ and download ODK Briefcase vN.N.N.jar.
Double click the file to start. If that fails, try running 'java -jar path_to_jar' from the command line.
Alternatively, for command line operation, type java -jar path_to_jar -help for help on the command line interface.

Upon first starting, Briefcase will ask for the directory under which to create the ODK Briefcase Storage area. An ODK Briefcase Storage folder will be created here to hold all the blank forms and finalized forms (submissions). Once created, you can copy and transport this storage folder across systems, just like a briefcase of paper documents.

Usage

When pulling forms from ODK Aggregate, you can either leave the username blank if the anonymousUser is granted Data Viewer permissions, or specify an ODK Aggregate user (Account Type 'ODK') that has been granted Data Viewer permissions. ODK Briefcase does not work with Google accounts (Account Type 'Google').

When pushing forms to ODK Aggregate, you can either leave the username blank if the anonymousUser is granted Form Manager permissions, or specify an ODK Aggregate user (Account Type 'ODK') that has been granted Form Manager permissions. ODK Briefcase does not work with Google accounts (Account Type 'Google').

ODK Briefcase will abort a pull attempt if the local copy of the form definition file differs in any way from the copy being pulled from the server or ODK Collect device. In this case, if the changes only affect the question text and do not alter the structure of the collected data (or change the formId or version), you can create a new ODK Briefcase Storage area, pull data into that, then manually copy the instances from there into your original storage area. If you do this, and need to continue to incrementally pull data from that ODK Aggregate, you should keep this new ODK Briefcase Storage area for future pull attempts; as long as that storage area retains all copies of the data already fetched, incremental pulls can be efficiently performed; if the copies are missing, the data will be re-fetched in its entirety, which can be expensive and time consuming.

Similarly, ODK Briefcase will abort a push attempt if the form definition already on the ODK Aggregate server differs in any way from the form definition in the ODK Briefcase Storage area. In this case, if these changes do not affect the data model, you can make a copy of the storage area, replace the copy's form definition file with that from the server, and then push the data up from this storage area copy.

Command Line Interface

In Briefcase v1.4.4 and later, there is a scriptable command line interface, that helps to automate downloading forms from ODK Aggregate (or ODK Collect) and exporting the forms to an Excel-compatible format like CSV. It can even decrypt those forms if they are encrypted.

The CLI also adds four features that are not in the Graphical User Interface. The CLI can specify a date range for export, speed up export by excluding media, append or overwrite an existing CSV, and specify separate storage and export directories.

To run the CLI, go to the command line (sometimes known as the Terminal), and run the following.
java -jar ODK\ Briefcase\ v1.4.4\ Production.jar --help;

Combine the individual commands described in the help to perform the actions needed. Examples are as follows.

Pull form data with id market_prices from Aggregate server at https://my_server.appspot.com and store data in Briefcase's storage directory on the ~/Desktop
java -jar ODK\ Briefcase\ v1.4.4\ Production.jar --form_id market_prices --storage_directory ~/Desktop --aggregate_url https://my_server.appspot.com --odk_username my_username --odk_password my_password;

Pull form data with id market_prices from Collect's odk folder at ~/Desktop/odk and store data in Briefcase's storage directory on the ~/Desktop
java -jar ODK\ Briefcase\ v1.4.4\ Production.jar --form_id market_prices --storage_directory ~/Desktop --odk_directory ~/Desktop/odk;

Export form data with id market_prices from Briefcase's storage directory on the ~/Desktop and write CSV file to ~/Desktop/market_prices.csv
java -jar ODK\ Briefcase\ v1.4.4\ Production.jar --form_id market_prices --storage_directory ~/Desktop --export_directory ~/Desktop --export_filename market_prices.csv;

Pull form data with id market_prices from Aggregate server at https://my_server.appspot.com and store data in Briefcase's storage directory on the ~/Desktop. Export only results from 2014/02/05 to 2014/02/06 and write the CSV file to ~/Desktop/market_prices_filtered.csv. Overwrite the CSV if it exists and exclude any media. Also, write any status or error messages to ~/Desktop/briefcase.log
java -jar ODK\ Briefcase\ v1.4.4\ Production.jar --form_id market_prices --storage_directory ~/Desktop --aggregate_url https://my_server.appspot.com --odk_username my_username --odk_password my_password --export_directory ~/Desktop --export_filename market_prices_filtered.csv --overwrite_csv_export --exclude_media_export --export_start_date 2014/02/05 --export_end_date 2014/02/06 >~/Desktop/briefcase.log 2>&1;

To script the CLI, write the commands you would like to execute in a scripting language (e.g., Bash, Batch, Python, Ruby) and use a scheduler (e.g., Cron, Windows Task Scheduler) to schedule the tasks.

As a Replacement for ODK Aggregate

ODK Briefcase can pull data directly off ODK Collect devices, but does not support pushing blank forms to ODK Collect. To use ODK Briefcase as a replacement for ODK Aggregate, you must manually install the forms on your ODK Collect device.

To manually install forms onto your ODK Collect device:

Connect the device to your computer with a USB cable and either, for Android 2.x devices, turn on USB storage (on the Android device), or, for Android 4.x devices, choose to connect it as a Media device (on the computer).
On the computer, copy the form definition and media folder to the /odk/forms directory. For Mac OS/X, if it is an Android 4.x device, you will need to use Android File Transfer to accomplish this.
On Android 2.x, turn off USB storage.
Run ODK Collect
Choose "Fill Blank Form".

ODK Collect will scan the directory for new forms, parse them, and make them available in this list. It can take several moments for large forms to appear. When ODK Collect has completed scanning for new forms, it will display Finished scanning. All forms loaded. in the second line at the top of this screen.

The steps for pulling data directly into ODK Briefcase from your Android device differs between the Android 2.x and Android 4.x devices.

Pulling data directly from an Android 2.x device

These steps apply to Android 2.x devices. They are as follows:

Ensure that all filled-in forms on ODK Collect are finalized. If they are not finalized, you should either delete them via ODK Collect's 'Delete Saved Form' button or make a back-up copy of the '/sdcard/odk' directory on your device, delete the incomplete forms in ODK Collect, follow the steps below and then restore to that back-up copy. If you want to do that, the OI File Manager app is very useful.
Connect your Android device to your computer using a USB cable; on the Android device, choose to mount the drive (the mounted device only shows the files under the '/sdcard' directory, so when you are selecting 'odk', you are actually choosing the '/sdcard/odk' directory on the device, as seen by the OI File Manager application). Again, these instructions only apply to Android 2.x or earlier systems.
Open ODK Briefcase, choose the "Pull" tab and on the "Pull data from:" drop-down, select "Mounted Android SD Card".
Click "Choose..." and select the appropriate mounted SD card (it will appear as a drive on your Windows PC).
Select the forms to pull and click the "Pull" button and the filled-in submissions that were copied off of the device will be loaded into ODK Briefcase's storage location.
Confirm that the filled-in submissions have been successfully pulled into ODK Briefcase (e.g., by confirming a successful pull status or by verifying the data appear in a CSV export file).
On the Android device, open ODK Collect and delete the filled-in forms via ODK Collect's "Delete Saved Form" button, selecting everything on the "Saved Forms" tab, and choosing "Delete Selected".

Notes:

The first step is important; ODK Briefcase does not discriminate between incomplete and finalized forms on the device. It will pull all data off of the device. This can cause problems during later pushes, and, especially, if you are encrypting your finalized forms. To keep your data set clean, you must ensure that all forms are complete before being pulled off of the device.

The last step is important; ODK Briefcase cannot discriminate between duplicates of the same filled-in form. After you pull the data into ODK Briefcase, it is important that you delete it from ODK Collect. Otherwise, the next time you follow this process, you will end up with two copies of the filled-in forms from the first pull, etc.

Pulling data directly from an Android 4.x device¶

ODK Briefcase does not work with the USB-mounted Media device (MTP) protocol, which replaced the USB-mounted drive protocol on Android 4.x devices. Unfortunately, MTP cannot be supported because there are no reliable cross-platform libraries for the MTP protocol. Additionally, the mounting mechanisms do not always reliably report all the available files to the PC operating system (MacOSX and/or Windows).

The work-around is manually create a zip of the '/sdcard/odk' directory (where ODK Collect stores all its data), copy that file from the Android device to your computer, unzip it, and use the 'Custom Path to ODK Directory' option within ODK Briefcase to pull that data into ODK Briefcase.

The recommended steps are:

Ensure that all filled-in forms on ODK Collect are finalized. If they are not finalized, you should either delete them via ODK Collect's 'Delete Saved Form' button or make a back-up copy of the '/sdcard/odk' directory on your device, delete the incomplete forms in ODK Collect, follow the steps below and then restore to that back-up copy. If you want to do that, the OI File Manager app is very useful.
Use an application like OI File Manager to create a zip of the entire 'odk' directory
Connect your Android device to your computer using a USB cable; on your computer, choose to mount it as a 'Media device.' Again, these instructions only apply to Android 4.x or later systems.
Copy that zip file from the mounted MTP/Android device to a local hard drive. For Mac OS/X, you will need to use Android File Transfer. For Windows, you can just drag-and-drop the zip of the 'odk' directory into a folder on your PC (the mounted device only shows the files under the '/sdcard' directory, so when you are selecting 'odk', you are actually choosing the '/sdcard/odk' directory on the device, as seen by the OI File Manager application).
Once it is copied onto your local hard drive, unzip the file of the 'odk' directory.
Open ODK Briefcase, choose the "Pull" tab and on the "Pull data from:" drop-down, select "Custom Path to ODK Directory".
Select the unzipped 'odk' folder that you copied onto your local hard drive.
Click the "Pull" button and the filled-in submissions that were copied off of the device will be loaded into ODK Briefcase's storage location.
Confirm that the filled-in submissions have been successfully pulled into ODK Briefcase (e.g., by confirming a successful pull status or by verifying the data appear in a CSV export file)
On the Android device, open ODK Collect and delete the filled-in forms via ODK Collect's "Delete Saved Form" button, selecting everything on the "Saved Forms" tab, and choosing "Delete Selected".

Notes:

The second step is important; the MTP connection may show an out-of-date view of the files on your Android device. Power cycling your device ensures that it will show the current set of files on your device.

Useful Notes

The developer wiki has release notes.