The CommonsBlog

Don't Put All Your Eggs in One Basket

I am glad that the fight between Google and the developers of Terraria appears to have been resolved. However, the effects of the ban on Terraria’s primary developer remind me that not all solo developers realize the potential impacts of distributing apps through the Play Store.

If you are an ordinary person doing ordinary things, it is highly unlikely that Google will have a need to ban your Google account. Saying that you use Google for email and movies and calendars and music and file storage and games and document tools and so on is not completely ridiculous.

Once you decide to distribute your apps through Google, though, that situation changes. Now you are doing something that Google deems risky, and their bot-and-cheap-labor vetting process means that you are at an above-average risk of having your account be banned. And due to the opaque nature of their banning process, it is not safe to assume that you are being banned for any actual violation of any actual terms of service. In the end, they can ban you for any reason they feel like.

So, if you are going to distribute apps through the Play Store, you owe it to yourself to:

Stop using as many Google services as is practical, and
Aim to isolate your app-distribution Google account(s) from any personal account(s) that you may have

The biggest service by far to get off of is Gmail. Losing access to a movie you bought through Google Play is unfortunate. Losing a chunk of your ability to communicate with the outside world may have a far greater impact. There are plenty of other email providers out there. Gmail has not been “head and shoulders” better than the competition in quite some time. I abandoned Gmail many years ago and have not missed a beat. When I get dragged back into Gmail by a client, my reaction to it is decidedly “meh” — the competition has similar features and capabilities. And the competition is unlikely to ban you because of issues with your app.

Google Drive and Google Docs are the other major areas of risk. Basically, you are giving Google an easy way to hold your data hostage. Use other cloud storage providers. Use other document tools, including (gasp!) ones that run on your own hardware.

For whatever remains, you need to recognize that your access to those Google services could vanish at any point. That’s not even necessarily tied to your apps. For example, back when I “drank the Kool-Aid”, I set up commonsware.com as a business account with Google. Google apparently has no way to reverse that action. After I discontinued my business account, commonsware.com remains incapable of accessing most Google services that have ties to their business services (e.g., Docs, Drive). Life goes on.

One way to help minimize the risk is to keep your personal and “business” Google accounts separate. Even if you have not set up a business for your app and do not think of yourself in business terms, it is best to treat your interactions with Google as if you were running a business. Distribute your apps under a separate Google account. Try to minimize “mixing business with pleasure” by using your personal account for app-related activity. Ideally, Google’s banhammer will have no idea that your personal account has anything to do with this app that they dislike.

Most solo developers will not run into any problems. However, the semi-random nature of Google’s enforcement actions means that even developers that are sure they are doing the right thing might encounter problems. Do yourself a favor and try to minimize the effects of distributing apps through the Play Store… even if all you are doing is writing a puppy adoption app. Google does lots of good things, but not everything that Google does is good for you.

In other words, Google is not your friend.

—Feb 27, 2021

Random Musings on the Android 12 Developer Preview 1

Each time Google releases a new developer preview, I rummage through the API differences report the high-level overviews, and even the release blog post, to see if there are things that warrant more attention from developers. I try to emphasize mainstream features that any developer might reasonably use, along with things that may not get quite as much attention, because they are buried in the JavaDocs.

As with the previous four releases, Android 12 seems to be lacking a prominent user-facing feature. I have no idea what I will tell my mother that she’ll gain if her phone gets the update at some point. Admittedly, Google appears to be sandbagging a bit, holding out some changes for later releases, so perhaps something compelling to ordinary people will appear later.

But, as has been common in a similar timeframe, the interesting bits for developers are not covered in the DP1 release notes.

What Will Irritate Developers

In principle, I am a fan of requiring positive declaration of exported components. However, this desperately needs a Lint warning, though hopefully one is forthcoming, perhaps in the current canary release of Android Studio 2020.3.1 Arctic Fox My Goodness This Name Is Long. I am hopeful that the error message will improve, though, as tens of thousands of developers next year are going to encounter this change, and approximately zero of those developers will be reading the documentation. Helen of Troy had “the face that launched a thousand ships”; this sort of inscrutable error message for an all-but-guaranteed failure is “the error that launched a thousand Stack Overflow questions”.

I had no idea that nested Intents were much of a thing, making me wonder if this was a technique used by certain libraries (e.g., ad networks). That, and requiring mutability flags on PendingIntent, worry me less for apps directly and more for libraries that might not be receiving updates. Of course, depending on where those libraries were published, the libraries may be gone, anyway.

I foresee a blog post in my future on writing activity trampolines.

Perhaps the one that worries me most is the limit on starting foreground services from the background. In theory, this is fine. In practice, foreground services are already unreliable, with timing-related OS bugs yielding Context.startForegroundService() did not then call Service.startForeground() or similar sorts of errors. My concern is that, just as Google seems unconcerned about those errors, Google will not care about follow-on errors related to timing of when you can and cannot start a foreground service. Hopefully, my fears are unfounded.

Fortunately, all of the preceding changes in this section only take effect once your targetSdkVersion hits 31(?) or higher. That means it probably will affect those of us experimenting with DP1 for a few months, and then the bulk of developers sometime next year.

One change that might take effect more quickly — though I have not yet tested it — is that there are new permissions for using the camera and using the microphone in the background. These are dangerous permissions, ones that we have to request at runtime. It is also unclear how future camera and microphone toggles might behave and how apps will find out about the state changes.

And the new VERSION_CODES value is messed up, as usual.

What Is Neat But Confusing

The unified content API, for accepting media from paste, drag-and-drop, and other sources, is nice. However, it is unclear why this is a new framework API, since the documentation depicts it as being a wrapper around existing APIs with a backport already in AndroidX. If the backport is complete, why did we need a framework API? And if we needed a framework API, what is the gap between what Android 12 offers and what the backport offers?

PackageManager has gained some interesting, if under-documented, features:

You can request checksums of installed apps.
There is a new form of “property” that you can declare on a component or elsewhere in the manifest. It is unclear how this relates to the long-standing <meta-data> option in the manifest.
A package can have attributions in the <manifest>, defined in <attribution> elements. It is unclear if this ties into data access auditing, though, or what role these “attributions” otherwise play.

What Is Subtly Nice

The postponed foreground service notifications feature should help with user distraction. Often, we need a foreground service because we cannot be certain that a background service will complete in time… but if the work is completed quickly, we hassle the users with a short-lived notification. This change should help eliminate that.

Apps have improved options for blocking touch input passed through other windows, including a permission to be able to enable such a block and a window option to implement the block. Frankly, this is long overdue, as is banning apps from closing system dialogs.

Since the beginning, we have built up some heuristics for which types of Context are safe for UI operations and which ones are not. Finally, Google is formalizing the concept, with a dedicated API to tell you if this Context is OK for UI work. There is a corresponding option in StrictMode to warn you about incorrect Context usage.

PowerManager can now give us real-world estimates of remaining battery life, which might be useful for some apps to better scale back work in low-battery states.

And, as Romain Guy tweeted about, we have a new API to apply a RenderEffect to a View, which should give us an easy blur option.

What Needs Third-Party Love

ImageDecoder now supports decoding the full animations of GIF and WebP images. It will be interesting if image-loading libraries start adopting it on Android 12 and newer devices — my guess is that they will only if if provides obvious benefits over their existing code for handling these animations.

What’s Dead (Or Seriously Morphed)

At least in DP1, there are relatively few massive deprecations, like the deprecation of AsyncTask in Android 10.

However:

LocationManager is significantly changing its API for obtaining locations
Obtaining the WiFi connection info or IP address has moved to new locations in the SDK
DevicePolicyManager is overhauling its passphrase quality API, resulting in a bunch of deprecations
You can no longer use GET_INTENT_FILTERS when querying the PackageManager
If you were using FLAG_BLUR_BEHIND or TYPE_STATUS_BAR_PANEL on WindowManger.LayoutParams, those values have been removed from the SDK entirely

What Is the Human Resources Department Doing In My Developer Preview?

There is a new PeopleManager system service. Google added a lightweight concept of “conversations” to the Notification API over the past few releases. This appears to try extend that further.

This might tie into the conversation app widgets described in some Android 12 document leaks. There also is a new way to tie a NotificationChannel to conversations.

What Is the Insect Resources Department Doing In My Developer Preview?

There is a BugreportManager in Android 12, though it is unclear exactly how useful it will turn out to be.

What Makes Me Go “Hmmmmm”

And now, we get to a list of other things that caught my eye in the API differences report that did not get included in Google’s DP1 overview:

There is a new AppSearchManager that appears to be an on-device search engine. It will be interesting if this gets covered more in future developer previews.
The fused location provider has appeared in the SDK for the first time. It will be interesting to see if this means that you can start using this on non-Play devices.
We now have a new facet of a Configuration: fontWeightAdjustment. I do not see an option for this in the Settings app. It also remains to be seen if this triggers a new type of configuration change.
We can now control whether we get click events for a disabled View.
It appears as though an <intent-filter> can now filter Uri values based on a suffix and some sort of “advanced pattern”. It is unclear what is so advanced about that pattern, though.
There are two new Notification categories. CATEGORY_WORKOUT suggests that Google is going to have more explicit support for fitness-related apps, perhaps stemming from their acquisition of Fitbit. And I do not know what CATEGORY_LOCATION_SHARING means, such as “sharing with who?”.
If you played with Android 10’s device controls, there is new control type: the thumbnail. I am uncertain what the use case is, though security cameras might be one.
There is a new requireDeviceScreenOn attribute on… something. It is described as “Whether the device must be screen on before routing data to this service. The default is true.”. That default seems odd; otherwise, I would have thought that perhaps this attribute goes on <service> in the manifest. It is possible that this ties into a new option to require unlocking the device before processing a notification action.
There is a new AppUriAuthenticationPolicy, though it is unclear how it will be used.
We can have more sophisticated haptics, with multiple blended vibration patterns. Alas, the name of the class for this will spawn yet another round of off-color jokes.

What Happens Now?

While I do plan on writing an Elements of Android S along the lines of its Q and R predecessors, I plan on waiting to release something. I waste too much time chasing changes that eventually themselves change or outright vanish. So, my current plan is to wait to ship a Version 0.1 of the book until the last developer preview or first beta release. If you feel that this is a mistake, and I should publish something more rapidly, let me know!.

—Feb 21, 2021

Using Repository Artifact Safelists in Gradle

With JCenter going away, we are going to be peeking more at our repositories and artifacts. After all, we need to make sure that we will continue to get the libraries that we need from their new homes, for any that were published purely to JCenter.

The timing is interesting, as “supply-chain” attacks are on the rise. It is far too easy for somebody to publish a malware-laden library and have it be picked up automatically by developers. This can even affect private artifacts in private repositories.

Ideally, while we are cleaning up our Gradle scripts, we would lock down where we get our artifacts from. The good news is that modern versions of Gradle give us somewhat better options for this. The bad news is that implementing those options is rather painful.

The problem with the default way that we declare dependencies in an Android project is that we do not say where each dependency comes from. Gradle largely disassociates artifacts (e.g., all those implementation lines) from repositories. We do not say “get this artifact from this repository”. We just list the supported repositories and desired artifacts, and Gradle fulfills our requests on its own.

And, by default, all Gradle does is use a top-down search against each repository. In other words, it uses this algorithm:

For each artifact
  For each repository in the order they are listed
    If the repository offers this artifact, download it
  End
End

There is little stopping an artifact from being offered in more than one repository, which is at the root of some of these supply-chain attacks. Depending on the order of the repositories in your Gradle script, you might get different actual artifacts than another project requesting the same artifacts but using a different repository order.

Ideally, we would say “get this artifact from this repository”. We can do that, to an extent.

Starting with Gradle 5.1, we can safelist what artifacts we get from a given repository. To do this, we add a content {} closure to the repository declaration, and in there use include...() functions to stipulate what artifacts to obtain from that repository:

jcenter {
    content {
        includeModule("org.jetbrains.trove4j", "trove4j")
    }
}

Here, we are saying that the only thing that we want to obtain from JCenter is org.jetbrains.trove4j:trove4j. JCenter will not be used for other artifacts.

The three main safelist functions are:

includeModule(), where you provide the artifact group (org.jetbrains.trove4j) and artifact ID (trove4j)
includeGroup(), to support any artifact from a specified artifact group
includeGroupByRegex(), which allows you to specify a regular expression and support any artifact group that matches that expression (e.g., includeGroupByRegex("org\\.jetbrains\\..*"))

If all of your repository declarations include one or more include...() functions, then the build should work purely off of those safelists:

The artifacts that you are using will obtained from their associated repositories and nowhere else
No artifacts that are not on the safelist will be used in your build

See this blog post for a bit more on the options.

The problem is that the safelists not only need to handle your direct dependencies, but also all of the transitive dependencies.

That can get rather lengthy.

This sample project is based on the tutorial project that we build in Exploring Android. The project requests 33 artifacts for the module, plus three classpath entries for Gradle plugins.

After adding all of the necessary include...() functions, the top-level build.gradle file is over 150 lines long, mostly involving those functions. And that is with cheating and using includeGroup() and includeGroupByRegex(), both of which are a bit less secure than includeModule().

Basically, what happens is that you add include...() calls for all of your direct dependencies, then try doing a build. In particular, adding --refresh-dependencies to a command-line build (e.g., gradle --refresh-dependencies app:assembleDebug) will confirm that Gradle can download all of your dependencies. You will wind up with a bunch of errors:

* What went wrong:
A problem occurred configuring root project 'GradleSafelist'.
> Could not resolve all artifacts for configuration ':classpath'.
   > Could not resolve org.glassfish.jaxb:jaxb-runtime:2.3.1.
     Required by:
         project : > com.android.tools.build:gradle:4.1.2 > androidx.databinding:databinding-compiler-common:4.1.2
         project : > com.android.tools.build:gradle:4.1.2 > com.android.tools.build:builder:4.1.2 > com.android.tools:sdklib:27.1.2 > com.android.tools:repository:27.1.2
      > Could not resolve org.glassfish.jaxb:jaxb-runtime:2.3.1.
         > Could not parse POM https://repo.maven.apache.org/maven2/org/glassfish/jaxb/jaxb-runtime/2.3.1/jaxb-runtime-2.3.1.pom
            > Could not resolve com.sun.xml.bind.mvn:jaxb-runtime-parent:2.3.1.
               > Could not resolve com.sun.xml.bind.mvn:jaxb-runtime-parent:2.3.1.
                  > Could not parse POM https://repo.maven.apache.org/maven2/com/sun/xml/bind/mvn/jaxb-runtime-parent/2.3.1/jaxb-runtime-parent-2.3.1.pom
                     > Could not resolve com.sun.xml.bind.mvn:jaxb-parent:2.3.1.
                        > Could not resolve com.sun.xml.bind.mvn:jaxb-parent:2.3.1.
                           > Could not parse POM https://repo.maven.apache.org/maven2/com/sun/xml/bind/mvn/jaxb-parent/2.3.1/jaxb-parent-2.3.1.pom
                              > Could not find com.sun.xml.bind:jaxb-bom-ext:2.3.1.
   > Could not resolve com.google.auto.value:auto-value-annotations:1.6.2.
     Required by:
         project : > com.android.tools.build:gradle:4.1.2 > com.android.tools.build:bundletool:0.14.0
      > Could not resolve com.google.auto.value:auto-value-annotations:1.6.2.
         > Could not parse POM https://repo.maven.apache.org/maven2/com/google/auto/value/auto-value-annotations/1.6.2/auto-value-annotations-1.6.2.pom
            > Could not resolve com.google.auto.value:auto-value-parent:1.6.2.
               > Could not resolve com.google.auto.value:auto-value-parent:1.6.2.
                  > Could not parse POM https://repo.maven.apache.org/maven2/com/google/auto/value/auto-value-parent/1.6.2/auto-value-parent-1.6.2.pom
                     > Could not find com.google.auto:auto-parent:6.

The end leaf of each branch shows a transitive dependency that is not covered by your include...() functions — in this case, com.sun.xml.bind:jaxb-bom-ext:2.3.1 and com.google.auto:auto-parent:6. After adding those, you run the test again and get a bunch of fresh errors from the transitive dependencies of the transitive dependencies. And, as the shampoo instructions state, lather, rinse, repeat, until eventually you get no more errors.

This process sucked… and this is not a big project. With luck, we can create some tooling to help make generating these safelists easier.

But, this is a more secure build than what it started with. The more important your project, the more likely it is that you are going to want to explore options like this for ensuring that your artifacts come from where you expect them to.

—Feb 20, 2021

Notes on the Jetpack Compose alpha11 to alpha12 Upgrade Process

Upgrading an app from Compose alpha11 to alpha12 was more troublesome than normal. I took some notes along the way and wanted to pass them along.

Note: all of the Gradle samples show the classic Groovy approach — you will need to tweak those for build.gradle.kts if you are going that route.

The obvious change is moving your Compose Gradle plugin and all of the Compose runtime dependencies to 1.0.0-alpha12, along with your kotlinCompilerExtensionVersion in composeOptions. If you have been working with Compose for a while, this is a standard change, and hopefully you have consolidated all of those version references into a single constant.

Compose alpha12 also requires Kotlin 1.4.30, for its Gradle plugin and for its runtime dependencies. With luck, you have a single constant for that as well:

buildscript {
    ext {
        kotlinVersion = "1.4.30"
        composeVersion = '1.0.0-alpha12'
    }

    ...
}

The release notes also show having this Gradle snippet outside of your android {} closure in your modules:

tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).configureEach {
  kotlinOptions {
    jvmTarget = "1.8"
    freeCompilerArgs += ["-Xallow-jvm-ir-dependencies"]
  }
}

After doing those things, I then ran into a bunch of IDE errors akin to the following:

‘padding(Dp): Modifier’ is only available since Kotlin 1.4.30 and cannot be used in Kotlin 1.4

At first, I tried upgrading to “Android Studio Arctic Fox | 2020.3.1 Canary 6” (hereafter referred to as “Canary 6” for simplicity). That did not clear up the problem.

The solution is to upgrade the Kotlin plugin in the IDE to support 1.4.30. Unfortunately, this is not yet stable. So, you need to:

Open the Settings dialog in Canary 6
Go into “Languages & Frameworks” > Kotlin
Switch your “Update channel” to “Early Access Preview 1.4.x”
Choose to upgrade your Kotlin plugin to 203-1.4.30-RC-AS6682.9 (or newer, depending on when you are reading this)
Restart Android Studio afterwards

This appears to be undocumented, unless you count Kotlinlang Slack messages from Google developers as documentation.

You should also check all your third-party dependencies and make sure that you have their latest versions, as otherwise you may fail when you build the module. In my case, I needed to upgrade dev.chrisbanes.accompanist:accompanist-coil to 0.5.1.

That got me to the point where things would run, but you are likely to encounter a bunch of deprecations.

The one that will affect most of you is that setContent() on Activity will show up as deprecated. That is because it moved out of a mainline Compose dependency. You will need to add:

implementation "androidx.activity:activity-compose:1.3.0-alpha02"

to your module’s list of dependencies. Note the alpha02 — while alpha01 was released this past week, alpha02 shipped hours later to fix a significant bug.

Once you have that dependency in place, you can replace:

import androidx.compose.ui.platform.setContent

with:

import androidx.activity.compose.setContent

If you happen to be using the Jetpack ViewModel in your Compose UI code, you may run into a similar deprecation notice on viewModel(). That is another case where the extension function moved from a mainline Compose dependency to another one. You will need to add:

implementation "androidx.lifecycle:lifecycle-viewmodel-compose:1.0.0-alpha01"

to your module’s dependencies. Then, you can change:

import androidx.compose.ui.viewinterop.viewModel

to:

import androidx.lifecycle.viewmodel.compose.viewModel

There is a “long tail” of other deprecations that you might encounter. In my samples, one was that imageResource() is deprecated. The replacements, though, will vary by circumstance:

If you need a Bitmap, you will wind up changing imageResource(R.drawable.whatever) to imageFromResource(LocalContext.current.resources, R.drawable.whatever)
Otherwise, painterResource(R.drawable.whatever) will probably suffice

This upgrade process was more involved than usual, exacerbated by gaps in documentation. I will be honest: this worries me, with a possible move to “beta” versions coming in the next couple of weeks.

—Feb 14, 2021

Quieting the SQL Syntax Warnings

val db: SQLiteDatabase = TODO("open a database")

val st = db.compileStatement("ATTACH DATABASE ? AS plaintext KEY ''")

By default, Android Studio does not like this SQL statement. It puts a red undersquiggle below KEY, complaining that it is expecting a ; instead.

According to SQLite, Android Studio is correct.

This error is coming from a SQL “language injection”. Rebecca Franks wrote a great blog post about language injections a few weeks ago. There is a “SQLiteDatabase methods” language injection set up in my Android Studio 4.1.2 installation, and I assume that it shipped with Studio itself.

However, in my case, I’m right, and Android Studio is wrong.

That is because in this code I am not using plain SQLite — I am using SQLCipher for Android. SQLCipher for Android has an extended form of ATTACH DATABASE that takes a KEY option to supply a passphrase (shown here as the empty string '').

It is unclear why a language injection for android.database.sqlite.SQLiteDatabase is affecting calls to net.sqlcipher.database.SQLiteDatabase. Perhaps it is because net.sqlcipher.database.SQLiteDatabase implements androidx.sqlite.db.SupportSQLiteDatabase, the SQLite indirection API that allows you to use things like SQLCipher for Android with Room and SQLDelight. Or maybe this is a limitation of the scoping rules for language injections, and anything named SQLiteDatabase will be affected.

Regardless, the red undersquiggle was annoying me.

I could disable that language injection, but then I lose syntax highlighting and validation everywhere else. Instead, I want to be able to mark certain statements as being correct and suppress the warning, akin to how @SuppressLint() works. However, this is not a Lint check, so @SuppressLint itself does not help here.

Rebecca’s post gave me an idea, though, which turns out to work, in a couple of different ways.

In her post, she shows that you can use a //language= comment to cause a particular language injection to be applied in a spot it might not normally be applied. So, while this would result in no errors:

private const val ATTACH_THIS = "ATTACH DATABASE ? AS plaintext KEY ''"

…this would complain about KEY:

//language=sql
private const val ATTACH_THIS = "ATTACH DATABASE ? AS plaintext KEY ''"

In the first snippet, the IDE has no idea that this string is a SQL statement and so does not apply any language injections. In the second, we teach it that this statement contains SQL, so the IDE applies the appropriate language injection.

So, one workaround is to pull the offending SQL out of the SQLiteDatabase calls:

val db: SQLiteDatabase = TODO("open a database")

val st = db.compileStatement(ATTACH_THIS)

If we skip the //language=sql comment on the ATTACH_THIS declaration, then we will not get any complaints about KEY.

Another workaround is to override the language injection at the call site:

val db: SQLiteDatabase = TODO("open a database")

//language=text
val st = db.compileStatement("ATTACH DATABASE ? AS plaintext KEY ''")

Here, we tell the IDE “Treat this as plain text. Yes, yes, I know, it looks like it should be SQL. And, yeah, I do not really know if you know what ‘plain text’ is. Just skip any syntax validation on the string, please.”

(fortunately, the comment is shorter)

At least for now, these workarounds give you fine-grained ability to suppress SQL syntax warnings, in cases where Android Studio’s SQL parser either has bugs or is confused by extended SQL syntax.

—Feb 11, 2021

Older Posts