An Android Foreground Service Can Die Without Crashing

The short version

A service can be alive while only its work is dead.

Guarding just the pollOnce() call is not enough. An unexpected exception in delay() or in computing the backoff interval exits while (true) quietly and ends the coroutine that was driving the loop. The process is alive and the foreground notification is still up, so nothing appears in the crash reports.

The countermeasure has two parts: guard the whole loop body so it cannot be exited, and add a watchdog that watches the last success timestamp so that “the work stopped” is detected separately from whether the process is alive.

What it looks like

On an always-on kiosk device, only the config-distribution polling stops.

• The device itself is running. The screen is up
• No crash log. No force-stop, no reboot
• Only some devices go quiet, in an otherwise identical fleet. Not being total makes it harder to notice
• Neither the timing nor which device is predictable

Android’s standard monitoring only watches whether the process died. A case like this, where the process stays alive and only the inner processing loop ends, is outside ordinary crash detection.

Why

ConfigPoller’s periodic loop wrapped only the pollOnce() call in try/catch.

while (true) {
    delay(nextDelayMs(failureStreak, lastResult))
    lastResult = try {
        pollOnce()
    } catch (e: Throwable) {
        // this part is guarded
    }
    failureStreak = updateStreak(failureStreak, lastResult)
}

Exceptions thrown from pollOnce() are caught. But nextDelayMs(), which computes the argument to delay(), and updateStreak() were unguarded against an unexpected exception. An exception there punches through outside the try/catch, exits while (true) entirely, and the loop’s coroutine simply ends.

One coroutine ending is not treated by Android as an app crash. The foreground service notification remains and the process is alive. Only the job — polling — stops, unnoticed by anyone.

Fixing it

We fixed it in three layers.

1. Guard the whole loop body

while (true) {
    try {
        delay(nextDelayMs(failureStreak, lastResult))
        lastResult = try {
            pollOnce()
        } catch (e: Throwable) {
            if (e is CancellationException) throw e
            // ...
        }
        recordIfSuccess(lastResult)
        failureStreak = updateStreak(failureStreak, lastResult)
    } catch (c: CancellationException) {
        throw c
    } catch (e: Throwable) {
        Log.w(TAG, "poll loop iteration error (continuing)", e)
    }
}

Everything from delay to the backoff calculation — one whole iteration — goes inside try/catch. CancellationException alone is a coroutine’s legitimate stop signal and is rethrown; anything else is swallowed, logged, and the loop continues. while (true) can never again be exited except by an explicit cancel.

2. Measure “alive” by the last success, not by the process

We added one line writing a timestamp on each successful loop.

private fun recordIfSuccess(result: PollResult) {
    if (result == PollResult.SUCCESS) {
        runCatching { Config.setLastPollSuccessMs(context, System.currentTimeMillis()) }
    }
}

An AlarmManager-based watchdog reads this every 15 minutes. It uses setExactAndAllowWhileIdle, so it fires during Doze.

if (pollStaleMs > STALE_THRESHOLD_MS &&
    now - Config.lastForceRestartMs(context) > RESTART_COOLDOWN_MS
) {
    Log.w(TAG, "poll stale ${pollStaleMs / 60_000}min -> force restart service")
    Config.setLastForceRestartMs(context, now)
    BleService.forceRestart(context)
}

It watches “did polling succeed in the last 20 minutes”, not “does the service exist”. That means the same mechanism also picks up loops stopped by paths layer 1 could not fix — an OOM killer, an OEM power-saving feature. Forced re-creation carries a 10-minute cooldown so that it cannot become a restart loop.

3. Insurance for an actual crash

The two above address the “stops without crashing” path; the “crashes and dies” side got separate handling. We install an uncaught exception handler for all threads on Application that registers a recovery alarm three seconds out with AlarmManager just before the process dies, then delegates to the default crash handling.

Thread.setDefaultUncaughtExceptionHandler { thread, throwable ->
    runCatching {
        Watchdog.scheduleRestart(this, 3000L)
    }
    val prev = previous
    if (prev != null) prev.uncaughtException(thread, throwable)
    else {
        android.os.Process.killProcess(android.os.Process.myPid())
        exitProcess(10)
    }
}

An alarm registered with AlarmManager survives on the OS side even when the process dies, so after the process is gone the alarm fires and brings the service back up. Because some devices have OEM power-saving features that swallow AlarmManager exact alarms, a WorkManager job on a 15-minute period is wired to the same tick handler as an independent second path.

In summary

There were three ways to stop. ① the loop exits quietly on an exception, ② the process crashes outright, ③ neither, and the OS kills it. ① is closed by guarding the whole loop; ② and ③ are caught in a double net of “an alarm that recovers after a crash” and “a periodic liveness check”.

What they have in common is designing on the premise that “the service is alive” on Android does not guarantee “the work is being done”. Only by watching the last successful processing time rather than the presence of the process does a non-crashing stop become detectable.

Another way the same devices go dark under long-running operation is covered in FLAG_KEEP_SCREEN_ON Ignored: It’s the Screensaver.