Android is shifting to an “upstream first” development model for new Linux kernel features

When you see the words “Android” and “fragmentation” in the same sentence, your mind probably immediately jumps to the Android version distribution chart. There are a few entities that most people point their fingers at when complaining that Android OS updates are slow to roll out across the board, but there’s only so much Google can do to force OEMs to develop and roll out updates more quickly. What Google can do, however, is reduce the development time and thus the cost of rolling out updates.

The first major initiative in Google’s long-term project to reduce development burdens is Project Treble. Announced alongside Android 8.0 Oreo in 2017, Project Treble modularized Android by separating the OS framework from the vendor implementation (HALs and the device-specific Linux kernel fork). This made it easier for Android OEMs to rebase their OSes on top of the latest AOSP framework as they could boot the latest version without needing updated code from vendors. As a result, OEMs could ready their custom Android forks quicker than before, and by extension, roll out major OS updates more quickly.

The next step in Google’s plans was to streamline the delivery of updates to key Android components. Google called this initiative Project Mainline when it introduced it alongside Android 10 in 2019. Google essentially took control of key OS components and forbid OEMs from modifying them. They then set up a delivery mechanism via Google Play so they could remotely roll out updates to these key components without having to wait for OEMs to apply the patches themselves. Mainline greatly improved how quickly devices receive updated versions of important OS components, in turn improving the security of the Android ecosystem as a whole.

But what’s coming next is even more important, and is arguably the most important part of Google’s long-term strategy. When we pointed out earlier how Treble modularized Android by separating the OS framework from the vendor implementation, we included the “device-specific Linux kernel fork” as part of that vendor code. Anyone who’s familiar with Linux on desktops will recognize a problem there: Why is it lumped in with closed-source vendor code? The problem is that while Android devices do ship with the Linux kernel, that kernel features a lot of out-of-tree code.

How did we get there? The problem, as outlined by Google software engineer Todd Kjos at this year’s Linux Plumbers Conference (via ArsTechnica), is because the mainline Linux kernel is forked several times before it ships on an Android device. Google forks each mainline Linux kernel into an “Android Common Kernel” branch, which closely tracks the mainline release but adds a few Android-specific patches. SoC vendors like Qualcomm, MediaTek, and Samsung then fork that kernel for each SoC they make. OEMs then take that SoC-specific kernel and add additional patches to implement support for the specific hardware they want to ship.

Due to these changes, “as much as 50% of the code running on a device is out-of-tree code (not from upstream Linux or AOSP common kernels)“, according to Google. The large amount of out-of-tree code on these devices makes merging upstream changes a long and challenging process. This is harmful to device security, as OEMs need to work harder to implement patches to vulnerabilities that are discovered in the Linux kernel. Furthermore, this leaves most Android devices on years-old kernel releases, meaning they miss out on new Linux kernel features.

In an effort to address this problem, Google is working on the Android Generic Kernel Image (GKI), which is essentially a kernel compiled straight from an ACK branch. The GKI isolates SoC vendor and OEM customizations to plugin modules, eliminating out-of-tree code and allowing Google to push kernel updates directly to the end-user. For over a year, Google has been working on a way to deliver GKI updates via the Play Store, through the use of a Mainline module.

According to our sources, devices that launch with Android 12 and ship with Linux kernel 5.10 must deploy a Google-signed boot image. Google’s very own Pixel 6 series will launch with Android 12 out of the box and ship with Linux kernel 5.10, so the two phones may be the first mass market devices to ship with a GKI.

In addition, Google’s Todd Kjos revealed that the company plans to shift to an “upstream first” development model for new Linux kernel features. This will help Google ensure that new code lands first in the mainline Linux kernel, which will reduce future technical debt accrued from more out-of-tree code landing on Android devices.

At the Linux Plumbers Conference this week, Kjos said, “Since out-of-tree modules are really important for our use case, we expect we’ll always have a set of exports and some things that are different or in addition to what’s upstream, but this whole project is a multi-year project working towards getting rid of as many out-of-tree patches as we possibly can, and aligning as much as possible with upstream.” Google aims to complete its work towards upstreaming existing features and isolating vendor changes by the end of 2022 and, starting in 2023, the company plans to adopt this “upstream first” development model to avoid such issues in the future.

The post Android is shifting to an “upstream first” development model for new Linux kernel features appeared first on xda-developers.