Attacking Titan M with Only One Byte

During the last year and a half, we (Damiano Melotti, Maxime Rossi Bellom & Philippe Teuwen) studied the Titan M, a security chip introduced by Google in their Pixel smartphones, starting from the Pixel 3. Among other events, we presented our results at Black Hat EU 2021, together with a whitepaper that contains all the background knowledge we acquired in the first part of this research. In this blog post, we dive into CVE-2022-20233, the latest vulnerability we found on the chip's firmware. First, we show how we found this vulnerability, using emulation-based fuzzing with AFL++ in Unicorn mode. Then, we go over the exploitation and its inherent challenges, that eventually led us to obtain code execution on the chip.

The Titan M was introduced by Google in their Pixel devices in 2018. The main goal was to reduce the attack surface available to attackers, mitigating hardware tampering and side-channel attacks. Indeed, the chip lies on a separate system-on-a-chip (SoC) in the device, runs its own firmware, and communicates with the Application Processor (AP) through the SPI bus. It implements several APIs that guarantee a higher level of protection for the most security sensitive features of the smartphone, such as secure boot or a hardware-backed Keystore with StrongBox.

In the first steps of this research, we focused on reverse engineering the Titan M firmware, which is based on Embedded Controller (EC), a lightweight open source OS for micro-controllers. This OS is fairly simple, and is built around the concept of tasks, with a fixed stack size and no heap (so no complex dynamic allocation). This is an important point that will turn out to be useful later: the Titan M chip has essentially a static memory layout, therefore we can assume some objects to always be located at the same address.

The simplicity of the OS also helps when it comes to security, especially by completely eliminating some temporal memory safety bugs associated with dynamic allocation. In addition, thanks to it's Memory Protection Unit (MPU), the chip's memory is configured such that no Write and eXecute permissions are granted to a region at the same time. Secure boot is implemented thanks to some signature checks performed before loading the firmware. Despite these properties, no common exploit mitigation techniques can be found on the chip, except for a hardcoded stack canary that is placed at the end of the tasks' stacks to detect errors. This leaves Titan M fairly exposed to memory corruption vulnerabilities, therefore we decided to explore how to fuzz it and look for some of them.

As broadly known, fuzzing is an extremely effective way to find memory corruption bugs in code bases written in unsafe languages, such as C. However, fuzzing a security chip without access to the sources, and with plenty of hardware-dependent code, can be quite a challenge. We decided to explore two different techniques, namely black-box and emulation-based fuzzing.

We have already shown how to perform black-box fuzzing in our previous presentations, but briefly going over it again is useful to understand the differences with the other approach we explored. In general, black-box fuzzing of targets like Titan M can be quite simple: all we need is a channel to interact with what we want to test, and a way to tell if it crashes or reaches unexpected states. In our case, in the previous steps of this research, we had developed nosclient, a custom client that runs natively on Android and communicates directly with the kernel driver responsible for the communication with the chip. The possibility to send arbitrary messages gave us both the channel to communicate with Titan M, and the signal to infer what happened while processing them, represented by the return code from the library functions we use.

Most of the tasks communicating with Android use Protobuf to serialize messages. Therefore, we could leverage this grammar, accessible thanks to the open source definitions in the AOSP, to mutate these messages with libprotobuf-mutator. For Nugget, one of the tasks that is not using Protobuf, we used the evergreen Radamsa to generate our test cases.

This approach did bring some interesting results, which when it comes to fuzzing, can be measured in bugs. We could find again several bugs we already knew about, while fuzzing an older version of the firmware, including the buffer overflow we had exploited during last year's research. Moreover, we discovered two new crashes in the latest version of the firmware at the time, although both were caused by the same null pointer dereference, that was not considered serious enough to be a vulnerability and end up in an Android Security Bulletin.

Despite its relatively straightforward setup, and the advantage of testing in the real world, black-box fuzzing comes with a number of shortcomings. First of all, detection is extremely difficult, since we can only spot the bugs that cause crashes or faulty return codes. But most importantly, a well-known limitation is that black-box fuzzing tends to only exercise the targets' shallow states, given the lack of visibility over their internals. As a consequence, we might have been only scratching the surface of the Titan M firmware, which could also explain why all the detected crashes happened after just a few minutes of fuzzing.

An alternative approach that we focused on is emulation-based fuzzing. In short, given that the firmware is publicly available, why not try to emulate its execution on our laptop? Leveraging our background knowledge on how it works, coming from hours of reverse engineering, we can set up an emulation framework to run the firmware instruction by instruction, and analyze its behavior. This is also a very good feedback for a coverage-guided fuzzer, which can prioritize inputs leading to new instructions being reached and adjust its mutator accordingly.

There exist a number of different ways to implement a solution like this. We tried a few, but we ended up using the Unicorn emulator engine, and AFL++ as fuzzing framework. Unicorn is a QEMU-based project that only supports CPU emulation, without full-system emulation. Rather than a limitation, this was an advantage in our case, since we could set up our scripts very easily, and implement some ad-hoc tweaks to improve the detection of bugs or work around some issues. In addition, Unicorn integrates very well with AFL++, thanks to its unicorn mode, which essentially allows to fuzz pretty much anything that can be emulated with Unicorn. The only step required is the definition of the place_input_callback, a function that writes inputs in the target's memory at every interaction.

Thanks to this, we could also test other functions in the Titan M firmware, while not forgetting about reasonable attack scenarios. Nonetheless, after a couple of fuzzing campaigns around the SPI rescue feature that did not lead to any significant result, we decided to focus on the tasks again, replicating a similar experiment as the one we did in a black-box setting. AFL++ allows to have custom mutators, hence we plugged libprotobuf-mutator again and emulated three tasks of the firmware separately: Keymaster, Identity, and Weaver. We decided to ignore AVB due to the limited attack surface it exposes, with most of the interactions happening while the device is in bootloader mode.

First and foremost, we asked ourselves whether we could once again find the bugs we already knew about, to prove the effectiveness of this new solution. Fortunately, the answer was positive, with the exception of one bug, which takes us to the limitations of emulation-based fuzzing. As we all know, indeed, there is no free lunch, and also in this case, this approach brings some drawbacks.

To address the second issue, Unicorn allows to set a number of custom hooks to monitor certain memory accesses or specific fragments of code. We did implement some heuristics to catch some patterns of bugs, like broken calls to memcpy that ended up reading from the Boot ROM (that is mapped at address 0x0). On the other hand, this comes at a cost: hooks impact performance, and there is always a trade-off between the time spent reverse engineering to identify these patterns, and the time you actually let the fuzzer run freely.

But now, let's get into the tasty part. While fuzzing the Keymaster task, we discovered an interesting crash caused by a UC_ERR_WRITE_UNMAPPED, while processing an ImportKey request. This crash was occurring in a strb instruction, meaning that the firmware was trying to write 1 byte in an unmapped memory area. Note that the vulnerable firmware was introduced by Google's Pixel security update of May 2022.

A simple message that triggers the vulnerability is the following:

As we can see from the Protobuf definitions, ImportKey messages contain a KeyParameters field, which in turn is made of one or more KeyParameter objects. The bug occurs while processing the tags in a key parameter: the vulnerable function loops through the list of parameters, and for each one it checks if the tag is 0x20005, corresponding to DIGEST. When such parameters are found, the function takes the associated integer field, and after some checks it uses it as an offset of a stack buffer to set a byte to 1. By passing a large enough integer as parameter, it is possible to write out of bounds that one byte value of 0x01.

The code snippet below, both in assembly and decompiled view, shows these checks as well as the strb instruction causing the out of bounds write and subsequent crash. At that point, r1 is 0x1, r7 is the buffer address and r3 is the integer field of the current KeyParameter.

The vulnerability can actually be triggered as many times as the number of parameters with a DIGEST tag, with the only limitation of remaining within the maximum size of the KeyParameters list. This results in multiple, 1-byte, out-of-bounds writes. Such writes are however far from arbitrary, due to the checks on the offset. Without entering into details of the bit-wise operations, the least significant byte can only be 0x0, 0x2 or 0x4.

At this point, it may seem that this is a very minor and hardly exploitable issue. However, do not forget that Titan M's memory is completely static, as we mentioned earlier. In addition, the bug can be reached by different code paths, considering the various messages that contain a KeyParameters field. Provided that we can write at the right place, there are plenty of different things that can be done by setting a single byte to 1, from a simple DoS to altering a size variable somewhere in memory and causing a corruption elsewhere.

At this point, we wrote a small script to generate and highlight all the possible writable addresses in Ghidra. Then we started searching for interesting targets, and as it turned out, triggering the vulnerability just once was enough to compromise the chip.

During our tests, we understood that we could get to the vulnerable code with r1 containing 0x14019. After applying a 0xa204 offset to this (note that the least significant byte allows to pass the checks), we could reach 0x1e21d. This address is part of a structure that we called KEYMASTER_SPI_DATA, containing information about the exchange of Keymaster messages with Android. In particular, we can overwrite one byte of the memory location pointing to where incoming requests for the Keymaster task are stored: by default this is 0x192c8, but if we write 0x1 to 0x1e21d, the value becomes 0x101c8. As a result, the following Keymaster requests would be stored quite far from where they are supposed to be, with catastrophic consequences for the Titan M.

Before diving into the tale of how we exploited this vulnerability, let us take a moment to refresh how we can actually interact with the chip. The only tool we use is our nosclient, which allows us to forge arbitrary messages and send them directly to the driver. After sending a request, Titan M replies with a return code, and a response, which is empty in case of errors. With only this information available, developing any exploit is particularly challenging: not only we do not have access to any debugger capability, but we also cannot see any stack traces nor immediate side-effects of an attack.

This is where the UART console exposed by the chip becomes extremely handy. There are two ways of accessing it, and both of them present different challenges. The first way relies on a special debugging cable called SuzyQable. This is officially mentioned as the cable to debug Chrome OS microcontrollers, and since Titan M is based on the same operating system, it actually turns out to also work with it. To activate it, we only need to boot the Pixel smartphone in fastboot mode, and use the command fastboot oem citadel suzyq on. Then, when connecting the device using the SuzyQable, we can access the UART console as a tty interface on our laptop. In parallel, the cable also allows to communicate with the device through adb. Unfortunately though, as soon as the chip crashes or even after a reboot command is sent over this console, the channel is closed and the interface needs to be activated again going back to fastboot mode. This is very impractical, which is the reason why we went with the second way.

The second solution can be seen as the MacGyver one. The Titan M exposes the UART pins on the motherboard, so all we needed to do was solder two wires and obtain the very same shell as the one we could get with the cable. In this case, the console never closes and stays active no matter what happens to the device or the chip.

The Titan M console is extremely basic and allows simple interactions with the chip, to investigate statistics, versions, and similar things. Most importantly though, it is the place where debugging logs are printed. Indeed, going back to exploit development, this clearly does not provide any information on what goes wrong when something does not work, but it is still very useful: for example, we could always try to jump to a function that prints something and verify that our exploit is working fine up to that point.

As we said, thanks to the out of bounds write, we changed the address where incoming Keymaster requests were stored. This location is still in the memory space of the chip, thus an incoming message would end up overwriting other data in the Titan M's memory. For this reason, we tried sending larger and larger commands, monitoring the UART logs. After a bit of testing, we realized that if we placed a valid code address after 556 bytes of payload, we could jump to such an address, practically redirecting execution to a function of our choice. We were able to detect that thanks to some logs being printed in the UART console.

We suppose that this portion of memory was being used by some task (probably idle), which may have pushed a return address to its stack, that was now overwritten. At this point, we knew we could jump to an arbitrary function. How can we translate that into an actual exploit?

Given the memory protections in place, we cannot write our own shellcode and jump to it, as writable memory is not executable. Instead, we can rely on a code-reuse attack, and mount an exploit based on Return Oriented Programming. The objective was to build a read-anywhere primitive: this way, we could leak the secrets Titan M is protecting with a crafted command from nosclient.

Unfortunately, we could not write our ROP chain exactly where we wrote our first gadget, because it would corrupt some memory that would make the chip crash before our exploit would be complete. Consequently, the first challenge was to pivot the stack. Ideally, we wanted to decrease the stack pointer: assuming the stack to be moving towards larger addresses, we wanted to rely on the payload we could place in those 556 bytes, which we knew we could set as we wanted without causing any nasty side-effect. We found only one gadget to do so, which however also does a few more things:

Despite that, we wrote a ROP chain that would call it several times, while making sure that r0 was pointing to a memory area we also controlled, and where we wrote some gadgets to make the ldr and blx instructions work as we wanted. In the picture below, we graphically show this approach for the first iterations (it is actually done several more times). The red arrows follow the execution flow, and the blue ones the stack pointer. Note that the scale is not respected.

After having climbed up the stack a few times, we could proceed with the actual attack. The only way we could return our leaked bytes back to Android was to copy them in the response of an SPI command. For this to happen, we needed to be in the context of the command handler, which is called taking its address from memory after parsing the command code. We could therefore overwrite its address, but how should we set it? As replacing the actual code was impossible, all we could do was to make it jump into some memory area that we could also control, and write there more gadgets for our ROP chain. This is done in the space we created in the first step, making sure to ignore the slots we used in the previous ROP chain (basically assigning them to registers that we do not use).

This step was probably the most challenging one, as it entails finding the right gadget to make the stack pivot to a memory area that remains under our control when the handler is triggered. Unfortunately again, no such gadget existed, so we had to be creative. The way we solved this was quite tricky: by emulating the execution of the Keymaster DestroyAttestationIds command handler, we understood that there was one single 32-bit slot in its stack that was not overwritten by some previous functions in the call chain. Therefore, we wrote one single gadget there, that would act as a trampoline to actually move the stack far enough from the Keymaster stack and reach an area where our payload would be preserved.

Wrapping up so far, the exploit goes as follows:

Once we got here, finishing the exploit was relatively straightforward, as we controlled execution from the handler context. We called memcpy with the user-provided arguments, to copy what we wanted to read in the Keymaster SPI response buffer. We then jumped back to the Keymaster stack, like the normal command handler would have done.

Thanks to the leak functionality that we built with this exploit, we can now read arbitrary memory on the chip. This means we can now have access to any readable address. As a consequence, we can dump the secrets stored in the chip (such as the Root of Trust sent by the Pixel bootloader when the Titan M is updated). Moreover, we can access the Boot ROM, that was not previously available. This is possible despite the slightly customized version of memcpy present in the chip, which checks if the src or dst buffers equal 0x0. In fact, instead of jumping directly to the entry point of the function, we skip those checks and jump directly to the basic block where the copying happens.

One of the most interesting consequences of this attack is the ability to retrieve any StrongBox protected key, defeating the highest level of protection of the Android Keystore. Similarly to what happens in TrustZone, these keys can only be used inside Titan M, while they are stored in an encrypted key blob on the device.

The procedure is the following:

To demonstrate the effectiveness of this attack we also developed a dummy application that simply creates a StrongBox-protected AES key and encrypts a string with it. With our exploit and the approach explained above, we can leak the corresponding key from the Titan M, and by re-using the same initialization vector we can successfully decrypt the string offline (or rather "off phone").

As a reminder, there are two conditions to perform this attack. First, we need to be able to send commands to the chip, either from a rooted device (required to use nosclient), or physically accessing the SPI bus.

Then, we need a way to access the key blobs on the Android file system, which can be done again by being root, or with some exploit to bypass File Based Encryption or the uid access control.

We reported this vulnerability to Google, and of course the best mitigation is the patch provided in the June security bulletin. However, we do want to point out an interesting feature that would have made the StrongBox key blob leak impossible. Indeed, an application can create a key that is authentication-bound, specifying setUserAuthenticationRequired(true) when building it with KeyGenParameterSpec. This way, users need to authenticate before using the key and the key blob is encrypted a second time using a special key derived from the user password that we do not have.

The Titan M chip is the most secure component of Google's Pixel phones and the anchor to which all the security of the device is ultimately linked. Despite good architectural decisions and many efforts made to minimize its attack surface, making exploitation of vulnerabilities very hard, and the lack of appropriate tools to debug the chip, in our project we managed to:

The vulnerability was reported to Google in May 2022 and a fix was released in the Pixel Security update of June 2022.

To provide transparency of our vulnerability disclosure process and illustrate the complexity of coordinating the reporting, triaging, and fixing bugs in embedded software, a detailed timeline of all relevant events of the reporting process is provided below.