Lennert Wouters
Towards the end of May 2021 Starlink launched in Belgium so we were finally able to get our hands on a Dishy McFlatface. In this blog post we will cover some initial exploration of the hardware and we will explain how we dumped and extracted the firmware. Note that this blog post does not discuss any specific vulnerabilities, we merely document techniques that can be used by others to research the Starlink User Terminal (UT). Towards the end of this blog post we will include some interesting findings from the firmware. Note that SpaceX actively encourages people to find and report security issues through their bug bounty program: https://bugcrowd.com/spacex
We first set up our UT on a flat section of our university building’s roof and played around with it for a few hours, giving the UT and router the chance to perform a firmware update. We did run a few mandatory speed tests and were seeing as much as 268 Mbps download and 49 Mbps upload.
Teardown: Level 1
After a few hours of playing around it was time to get our hands dirty and disassemble the UT.
There have been a few teardown videos of the UT but none of them went into the details we were interested in: the main SoC and firmware. Nevertheless, these prior teardowns ([1, 2, 3]) of the dish contain a lot of useful information that allowed us to disassemble our dish without too much damage. It appears that there a few hardware revisions of the UT out there by now, certain parts of the teardown process can differ depending on the revision, something we learned the hard way. One of the aforementioned teardown videos shows the Ethernet and motor control cables to be detached from the main board before the white plastic cover is removed. On our UT, a tug on the motor control cables pulled the entire connector from the PCB; luckily it appears we can repair the damage. In other words, do not pull on those cables but first remove the back plastic cover, for those of you in the same boat: JST BM05B-ZESS-TBT.
After removing the back plastic cover we can see a metal shield covering the PCB, with the exception of a small cut-out containing the connectors for the Ethernet cable and motor control cable. There is one additional, unpopulated connector (4 pin JST SH 1.0mm), that we assumed would contain a UART debug interface as was shown in [4]. Note that the early teardown videos had an additional connector that is no longer present on our UT.
The UART interface
After hooking up a USB to serial converter we could get some information on the UT’s boot process. The output contains information regarding the early stage bootloaders before showing the following output.
We can see that the UT is using the U-Boot bootloader, and that typing ‘falcon’ may interrupt the boot process. While this may give us access to a U-Boot CLI we can also see that the serial input is configured as ‘nulldev’. Unsurprising, spamming the serial interface with ‘falcon’ during boot did not yield any result.
Continuing through the boot process we can see that U-Boot loads a kernel, ramdisk and Flattened Device Tree (FDT) from a Flattened uImage Tree (FIT) image that is stored on an embedded MultiMediaCard (eMMC). We can also see that the integrity (SHA256) and authenticity (RSA 2048) of the kernel, ramdisk and FDT is being checked. While we would have to perform some more tests it appears that a full trusted boot chain (TF-A) is implemented from the early stage ROM bootloader all the way down to the Linux operating system.
The remainder of the boot process contains some other interesting pieces of information. For example, we can see the kernel command line arguments and with that the starting addresses and lengths of some partitions. Additionally, we can see that the SoC contains 4 CPU cores.
Finally, when the UT completes its boot process we are greeted with a login prompt:
Development login enabled: no
SpaceX User Terminal.
user1 login:While making a few attempts at guessing valid login credentials we started realising that this UART interface would be unlikely to result in an easy win. We had to go deeper.
Teardown: Level 2
The back metal cover of the UT is glued to the assembly around the outer edge and additional glue is applied between the ribs in the metal cover and the underlying PCB. To loosen the glue at the edge of the metal cover we used a heat gun, prying tools, isopropyl alcohol and a lot of patience. Specifically, we first applied heat to a small section, used a prying tool to loosen that section, added IPA to help dissolve the glue and another round of the prying tool. Having removed the metal cover we are greeted by an enormous PCB measuring approximately 55 cm in diameter.
The parts of interest to us are shown in the picture below. The flip-chip BGA package with the metal lid is the main SoC on this board (marking: ST GLLCCOCA6BF). Unsurprisingly the SoC is connected to some volatile DRAM storage and non-volatile flash storage in the form of an eMMC chip.
Identifying eMMC test points
An embedded MultiMediaCard (eMMC) contains flash storage and a controller and is quite similar to an SD-card. The UT contains a Micron eMMC chip with package marking JY976, Micron offers a convenient tool to decode these package markings to the actual part number: https://www.micron.com/support/tools-and-utilities/fbga. The eMMC chip in question has part number MTFC4GACAJCN-1M and contains 4GB of flash storage in a BGA-153 package. In most scenarios we would desolder such an eMMC chip, reball it and dump it using a BGA socket. However, in this case we first attempted to dump the eMMC in-circuit to minimize the odds of damaging our UT and the eMMC chip.
eMMC chips are similar to SD cards in that they share a similar interface; the eMMC chip does support up to 8 data lines whereas SD-cards support up to 4 data lines. Both eMMC chips and SD-Cards support the use of only a single data line at the cost of lower read/write speeds.
To read the eMMC chip in-circuit we have to identify the clock (CLK), command (CMD) and data 0 (D0) signals. The 10 test points above the main SoC drew our attention, as 10 test points could be a CMD, CLK and 8 data lines. Additionally, all of these test points have a 30 Ohm series resistor connected to them which is relatively common for eMMC connections. We soldered a short wire to each test point, allowing us to create a logic analyser capture during the UT boot process. Using such a capture it is relatively straightforward to identify the required signals. The CLK signal will be the only repetitive signal, CMD is the signal that is first active after the clock starts toggling and D0 is the first data line to send out data. Determining the remaining 7 data lines is luckily unnecessary to dump the eMMC contents.
Dumping the eMMC in-circuit
To dump the eMMC chip we can connect a reader (that supports 1.8V IO) to the identified test points. Commercial readers that are mostly aimed at phone repair exist and should work well for this purpose (e.g. easy-JTAG and Medusa Pro). Alternatively you can use a regular USB SD-card reader (one that supports 1-bit mode) with an SD-card breakout with integrated level-shifters (e.g. https://shop.exploitee.rs/shop/p/low-voltage-emmc-adapter ). You can also whip something up yourself if you have some parts laying around. The picture below shows a standard USB SD-card reader connected to a TI TXS0202EVM level-shifter breakout board.
We only provide power to the eMMC to prevent the main SoC from interfering. The eMMC can be powered through two nearby decoupling capacitors, 3.3V is provided by the SD card reader and 1.8V is provided using a lab power supply. Once everything is hooked up properly we can create a disk image for later analysis.
Note that reading eMMC in circuit is not always an easy task; wires that are slightly too long can already prevent reading from succeeding. In this case it was rather straightforward and the system appears to function normally even with these relatively long wires attached.
Unpacking the raw eMMC dump
Unfortunately, Binwalk was not able to extract the full filesystem hence a manual analysis was required.
From the boot log it was clear that U-Boot was loading 49152 blocks of data starting at block 98304. Meaning that U-Boot is reading 0x1800000 bytes (blocksize of 512 (0x200) bytes) starting from address 0x3000000. We also know from the U-Boot output that this chunk of data is a FIT image. However, when trying to read the FIT image header information using the dumpimage tool (part of the u-boot-tools package) we weren’t getting any useful information.
Luckily SpaceX released their modifications to U-Boot on Github for GPL compliance: https://github.com/SpaceExplorationTechnologies By looking at this code it became clear that certain parts of the firmware are stored in a custom format that contains Error Correcting Code (ECC) data.
Stripping Reed-Solomon ECC words
The file spacex_catson_boot.h contains interesting information related to how the device boots.
The following snippets show how data is being read from eMMC (mmc read8) and the definition for startkernel.
The definition of startkernel is particularly interesting as it shows how the address where the kernel was loaded is being passed to a command called unecc. From the unecc command definition it is quite clear that this functionality is performing error correction on the data read from the eMMC.
The unecc command calls the do_unecc function implemented in unecc.c. Eventually this will result in calling the ecc_decode_one_pass function defined in ecc.c.
ecc.h contains several relevant definitions:
In the end what it boils down to is that, in this implementation, an ECC protected block of memory starts with the magic header value SXECCv followed by a version byte (1). This magic value marks the start of the ECC protected data, but also the start of the header block. The header block itself contains (in addition to the magic value and version byte), 215 bytes of data, an asterisk (*), and 32 bytes of ECC code words.
The header block is followed by multiple data blocks. Each of these data blocks is 255 bytes long and contains 222 bytes of data followed by an asterisk symbol (*) and 32 bytes of ECC code words. The last data block contains a dollar sign ($) instead of the asterisk and is followed by a final footer block. This footer block starts with and exclamation mark (!) that is followed by the number of data bytes in the ECC protected block of memory (4-bytes) and MD5 digest over those data bytes.
At this point it should be clear why Binwalk did not succeed in extracting the kernel, initramfs and FDT. Binwalk is able to pick up on the magic values that indicate the start of a particular file, but each block of the file had additional data that prevented Binwalk from extracting it. We used a simple Python scrip to remove the extra ECC data before using Binwalk to extract the image. Similarly, we can now also use dumpimage to get more information on the FIT image.
The FIT image and board revisions
The following snippet contains some of the dumpimage output. The FIT image contains 13 boot configurations, all configurations use the same kernel and initramfs images but a different Flattened Device Tree (FDT).
From the U-Boot code (spacex_catson_uterm.c) it becomes clear that the boot configuration is decided based on the state of 5 GPIO pins.
The following picture shows where these pins are being pulled high/low to indicate the board revision. Note from the earlier serial bootlog that our UT boots using the rev2_proto2 configuration (case 0b00011). In a recent video Colin O’Flynn pulled some of these pins high/low and could observe that the UT tried booting using a different FIT configuration and thus different devicetree [5]. We compared some of the FDTs but did not spot any differences that would be interesting from a security perspective.
A first look at the firmware
The login prompt
Recall that after the boot process has completed we are greeted with a login prompt. For further research it would be useful to gain the ability to log in, allowing us to interact with the live system. However, by looking at the shadow file it becomes clear that none of the users are allowed to log in. During boot the UT does read a fuse to determine if it is development hardware or not. If the UT is unfused it will set a password for the root user, allowing log in. Starlink UTs that are sold to consumers are of course production fused, disabling the login prompt.
root:*:10933:0:99999:7:::
bin:*:10933:0:99999:7:::
daemon:*:10933:0:99999:7:::
sync:*:10933:0:99999:7:::
halt:*:10933:0:99999:7:::
uucp:*:10933:0:99999:7:::
operator:*:10933:0:99999:7:::
ftp:*:10933:0:99999:7:::
nobody:*:10933:0:99999:7:::
sshd:*:::::::Development hardware
Development hardware often finds its way into the wrong hands [6, 7]. The engineers at SpaceX considered this scenario and appear to try to actively detect unfused development hardware that is no longer under their control. Development hardware is geofenced to only work in certain predefined areas, most of which are clearly SpaceX locations. SpaceX is likely notified if development hardware is used outside these predefined geofences.
Interestingly, some of these geofences do not seem to have a clear connection to SpaceX. While we will not disclose these locations here, I will say that the SNOW_RANCH looks like a nice location to play with development hardware.
Secure element
From references in the firmware it became clear that (our revision of) the UT contains a STMicroelectronics STSAFE secure element. The purpose of the secure element is not entirely clear yet, but it may be used to remotely authenticate the UT.
The SoC
Some people have asked which processor is being used: the answer is a Quad-Core Cortex-A53 and each core has been assigned a specific task.
What’s next
That’s it for now. We will likely continue looking into the Starlink UT and provide more details in future blog posts if there is interest. At the time of writing we were able to obtain a root shell on the UT, but it’s too early to publicly share more information on that matter.
References
[1] MikeOnSpace – Starlink Dish TEARDOWN! – Part 1 – https://youtu.be/QudtSo5tpLk [2] Ken Keiter – Starlink Teardown: DISHY DESTROYED! – https://youtu.be/iOmdQnIlnRo [3] The Signal Path – Starlink Dish Phased Array Design, Architecture & RF In-depth Analysis – https://youtu.be/h6MfM8EFkGg [4] MikeOnSpace – Starlink Dish TEARDOWN! – Part 2 – https://youtu.be/38_KTq8j0Nw [5] Colin O’Flynn – Starlink Dishy (Rev2 HW) Teardown Part 1 – UART, Reset, Boot Glitches – https://youtu.be/omScudUro3s [6] Brendan I. Koerner – The Teens Who Hacked Microsoft’s Xbox Empire – https://www.wired.com/story/xbox-underground-videogame-hackers/ [7] Jack Rhysider – Darknet Diaries EP 45: XBOX UNDERGROUND (PART 1) – https://darknetdiaries.com/episode/45/