Emulate Your Way to Success
In the last episode of unlocking the Goke GK7101 SoC, we found ourselves faced with a big obstacle: a HAL layer in the form of I/O read/write calls that translated on-board peripherals’ register locations to their real addresses. The HAL’s underlying code is convoluted and much too hard to parse – it’s a large maze of twisty little if-then-elses, all alike. And since this SoC has tons of functionality, there are hundreds of register addresses to find.
But then, a surprise: a wild SDK appears! Somebody uploaded an official Goke SDK tarball to a certain open source repository site, and it has tons of code: the full Linux kernel as hacked up by Goke, their U-Boot source, a build system to make a full working system including root filesystem, and even some example applications that use their kernel drivers.
Weirdly, all of it is marked either GPL or, in a very few cases, public domain. You have to wonder why they’re not just putting this thing up for download; it’s literally all they have to do to abide by the GPL.
Sure enough, lots of information about the HAL is to be found. The struct
with the read and write calls is called hw_ops:
struct hw_ops
{
int (*get_version)(void);
unsigned int (*reserved)(unsigned int );
unsigned char (*hw_readb)(unsigned int );
unsigned short (*hw_readw)(unsigned int );
unsigned int (*hw_readl)(unsigned int );
void (*hw_writeb)(unsigned char , unsigned int );
void (*hw_writew)(unsigned short , unsigned int );
void (*hw_writel)(unsigned int , unsigned int );
unsigned int (*flash_read)(void);
void (*flash_write)(unsigned int);
unsigned char (*usb_readb)(unsigned int ptr, unsigned int offset);
unsigned short (*usb_readw)(unsigned int ptr, unsigned int offset);
unsigned int (*usb_readl)(unsigned int ptr, unsigned int offset);
void (*usb_writeb)(unsigned int ptr, unsigned int offset, unsigned char value);
void (*usb_writew)(unsigned int ptr, unsigned int offset, unsigned short value);
void (*usb_writel)(unsigned int ptr, unsigned int offset, unsigned int value);
unsigned int (*dma_readl)(unsigned int ptr);
void (*dma_writel)(unsigned int ptr, unsigned int value);
#if SPI_API_MODE
unsigned char (*spi_readb)(unsigned int ptr);
unsigned short (*spi_readw)(unsigned int ptr);
unsigned int (*spi_readl)(unsigned int ptr);
void (*spi_writeb)(unsigned int ptr, unsigned char value);
void (*spi_writew)(unsigned int ptr, unsigned short value);
void (*spi_writel)(unsigned int ptr, unsigned int value);
#endif
};
Keeping in mind that all these are 32-bit function pointers, our 0x10 and
0x1c offsets correspond to hw_readl and hw_writel, respectively.
The first function, get_version, looks like this in Ghidra:
********************************************
* FUNCTION *
********************************************
uint32_t __stdcall get_version(void)
uint32_t r0:4 <RETURN>
get_version XREF[2]: hal_init:c00120b0(*),
c0012160(*)
c00121a8 ldr r0, [DAT_c00121b0] = 20151223h
c00121ac bx lr
DAT_c00121b0 XREF[1]: get_version:c00121a8(R
c00121b0 unde 20151223h
That simply returns the number 0x20151223. We’ve seen that before, in the
vendor kernel’s boot log:
[ 0.000000] hal version = 20151223
What about the source to the actual HAL code? That gets put into memory by U-Boot, where the kernel then fences it off with the MMU, and uses it in-place. The U-Boot source has this:
static inline unsigned int gk_hal_init (int cp_flag)
{
unsigned int rval = 0;
unsigned int *haladdress;
haladdress = (unsigned int *)CONFIG_GK_HAL_ADDR;
*(volatile u32 *) (CONFIG_U2K_HAL_ADDR) = (u32)haladdress;
if(1==cp_flag)
memcpy(haladdress,hal_data,sizeof(hal_data));
hal_function_t hal_init = (hal_function_t) (haladdress) ;
g_hw = (struct hw_ops *)hal_init (0, 0, 0x90000000, 0xA0000000, 0) ;
return rval ;
}
It’s copied from hal_data to CONFIG_GK_HAL_ADDR, which is defined to
0xc0012000. So what’s hal_data?
int hal_data[]={
0xe92d45f8,0xe1a04000,0xe1a05001,0xe1a06002,0xe1a0a003,0xe59d7020,0xe3570000,0x0a00000e,
0xe59f8148,0xe58802f0,0xe5881320,0xe59832f8,0xe3530000,0x0a000002,0xe2880e2f,0xe3a01001,
...
For whatever reason, the HAL source is missing. Nice to have their kernel source, but we’re no closer to solving this massive HAL problem.
Breaking it down, getting the real register locations involves:
- identify “fake” register address.
- call the HAL’s
hw_readl()function. - find out what that function translated the address to.
Now that we have source code, it’s at least gotten a lot easier to get a list of fake registers – and names to describe them, which is very nice to have. Here’s the UART definition:
#define UART_RB_OFFSET 0x04
#define UART_TH_OFFSET 0x04
#define UART_DLL_OFFSET 0x04
#define UART_IE_OFFSET 0x00
#define UART_DLH_OFFSET 0x00
#define UART_II_OFFSET 0x08
#define UART_FC_OFFSET 0x08
#define UART_LC_OFFSET 0x18
#define UART_MC_OFFSET 0x0c
#define UART_LS_OFFSET 0x14
#define UART_MS_OFFSET 0x10
#define UART_SC_OFFSET 0x1c /* Byte */
#define UART_SRR_OFFSET 0x88
[...]
/* UART[x]_LS_REG */
#define UART_LS_FERR 0x80
#define UART_LS_TEMT 0x40
#define UART_LS_BI 0x20
#define UART_LS_FE 0x10
#define UART_LS_THRE 0x08
#define UART_LS_DR 0x04
#define UART_LS_PE 0x02
#define UART_LS_OE 0x01
That matches the code in the vendor kernel’s decompressor exactly, including the LSR’s DR bit being at bit 2.
But how can we call hw_readl() for all of those registers, and get the
results back in a way we can use in our code? We have the HAL code, but
it’s not like we can run the HAL without the GK7101… or can we?
It would actually be possible to run the code on any other ARM core sharing that ISA, if not for the fact that it interacts with actual memory locations: these HAL functions don’t just return the locations – they perform the actual reads and writes. So what we need is an ARM platform that will let us call the function but abort before doing the actual read operation.
Of course an architecture with an MMU makes that easy: by simply not mapping any memory except the part that runs the code, memory accesses to these unmapped locations generate an exception – that is, they run the instruction located at the ARM vector called Data Abort.
This would be even easier to do without having to mess with actual exception vectors on raw hardware, with custom handlers that somehow log what happened. What we need is something like QEMU – which supports ARM – to start up a little VM that only loads the HAL code, calls one function in it, and reports in some detail about a Data Abort exception. That’s actually a bit messy for QEMU; it’s intended for a much higher level of control. We need something lower-level than QEMU, but that’s somehow still easy to use and control. That seems like a big ask – would something like that even exist?
Enter the magnificent Unicorn. The authors took the basic low-level VM code in QEMU, and built a very different thing around it. Unicorn takes the form of a library, with an API that lets you create a VM, map memory into it, set callbacks to your code for exceptions, run code with a granularity down to one instruction, and read/write registers. It’s utterly easy to use, well documented, and exactly what we need. The API is in C, but has bindings for tons of languages, notably Python.
What we need to do, in order:
- Create an ARM VM.
- Map some memory into it, obviously not in the region where I/O might be.
We’ll use
0xc0000000, since the HAL lives in that region. - Assign some of that memory to the stack, and set the virtual CPU’s stack
pointer (the
spregister). - Drop the HAL code in there, at address
0xc0012000– we have no guarantees that all that code is relocatable, so this will avoid problems. - Set up a callback for the Data Abort exception: the exception called when code tries to access memory the MMU doesn’t have a mapping for. We’ll just print the “bad” address from there, and that’ll give us the final address as determined by the HAL.
- Call the
hal_init()function – recall it sets up some variables used by the address translation stuff later.
We’ll also need a little bit of code to call hw_readl() with a supplied
address we want translated. Something like this should do it:
ldr r4, [pc, #4] // load hw_readl() pointer
ldr r0, [pc, #4] // load argument to hw_readl()
blx r4 // call hw_readl()
// --- code ends here ---
.long 0 // pointer to hw_readl() goes here
.long 0 // test address goes here
The Unicorn application is here. It’s a bit messy, with lots of debugging code, but it works. You give it a HAL code blob as extracted from the device and a test address:
sumner: ./emu-hal halcode-fromdevice 0xa0005014
0xa0005014 -> 0x70005014
Using the UART register offsets we found in the source code, we can thus map the whole UART block:
sumner: for reg in 04 00 08 18 0c 14 10 1c 88; do ./emu-hal halcode-fromdevice 0xa00050$reg; done
0xa0005004 -> 0x70005000
0xa0005000 -> 0x70005004
0xa0005008 -> 0x70005008
0xa0005018 -> 0x7000500c
0xa000500c -> 0x70005010
0xa0005014 -> 0x70005014
0xa0005010 -> 0x70005018
0xa000501c -> 0x7000501c
0xa0005088 -> 0x70005088
Success! We’ll need to extract register offsets for all subsystems from the source code, but that’s a luxury problem: a bit of parsing with a Python script is peanuts compared to having to extract them from decompiled drivers.
Next up, we’ll figure out the registers for the interrupt and timer
subsystems, so we can get a kernel to boot. We can use the early_print()
facility to debug that, since a proper UART driver won’t work until
we have interrupts.