Reverse engineering a dynamic library on the Xperia Play

Welcome to part two of my journey to completely reverse the PSX emulator on the Xperia Play. When we last left off, I managed to figure out the image.ps format and the basic order of execution of the emulator. It’s been a week now, and I have more stuff to reveal.

Decrypting the data

One of the main problems was that most of the important files are encrypted. More specifically, these three files: ps1_rom.bin (BIOS), libdefault.so (the emulator), and image_ps_toc (then unknown data). As I mentioned before, Sony used what’s called white box cryptography, which means obfuscating the code to hide the decryption keys. But, we don’t need the keys, we just need the decrypted data. The obvious way of extracting the decrypted data is dumping it from the RAM. However, the Android kernel I’m using doesn’t support reading /dev/kmem and I don’t want to mess with recompiling the kernel. I’ve also tried dumping with GDB, and it did work, but the data isn’t complete and is messy. I used a more unorthodox method of obtaining the decrypted data. After hours of reading and mapping in IDA Pro, I figured out that everything that is decrypted goes through one public function, uncompress(), a part of zlib. This is important, because this means everything that is decrypted is sent to zlib and zlib is open source. That means, I just need to recompile zlib with some extra code in uncompress() that will dump the input and output data. A simple fwrite() will do, as the data is already in a clean, memcpy-able form. (I forgot about LD_PRELOAD at the time, but that might have worked easier). After some trouble getting NDK to compile zlib, I have dumps of both the compressed/decrypted and uncompressed forms of all encrypted content.

Analyzing the dumps

The next thing is to find out the meaning of the data that we worked so hard to get. ps1_rom.bin is easy. Surprisingly, it is NOT a PS1 BIOS file, but actually part of a PS2 BIOS dump (part, being only the PS1 part of the PS2 BIOS). Does this mean a PS2 emulator is coming for the Play? I don’t know. Next, we have libdefault.so. Plugging it into IDA Pro reveals the juicy details of the PS1 emulator. It’s really nothing interesting, but if we ever want multi-disk support or decrypting the manuals, this would be the place to look. Finally, we have image_ps_toc (as it is called in the symbols file). I am actually embarrassed to say it took almost a day for me to figure out that it’s a table of contents file. I did guess so at first, but I couldn’t see a pattern, but after a night’s sleep, I figured out the format of the uncompressed image_ps_toc file. (Offsets are in hex, data are little-endian)

0×4 byte header

0×4 byte uncompressed image size

0×12 byte constant (I’m guessing it may have something to do with number of disks and where to cut off)

0×4 byte number of entries

Each entry:

0×4 byte offset in image.ps, where the compressed image is split

Image.ps format

I actually forgot to mention this in my last post. The “rom” that is loaded by the emulator is a file named image.ps. It is found on the SD card inside the ZPAK. It is unencrypted, and if you delete it, it will be downloaded again from Sony’s servers unencrypted. How it works is that an PSX ISO is taken and split into 0×9300 (about 38kb) sections, and each section is compressed using deflate (zlib again) and placed inside image.ps (with a 0×14 byte header). The offsets of each section is stored in the toc file (and encrypted) because although uncompressed, they’re perfect 38kb sections, compressed, they’re variable sized. I already wrote a tool to convert image.ps to an ISO and back again/

Putting it all back together

Now that we’ve tore apart, analyzed, and understood every element of the PSX emulator on the Xperia Play, what do we do? The ultimate goal is to convert any PSX game to run on the Xperia Play, but how do we do that. There are two main challenges. First of all, libjava-activity.so, which loads everything, expects data to be encrypted. Once again, we need keys. Also, I’m pretty sure it uses a custom encryption technique called “TFIT AES Cipher”, because I was not able to find information on it anywhere else. However, since we have the decrypted files, we can patch the library to load the decrypted files directly from memory, and I was halfway into doing that when I realized two more problems. Secondly, if I were to patch the library to load decrypted data, that means every user needs to decrypt the files themselves (because I won’t distribute copyrighted code). Third, image_ps_toc is variable sized, which means if the image is too big, it’ll break the offsets and refuse to load.

Currently, I’m trying to find the easiest and most legal way of allowing custom image_ps_toc files to work. (Bonus points if I can also load custom BIOS files). What I hope for is to write a wrapper library, libjava-activity-wrapper.so, which loads libjava-activity.so and patches GetImageTOC and GetImageTOCLength to load from a file instead of memory. This means I have to deal with Java and JNI again (ugh), and also do some weird stuff with pointers and memcpy (double ugh). The JNI methods in the library do not have their symbols exported, so I have to find a way of manually load them.

Bonus: blind patching a binary

When trying to patch a method for an ARM processor, it’s a bit of a pain and I’m too lazy to read about proper GDB debugging techniques. In additions, Sony wasn’t kind enough to compile everything with debugging symbols. However, I came up with a hacky-slashy way of changing instructions and seeing what happens. First, open up IDA Pro and find the function you want to modify. For example, I want decrypt_executable() to bypass decryption and just copy data plain. Find the instruction to change, and the opcode to change it to. For example, I want to change a BL instruction to NOP and CMP to CMN (don’t jump to decryption process and negate the “return == 0″). I have ARM’s NOP memorized by now 00 00 A0 E1. I don’t know what CMN’s opcode is, but if I look around I can find CMN somewhere and I see it’s just a change from a 7 to a 5. After everything’s done, copy it over to the phone and run it. If it crashes (and it should), look at the dump. The only important part is the beginning:

I/DEBUG   (  105): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000054
I/DEBUG   (  105):  r0 002d9508  r1 413c103c  r2 2afcc8d0  r3 002d93d8
I/DEBUG   (  105):  r4 00000004  r5 002d93e0  r6 6ca9dd68  r7 00000000
I/DEBUG   (  105):  r8 7e9dd478  r9 2cbffc70  10 0000aca0  fp 6caa4d48
I/DEBUG   (  105):  ip 002d93e8  sp 7e9dd0c0  lr 00000001  pc 4112d01c  cpsr 40000010

The error message doesn’t help at all “SIGSEGV,” but we have a dump of all the registers in the CPU. The important one is the PC (program counter), which shows what offset the last instruction was at offset 0x4112d01c in the memory. To get the program offset, just cat /proc/{pid}/maps to find where libjava-activity.so is loaded in memory. Subtract the offsets, and pop it into IDA pro. Now figure out why it crashed and try again. I need to learn proper debugging techniques one day.

Recovering a formatted or corrupt Kindle 2

One day, while playing around with a Kindle 2, I accidentally deleted the /lib folder. Oops. Now, no command beyond “ls” and “rm” work. If this was a computer, I could have simply inserted a installation DVD and copied the files over, but this was an eBook reader, and I was in for a world of pain. This was a month ago, and I’ve finally recovered the Kindle. I’m posting what I did online to save anyone else who’s in the same boat a ton of time. This tutorial is only designed for the Kindle 2, but it MAY work for the DX. It will NOT work for the Kindle 3, but directions should be similar.

 

First

If you’ve think you “bricked” your Kindle, don’t panic yet. There could be a easy solution. Chances are, if you can see the startup progress bar loading, the solution should be easier (although I can’t tell you exactly what your problem is). I would follow Amazon’s troubleshooting directions first. Only proceed if you are absolutely sure nothing else can be done.

Overview

Here’s what you’ll need

  1. TTL to RS232 or USB connector. I used this one. For that, use the jumper on pin 1 and 2 on the side (with the three pins, pin 1 is towards the USB port). Connect Kindle Tx -> Tx, Rx -> Rx, GND -> GND, VDD -> VDD
  2. Windows with HyperTerminal (I tried Kermit on Linux, but it couldn’t send files. HyperTerminal is the only program I’ve tested that works sending files to the Kindle)
  3. Linux or Unix-based computer with “dd” and “ssh”
  4. My custom recovery kernel which allows jailbreak signed recovery packages and exporting MMC0 without a password. If you want to know how I’ve made it in technical details, see the appendix.

Here’s what we’ll be doing:

  1. Attaching the recovery port
  2. Flashing the custom patched recovery kernel
  3. Obtaining a backup of Kindle system files
  4. Restoring your Kindle

Attaching the recovery port

First open up the Kindle 2 to reveal the PCB board. You should remove both the metal casing and the white plastic with all the screws. On the top, to the left of the headphone jack, you should see four pads labeled J4. Either solder (recommended) or tape (make sure it isn’t lose!) four wires to these pads. The order of these ports (left to right, where left is towards the volume switch) are: VDD, Rx, Tx, GND. Connect these lines to your TTL adapter and connect the adapter to your computer.

Flashing the custom patched recovery kernel

Open up HyperTerminal, and connect to your adapter. Make sure to click “Configure” and fill in the settings. The settings are: BPS: 115200, Data bits: 8, Parity: none, Stop bits: 1, Flow control: none. Then, restart your Kindle either by removing & reconnecting the battery, holding the sleep switch for 30 seconds, or tapping the reset switch on the PCB. Press Enter when text pops up in HyperTerminal. You only have one second, so be quick. In uBook, type in “run prg_kernel_serial” (make sure to type fast or uBoot will timeout). Then right click on the text, and choose “Send File”. Under protocol, choose Ymodem-G and for the file, select my custom kernel. Wait a few minutes for it to upload and install, then type in “bootm 0xa0060000″ to boot the kernel. The Kindle has two kernels that it alternates on each boot, so if you want to use my recovery kernel, you need to either flash the other kernel also or type in “bootm 0xa0060000″ into uboot on startup. Hold down Enter either on your computer or on the Kindle to enter the recovery menu. The recovery menu times out in 10 seconds, so you need to be quick. First type “I” to recreate all the partitions, then type “3″ to export the MMC. Again, these can be typed from either your keyboard in HyperTerminal, or the Kindle keypad. If you do not have access to HyperTerminal because you are in Linux restoring, you can get back here by holding Enter on the Kindle keypad and pressing 3 on the recovery menu.

Obtaining a backup of Kindle system files

Let’s put your broken Kindle aside. You need a working copy of Kindle’s system files. I cannot provide this for legal reasons, but if you obtain another Kindle 2 (preferably the same model and version as your broken one, but others MAY work [not Kindle 3 though... yet]), jailbreak it and install the usbNetwork hack for SSH access. Make sure that Kindle has at least 500MB of free space on the FAT partition to store the backup image. Once you SSH’d into the working Kindle (there are tons of tutorials around on this), issue the following command:

dd if=/dev/mmcblk0p1 of=/mnt/us/rootfs.img bs=1024

Note that this will only make a copy of the OS files. All personal information, passwords, books, etc are not copied. You can tell your friend that. This may take five to fifteen minutes to run, but when the command returns with the blocks written, you can disable usbNetwork and enable USB storage again. Copy the rootfs.img file over to your recovery computer and prepare to restore.

Restoring your Kindle

Back to your broken Kindle. You need to reformat the root and copy over the backup files. I moved the Kindle over to a Linux computer because it is easier. You can also use OSX or maybe even cygwin, but I haven’t tested. In shell, type in the following commands:

sudo su # Become root, so you don’t need to sudo everything

fdisk -l # Look for your Kindle’s identifier, it should be something like /dev/sdc, it should be a 2GB drive with 4 partitions. I will use /dev/sdX to represent this drive

mkfs.ext3 /dev/sdX2 # Make a ext3 partition for /var/local

dd if=/path/to/rootfs.img of=/dev/sdX1 bs=1MiB # This will take a long time to finish

Note that an alternative method is to gzip rootfs.img and place it into a recovery package created using kindle_update_tool.py, but I’ll leave that as an exercise for the reader.

Appendix

So, what is in the magical Kindle recovery kernel? It’s actually just a regular Kindle kernel recompiled with a modified initramfs with a patched recovery script. Using the regular kernel, you’ll run into two difficulties when trying to recover. First, if you press 3 to export MMC0, you’ll get a password prompt. Good luck brute forcing it. Second, if you build a custom recovery package using kindle_update_tool.py m –k2 –sign –fb, it will not work because of signature checks. What I did was patch the two checks.

First, I extracted the recovery-utils file by gunzipping uImage (with the extra stuff stripped out), and gunzipped initramfs.cpio from that. Then I extracted initramfs.cpio and found recovery-utils under /bin.

Next, the easy part is patching the updater package signature checks. What I did is extract the updater RSA public key file from the Kindle, found under /etc/uks and used OpenSSL to extract the public key from it (n and e). Then I opened recovery-utils with a hex editor, searched for the public key, and replaced it with the jailbreak key (found in kindle_update_tool.py).

Finally, the challenging part was to patch the password check from export MMC0. First I opened recovery-utils with IDA Pro. Then I located the check_pass function. I worked backwards from that and saw it was called from loc_94A8. Here’s a snippet of the check along with my interpretation of what it does:

BL check_pass # Call the check_pass function

CMP R0, #0 # check_pass sets register R0 with the return value, we will check if R0 equals 0×0

BEQ loc_9604 # If the previous instruction is true, then password check has failed

LDR R0, =aDevMmcblk0 ; “/dev/mmcblk0″ # We did not jump out, so CMP R0, #0 is false

BL storage_export # Call this function

It’s always easy to patch the least amount of instructions with the least amount of changes, so this is what I did. (Note that IDA Pro doesn’t allow editing instructions directly, so I have to find the machine code in hex to know what to replace. Luckily, I have tons to instructions to look at and see what their corresponding machine codes are).

NOP # Instead of calling check_pass, I did MOV R0, R0 which is the same as NOP

CMN R0, R0 # Negative compare R0 with itself. Basically, always return false.

… rest is the same

Now, I saved the file and luckily, it was the same size. So I didn’t have to recreate the initramfs.cpio, I just replaced the file inside with my hex editor (note that cpio files do not have checksum checks unlike tar files). I copied this to the kernel source folder and compiled the kernel. Lucky for you, I have already done all of this so you don’t have to.

Compiling the Linux kernel for Amazon Kindle

So, I recently bought a Kindle 2. As usual, the minute it arrived, I ripped it apart, poked every chip, and then started to reverse engineer the damn thing. Wait. I didn’t have to! I found this out days late, after messing with IDA Pro. Amazon has generously released most of the back end code for the Kindle as open source. (The front end, aka the stuff you see, is written in Java and we might get to that another day). So I decided to compile my own Kindle kernel. Why? Why not. Here’s how:

Part 1: Prerequisites

  • Get a root shell of your Kindle. If you don’t know, Google “usbNetworking”
  • A Linux computer for compiling code
  • Amazon’s sources for your version of the Kindle: http://www.amazon.com/gp/help/customer/display.html?nodeId=200203720
  • An ARM cross-compiler. You can compile Amazon’s code, or if you’re lazy, use CodeSourcery’s precompiled toolchain: http://www.codesourcery.com/sgpp/lite/arm
  • The following packages, get them from your distro’s repo: libncurses-dev (for menuconfig), uboot-mkimage (for making the kernel image), and module-init-tools (depmod)

Part 2: Compiling the kernel

  1. Extract the source to anywhere. If you can’t decide, use “~/src/kernel/” and “cd” to the source files.
  2. Now, you need to configure for the Kindle, type “make mario_mx_defconfig
  3. Edit the “.config” file and look for the line that starts with “CONFIG_INITRAMFS_SOURCE“. We don’t need that, delete that line or comment (#) it out.
  4. Here’s the part were you make all your modifications to the kernel. You might want to do “make menuconfig” and add extra drivers/modules. I’ll wait while you do that.
  5. Back? Let’s do the actual compiling. Type the following: “make ARCH=arm CROSS_COMPILE=~/CodeSourcery/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi- uImage”. This will make the kernel image. I assume you installed CodeSourcery’s cross compiler to your home folder (default). If your cross compiler is elsewhere, change the command to match it.
  6. Compile the modules into a compressed TAR archive (for easy moving to the kindle): “make ARCH=arm CROSS_COMPILE=~/CodeSourcery/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi- targz-pkg” (again, if your cross compiler is installed to a different location, change it).
  7. For some reason, depmod refuses to run with the compile script, so we’re going to do it manually. Do the following “depmod -ae -F System.map -b tar-install -r 2.6.22.19-lab126 -n > modules.dep” Change 2.6.22.19-lab126 to your compiled kernel version.
  8. Open modules.dep up with a text editor and do a search & replace. Replace all instances of “kernel/” with “/lib/modules/2.6.22.19-lab126/kernel/” (again, use your version string). I’m not sure this is needed, but better safe then brick.
  9. Now copy arch/arm/boot/uImage, linux-2.6.22.19-lab126.tar.gz (or whatever your version is), and modules.dep to an easy to access location.

Part 3: Installing on Kindle

  1. Connect the Kindle to your computer, and open up the storage device. Copy the three files you moved from the previous part to your Kindle via USB.
  2. This part is mostly commands, so get a root shell to your Kindle, and do the following commands line by line. Again, anywhere the version string “2.6.22.19-lab126” is used, change it to your kernel’s version. Explanation follows.

mv /mnt/us/linux-2.6.22.19-lab126.tar.gz /mnt/us/modules.dep /mnt/us/uImage /tmp

mv /lib/modules /lib/modules.old

cd /tmp & tar xvzf /tmp/linux-2.6.22.19-lab126.tar.gz

mv lib/modules /lib/

chmod 644 modules.dep

mv modules.dep /lib/modules/2.6.22.19-lab126/

/test/flashtools/update-kernel-both uImage

sync

shutdown -r now

Wow, that’s a lot of commands. What did that do? Well, line by line:

  1. Move the files we compiled to the temp folder. That way, we don’t have to clean up.
  2. Back up the old kernel modules
  3. Go to the temp folder and untar the modules
  4. Install the modules
  5. Correct the permissions for the modules.dep file (in case something happened after copying from your computer)
  6. Move the module dependencies list to it’s correct folder.
  7. Flash the kernel (I don’t know why it has to be flashed twice to two different partitions, but if you don’t, it won’t load, maybe sig checks?)
  8. Make sure everything is finished writing
  9. Reboot