Reverse engineering a dynamic library on the Xperia Play

Welcome to part two of my journey to completely reverse the PSX emulator on the Xperia Play. When we last left off, I managed to figure out the image.ps format and the basic order of execution of the emulator. It’s been a week now, and I have more stuff to reveal.

Decrypting the data

One of the main problems was that most of the important files are encrypted. More specifically, these three files: ps1_rom.bin (BIOS), libdefault.so (the emulator), and image_ps_toc (then unknown data). As I mentioned before, Sony used what’s called white box cryptography, which means obfuscating the code to hide the decryption keys. But, we don’t need the keys, we just need the decrypted data. The obvious way of extracting the decrypted data is dumping it from the RAM. However, the Android kernel I’m using doesn’t support reading /dev/kmem and I don’t want to mess with recompiling the kernel. I’ve also tried dumping with GDB, and it did work, but the data isn’t complete and is messy. I used a more unorthodox method of obtaining the decrypted data. After hours of reading and mapping in IDA Pro, I figured out that everything that is decrypted goes through one public function, uncompress(), a part of zlib. This is important, because this means everything that is decrypted is sent to zlib and zlib is open source. That means, I just need to recompile zlib with some extra code in uncompress() that will dump the input and output data. A simple fwrite() will do, as the data is already in a clean, memcpy-able form. (I forgot about LD_PRELOAD at the time, but that might have worked easier). After some trouble getting NDK to compile zlib, I have dumps of both the compressed/decrypted and uncompressed forms of all encrypted content.

Analyzing the dumps

The next thing is to find out the meaning of the data that we worked so hard to get. ps1_rom.bin is easy. Surprisingly, it is NOT a PS1 BIOS file, but actually part of a PS2 BIOS dump (part, being only the PS1 part of the PS2 BIOS). Does this mean a PS2 emulator is coming for the Play? I don’t know. Next, we have libdefault.so. Plugging it into IDA Pro reveals the juicy details of the PS1 emulator. It’s really nothing interesting, but if we ever want multi-disk support or decrypting the manuals, this would be the place to look. Finally, we have image_ps_toc (as it is called in the symbols file). I am actually embarrassed to say it took almost a day for me to figure out that it’s a table of contents file. I did guess so at first, but I couldn’t see a pattern, but after a night’s sleep, I figured out the format of the uncompressed image_ps_toc file. (Offsets are in hex, data are little-endian)

0×4 byte header

0×4 byte uncompressed image size

0×12 byte constant (I’m guessing it may have something to do with number of disks and where to cut off)

0×4 byte number of entries

Each entry:

0×4 byte offset in image.ps, where the compressed image is split

Image.ps format

I actually forgot to mention this in my last post. The “rom” that is loaded by the emulator is a file named image.ps. It is found on the SD card inside the ZPAK. It is unencrypted, and if you delete it, it will be downloaded again from Sony’s servers unencrypted. How it works is that an PSX ISO is taken and split into 0×9300 (about 38kb) sections, and each section is compressed using deflate (zlib again) and placed inside image.ps (with a 0×14 byte header). The offsets of each section is stored in the toc file (and encrypted) because although uncompressed, they’re perfect 38kb sections, compressed, they’re variable sized. I already wrote a tool to convert image.ps to an ISO and back again/

Putting it all back together

Now that we’ve tore apart, analyzed, and understood every element of the PSX emulator on the Xperia Play, what do we do? The ultimate goal is to convert any PSX game to run on the Xperia Play, but how do we do that. There are two main challenges. First of all, libjava-activity.so, which loads everything, expects data to be encrypted. Once again, we need keys. Also, I’m pretty sure it uses a custom encryption technique called “TFIT AES Cipher”, because I was not able to find information on it anywhere else. However, since we have the decrypted files, we can patch the library to load the decrypted files directly from memory, and I was halfway into doing that when I realized two more problems. Secondly, if I were to patch the library to load decrypted data, that means every user needs to decrypt the files themselves (because I won’t distribute copyrighted code). Third, image_ps_toc is variable sized, which means if the image is too big, it’ll break the offsets and refuse to load.

Currently, I’m trying to find the easiest and most legal way of allowing custom image_ps_toc files to work. (Bonus points if I can also load custom BIOS files). What I hope for is to write a wrapper library, libjava-activity-wrapper.so, which loads libjava-activity.so and patches GetImageTOC and GetImageTOCLength to load from a file instead of memory. This means I have to deal with Java and JNI again (ugh), and also do some weird stuff with pointers and memcpy (double ugh). The JNI methods in the library do not have their symbols exported, so I have to find a way of manually load them.

Bonus: blind patching a binary

When trying to patch a method for an ARM processor, it’s a bit of a pain and I’m too lazy to read about proper GDB debugging techniques. In additions, Sony wasn’t kind enough to compile everything with debugging symbols. However, I came up with a hacky-slashy way of changing instructions and seeing what happens. First, open up IDA Pro and find the function you want to modify. For example, I want decrypt_executable() to bypass decryption and just copy data plain. Find the instruction to change, and the opcode to change it to. For example, I want to change a BL instruction to NOP and CMP to CMN (don’t jump to decryption process and negate the “return == 0″). I have ARM’s NOP memorized by now 00 00 A0 E1. I don’t know what CMN’s opcode is, but if I look around I can find CMN somewhere and I see it’s just a change from a 7 to a 5. After everything’s done, copy it over to the phone and run it. If it crashes (and it should), look at the dump. The only important part is the beginning:

I/DEBUG   (  105): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000054
I/DEBUG   (  105):  r0 002d9508  r1 413c103c  r2 2afcc8d0  r3 002d93d8
I/DEBUG   (  105):  r4 00000004  r5 002d93e0  r6 6ca9dd68  r7 00000000
I/DEBUG   (  105):  r8 7e9dd478  r9 2cbffc70  10 0000aca0  fp 6caa4d48
I/DEBUG   (  105):  ip 002d93e8  sp 7e9dd0c0  lr 00000001  pc 4112d01c  cpsr 40000010

The error message doesn’t help at all “SIGSEGV,” but we have a dump of all the registers in the CPU. The important one is the PC (program counter), which shows what offset the last instruction was at offset 0x4112d01c in the memory. To get the program offset, just cat /proc/{pid}/maps to find where libjava-activity.so is loaded in memory. Subtract the offsets, and pop it into IDA pro. Now figure out why it crashed and try again. I need to learn proper debugging techniques one day.

Porting Kindle 3.1: Part 2 – Update encryption

Overview

So, on the topic of Kindle (I swear, it’s becoming an obsession). I am currently in the process of porting the Kindle 3.1 software to Kindle 2 and DX. I will make a series of posts describing my process while describing how various parts of the Kindle operating system works. Now I’ve tested 3.1 on my Kindle 2 and it works perfectly fine. All features work (audio, TTS, book reading), and the new features also work without major slowdowns (new PDF reader, new browser, etc).

Where’s part one you ask? Well, part one was getting the 3.1 OS to work on the Kindle 2, the rest is making an easy installer. That is a long story that involves custom partition tables, manually creating tar files (checksums are a pain), remote debugging, and more. It’s a lot of stuff and most aren’t very useful because nobody should have to repeat the process, which is why I’m creating a easy to use installer. If I have time one day, I may write it down for documentation purposes.

First of all, I will write down the game plan. What I plan to do is create an installer with the least amount of steps for the user. I’m hoping for a two part or three part installer. (Can’t be one part because you need a copy of the OS, and distributing it is most likely frowned upon by Amazon). How the installer should work is:

  1. User copies a image-creator package on a jail-broken Kindle 2. This package will backup the original OS, and generate a new ext3 image with some required files from the Kindle 2 (drivers and such). It will also update the kernel to support recovery packages.
  2. User keeps backup in a safe place and copies the image-creator package and the image generated from the K2 on a jail-broken Kindle 3 and runs the package. The image-creator will scan the filesystem making sure all files exist are are unmodified, then copies the files to the ext3 image. It will then take the ext3 image and generate a Kindle 2 recovery package with the 3.1 OS.
  3. User copies the recovery package generated from the Kindle 3 and copies it to the Kindle 2 and restarts. The Kindle will write the ext3 image to the root partition.

Update Encryption

Now, Igor Skochinsky wrote a nice post a couple of years ago on the Kindle update encryption algorithm. Basically, to encrypt an update, you take each byte of the file and shift the bits four to the left and OR it with the same bits shifted four to the right. Then you AND the result by 0xFF and XOR it by 0x7A. (Sounds like some computer dance move). Well, Igor also wrote a nice Python script that does the encrypting and decrypting, but I didn’t want to port Python to Kindle, so I decided to modify Amazon’s update decryption script “dm” and reverse it to make a encryption script “md”. I opened up IDA Pro and looked for the encryption. Here it is nicely commented by me into psudocode:

BL getchar // get byte to modify

EOR R3, R0, #0x7A // R3 = R0 ^ 0x7A

CMN R0, #1 // if !(R0 == 1), we are at the end of the file …

MOV R0, R3,LSR#4 // R0 = R3 >> 4

AND R0, R0, #0xF // R0 = R0 & 0xF

ORR R0, R0, R3,LSL#4 // R0 = R0 | R3 << 4

BNE loc_8470 // … then jump to end of program

MOV R0, #0 // clear R0 register

ADD SP, SP, #4 // don’t care

LDMFD SP!, {PC} // don’t care

It was a simple matter of reversing the instructions and registers, but like I said before, IDA Pro does not allow changing instructions directly, so I had to mess around with the machine code in the hex editor until I made the instructions I want. Here’s the modified function nicely commented by me in human.

BL getchar // get byte to modify

CMN R0, #1 // if byte is 0×01, then …

MOV R3, R0,LSR#4 // set R0 to R0 right shift 4

AND R3, R3, #0xF // set R4 to R4 logical AND 0xF

ORR R3, R3, R0,LSL#4 // set R3 to R3 logical OR ( R0 left shift 4 )

EOR R0, R3, #0x7A // set R0 to R3 logical exclusive OR 0x7A

BNE loc_8470 // … exit program

MOV R0, #0 // clear register R0

ADD SP, SP, #4 // don’t care

LDMFD SP!, {PC} // don’t care

If you want to try it out, here’s the bspatch from “dm” to “md”. MD5 of dm is 6725ac822654b97355facd138f86d438 and after patching, md should be 3b650bcf4021b41d70796d93e1aad658. You can copy md to your Kindle’s /usr/sbin and test it out:

echo ‘hello world’ | md > hello.bin # “md” encrypt ‘hello world’ and output to hello.bin

cat hello.bin | dm # “dm” decrypt hello.bin and it should output ‘hello world’

Now that we can create update packages from the Kindle, I can start working on the Kindle 2 image-creator script.

Recovering a formatted or corrupt Kindle 2

One day, while playing around with a Kindle 2, I accidentally deleted the /lib folder. Oops. Now, no command beyond “ls” and “rm” work. If this was a computer, I could have simply inserted a installation DVD and copied the files over, but this was an eBook reader, and I was in for a world of pain. This was a month ago, and I’ve finally recovered the Kindle. I’m posting what I did online to save anyone else who’s in the same boat a ton of time. This tutorial is only designed for the Kindle 2, but it MAY work for the DX. It will NOT work for the Kindle 3, but directions should be similar.

 

First

If you’ve think you “bricked” your Kindle, don’t panic yet. There could be a easy solution. Chances are, if you can see the startup progress bar loading, the solution should be easier (although I can’t tell you exactly what your problem is). I would follow Amazon’s troubleshooting directions first. Only proceed if you are absolutely sure nothing else can be done.

Overview

Here’s what you’ll need

  1. TTL to RS232 or USB connector. I used this one. For that, use the jumper on pin 1 and 2 on the side (with the three pins, pin 1 is towards the USB port). Connect Kindle Tx -> Tx, Rx -> Rx, GND -> GND, VDD -> VDD
  2. Windows with HyperTerminal (I tried Kermit on Linux, but it couldn’t send files. HyperTerminal is the only program I’ve tested that works sending files to the Kindle)
  3. Linux or Unix-based computer with “dd” and “ssh”
  4. My custom recovery kernel which allows jailbreak signed recovery packages and exporting MMC0 without a password. If you want to know how I’ve made it in technical details, see the appendix.

Here’s what we’ll be doing:

  1. Attaching the recovery port
  2. Flashing the custom patched recovery kernel
  3. Obtaining a backup of Kindle system files
  4. Restoring your Kindle

Attaching the recovery port

First open up the Kindle 2 to reveal the PCB board. You should remove both the metal casing and the white plastic with all the screws. On the top, to the left of the headphone jack, you should see four pads labeled J4. Either solder (recommended) or tape (make sure it isn’t lose!) four wires to these pads. The order of these ports (left to right, where left is towards the volume switch) are: VDD, Rx, Tx, GND. Connect these lines to your TTL adapter and connect the adapter to your computer.

Flashing the custom patched recovery kernel

Open up HyperTerminal, and connect to your adapter. Make sure to click “Configure” and fill in the settings. The settings are: BPS: 115200, Data bits: 8, Parity: none, Stop bits: 1, Flow control: none. Then, restart your Kindle either by removing & reconnecting the battery, holding the sleep switch for 30 seconds, or tapping the reset switch on the PCB. Press Enter when text pops up in HyperTerminal. You only have one second, so be quick. In uBook, type in “run prg_kernel_serial” (make sure to type fast or uBoot will timeout). Then right click on the text, and choose “Send File”. Under protocol, choose Ymodem-G and for the file, select my custom kernel. Wait a few minutes for it to upload and install, then type in “bootm 0xa0060000″ to boot the kernel. The Kindle has two kernels that it alternates on each boot, so if you want to use my recovery kernel, you need to either flash the other kernel also or type in “bootm 0xa0060000″ into uboot on startup. Hold down Enter either on your computer or on the Kindle to enter the recovery menu. The recovery menu times out in 10 seconds, so you need to be quick. First type “I” to recreate all the partitions, then type “3″ to export the MMC. Again, these can be typed from either your keyboard in HyperTerminal, or the Kindle keypad. If you do not have access to HyperTerminal because you are in Linux restoring, you can get back here by holding Enter on the Kindle keypad and pressing 3 on the recovery menu.

Obtaining a backup of Kindle system files

Let’s put your broken Kindle aside. You need a working copy of Kindle’s system files. I cannot provide this for legal reasons, but if you obtain another Kindle 2 (preferably the same model and version as your broken one, but others MAY work [not Kindle 3 though... yet]), jailbreak it and install the usbNetwork hack for SSH access. Make sure that Kindle has at least 500MB of free space on the FAT partition to store the backup image. Once you SSH’d into the working Kindle (there are tons of tutorials around on this), issue the following command:

dd if=/dev/mmcblk0p1 of=/mnt/us/rootfs.img bs=1024

Note that this will only make a copy of the OS files. All personal information, passwords, books, etc are not copied. You can tell your friend that. This may take five to fifteen minutes to run, but when the command returns with the blocks written, you can disable usbNetwork and enable USB storage again. Copy the rootfs.img file over to your recovery computer and prepare to restore.

Restoring your Kindle

Back to your broken Kindle. You need to reformat the root and copy over the backup files. I moved the Kindle over to a Linux computer because it is easier. You can also use OSX or maybe even cygwin, but I haven’t tested. In shell, type in the following commands:

sudo su # Become root, so you don’t need to sudo everything

fdisk -l # Look for your Kindle’s identifier, it should be something like /dev/sdc, it should be a 2GB drive with 4 partitions. I will use /dev/sdX to represent this drive

mkfs.ext3 /dev/sdX2 # Make a ext3 partition for /var/local

dd if=/path/to/rootfs.img of=/dev/sdX1 bs=1MiB # This will take a long time to finish

Note that an alternative method is to gzip rootfs.img and place it into a recovery package created using kindle_update_tool.py, but I’ll leave that as an exercise for the reader.

Appendix

So, what is in the magical Kindle recovery kernel? It’s actually just a regular Kindle kernel recompiled with a modified initramfs with a patched recovery script. Using the regular kernel, you’ll run into two difficulties when trying to recover. First, if you press 3 to export MMC0, you’ll get a password prompt. Good luck brute forcing it. Second, if you build a custom recovery package using kindle_update_tool.py m –k2 –sign –fb, it will not work because of signature checks. What I did was patch the two checks.

First, I extracted the recovery-utils file by gunzipping uImage (with the extra stuff stripped out), and gunzipped initramfs.cpio from that. Then I extracted initramfs.cpio and found recovery-utils under /bin.

Next, the easy part is patching the updater package signature checks. What I did is extract the updater RSA public key file from the Kindle, found under /etc/uks and used OpenSSL to extract the public key from it (n and e). Then I opened recovery-utils with a hex editor, searched for the public key, and replaced it with the jailbreak key (found in kindle_update_tool.py).

Finally, the challenging part was to patch the password check from export MMC0. First I opened recovery-utils with IDA Pro. Then I located the check_pass function. I worked backwards from that and saw it was called from loc_94A8. Here’s a snippet of the check along with my interpretation of what it does:

BL check_pass # Call the check_pass function

CMP R0, #0 # check_pass sets register R0 with the return value, we will check if R0 equals 0×0

BEQ loc_9604 # If the previous instruction is true, then password check has failed

LDR R0, =aDevMmcblk0 ; “/dev/mmcblk0″ # We did not jump out, so CMP R0, #0 is false

BL storage_export # Call this function

It’s always easy to patch the least amount of instructions with the least amount of changes, so this is what I did. (Note that IDA Pro doesn’t allow editing instructions directly, so I have to find the machine code in hex to know what to replace. Luckily, I have tons to instructions to look at and see what their corresponding machine codes are).

NOP # Instead of calling check_pass, I did MOV R0, R0 which is the same as NOP

CMN R0, R0 # Negative compare R0 with itself. Basically, always return false.

… rest is the same

Now, I saved the file and luckily, it was the same size. So I didn’t have to recreate the initramfs.cpio, I just replaced the file inside with my hex editor (note that cpio files do not have checksum checks unlike tar files). I copied this to the kernel source folder and compiled the kernel. Lucky for you, I have already done all of this so you don’t have to.

Quickguide: Bypassing Lenovo S10 BIOS Whitelist

Lenovo loves to assert their dominance to you by whitelisting what WWAN (3G modem) card you can install in your laptop. There has been a way to bypass or remove the whitelist on most models, except the S10. Now I found a great guide here: http://www.sbbala.com/DellWWAN/Whitelist.htm that shows you how the remove the whitelist, but as many found out, it doesn’t always work. The problem is that… well, I don’t know what the problem is, but I’m guessing there’s additional checks. I’ve been trying to find the format of the S10 whitelist, but I’m having no luck, so we’ll do it the easy way. Brute force. Put your WWAN card into every whitelist entry. It’ll have to work then, right?

Now this is a “quickguide” which means I won’t spoon feed you. This is mostly because I don’t have the time to write a full guide, but maybe if I ever find the format of the whitelist or find a way to disable it completely, I’ll write an actual guide.

Basically, follow sbbala’s guide up until “Save and now you can close the hex-editor.” Instead of pulling out after replacing one entry, we’re going to replace a couple of others in MISER00.ROM. Take the PID/VID (little-endian reversed) and replace the follow entries with it:

DB 0B 00 19 (this one was in the guide)

D1 12 01 10 (this one will appear twice, replace both)

D1 12 03 10

C6 05 01 92

D2 19 F1 FF (this one will appear twice, replace both)

Now, I’m sure there are more devices in the whitelist, but for safety reasons, the ones I choose are 1) WWAN cards (I don’t want to accidentally remove the camera from the whitelist), and 2) in the Linux VID/PID list. If this doesn’t work, then try looking and replacing some more values in the whitelist. Although I haven’t completely reversed the whitelist format yet, I THINK it’s something like this. 1 Byte: FA followed by 4 bytes VID (little-endian) followed by 4 bytes PID (little-endian) followed by X bytes of don’t-know-what. The offset is different for every BIOS version, but it’s always in MISER00.ROM and is before DB 0B 00 19 and a bit after a bunch of 00s.

Creating a PSP FreeCheat Memory Patch

FreeCheat is a memory editor and cheat device (like Action Replay) for the PSP. It includes features like a live in-game memory viewer and searcher. One of the feature that intrigued me is the memory patcher. I had no idea what it does, but I assume it does what it says: patches the memory. Problem is: I’ve searched everywhere, but there seems to be no information on how to create a FreeCheat memory patch for the PSP (only .pat files for Monster Hunter). Well, it’s not that hard. After some trial and error, I’ve found out how to create a FreeCheat .pat memory patch. Note that the following should only be attempted by a person with enough technical knowledge to understand it.

To create a memory patch, first you need to find out what you want to patch. I suggest using FreeCheat’s own memory searcher to find the memory location. Another method if using FreeCheat to dump the memory to a file, and open it on your computer with a hex editor. Once you find something you want to replace, look at the address. On FreeCheat, this is the hex number on the bottom left of the memory viewer box. On your hex editor, it should be listed as “address” or “offset”. This should be between 0×0000000 and 0x017FFFFF. Now take this number and add 0×08800000 (hex math please) to it.

You can now create a new file in your hex editor to be the patch. The first four bytes in the file is the memory offset (that you found) in big endian form. The problem is that the offset you found is a little endian number. You need to convert it to a big endian number. Most hex editors allows something like this. I use 0xED on OSX, so on there (make sure it’s set to Edit->Number Mode->Little Endian!), I would type in 00000000, highlight it, and under “32 bit unsigned”, I would paste in the offset I found and it would convert it automatically. Then in the rest of the hex document, fill in whatever you want to replace the memory with. Save this as a .pat file and copy it to your PSP at /FreeCheat/PATCH and on the PSP, open up FreeCheat, go to MEM Manager and Load MEM Patch.