For the last couple of months, molecule (composed of I along with Davee, Proxima, and xyz) have been working hard to bring you an easy-to-use homebrew solution. The result is HENkaku (変革), the first HEN for the Vita. Since the release of Rejuvenate a year ago, developers have created tons of wonderful emulators, games, and apps for the Vita. Unfortunately, Rejuvenate is hard to set up, has many annoying limitations, and supports only an older firmware version. As a result, we recommended Rejuvenate only to developers who wish for an unofficial way to write apps for the Vita. When I first announced Rejuvenate and the call for an open toolchain, I emphasized that the SDK must be binary compatible with the Vita’s native loader. I published the specifications document and some gracious developers took up the task and wrote vita-toolchain. At the time, there were some pushback on why I was adamant on binary compatibility when the loader was also written by us. Well, the reason was this: developers (mostly) do not have to make any changes to their code. If your homebrew ran on Rejuvenate, it will run with HENkaku with minimal work. We ask developers to build their code with the latest toolchain now for HENkaku compatibility.
I think one of the biggest challenge for system engineers is designing security. Recently, at the 32c3 conference, plutoo, derrek, and smea presented a series of hacks that completely defeated the security of the 3DS. As a result, people have implemented boot-time unsigned code execution (called “arm9loaderhax” in the 3DS community; other communities might relate this with ”untethered jailbreak” or “bootloader unlock”). What I want to do today is not to reveal anything novel, but look at the security of the 3DS as a whole and see what went wrong. In this deep dive, I will hypothesize the design decisions that led to the cryptosystem found on the 3DS. Then I will present the flaws that led to “arm9loaderhax.” Finally, I will summarize the findings and provide a few tips to fellow engineers in hopes that these kinds of mistakes will not be made again. (Extra details are provided in parenthesis, they are for people with deeper knowledge of the 3DS and are not required to understand the rest of the article.)
I’ve seen many CFWs (custom firmware; actually they’re just modified firmwares) for the 3DS but there seems to be a lack of organization and design in most of them. I believe that without a proper framework for patching the system, writing mods for the 3DS is extremely difficult and usually requires an in depth knowledge of the system even to make simple modifications. So here I present a plan that I hope developers will pick up on and contribute to.
It all started when I wanted to analyze some MeP code. Usually, I do all my disassembly in IDA Pro, but this is one of the few processors that isn’t supported by IDA. Luckily, there is objdump for this obscure architecture. After fumbling around for a bit, I was convinced that porting the disassembler to IDA would be a much better use of my time than manually drawing arrows and annotating the objdump output.