Software reverse engineering, the art of pulling programs apart to figure out how they work, is what makes it possible for sophisticated hackers to scour code for exploitable bugs. It's also what allows those same hackers' dangerous malware to be deconstructed and neutered. Now a new encryption trick could make both those tasks much, much harder.
At the SyScan conference next month in Singapore, security researcher Jacob Torrey plans to present a new scheme he calls Hardened Anti-Reverse Engineering System, or HARES. Torrey's method encrypts software code such that it's only decrypted by the computer's processor at the last possible moment before the code is executed. This prevents reverse engineering tools from reading the decrypted code as it's being run. The result is tough-to-crack protection from any hacker who would pirate the software, suss out security flaws that could compromise users, and even in some cases understand its basic functions.
"This makes an application completely opaque," says Torrey, who works as a researcher for the New York State-based security firm Assured Information Security. "It protects software algorithms from reverse engineering, and it prevents software from being mined for vulnerabilities that can be turned into exploits."
A company like Adobe or Autodesk might use HARES as a sophisticated new form of DRM to protect their pricey software from being illegally copied. On the other hand, it could also mean the start of a new era of well-armored criminal or espionage malware that resists any attempt to determine its purpose, figure out who wrote it, or develop protections against it. As notable hacker the Grugq wrote on twitter when Torrey's abstract was posted to SyScan's schedule, HARES could mean the "end of easy malware analysis. :D"
To keep reverse engineering tools in the dark, HARES uses a hardware trick that's possible with Intel and AMD chips called a Translation Lookaside Buffer (or TLB) Split. That TLB Split segregates the portion of a computer's memory where a program stores its data from the portion where it stores its own code's instructions. HARES keeps everything in that "instructions" portion of memory encrypted such that it can only be decrypted with a key that resides in the computer's processor. (That means even sophisticated tricks like a "cold boot attack," which literally freezes the data in a computer's RAM, can't pull the key out of memory.) When a common reverse engineering tool like IDA Pro reads the computer's memory to find the program's instructions, that TLB split redirects the reverse engineering tool to the section of memory that's filled with encrypted, unreadable commands.
"You can specifically say that encrypted memory shall not be accessed from other regions that aren’t encrypted," says Don Andrew Bailey, a well-known security researcher for Lab Mouse Security, who has reviewed Torrey's work.
Many hackers begin their reverse engineering process with a technique called "fuzzing." Fuzzing means they enter random data into the program in the hopes of causing it to crash, then analyze those crashes to locate more serious exploitable vulnerabilities. But Torrey says that fuzzing a program encrypted with HARES would render those crashes completely unexplainable. "You could fuzz a program, but even if you got a crash, you wouldn’t know what was causing it," he says. "It would be like doing it blindfolded and drunk."
>"Imagine trying to figure out what Stuxnet did if you couldn’t look at it."
Torrey says he intends HARES to be used for protection against hacking---not for creating mysterious malware that can't be dissected. But he admits that if HARES works, it will be adopted for offensive hacking purposes, too. "Imagine trying to figure out what Stuxnet did if you couldn’t look at it," he says. "I think this will change how [nation-state] level malware can be reacted to."
HARES's protections aren't quite invincible. Any program that wants to use its crypto trick needs to somehow place a decryption key in a computer's CPU when the application is installed. In some cases, a super-sophisticated reverse engineer could intercept that key and use it to read the program's hidden commands. But snagging the key would require him or her to plan ahead, with software that's ready to look for it. And in some cases where software comes pre-installed on a computer, the key could be planted in the CPU ahead of time by an operating system maker like Apple or Microsoft to prevent its being compromised. "There are some concerns with this from a technical point of view," says Bailey. "But it’s way better than anything we have out there now."
