Reverse engineering is the systematic approach to determine the inner working of a system by inspecting the system’s logic flow and responses to specific inputs.
For the Cyber Security Challenges you will be given some sample applications to reverse engineer in order to uncover hidden keys or serial numbers within the applications or in order to unlock hidden execution paths or bypass expected execution paths.
In order to achieve this you must first understand the control flow of the application and how one can analyse a compiled application.
Common Reverse Engineering steps
Test the input response of the application. Test how the system responds when presented with unexpected inputs (submitting an Integer in a String field or visa versa). Determine what controls are in place to prevent unexpected user behaviour. Map out these controls to start identifying control flows within the application. Fuzzing could be a viable technique to identify insecure input validation and possible injection paths.
Another common activity during the reverse engineering process is to decompile the source code of an application. Decompiling an application attempts to convert the binary code into the higher order language it was written in. Common decompiling software include: APKTool, Dex2Jar and Java-Decompiler. Higher order languages tend to be more human readable but depending on how the source code was obfuscated, the decompiled code might still be difficult to decipher. It is recommended you step through the code blocks to determine the logical flow of the program. This might be combined with the control flows identified during the previous step.
If a program cannot be decompiled a Hex Editor can be used to inspect the compiled code. The “strings” command can also be used on a compiled application to extract all strings found within the application, in some cases this might lead to some information regarding error messages and code control checks.
Another oprion would be to perform active analysis and step through the code with a debugger like OllyDbg. A debugging application allows you to perform dynamic analysis and set break points to step through the execution of a program.