As mentioned above, the development of commercial penetration testing applications and layered host security defenses have significantly increased the complexity of exploit payloads. This section describes the exploit payload systems found in the major available commercial and open-source penetration testing frameworks.
Syscall proxying[3] is a simple and flexible technique for simulating remote execution. Syscall proxying implements a fat client, thin server remote procedure call protocol based on transferring stack frames. To execute a remote system call, the syscall proxy client marshalls the required argument register values and memory buffers into a single buffer. The syscall proxy server writes this marshalled stack buffer directly to its stack segment, pops the general register values from it, and executes the system call specified. After the system call returns, the server pushes all the general purpose registers onto the stack and transfers the stack buffer back to the client. This general mechanism requires very little server-side logic, but provides a great deal of flexibility, allowing the remote client to open files, make network connections, and even exploit other vulnerabilities on the same or other remote systems.
There are some limitations, however, with the syscall proxy's simple server-side logic. For example, it cannot handle the fork system call. In addition, file or socket input and output with the read or write system calls requires transferring the buffer twice because allocating stack space requires sending unused data in the marshalled stack buffer. Similarly, a round-trip for each system call can add significant latency to input/output intensive operations. Finally, all communication with the syscall proxy server is performed unencrypted. This can be a concern when a remote penetration tester is gaining access to sensitive hosts or files. CORE IMPACT [4], a commercial penetration testing application that employs syscall proxying, addresses these issues by also employing more complex agents with richer network transports that perform encryption. The initial syscall proxy server can be used to deploy a more capable remote agent, but this requires writing an executable to the remote system's hard drive.
Immunity's CANVAS [5], a competing penetration-testing application, uses a custom compiler-based payload construction system called MOSDEF [6]. The MOSDEF compiler takes a subset of the C programming language and generates highly position-independent machine code suitable for dynamic injection into remote processes. The remotely executed payload can make use of system functions or other locally compiled functions. Loops, conditional execution paths, and function calls are all executed inside the remote compromised process and only minimal necessary information is transferred to or from the remote process. This allows the payload author to create complex payloads that can, for example, transfer an entire remote file if successful, or only a smaller error code if an error occurred.
A compiler-style approach such as MOSDEF addresses many of the limitations of syscall proxying at the expense of implementation complexity. A MOSDEF program can create multiple processes and perform a significant amount of work on the client without requiring network round-trips or data transfers. The low-level flexibility of C also allows a MOSDEF program to perform byte-level memory manipulations easily. This is an important capability as performing a heap metadata exploit typically corrupts the heap to the point of causing any subsequent heap operations to crash the program. A MOSDEF program can systematically repair the heap before executing higher-level functions that may require heap memory allocation. MOSDEF, however, does not encrypt the data in transit or payload code. Immunity also provides a remote access trojan called Hydrogen that uses RSA and Twofish to protect communications. The Hydrogen server executable contains an embedded RSA key that is used to securely send the randomly generated Twofish session key to the client. The Hydrogen server is typically copied to the remote system using MOSDEF and executed [7].
An alternative approach is remote library injection. Remote library injection uses special linking and loading techniques to inject arbitrary shared libraries into a remote compromised process address space. The remote library injection approach has been implemented for both Linux ELF [8] and Windows DLL [9] and both implementations are included in the Metasploit open-source exploit development framework [10]. The benefit of this approach is that arbitrarily complex payloads or existing software can be easily re-linked and remotely executed. The existing implementations load the shared library purely in memory without writing the library to the remote system's hard drive.
The Metasploit project has implemented several payloads using DLL injection, including an injected VNC server [11] for remote graphical control of the host, and the Meterpreter [12], a dynamically extensible server-side scripted environment. The Meterpreter consists of the base Meterpreter library that is remotely injected into the victim process, the interactive shell running on the attacker's host, and custom extensions. Meterpreter extensions add new functions to the Meterpreter shell that are implemented as a combination of local scripting language code and a remotely injected library. The use of a local scripting language (Ruby in Metasploit 3.0 and Perl in prior versions) allows payload functionality to be rapidly developed and deployed on-the-fly. Meterpreter can optionally XOR-encode packets in order to evade Intrusion Detection System signatures.
In a different application but similar spirit, Adam Young and Moti Yung have investigated many ways that strong cryptography can augment and prevent computer viruses [13].