When it comes to programming an enclave, there are two options available:
Intel SGX Software Development Kit
This approach involves utilizing the Intel SGX Software Development Kit (SDK) to build an enclaved program. It requires partitioning the application and implementing secure communication mechanisms independently. Developers have full control over the implementation but need to handle all aspects of enclave development themselves.
Library Operating System
Library Operating Systems (libOSs) have gained attention as an alternative to virtualization. The concept behind libOSes is that applications don't necessarily require the entire functionality of a host Operating System. Operating Systems act as a middle layer between hardware and applications, facilitating application development. In the context of enclaves, libOSes allow wrapping an existing program with minimal modifications.
LibOSes offer several benefits, including security isolation, host platform compatibility, and migration support. By refactoring a traditional OS kernel into an application library, duplicate functionality and overheads are minimized. This lean approach makes libOSes particularly appealing for cloud virtualization. However, it's important to note that the lack of certain OS functions may limit the applicability of enclaving any type of application.
Normal application stack (left) vs. libOSed application stack (right)
The decision on how much functionality to include within the enclave depends on the chosen libOS. At one extreme, a library OS can pull most of the application-supporting code of the OS into the enclave. At the other extreme, thin "shim" layers wrap an API layer like the system call table. Pulling more code into the enclave increases the Trusted Computing Base (TCB) size but reduces the complexity and attack surface between the enclave and the untrusted OS. Performance implications depend on reducing enclave border crossings and managing the size of the Enclave Page Cache.
Best Practice
Considering the two approaches, developers need to consider security, performance, and usability when developing enclaved applications. The SDK approach is preferred for security-sensitive functions that require a small Trusted Computing Base (TCB) but may limit the usability due to the lack of certain functionalities. It is suitable for implementing key or identity management systems, cryptographic wallets, or HSM/TPM modules.
On the other hand, the libOS approach is recommended for comprehensive or closed-source applications that do not require recompiling the code. However, the choice of libOS impacts performance and usability. If the libOS doesn't provide a required host OS function to maintain a small TCB, developers may need to patch out that function from the program code, affecting the usability. This approach is suitable for off-the-shelf and proprietary applications such as databases, servers, and API endpoints.
Limitations
It's important to acknowledge that not all applications are suitable for SGX-based enclaving. Applications relying on shared memory principles cannot run inside an enclave, as it would compromise the SGX security model by allowing potentially malicious processes to enter the enclave. Prominent examples include PostgresDB and SAP's fork Hana.
