K8s + HashiCorp Vault on Azure DCsv3

Once your machine is prepared for SGX and you have installed kubernetes related dependencies, you can proceed:

1. Clone the repository for the Vault SGX Plugin by executing the command:

git clone https://github.com/enclaive/vault-sgx-plugin

Navigate to the cloned repository:

cd vault-sgx-plugin

Switch to the "workaround" branch:

git checkout workaround

Note: If you are using Azure DCs_v3, please remove the enclaive-sgx-pccs-config configMap and the qcnl-conf volume from all YAML files. This is because Azure DCs_v3 uses the indirect method for platform registration, and the K8S PCCS cannot retrieve certificates from the Intel PCS. Instead, Azure's global PCCS should be used.

Note: Remove nodeName selector or replace it to the selector that can select SGX available nodes correctly.

1. The pccs/pccs.yaml YAML file requires an API key from the Intel PCS API. Add this API key to the environment variable APIKEY in the YAML file.

Add your API key here

To subscribe to Intel Provisioning Certificate Service and receive an API key, visit the following link: Intel Provisioning Certificate Service and click on "Subscribe", then log in / create a new account and you will find the key.

Step 1

Step 2

Step 3

1. Apply the YAML file for the PCCS (Provisioning Certificate Client Service) by running the command:

kubectl apply -f pccs/pccs.yaml

1. Apply the YAML file for Vault by executing:

kubectl apply -f vault/vault.yaml

Wait for both of the pods to be ready:

$ kubectl get po
NAME                 READY   STATUS    RESTARTS   AGE
enclaive-sgx-pccs    1/1     Running   0          99s
enclaive-vault-sgx   1/1     Running   0          90s

1. To facilitate easy access to the Vault API for demonstration purposes, open another bash, connect to the control node and create a port forward by running:

kubectl port-forward svc/enclaive-vault-sgx 8200:8200

Note: This can also be placed behind a load balancer in a production environment.

1. Create a directory to store certificates:

mkdir -p certs

This directory will serve as the storage for certificates.

1. Set the deployment namespace used for certificate setup:

export ENCLAIVE_NAMESPACE="default"

1. Set the VAULT_CACERT environment variable to the path of the SGX PKI certificate file:

export VAULT_CACERT="certs/attest.pem"

Note: The SGX PKI certificates will be written by the "client" binary, including the attested temporary Vault certificate.

1. Set the VAULT_ADDR environment variable to the address of the Vault API:

export VAULT_ADDR="https://127.0.0.1:8200"

1. The measurement value can be extracted from the SIGSTRUCT in the Docker container by running:

export MEASUREMENT=$(kubectl exec -i enclaive-vault-sgx -- cat vault.sig | xxd -s 0x3c0 -l 32 -p -c 32)
echo $MEASUREMENT

This measurement value represents the MRENCLAVE for the current Vault release.

1. Build client binary from enclaive/vault-sgx-plugin/cmd/client/:

make client

1. Use the following command for attestation:

./client verify -ref $MEASUREMENT

Note: To ensure seamless operation and attestation availability upon restart, it is essential to deploy and configure the vault container before any other containers. To achieve this, enclaved applications utilize initContainers, which wait for the vault's health endpoint to signal an unsealed state. The vault deployment file utilizes an environment variable called ENCLAVE_DEPLOYMENT, which serves as a unique deployment ID for the domain, equating to the namespace.

The setup of the PCCS and Vault client is complete. You can now use the Vault client as follows:

1. Initialize the Vault and keep a note of the Unseal Key and Initial Root Token:

$ vault operator init -key-shares=1 -key-threshold=1
Unseal Key 1: oihiLER2EgGGGhuWcCsGP7V0ITM4A307sut9IdjMFKM=

Initial Root Token: hvs.Y28b62RHQkpBxvgY7biQl6nC

Vault initialized with 1 key shares and a key threshold of 1. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 1 of these keys to unseal it
before it can start servicing requests.

1. Unseal the vault with the key we just noted:

$ vault operator unseal
Unseal Key (will be hidden): 
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    1
Threshold       1
Version         1.13.1
Build Date      2023-03-23T12:51:35Z
Storage Type    file
Cluster Name    vault-cluster-afe31fbc
Cluster ID      6ba4cdf8-5ea1-e3f9-f6bf-09f14a439009
HA Enabled      false

1. Log in to the Vault with the Initial Root Token:

$ vault login
Token (will be hidden): 
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                hvs.Y28b62RHQkpBxvgY7biQl6nC
token_accessor       UaTmPfJvcuoqlWtY7GEzt8Ev
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

1. The SHA256 parameter is an expected parameter for Vault, providing integrity over the plugin binary. Extract the hash from the Docker container by running:

export HASH=$(kubectl exec -i enclaive-vault-sgx -- sha256sum plugins/vault-plugin-auth-sgx | cut -f1 -d' ')
echo $HASH

1. Register the Vault SGX plugin using the following command:

vault plugin register -sha256="$HASH" auth vault-plugin-auth-sgx

1. Enable the secret engines and set up the PKI by running the following command:

./client.sh enable

This command requires the ENCLAIVE_NAMESPACE variable to be set and will generate client certificates used for demonstration or external connections to Vault.

1. Register the Redis-SGX container with a secret using the following command:

./client.sh create

The script has example secrets with the expected measurement already hardcoded. It uses only standard Vault commands from the CLI.

With these steps completed, your cluster is now ready to attest an application, check the Use Cases for the specific guidance on the applications you want to deploy.

🌍Tutorials