Monitoring vHSM with Grafana

Learn to monitor the performance and usage of vHSM server with Grafana.

You can gain insights into vHSM's performance and usage to support proactive incident response and understand business workloads. Operators and security teams must monitor for conditions that may impact performance or signal security issues requiring immediate attention.

Monitoring vHSM with Grafana involves these steps:

1. Configure your local system

2. Install vHSM server

3. Install Prometheus

4. Install Grafana

5. Monitor in Grafana

Prerequisites

• Install Docker

• Set the environment variable for ENCLAIVE_LICENCE

• Download the vHSM docker image

• Download the vHSM CLl

Configure your local system for Docker

1. Create the required directories and set the environment variable for these paths.

mkdir -p ~/learn-vhsm-monitoring/{vhsm-config,vhsm-data} \
    ~/learn-vhsm-monitoring/grafana-config \
    ~/learn-vhsm-monitoring/prometheus-config && \
    export LEARN_VHSM=~/learn-vhsm-monitoring

1. Create a Docker network named learn-vhsm that is used by all containers.

docker network create --attachable --subnet 10.42.74.0/24 learn-vhsm

1. Set the environment variable for ENCLAIVE_LICENCE .

export ENCLAIVE_LICENCE=<your-licence-key>

Install vHSM server

1. Create a config.json file in $LEARN_VHSM/vhsm-config/that provides the configuration for the vHSM server . This config starts the vHSM server with a non-TLS TCP listener on port 8200, stores data in /vhsm/data, and enables Prometheus telemetry with 12h retention and no hostname.

   ui            = true
   # Configure the storage backend
   storage "file" {
                   path = "/vhsm/data"
   }
   listener "tcp" {
                   address       = "0.0.0.0:8200"
                   tls_disable = true
                   }
   api_addr  = "http://127.0.0.1:8200"
   
   telemetry {
     disable_hostname = true
     prometheus_retention_time = "12h"
   }

Note: TLS is disabled in this configuration for simplicity, but in production, always enable it to secure communication. This requires a certificate and key file on each vHSM server.

1. Start the vHSM server in a Docker container

docker run --rm -it \
  --cap-add=IPC_LOCK \
  -p 8200:8200 \
  -e ENCLAIVE_LICENCE="$ENCLAIVE_LICENCE" \
  -v $LEARN_VHSM/vhsm-config/config.json:/vhsm/config/config.json \
  -v $LEARN_VHSM/vhsm-data:/vhsm/data \
  harbor.enclaive.cloud/enclaive-dev/vhsm:latest \
  server -config=/vhsm/config/config.json

Note: Verify the status of the vhsm server and initialize or unseal the vHSM server.

Install Prometheus

Define Prometheus ACL Policy

1. The /sys/metrics endpoint in vHSM requires authentication. To let Prometheus access it, create a prometheus-metricsACL policy with read access to the endpoint.

vhsm policy write prometheus-metrics - << EOF
path "/sys/metrics" {
  capabilities = ["read"]
}
EOF

The output is:

Success! Uploaded policy: prometheus-metrics

1. Create a token with the prometheus-metrics policy for Prometheus to access vHSM metrics, and save its ID to the prometheus-token file in the Prometheus config directory.

vhsm token create \
  -field=token \
  -policy prometheus-metrics \
  > $LEARN_VHSM/prometheus-config/prometheus-token

Note:

n production, vHSM uses auth methods to issue tokens, but this example issues one directly for simplicity. The server is now ready to expose telemetry metrics, and the Prometheus token is set.

1. Create prometheus.yml in $LEARN_VHSM/prometheus-config/ to define a vhsm scrape job with the vHSM API endpoint, token path, and server IP with port.

scrape_configs:
  - job_name: vhsm
    metrics_path: /v1/sys/metrics
    params:
      format: ['prometheus']
    scheme: http
    authorization:
      credentials_file: /etc/prometheus/prometheus-token
    static_configs:
    - targets: ['host.docker.internal:8200']

1. Pull the Prometheus image.

docker pull prom/prometheus

1. Start the Prometheus container using volume mounts that point to the configuration file and token file.

docker run \
    --detach \
    --ip 10.42.74.110 \
    --name learn-prometheus \
    --network learn-vhsm \
    -p 9090:9090 \
    --rm \
    --volume $LEARN_VHSM/prometheus-config/prometheus.yml:/etc/prometheus/prometheus.yml \
    --volume $LEARN_VHSM/prometheus-config/prometheus-token:/etc/prometheus/prometheus-token \
    prom/prometheus

1. Verify that Prometheus is ready to receive requests.

docker logs learn-prometheus 2>&1 | grep -i "server is ready"

The log should contain an entry like this one.

ts=2025-04-06T14:19:01.375Z caller=main.go:1133 level=info msg="Server is ready to receive web requests."

Install Grafana

Create a Grafana config datasource.yml file in $LEARN_VHSM/grafana-config/to set Prometheus as the data source.

# config file version
apiVersion: 1

datasources:
- name: vhsm
  type: prometheus
  access: server
  orgId: 1
  url: http://10.42.74.110:9090
  password:
  user:
  database:
  basicAuth:
  basicAuthUser:
  basicAuthPassword:
  withCredentials:
  isDefault:
  jsonData:
     graphiteVersion: "1.1"
     tlsAuth: false
     tlsAuthWithCACert: false
  secureJsonData:
    tlsCACert: ""
    tlsClientCert: ""
    tlsClientKey: ""
  version: 1
  editable: true

1. Pull the latest Grafana image.

docker pull grafana/grafana:latest

1. Start the Grafana container.

docker run \
    --detach \
    --ip 10.42.74.120 \
    --name learn-grafana \
    --network learn-vhsm \
    -p 3000:3000 \
    --rm \
    --volume $LEARN_VHSM/grafana-config/datasource.yml:/etc/grafana/provisioning/datasources/prometheus_datasource.yml \
    grafana/grafana

1. Verify that the Grafana container is ready.

docker logs learn-grafana 2>&1 | grep "HTTP Server Listen"

The log should contain an entry similar to:

logger=http.server t=2025-04-06T14:19:43.478883954Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=http subUrl= socket=

1. You can also optionally check once more to verify that all containers are up and running.

 docker ps --format "table {{.Names}}\t{{.Status}}"

The output should resemble this example:

NAMES                  STATUS
learn-grafana          Up About a minute
learn-prometheus       Up 2 minutes
compassionate_banzai   Up About an hour

Monitor in Grafana

To monitor your vHSM server in Grafana UI you need to download the vhsm-test.json example file.

3KB

vhsm-test.json

Open

1. Open your browser and go to: http://localhost:3000

2. Login to Grafana with the Username: admin and Password: admin

Note: you'll be prompted to change it after first login.

1. In the Dashboards page select New → Import

2. Choose one of the following:

   • Upload JSON file (click Upload JSON file and select your vhsm-test.json file), or

   • Paste JSON content into the textbox.

3. Select the Prometheus data source when prompted.

4. Click Import.

You will be redirected to the imported dashboard where you can now:

• View the vHSM cluster health

• Monitor Audit Logs