vhsm debug

Learn to debug vHSM server by starting a process that collects and probes information for a specific duration.

The debug command starts a process that monitors a vHSM server, collecting and probing information for a specified duration. This helps operators retrieve and share consistent diagnostic information about the server state.

Note: The command honors the same authentication variables as the base command, including the token stored from a previous login or the environment variables VAULT_TOKEN and VAULT_ADDR. The permissions granted by the token determine the extent of information collected.

If the command is interrupted, the collected data is persisted to an output directory.

---

Permissions

The ability to fetch debug data depends on the provided token. Some targets, such as server-status, query unauthenticated endpoints, while others require ACL permissions.

Required Policy for Full Debugging

path "auth/token/lookup-self" {
  capabilities = ["read"]
}

path "sys/pprof/*" {
  capabilities = ["read"]
}

path "sys/config/state/sanitized" {
  capabilities = ["read"]
}

path "sys/monitor" {
  capabilities = ["read"]
}

path "sys/host-info" {
  capabilities = ["read"]
}

path "sys/in-flight-req" {
  capabilities = ["read"]
}

Examples

Start debug with default settings:

vhsm debug

Run debug for 1 minute, with custom intervals and no compression:

vhsm debug -duration=1m -interval=10s -metrics-interval=5s -compress=false

Capture only specific targets:

vhsm debug -target=host -target=metrics

Output

Finished capturing information, bundling files...
Success! Bundle written to: vault-debug-2025-03-04T08-28-46Z.tar.gz

Command Options

Flag	Type	Default Value	Description
-compress	bool	true	Toggles compression of the output package.
-duration	int or string	2m	Duration for which the command runs.
-interval	int or string	30s	Interval for collecting profiling data and server state.
-log-format	string	standard	Log format for log target (standard or json).
-metrics-interval	int or string	10s	Interval for collecting metrics data.
-output	string	Auto-generated filename	Specifies output path for the debug package.
-target	string	all targets	Specifies targets to capture (can be used multiple times).

Capture Targets

The -target flag can be used multiple times to specify which information to capture. By default, all available targets are collected.

Target	Description
config	Sanitized version of the configuration state.
host	Information about the server instance, including CPU, memory, and disk.
metrics	Telemetry information.
pprof	Runtime profiling data, including heap, CPU, goroutine, and trace profiling.
replication-status	Replication status.
server-status	Health and seal status.

Note:

• The config, host, metrics, and pprof targets are only queried on active and performance standby nodes.

• The host target is unavailable on OpenBSD due to system library limitations.

Output Layout

The collected data is stored in a structured directory format. Profiling data is captured at each interval in separate files, while other targets store data as JSON arrays.

Example Directory Structure

vault-debug-2025-03-04T21-44-49Z/
├── 2025-03-04T21-44-49Z
│   ├── goroutine.prof
│   ├── heap.prof
│   ├── profile.prof
│   └── trace.out
├── 2025-03-04T21-45-19Z
│   ├── goroutine.prof
│   ├── heap.prof
│   ├── profile.prof
│   └── trace.out
├── config.json
├── host_info.json
├── index.json
├── metrics.json
├── replication_status.json
├── server_status.json