Home
Documentation Tutorials Try Cloud!

operator

The operator command groups subcommands for operators interacting with Vault. Most users will not need to interact with these commands.

Examples

Initialize a new Vault cluster:

$ vault operator init
Unseal Key 1: sP/4C/fwIDjJmHEC2bi/1Pa43uKhsUQMmiB31GRzFc0R
Unseal Key 2: kHkw2xTBelbDFIMEgEC8NVX7NDSAZ+rdgBJ/HuJwxOX+
Unseal Key 3: +1+1ZnkQDfJFHDZPRq0wjFxEuEEHxDDOQxa8JJ/AYWcb
Unseal Key 4: cewseNJTLovmFrgpyY+9Hi5OgJlJgGGCg7PZyiVdPwN0
Unseal Key 5: wyd7rMGWX5fi0k36X4e+C4myt5CoTmJsHJ0rdYT7BQcF

Initial Root Token: 6662bb4a-afd0-4b6b-faad-e237fb564568

# ...

Force a Vault to resign leadership in a cluster:

$ vault operator step-down
Success! Stepped down: https://127.0.0.1:8200

Rotate Vault's underlying encryption key:

$ vault operator rotate
Success! Rotated key

Key Term        2
Install Time    01 Jan 07 12:30 UTC

Usage

Usage: vault operator <subcommand> [options] [args]

  # ...

Subcommands:
    generate-root    Generates a new root token
    init             Initializes a server
    key-status       Provides information about the active encryption key
    rekey            Generates new unseal keys
    rotate           Rotates the underlying encryption key
    seal             Seals the Vault server
    step-down        Forces Vault to resign active duty
    unseal           Unseals the Vault server

For more information, examples, and usage about a subcommand, click on the name of the subcommand in the sidebar.

operator diagnose

The operator diagnose command should be used primarily when vault is down or partially inoperational. The command can be used safely regardless of the state vault is in, but may return meaningless results for some of the test cases if the vault server is already running.

Note: if you run the diagnose command proactively, either before a server starts or while a server is operational, please consult the documentation on the individual checks below to see which checks are returning false error messages or warnings.

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

Output layout

The operator diagnose command will output a set of lines in the CLI. Each line will begin with a prefix in parenthesis. These are:.

  • [ success ] - Denotes that the check was successful.

  • [ warning ] - Denotes that the check has passed, but that there may be potential issues to look into that may relate to the issues vault is experiencing. Diagnose warns frequently. These warnings are meant to serve as starting points in the debugging process.

  • [ failure ] - Denotes that the check has failed. Failures are critical issues in the eyes of the diagnose command.

In addition to these prefixed lines, there may be output lines that are not prefixed, but are color-coded purple. These are advice lines from Diagnose, and are meant to offer general guidance on how to go about fixing potential warnings or failures that may arise.

Warn or fail prefixes in nested checks will bubble up to the parent if the prefix superceeds the parent prefix. Fail superceeds warn, and warn superceeds ok. For example, if the TLS checks under the Storage check fails, the [ failure ] prefix will bubble up to the Storage check.

Command options

  • -config (string; "") - The path to the vault configuration file used by the vault server on startup.

Diagnose checks

The following section details the various checks that Diagnose runs. Check names in documentation will be separated by slashes to denote that they are nested, when applicable. For example, a check documented as A / B will show up as B in the operator diagnose output, and will be nested (indented) under A.

Vault diagnose

Vault Diagnose is the top level check that contains the rest of the checks. It will report the status of the check

Check operating system / check open file limit

Check Open File Limit verifies that the open file limit value is set high enough for vault to run effectively. We recommend setting these limits to at least 1024768.

This check will be skipped on openbsd, arm, and windows.

Check operating system / check disk usage

Check Disk Usage will report disk usage for each partition. For each partition on a prod host, we recommend having at least 5% of the partition free to use, and at least 1 GB of space.

This check will be skipped on openbsd and arm.

Parse configuration

Parse Configuration will check the vault server config file for syntax errors. It will check for extra values in the configuration file, repeated stanzas, and stanzas that do not belong in the configuration file (for example a "tcpp" listener as opposed to a tcp listener).

Currently, the storage stanza is not checked.

Check storage / create storage backend

Create Storage Backend ensures that the storage stanza configured in the vault server config has enough information to create a storage object internally. Common errors will have to do with misconfigured fields in the storage stanza.

Check storage / check consul TLS

Check Consul TLS verifies TLS information included in the storage stanza if the storage type is consul. If a certificate chain is provided, Diagnose parses the root, intermediate, and leaf certificates, and checks each one for correctness.

Check storage / check consul direct storage access

Check Consul Direct Storage Access is a consul-specific check that ensures Vault is not accessing the consul server directly, but rather through a local agent.

Check storage / check raft folder permissions

Check Raft Folder Permissions computes the permissions on the raft folder, checks that a boltDB file has been initialized within the folder previously, and ensures that the folder is not too permissive, but at the same time has enough permissions to be used. The raft folder should not have other permissions, but should have group rw or owner rw, depending on different setups. This check also warns if it detects a symlink being used.

Note that this check will warn that a raft file has not been created if diagnose is run without any pre-existing server runs.

This check will be skipped on windows.

Check storage / check raft folder ownership

Check Raft Folder Ownership ensures that vault does not need to run as root to access the boltDB folder.

Note that this check will warn that a raft file has not been created if diagnose is run without any pre-existing server runs.

This check will be skipped on windows.

Check storage / check for raft quorum

Check For Raft Quorum uses the FSM to ensure that there were an odd number of voters in the raft quorum when vault was last running.

Note that this check will warn that there are 0 voters if diagnose is run without any pre-existing server runs.

Check storage / check storage access

Check Storage Access will try to write a dud value, named diagnose/latency/<uuid>, to storage. Ensure that there is no important data at this location before running diagnose, as this check will overwrite that data. This check will then try to list and read the value it wrote to ensure the name and value is as expected.

Check Storage Access will warn if any operation takes longer than 100ms, and error out if the entire check takes longer than 30s.

Check service discovery / check consul service discovery TLS

Check Consul Service Discovery TLS verifies TLS information included in the service discovery stanza if the storage type is consul. If a certificate chain is provided, Diagnose parses the root, intermediate, and leaf certificates, and checks each one for correctness.

Check service discovery / check consul direct service discovery

Check Consul Direct Service Discovery is a consul-specific check that ensures Vault is not accessing the consul server directly, but rather through a local agent.

Create Vault server configuration seals

Create Vault Server Configuration Seals creates seals from the vault configuration stanza and verifies they can be initialized and finalized.

Check transit seal TLS

Check Transit Seal TLS checks the TLS client certificate, key, and CA certificate provided in a transit seal stanza (if one exists) for correctness.

Create core configuration / initialize randomness for core

Initialize Randomness for Core ensures that vault has access to the randReader that the vault core uses.

HA storage

This check and any nested checks will be the same as the Check Storage checks. The only difference is that the checks here will be run on whatever is specified in the ha_storage section of the vault configuration, as opposed to the storage section.

Determine redirect address

Ensures that one of the VAULT_API_ADDR, VAULT_REDIRECT_ADDR, or VAULT_ADVERTISE_ADDR environment variables are set, or that the redirect address is specified in the vault configuration.

Check cluster address

Parses the cluster address from the VAULT_CLUSTER_ADDR environment variable, or from the redirect address or cluster address specified in the vault configuration, and checks that the address is of the form host:port.

Check core creation

Check Core Creation verifies the logical configuration checks that vault does when it creates a core object. These are runtime checks, meaning any errors thrown by this diagnose test will also be thrown by the vault server itself when it is run.

Start listeners / check listener TLS

Check Listener TLS verifies the server certificate file and key are valid and matching. It also checks the client CA file, if one is provided, for a valid certificate, and performs the standard runtime listener checks on the listener configuration stanza, such as verifying that the minimum and maximum TLS versions are within the bounds of what vault supports.

Like all the other Diagnose TLS checks, it will warn if any of the certificates provided are set to expire within the next month.

Start listeners / create listeners

Create Listeners uses the listener configuration to initialize the listeners, erroring with a server error if anything goes wrong.

Check autounseal encryption

Check Autounseal Encryption will initialize the barrier using the seal stanza, if the seal type is not a shamir seal, and use it to encrypt and decrypt a dud value.

Check server before runtime

Check Server Before Runtime achieves parity with the server run command, running through the runtime code checks before the server is initialized to ensure that nothing fails. This check will never fail without another diagnose check failing.

operator generate-root

The operator generate-root command generates a new root token by combining a quorum of share holders. With the -dr-token or -recovery-token options, it can generate a DR operation token or a recovery token in the same way.

One of the following must be provided to start the root token generation:

  • A base64-encoded one-time-password (OTP) provided via the -otp flag. Use the -generate-otp flag to generate a usable value. The resulting token is XORed with this value when it is returned. Use the -decode flag to output the final value.

  • A file containing a PGP key or a keybase username in the -pgp-key flag. The resulting token is encrypted with this public key.

An unseal key may be provided directly on the command line as an argument to the command. If key is specified as "-", the command will read from stdin. If a TTY is available, the command will prompt for text.

Please see the generate root guide for step-by-step instructions.

Examples

Generate an OTP code for the final token:

$ vault operator generate-root -generate-otp

Start a root token generation:

$ vault operator generate-root -init

Enter an unseal key to progress root token generation:

$ vault operator generate-root

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

Command options

  • -cancel (bool: false) - Reset the root token generation progress. This will discard any submitted unseal keys or configuration.

  • -decode (string: "") - Decode and output the generated root token. This option requires the -otp flag be set to the OTP used during initialization. If value is "-" then read the encoded token from stdin.

  • -generate-otp (bool: false) - Generate and print a high-entropy one-time-password (OTP) suitable for use with the "-init" flag.

  • -init (bool: false) - Start a root token generation. This can only be done if there is not currently one in progress.

  • -nonce (string; "")- Nonce value provided at initialization. The same nonce value must be provided with each unseal key.

  • -otp (string: "") - OTP code to use with -decode or -init.

  • -pgp-key (keybase or pgp)- Path to a file on disk containing a binary or base64-encoded public PGP key. This can also be specified as a Keybase username using the format keybase:<username>. When supplied, the generated root token will be encrypted and base64-encoded with the given public key.

  • -status (bool: false) - Print the status of the current attempt without providing an unseal key. The default is false.

  • -dr-token (bool: false) - Generate DR operation token

  • -recovery-token (bool: false) - Generate recovery token

operator init

The operator init command initializes a Vault server. Initialization is the process by which Vault's storage backend is prepared to receive data. Since Vault servers share the same storage backend in HA mode, you only need to initialize one Vault to initialize the storage backend. This command cannot be run against already-initialized Vault cluster.

During initialization, Vault generates a root key, which is stored in the storage backend alongside all other Vault data. The root key itself is encrypted and requires an unseal key to decrypt it.

The default Vault configuration uses Shamir's Secret Sharing to split the root key into a configured number of shards (referred as key shares, or unseal keys). A certain threshold of shards is required to reconstruct the root key, which is then used to decrypt the Vault's encryption key.

Refer to the Seal/Unseal documentation for further details.

Examples

Start initialization with the default options:

$ vault operator init

Initialize, but encrypt the unseal keys with pgp keys:

$ vault operator init \
    -key-shares=3 \
    -key-threshold=2 \
    -pgp-keys="keybase:hashicorp,keybase:jefferai,keybase:sethvargo"

Initialize Auto Unseal with a non-default threshold and number of recovery keys, and encrypt the recovery keys with pgp keys:

$ vault operator init \
    -recovery-shares=7 \
    -recovery-threshold=4 \
    -recovery-pgp-keys="keybase:jeff,keybase:chris,keybase:brian,keybase:calvin,keybase:matthew,keybase:vishal,keybase:nick"

Encrypt the initial root token using a pgp key:

$ vault operator init -root-token-pgp-key="keybase:hashicorp"

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "") - Print the output in the given format. Valid formats are "table", "json", or "yaml". The default is table. This can also be specified via the VAULT_FORMAT environment variable.

Common options

  • -key-shares (int: 5) - Number of key shares to split the generated master key into. This is the number of "unseal keys" to generate. This is aliased as -n.

  • -key-threshold (int: 3) - Number of key shares required to reconstruct the root key. This must be less than or equal to -key-shares. This is aliased as -t.

  • -pgp-keys (string: "...") - Comma-separated list of paths to files on disk containing public PGP keys OR a comma-separated list of Keybase usernames using the format keybase:<username>. When supplied, the generated unseal keys will be encrypted and base64-encoded in the order specified in this list. The number of entries must match -key-shares, unless -stored-shares are used.

  • -root-token-pgp-key (string: "") - Path to a file on disk containing a binary or base64-encoded public PGP key. This can also be specified as a Keybase username using the format keybase:<username>. When supplied, the generated root token will be encrypted and base64-encoded with the given public key.

  • -status (bool": false) - Print the current initialization status. An exit code of 0 means the Vault is already initialized. An exit code of 1 means an error occurred. An exit code of 2 means the Vault is not initialized.

Consul options

  • -consul-auto (bool: false) - Perform automatic service discovery using Consul in HA mode. When all nodes in a Vault HA cluster are registered with Consul, enabling this option will trigger automatic service discovery based on the provided -consul-service value. When Consul is Vault's HA backend, this functionality is automatically enabled. Ensure the proper Consul environment variables are set (CONSUL_HTTP_ADDR, etc). When only one Vault server is discovered, it will be initialized automatically. When more than one Vault server is discovered, they will each be output for selection. The default is false.

  • -consul-service (string: "vault") - Name of the service in Consul under which the Vault servers are registered.

HSM and KMS options

  • -recovery-pgp-keys (string: "...") - Behaves like -pgp-keys, but for the recovery key shares. This is only available with Auto Unseal seals (HSM, KMS and Transit seals).

  • -recovery-shares (int: 5) - Number of key shares to split the recovery key into. This is only available with Auto Unseal seals (HSM, KMS and Transit seals).

  • -recovery-threshold (int: 3) - Number of key shares required to reconstruct the recovery key. This is only available with Auto Unseal seals (HSM, KMS and Transit seals).

  • -stored-shares (int: 0) - Number of unseal keys to store on an HSM. This must be equal to -key-shares.

Recovery keys: Refer to the Seal/Unseal documentation to learn more about recovery keys.

operator key-status

The operator key-status provides information about the active encryption key. Specifically, the current key term and the key installation time.

Examples

Get the key status:

$ vault operator key-status
Key Term          2
Install Time      01 Jan 17 12:30 UTC
Encryption Count  4494

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

operator members

The operator members lists the active node and the peers that it's heard from since it became active.

Examples

Get the key status:

$ vault operator members
Host Name            API Address              Cluster Address           Active Node    Version    Upgrade Version    Redundancy Zone    Last Echo
---------            -----------              ---------------           -----------    -------    ---------------    ---------------    ---------
josh-C02ZT9DYMD6R    http://127.0.0.1:8200    https://127.0.0.1:8201    true           1.11.0     1.11.0             a                  n/a
josh-C02ZT9DYMD6R    http://127.0.0.2:8200    https://127.0.0.2:8201    false          1.11.0     1.11.0             a                  2022-05-23T15:51:19-07:00
josh-C02ZT9DYMD6R    http://127.0.0.3:8200    https://127.0.0.3:8201    false          1.11.0     1.11.0             b                  2022-05-23T15:51:19-07:00
josh-C02ZT9DYMD6R    http://127.0.0.4:8200    https://127.0.0.4:8201    false          1.11.0     1.11.0             b                  2022-05-23T15:51:22-07:00
josh-C02ZT9DYMD6R    http://127.0.0.5:8200    https://127.0.0.5:8201    false          1.11.0     1.12.0             a                  2022-05-23T15:51:20-07:00
~

Note that in the above output.

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

operator migrate

The operator migrate command copies data between storage backends to facilitate migrating Vault between configurations. It operates directly at the storage level, with no decryption involved. Keys in the destination storage backend will be overwritten, and the destination should not be initialized prior to the migrate operation. The source data is not modified, with the exception of a small lock key added during migration.

This is intended to be an offline operation to ensure data consistency, and Vault will not allow starting the server if a migration is in progress.

Examples

Migrate all keys:

$ vault operator migrate -config migrate.hcl

2018-09-20T14:23:23.656-0700 [INFO ] copied key: data/core/seal-config
2018-09-20T14:23:23.657-0700 [INFO ] copied key: data/core/wrapping/jwtkey
2018-09-20T14:23:23.658-0700 [INFO ] copied key: data/logical/fd1bed89-ffc4-d631-00dd-0696c9f930c6/31c8e6d9-2a17-d98f-bdf1-aa868afa1291/archive/metadata
2018-09-20T14:23:23.660-0700 [INFO ] copied key: data/logical/fd1bed89-ffc4-d631-00dd-0696c9f930c6/31c8e6d9-2a17-d98f-bdf1-aa868afa1291/metadata/5kKFZ4YnzgNfy9UcWOzxxzOMpqlp61rYuq6laqpLQDnB3RawKpqi7yBTrawj1P
...

Migration is done in a consistent, sorted order. If the migration is halted or exits before completion (e.g. due to a connection error with a storage backend), it may be resumed from an arbitrary key prefix:

$ vault operator migrate -config migrate.hcl -start "data/logical/fd"

Configuration

The operator migrate command uses a dedicated configuration file to specify the source and destination storage backends. The format of the storage stanzas is identical to that used to configure Vault, with the only difference being that two stanzas are required: storage_source and storage_destination.

storage_source "mysql" {
  username = "user1234"
  password = "secret123!"
  database = "vault"
}

storage_destination "consul" {
  address = "127.0.0.1:8500"
  path    = "vault/"
}

Migrating to integrated raft storage

Example configuration

The below configuration will migrate away from Consul storage to integrated raft storage. The raft data will be stored on the local filesystem in the defined path. node_id can optionally be set to identify this node. cluster_addr must be set to the cluster hostname of this node. For more configuration options see the raft storage configuration documentation.

If the original configuration uses "raft" for ha_storage a different path needs to be declared for the path in storage_destination and the new configuration for the node post-migration.

storage_source "consul" {
  address = "127.0.0.1:8500"
  path    = "vault"
}

storage_destination "raft" {
  path = "/path/to/raft/data"
  node_id = "raft_node_1"
}
cluster_addr = "http://127.0.0.1:8201"

Run the migration

Vault will need to be offline during the migration process. First, stop Vault. Then, run the migration on the server you wish to become a the new Vault node.

$ vault operator migrate -config migrate.hcl

2018-09-20T14:23:23.656-0700 [INFO ] copied key: data/core/seal-config
2018-09-20T14:23:23.657-0700 [INFO ] copied key: data/core/wrapping/jwtkey
2018-09-20T14:23:23.658-0700 [INFO ] copied key: data/logical/fd1bed89-ffc4-d631-00dd-0696c9f930c6/31c8e6d9-2a17-d98f-bdf1-aa868afa1291/archive/metadata
2018-09-20T14:23:23.660-0700 [INFO ] copied key: data/logical/fd1bed89-ffc4-d631-00dd-0696c9f930c6/31c8e6d9-2a17-d98f-bdf1-aa868afa1291/metadata/5kKFZ4YnzgNfy9UcWOzxxzOMpqlp61rYuq6laqpLQDnB3RawKpqi7yBTrawj1P
...

After migration has completed, the data is stored on the local file system. To use the new storage backend with Vault, update Vault's configuration file as described in the raft storage configuration documentation. Then start and unseal the vault server.

Join additional nodes

After migration the raft cluster will only have a single node. Additional peers should be joined to this node.

If the cluster was previously HA-enabled using "raft" as the ha_storage, the nodes will have to re-join to the migrated node before unsealing.

Usage

The following flags are available for the operator migrate command.

  • -config (string: <required>) - Path to the migration configuration file.

  • -start (string: "") - Migration starting key prefix. Only keys at or after this value will be copied.

  • -reset - Reset the migration lock. A lock file is added during migration to prevent starting the Vault server or another migration. The -reset option can be used to remove a stale lock file if present.

  • -max-parallel int: 10 - Allows the operator to specify the maximum number of lightweight threads (goroutines) which may be used to migrate data in parallel. This can potentially speed up migration on slower backends at the cost of more resources (e.g. CPU, memory). Permitted values range from 1 (synchronous) to the maximum value for an integer. If not supplied, a default of 10 parallel goroutines will be used.

Note: The maximum number of concurrent requests handled by a storage backend is ultimately governed by the storage backend configuration setting, which enforces a maximum number of concurrent requests (max_parallel).

operator raft

This command groups subcommands for operators to manage the Integrated Storage Raft backend.

Usage: vault operator raft <subcommand> [options] [args]

 This command groups subcommands for operators interacting with the Vault
 integrated Raft storage backend. Most users will not need to interact with these
 commands. Here are a few examples of the Raft operator commands:

Subcommands:
    join           Joins a node to the Raft cluster
    list-peers     Returns the Raft peer set
    remove-peer    Removes a node from the Raft cluster
    snapshot       Restores and saves snapshots from the Raft cluster

join

This command is used to join a new node as a peer to the Raft cluster. In order to join, there must be at least one existing member of the cluster. If Shamir seal is in use, then unseal keys are to be supplied before or after the join process, depending on whether it's being used exclusively for HA.

If raft is used for storage, the node must be joined before unsealing and the leader-api-addr argument must be provided. If raft is used for ha_storage, the node must be first unsealed before joining and the leader-api-addr must not be provided.

Usage: vault operator raft join [options] <leader-api-addr>

  Join the current node as a peer to the Raft cluster by providing the address
  of the Raft leader node.

      $ vault operator raft join "http://127.0.0.2:8200"

The join command also allows operators to specify cloud auto-join configuration instead of a static IP address or hostname. When provided, Vault will attempt to automatically discover and resolve potential leader addresses based on the provided auto-join configuration.

Vault uses go-discover to support the auto-join functionality. Please see the go-discover README for details on the format.

By default, Vault will attempt to reach discovered peers using HTTPS and port 8200. Operators may override these through the --auto-join-scheme and --auto-join-port CLI flags respectively.

Usage: vault operator raft join [options] <auto-join-configuration>
  Join the current node as a peer to the Raft cluster by providing cloud auto-join
  metadata configuration.
    $ vault operator raft join "provider=aws region=eu-west-1 ..."

Parameters

The following flags are available for the operator raft join command.

  • -leader-ca-cert (string: "") - CA cert to communicate with Raft leader.

  • -leader-client-cert (string: "") - Client cert to authenticate to Raft leader.

  • -leader-client-key (string: "") - Client key to authenticate to Raft leader.

  • -retry (bool: false) - Continuously retry joining the Raft cluster upon failures. The default is false.

Note: Please be aware that the content (not the path to the file) of the certificate or key is expected for these parameters: -leader-ca-cert, -leader-client-cert, -leader-client-key.

list-peers

This command is used to list the full set of peers in the Raft cluster.

Usage: vault operator raft list-peers

  Provides the details of all the peers in the Raft cluster.

      $ vault operator raft list-peers

Example output

{
 ...
  "data": {
    "config": {
      "index": 62,
      "servers": [
        {
          "address": "127.0.0.2:8201",
          "leader": true,
          "node_id": "node1",
          "protocol_version": "3",
          "voter": true
        },
        {
          "address": "127.0.0.4:8201",
          "leader": false,
          "node_id": "node3",
          "protocol_version": "3",
          "voter": true
        }
      ]
    }
  }
}

remove-peer

This command is used to remove a node from being a peer to the Raft cluster. In certain cases where a peer may be left behind in the Raft configuration even though the server is no longer present and known to the cluster, this command can be used to remove the failed server so that it is no longer affects the Raft quorum.

Usage: vault operator raft remove-peer <server_id>

  Removes a node from the Raft cluster.

      $ vault operator raft remove-peer node1

Note

Once a node is removed, its Raft data needs to be deleted before it may be joined back into an existing cluster. This requires shutting down the Vault process, deleting the data, then restarting the Vault process on the removed node.

snapshot

This command groups subcommands for operators interacting with the snapshot functionality of the integrated Raft storage backend. There are 2 subcommands supported: save and restore.

Usage: vault operator raft snapshot <subcommand> [options] [args]

  This command groups subcommands for operators interacting with the snapshot
  functionality of the integrated Raft storage backend.

Subcommands:
    restore    Installs the provided snapshot, returning the cluster to the state defined in it
    save       Saves a snapshot of the current state of the Raft cluster into a file

snapshot save

Takes a snapshot of the Vault data. The snapshot can be used to restore Vault to the point in time when a snapshot was taken.

Usage: vault operator raft snapshot save <snapshot_file>

  Saves a snapshot of the current state of the Raft cluster into a file.

      $ vault operator raft snapshot save raft.snap

Note: Snapshot is not supported when Raft is used only for ha_storage.

snapshot restore

Restores a snapshot of Vault data taken with vault operator raft snapshot save.

Usage: vault operator raft snapshot restore <snapshot_file>

  Installs the provided snapshot, returning the cluster to the state defined in it.

      $ vault operator raft snapshot restore raft.snap

autopilot

This command groups subcommands for operators interacting with the autopilot functionality of the integrated Raft storage backend. There are 3 subcommands supported: get-config, set-config and state.

For a more detailed overview of autopilot features, see the concepts page.

Usage: vault operator raft autopilot <subcommand> [options] [args]

This command groups subcommands for operators interacting with the autopilot
functionality of the integrated Raft storage backend.

Subcommands:
    get-config    Returns the configuration of the autopilot subsystem under integrated storage
    set-config    Modify the configuration of the autopilot subsystem under integrated storage
    state         Displays the state of the raft cluster under integrated storage as seen by autopilot

autopilot state

Displays the state of the raft cluster under integrated storage as seen by autopilot. It shows whether autopilot thinks the cluster is healthy or not, and how many nodes could fail before the cluster becomes unhealthy ("Failure Tolerance").

State includes a list of all servers by nodeID and IP address. Last Index indicates how close the state on each node is to the leader's.

A node can have a status of "leader", "voter", and "non-voter".

Usage: vault operator raft autopilot state

  Displays the state of the raft cluster under integrated storage as seen by autopilot.

    $ vault operator raft autopilot state

Example output

Healthy:                      true
Failure Tolerance:            1
Leader:                       raft1
Voters:
   raft1
   raft2
   raft3
Servers:
   raft1
      Name:            raft1
      Address:         127.0.0.1:8201
      Status:          leader
      Node Status:     alive
      Healthy:         true
      Last Contact:    0s
      Last Term:       3
      Last Index:      38
   raft2
      Name:            raft2
      Address:         127.0.0.2:8201
      Status:          voter
      Node Status:     alive
      Healthy:         true
      Last Contact:    2.514176729s
      Last Term:       3
      Last Index:      38

autopilot get-config

Returns the configuration of the autopilot subsystem under integrated storage.

Usage: vault operator raft autopilot get-config

  Returns the configuration of the autopilot subsystem under integrated storage.

    $ vault operator raft autopilot get-config

autopilot set-config

Modify the configuration of the autopilot subsystem under integrated storage.

Usage: vault operator raft autopilot set-config [options]

  Modify the configuration of the autopilot subsystem under integrated storage.

      $ vault operator raft autopilot set-config -server-stabilization-time 10s

Flags applicable to this command are the following:

  • cleanup-dead-servers (bool) - Controls whether to remove dead servers from the Raft peer list periodically or when a new server joins. This requires that min-quorum is also set. Defaults to false.

  • last-contact-threshold (string) - Limit on the amount of time a server can go without leader contact before being considered unhealthy. Defaults to 10s.

  • dead-server-last-contact-threshold (string) - Limit on the amount of time a server can go without leader contact before being considered failed. This takes effect only when cleanup_dead_servers is set as true. Defaults to 24h.

    Note: A failed server that autopilot has removed from the raft configuration cannot rejoin the cluster without being reinitialized.

  • max-trailing-logs (int) - Amount of entries in the Raft Log that a server can be behind before being considered unhealthy. Defaults to 1000.

  • min-quorum (int) - Minimum number of servers that should always be present in a cluster. Autopilot will not prune servers below this number. This should be set to the expected number of voters in your cluster. There is no default.

  • server-stabilization-time (string) - Minimum amount of time a server must be in a healthy state before it can become a voter. Until that happens, it will be visible as a peer in the cluster, but as a non-voter, meaning it won't contribute to quorum. Defaults to 10s.

operator rekey

The operator rekey command generates a new set of unseal keys. This can optionally change the total number of key shares or the required threshold of those key shares to reconstruct the root key. This operation is zero downtime, but it requires the Vault is unsealed and a quorum of existing unseal keys are provided.

An unseal key may be provided directly on the command line as an argument to the command. If key is specified as "-", the command will read from stdin. If a TTY is available, the command will prompt for text.

Please see the rotating and rekeying for step-by-step instructions.

Examples

Initialize a rekey:

$ vault operator rekey \
    -init \
    -key-shares=15 \
    -key-threshold=9

Initialize a rekey when Auto Unseal is used for the Vault cluster:

$ vault operator rekey \
    -target=recovery \
    -init \
    -key-shares=15 \
    -key-threshold=9

Initialize a rekey and activate the verification process:

$ vault operator rekey \
    -init \
    -key-shares=15 \
    -key-threshold=9 \
    -verify

Rekey and encrypt the resulting unseal keys with PGP:

$ vault operator rekey \
    -init \
    -key-shares=3 \
    -key-threshold=2 \
    -pgp-keys="keybase:hashicorp,keybase:jefferai,keybase:sethvargo"

Rekey an Auto Unseal vault and encrypt the resulting recovery keys with PGP:

$ vault operator rekey \
    -target=recovery \
    -init \
    -pgp-keys=keybase:grahamhashicorp
    -key-shares=1
    -key-threshold=1

Store encrypted PGP keys in Vault's core:

$ vault operator rekey \
    -init \
    -pgp-keys="..." \
    -backup

Retrieve backed-up unseal keys:

$ vault operator rekey -backup-retrieve

Delete backed-up unseal keys:

$ vault operator rekey -backup-delete

Perform the verification of the rekey using the verification nonce:

$ vault operator rekey -verify -nonce="..."

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

Command options

  • -cancel (bool: false) - Reset the rekeying progress. This will discard any submitted unseal keys or configuration. The default is false.

  • -init (bool: false) - Initialize the rekeying operation. This can only be done if no rekeying operation is in progress. Customize the new number of key shares and key threshold using the -key-shares and -key-threshold flags.

  • -key-shares (int: 5) - Number of key shares to split the generated master key into. This is the number of "unseal keys" to generate. This is aliased as -n

  • -key-threshold (int: 3) - Number of key shares required to reconstruct the root key. This must be less than or equal to -key-shares. This is aliased as -t.

  • -nonce (string: "") - Nonce value provided at initialization. The same nonce value must be provided with each unseal key.

  • -pgp-keys (string: "...") - Comma-separated list of paths to files on disk containing public PGP keys OR a comma-separated list of Keybase usernames using the format keybase:<username>. When supplied, the generated unseal keys will be encrypted and base64-encoded in the order specified in this list.

  • -status (bool: false) - Print the status of the current attempt without providing an unseal key. The default is false.

  • -target (string: "barrier") - Target for rekeying. "recovery" only applies when HSM support is enabled or using Auto Unseal.

  • -verify (bool: false) - Indicate during the phase -init that the verification process is activated for the rekey. Along with -nonce option it indicates that the nonce given is for the verification process.

Backup options

  • -backup (bool: false) - Store a backup of the current PGP encrypted unseal keys in Vault's core. The encrypted values can be recovered in the event of failure or discarded after success. See the -backup-delete and -backup-retrieve options for more information. This option only applies when the existing unseal keys were PGP encrypted.

  • -backup-delete (bool: false) - Delete any stored backup unseal keys.

  • -backup-retrieve (bool: false) - Retrieve the backed-up unseal keys. This option is only available if the PGP keys were provided and the backup has not been deleted.

operator rotate

The operator rotate rotates the underlying encryption key which is used to secure data written to the storage backend. This installs a new key in the key ring. This new key is used to encrypted new data, while older keys in the ring are used to decrypt older data.

This is an online operation and does not cause downtime. This command is run per-cluster (not per-server), since Vault servers in HA mode share the same storage backend.

Examples

Rotate Vault's encryption key:

$ vault operator rotate
Key Term        3
Install Time    01 May 17 10:30 UTC

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

operator seal

The operator seal seals the Vault server. Sealing tells the Vault server to stop responding to any operations until it is unsealed. When sealed, the Vault server discards its in-memory root key to unlock the data, so it is physically blocked from responding to operations unsealed.

If an unseal is in progress, sealing the Vault will reset the unsealing process. Users will have to re-enter their portions of the root key again.

This command does nothing if the Vault server is already sealed.

For more information on sealing and unsealing, please the seal concepts page.

Examples

Seal a Vault server:

$ vault operator seal
Success! Vault is sealed.

Usage

There are no flags beyond the standard set of flags included on all commands.

operator step-down

The operator step-down forces the active Vault node within an HA cluster to step down from active duty. When executed against a non-active node, i.e. a standby or performance standby node, the request will be forwarded to the active node. While the affected node will have a delay before attempting to acquire the leader lock again, if no other Vault nodes acquire the lock beforehand, it is possible for the same node to re-acquire the lock and become active again. Due to the locking architecture, there may be a period of time before another node grabs the lock and finishes transitioning to active operation where the cluster has no active node. During the period where none of the nodes are active, forwarding of requests to the leader node will fail.

Examples

Force a Vault server to step down as the leader:

$ vault operator step-down
Success! Stepped down: http://127.0.0.1:8200

Usage

There are no flags beyond the standard set of flags included on all commands.

operator unseal

The operator unseal allows the user to provide a portion of the root key to unseal a Vault server. Vault starts in a sealed state. It cannot perform operations until it is unsealed. This command accepts a portion of the master key (an "unseal key").

The unseal key can be supplied as an argument to the command, but this is not recommended as the unseal key will be available in your history:

$ vault operator unseal IXyR0OJnSFobekZMMCKCoVEpT7wI6l+USMzE3IcyDyo=

Instead, run the command with no arguments and it will prompt for the key:

$ vault operator unseal
Key (will be hidden): IXyR0OJnSFobekZMMCKCoVEpT7wI6l+USMzE3IcyDyo=

For more information on sealing and unsealing, please the seal concepts page.

Examples

Provide an unseal key:

$ vault operator unseal
Key (will be hidden):
Sealed: false
Key Shares: 1
Key Threshold: 1
Unseal Progress: 0

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

Command options

  • -migrate (bool: false) - Indicate that this share is provided with the intent that it is part of a seal migration process.

  • -reset (bool: false) - Discard any previously entered keys to the unseal process.

operator usage

The operator usage command allows an administrator to retrieve a client count report for the default reporting period, or for a specific time range of months.

The command output will list clients by distinct entities, non-entity tokens, and total active clients for each namespace, and the total for the entire cluster. It may indicate that no data is available for the requested time range, which may be because the client count reporting is disabled, the time range is too far in the past, or no data has yet been collected since the feature was enabled.

Refer to the Vault Usage Metrics tutorial to learn more about usage metrics, including required policy and data collection details.

Examples

Retrieve the client counts for the default reporting period:

$ vault operator usage
Period start: 2019-11-01T00:00:00Z
Period end: 2020-10-31T23:59:59Z

Namespace path   Distinct entities   Non-Entity tokens   Active clients
--------------   -----------------   -----------------   --------------
[root]           1581                332                 1913

Total            1581                332                 1913

Retrieve the client counts for a specific month in the past:

$ vault operator usage -start-time=2020-01 -end-time=2020-01
Period start: 2020-01-01T00:00:00Z
Period end: 2020-01-31T23:59:59Z

Namespace path   Distinct entities   Non-Entity tokens   Active clients
--------------   -----------------   -----------------   --------------
[root]           954                 176                 1130

Total            954                 176                 1130

Usage

The following flags are available in addition to the standard set of flags included on all commands.

Output options

  • -format (string: "table") - Print the output in the given format. Valid formats are "table", "json", or "yaml". This can also be specified via the VAULT_FORMAT environment variable.

Command options

  • -start-time (date) - Start month of the report to generate. May be given as YYYY-MM-DD, YYYY-MM-DD, a full RFC3339 timestamp, or a Unix epoch timestamp. Defaults to the configurable default_report_months prior to end-time.

  • -end-time (date: previous month) - End month of the report to generate. Defaults to the end of the previous calendar month.

The output shows the exact time range being reported, which may not match the input parameters if a full month is not available, or if the available reports are a subset of the months requested.

PreviousnamespaceNextpatch

Last updated