Using Consul as a KV store
Last updated: 3 minutes read.
HashiCorp Consul is a service networking solution that is used to connect and configure applications across dynamic, distributed infrastructure. Consul KV is a simple Key-Value (KV) store provided as a core feature of Consul that can be used to store and retrieve Tyk Gateway configuration across multiple data centres.
How to configure Tyk to access Consul
Configuring Tyk Gateway to read values from Consul is straightforward - you simply configure the connection in your Tyk Gateway config file (tyk.conf) by adding the kv section as follows:
{
"kv": {
"consul": {
"address": "localhost:8025",
"scheme": "http",
"datacenter": "dc-1",
"http_auth": {
"username": "",
"password": ""
},
"wait_time": 10,
"token": "",
"tls_config": {
"address": "",
"ca_path": "",
"ca_file": "",
"cert_file": "",
"key_file": "",
"insecure_skip_verify": false
}
}
}
}
| Key | Description |
|---|---|
| address | The location of the Consul server |
| scheme | The URI scheme for the Consul server, e.g. http |
| datacenter | Consul datacenter (agent) identifier |
| http_auth | Username and password for Tyk to log into Consul using HTTP Basic Auth (if required by your Consul service) |
| wait_time | Limits how long a watch will block in milliseconds (if enabled in your Consul service) |
| token | Used to provide a per-request access token to Consul (if required by your Consul service) |
| tls_config | Configuration for TLS connection to Consul (if enabled in your Consul service) |
Alternatively, you can configure it using the equivalent environment variables.
Where to store data in Consul
When you want to reference KV data from Tyk Gateway config or transform middleware, you can store your KV pairs wherever you like within the Consul KV store. You can provide the Consul path to the key in the reference using the notation appropriate to the calling location.
From Tyk Gateway 5.3.0, you can reference KV data from any string field in the API definition. For these you should create a folder named tyk-apis in the root of your Consul KV store and store all keys in a flat structure there (sub-directories not currently supported). You should not include the tyk-apis path in the reference so, for example, given a key-value pair "foo":"bar" stored in tyk-apis in Consul, you would reference this from the API definition using consul://foo.
How to access data stored in Consul
The notation used to refer to a KV pair stored in Consul depends upon the location of the reference as follows.
Tyk Gateway configuration file
As described here, from Tyk Gateway’s configuration file (tyk.conf) you can retrieve values from Consul using the following notation:
consul://path/to/KEY
API definition
The Target URL and Listen Path key-value pairs can be stored in any directory in the Consul KV store as they are accessed using a different mechanism than other fields in the API definition. If storing these in a sub-directory, you can retrieve the values from Consul by providing the directory path within Consul KV using the following notation:
consul://path/to/KEY
For certain transformation middleware because the secret resolution happens during the request context, a different notation is used to retrieve values from Consul:
$secret_consul.KEY
From Tyk Gateway v5.3.0 onwards, you can store KV pairs to be used in any string field in the API definition in the Consul KV store. You can retrieve these values from Consul, noting that you do not provide the directory path (/tyk-apis) when accessing data for these fields, using the following notation:
consul://KEY