Rich Plugins Data Structures
This page describes the data structures used by Tyk rich plugins, for the following plugin drivers:
- Python (built-in)
- Lua (built-in)
- gRPC (external, compatible with any supported gRPC language)
The Tyk Protocol Buffer definitions are intended for users to generate their own bindings using the appropriate gRPC tools for the required target language. The remainder of this document illustrates a class diagram and explins the attributes of the protobuf messages.
Class Diagram
The class diagram below illustrates the structure of the Object message, dispatched by Tyk to a gRPC server that handles custom plugins.
Object
The Coprocess.Object
data structure wraps a Coprocess.MiniRequestObject
and Coprocess.ResponseObject
It contains additional fields that are useful for users that implement their own request dispatchers, like the middleware hook type and name.
It also includes the session state object (SessionState
), which holds information about the current key/user that’s used for authentication.
message Object {
HookType hook_type = 1;
string hook_name = 2;
MiniRequestObject request = 3;
SessionState session = 4;
map<string, string> metadata = 5;
map<string, string> spec = 6;
ResponseObject response = 7;
}
Field Descriptions
hook_type
Contains the middleware hook type: pre, post, custom auth.
hook_name
Contains the hook name.
request
Contains the request object, see MiniRequestObject
for more details.
session
Contains the session object, see SessionState
for more details.
metadata
Contains the metadata. This is a dynamic field.
spec
Contains information about API definition, including APIID
, OrgID
and config_data
.
response
Contains information populated from the upstream HTTP response data, for response hooks. See ResponseObject for more details. All the field contents can be modified.
MiniRequestObject
The Coprocess.MiniRequestObject
is the main request data structure used by rich plugins. It’s used for middleware calls and contains important fields like headers, parameters, body and URL. A MiniRequestObject
is part of a Coprocess.Object
.
message MiniRequestObject {
map<string, string> headers = 1;
map<string, string> set_headers = 2;
repeated string delete_headers = 3;
string body = 4;
string url = 5;
map<string, string> params = 6;
map<string, string> add_params = 7;
map<string, string> extended_params = 8;
repeated string delete_params = 9;
ReturnOverrides return_overrides = 10;
string method = 11;
string request_uri = 12;
string scheme = 13;
bytes raw_body = 14;
}
Field Descriptions
headers
A read-only field for reading headers injected by previous middleware. Modifying this field won’t alter the request headers See set_headers
and delete_headers
for this.
set_headers
This field appends the given headers (keys and values) to the request.
delete_headers
This field contains an array of header names to be removed from the request.
body
Contains the request body. See ReturnOverrides
for response body modifications.
raw_body
Contains the raw request body (bytes).
url
The request URL.
params
A read-only field that contains the request params. Modifying this value won’t affect the request params.
add_params
Add paramaters to the request.
delete_params
This field contains an array of parameter keys to be removed from the request.
return_overrides
See ReturnOverrides
for more information.
method
The request method, e.g. GET, POST, etc.
request_uri
Raw unprocessed URL which includes query string and fragments.
scheme
Contains the URL scheme, e.g. http
, https
.
ResponseObject
The ResponseObject
exists within an object for response hooks. The fields are populated with the upstream HTTP response data. All the field contents can be modified.
syntax = "proto3";
package coprocess;
message ResponseObject {
int32 status_code = 1;
bytes raw_body = 2;
string body = 3;
map<string, string> headers = 4;
repeated Header multivalue_headers = 5;
}
message Header {
string key = 1;
repeated string values = 2;
}
Field Descriptions
status_code
This field indicates the HTTP status code that was sent by the upstream.
raw_body
This field contains the HTTP response body (bytes). It’s always populated.
body
This field contains the HTTP response body in string format. It’s not populated if the raw_body
contains invalid UTF-8 characters.
headers
A map that contains the headers sent by the upstream.
multivalue_headers
A list of headers, each header in this list is a structure that consists of two parts: a key and its corresponding values.
The key is a string that denotes the name of the header, the values are a list of strings that hold the content of the header, this is useful when the header has multiple associated values.
This field is available for Go, Python and Ruby since tyk v5.0.4 and 5.1.1+.
ReturnOverrides
The ReturnOverrides
object, when returned as part of a Coprocess.Object
, overrides the response of a given HTTP request. It also stops the request flow and the HTTP request isn’t passed upstream. The fields specified in the ReturnOverrides
object are used as the HTTP response.
A sample usage for ReturnOverrides
is when a rich plugin needs to return a custom error to the user.
syntax = "proto3";
package coprocess;
message ReturnOverrides {
int32 response_code = 1;
string response_error = 2;
map<string, string> headers = 3;
bool override_error = 4;
string response_body = 5;
}
Field Descriptions
response_code
This field overrides the HTTP response code and can be used for error codes (403, 500, etc.) or for overriding the response.
response_error
This field overrides the HTTP response body.
headers
This field overrides response HTTP headers.
override_error
This setting provides enhanced customization for returning custom errors. It should be utilised alongside response_body
for optimal effect.
response_body
This field serves as an alias for response_erro
r and holds the HTTP response body.
SessionState
A SessionState
data structure is created for every authenticated request and stored in Redis. It’s used to track the activity of a given key in different ways, mainly by the built-in Tyk middleware like the quota middleware or the rate limiter.
A rich plugin can create a SessionState
object and store it in the same way built-in authentication mechanisms do. This is what a custom authentication middleware does. This is also part of a Coprocess.Object
.
Returning a null session object from a custom authentication middleware is considered a failed authentication and the appropriate HTTP 403 error is returned by the gateway (this is the default behaviour) and can be overridden by using ReturnOverrides
.
Field Descriptions
last_check
No longer used.
allowance
No longer in use, should be the same as rate
.
rate
The number of requests that are allowed in the specified rate limiting window.
per
The number of seconds that the rate window should encompass.
expires
An epoch that defines when the key should expire.
quota_max
The maximum number of requests allowed during the quota period.
quota_renews
An epoch that defines when the quota renews.
quota_remaining
Indicates the remaining number of requests within the user’s quota, which is independent of the rate limit.
quota_renewal_rate
The time in seconds during which the quota is valid. So for 1000 requests per hour, this value would be 3600 while quota_max
and quota_remaining
would be 1000.
access_rights
Defined as a map<string, APIDefinition>
instance, that maps the session’s API ID to an AccessDefinition. The AccessDefinition defines the access rights for the API in terms of allowed: versions and URLs(endpoints). Each URL (endpoint) has a list of allowed methods. For further details consult the tutorials for how to create a security policy for Tyk Cloud, Tyk Self Managed and Tyk OSS platforms.
org_id
The organisation this user belongs to. This can be used in conjunction with the org_id setting in the API Definition object to have tokens “owned” by organisations.
oauth_client_id
This is set by Tyk if the token is generated by an OAuth client during an OAuth authorisation flow.
basic_auth_data
This section contains a hashed representation of the basic auth password and the hashing method used.
For further details see BasicAuthData.
jwt_data
Added to sessions where a Tyk key (embedding a shared secret) is used as the public key for signing the JWT. The JWT token’s KID header value references the ID of a Tyk key. See JWTData for an example.
hmac_enabled
When set to true
this indicates generation of a HMAC signature using the secret provided in hmac_secret
. If the generated signature matches the signature provided in the Authorization header then authentication of the request has passed.
hmac_secret
The value of the HMAC shared secret.
is_inactive
Set this value to true to deny access.
apply_policy_id
The policy ID that is bound to this token.
Note
Although apply_policy_id
is still supported, it is now deprecated. apply_policies
is now used to list your policy IDs as an array. This supports the Multiple Policy feature introduced in the v2.4 - 1.4 release.
data_expires
A value, in seconds, that defines when data generated by this token expires in the analytics DB (must be using Pro edition and MongoDB).
monitor
Defines a quota monitor containing a list of percentage threshold limits in descending order. These limits determine when webhook notifications are triggered for API users or an organisation. Each threshold represents a percentage of the quota that, when reached, triggers a notification. See Monitor for further details and an example.
enable_detailed_recording
Set this value to true to have Tyk store the inbound request and outbound response data in HTTP Wire format as part of the analytics data.
metadata
Metadata to be included as part of the session. This is a key/value string map that can be used in other middleware such as transforms and header injection to embed user-specific data into a request, or alternatively to query the providence of a key.
tags
Tags are embedded into analytics data when the request completes. If a policy has tags, those tags will supersede the ones carried by the token (they will be overwritten).
alias
As of v2.1, an Alias offers a way to identify a token in a more human-readable manner, add an Alias to a token in order to have the data transferred into Analytics later on so you can track both hashed and un-hashed tokens to a meaningful identifier that doesn’t expose the security of the underlying token.
last_updated
A UNIX timestamp that represents the time the session was last updated. Applicable to Post, PostAuth and Response plugins. When developing CustomAuth plugins developers should add this to the SessionState instance.
id_extractor_deadline
This is a UNIX timestamp that signifies when a cached key or ID will expire. This relates to custom authentication, where authenticated keys can be cached to save repeated requests to the gRPC server. See id_extractor and Auth Plugins for additional information.
session_lifetime
UNIX timestamp that denotes when the key will automatically expire. Any·subsequent API request made using the key will be rejected. Overrides the global session lifetime. See Key Expiry and Deletion for more information.
AccessDefinition
message AccessDefinition {
string api_name = 1;
string api_id = 2;
repeated string versions = 3;
repeated AccessSpec allowed_urls = 4;
}
Defined as an attribute within a SessionState instance. Contains the allowed versions and URLs (endpoints) for the API that the session request relates to. Each URL (endpoint) specifies an associated list of allowed methods. See also AccessSpec.
Field Descriptions
api_name
The name of the API that the session request relates to.
api_id
The ID of the API that the session request relates to.
versions
List of allowed API versions, e.g. "versions": [ "Default" ]
.
allowed_urls
List of AccessSpec instances. Each instance defines a URL (endpoint) with an associated allowed list of methods. If all URLs (endpoints) are allowed then the attribute is not set.
AccessSpec
Defines an API’s URL (endpoint) and associated list of allowed methods
message AccessSpec {
string url = 1;
repeated string methods = 2;
}
Field Descriptions
url
A URL (endpoint) belonging to the API associated with the request session.
methods
List of allowed methods for the URL (endpoint), e.g. "methods": [ "GET". "POST", "PUT", "PATCH" ]
.
BasicAuthData
The BasicAuthData
contains a hashed password and the name of the hashing algorithm used. This is represented by the basic_auth_data
attribute in SessionState message.
"basicAuthData": {
"password": <a_hashed_password_presentation>,
"hash": <the_hashing_algorithm_used_to_hash_the_password>
}
Field Descriptions
password
A hashed password.
hash
Name of the hashing algorithm used to hash the password.
JWTData
Added to sessions where a Tyk key (embedding a shared secret) is used as the public key for signing the JWT. This message contains the shared secret.
"jwtData": {
"secret": "the_secret"
}
Field Descriptions
secret
The shared secret.
Monitor
Added to a session when monitor quota thresholds are defined within the Tyk key. This message contains the quota percentage threshold limits, defined in descending order, that trigger webhook notification.
message Monitor {
repeated double trigger_limits = 1;
}
Field Descriptions
trigger_limits
List of trigger limits defined in descending order. Each limit represents the percentage of the quota that must be reached in order for the webhook notification to be triggered.
"monitor": {
"trigger_limits": [80.0, 60.0, 50.0]
}