1. Home
  2. Tyk Open Source API Gateway v2.x
  3. API Management
  4. Transforms

Transforms

It is possible to modify inbound and outbound body data and header information on the fly using Tyk. This can either be done using the scriptable middleware, or can be achieved using dedicated middleware.

Modifying header information

Tyk enables you to modify header information before it leaves the proxy and is passed to your upstram API or when a response is proxied back to the client. This can be very useful in cases where you have an upstream API that has a single authentication key, and you want to add multi-user access to it without modifying it or adding clunky authentication methods to it to support new users.

Tyk also supports the ability to use information from the key or token holder in the transform (for example, converting a Tyk-generated token into a user id that your application can use).

Here is an example scenario:

You have an API called WidgetsAPI, that takes an x-widgets-secret header to allow access, this is an internal API used by your teams but you want to expose it to your customers and charge them for access.

You could either modify the API and add a whole user, key and access management system, or you could use Tyk to inject this header for you.

Using Tyk, you would set up your API Definition with these additions to the extended_paths.transform_headers filed:

"extended_paths": {
    "ignored": [],
    "white_list": [],
    "black_list": [],
    "cache": ["get"],
    "transform": [],
    "transform_headers": [
        {
            "delete_headers": ["authorization"],
            "add_headers": {"x-widgets-secret": "th-secret-widget-key-is-secret"},
            "path": "widgets/{id}",
            "method": "GET"
        },
        {
            "delete_headers": ["authorization"],
            "add_headers": {"x-widgets-secret": "th-secret-widget-key-is-secret"},
            "path": "widgets",
             "method": "GET"
        }
    ],
    "transform_response_headers": [
            {
                "delete_headers": ["x-server-secret"],
                "add_headers": {"x-server-id": "this-is-important"},
                "path": "widgets/{id}",
                "method": "GET"
            }
        ]
}

Now Tyk keys that you create with an Access Definition rule that is set to this API and version, can have quotas, throttling and access checks applied without needing to add any new code or functionality to your existing API.

Using Meta Data from a key object in header transforms

It is possible to inject information that is carried within the user session object into the header space as well. Each token or key has an attached session object which contains a meta_data field, this is a key/value map that allows for dynamic middleware and other components to intelligently act on identity information from the inbound request without exposing it.

To use this data in your header transform simply access the special $tyk_meta namespace, here is a working example:

Say in your session object you have included the following metadata:

"meta_data": {
    "uid": 12345,
    "username": "norman_bates"
}

To use this in your header transform, your API definition path would be:

"transform_headers": [
    {
        "delete_headers": [],
        "add_headers": {"user-id": "$tyk_meta.uid", "user-name": "$tyk_meta.username"},
        "path": "widgets/{id}",
        "method": "GET"
    },
]

Context variables for header injection

As of version 2.2 Tyk also allows context variables to be injected into headers using the $tyk_context. namespace.

The context variables that are available are:

  • request_data: If the inbound request contained any query data or form data, it will be available in this object, for the header injector, Tyk will format this data as key:value1,value2,valueN;key:value1,value2 etc.
  • path_parts: The components of the path, split on /, again these values are made available in the format of a comma delimited list
  • token: The inbound raw token (if bearer tokens are being used) of this user
  • path: The path that is being requested
  • remote_addr: The IP address of the connecting client

Injecting and Removing headers globally

In some cases you may wish to add a secure header to all outbound requests (e.g. to verify that traffic is coming from the gateway), to do so, add this to your version block:

"version_data": {
    "versions": {
      "Default": {
        ...
        "global_headers": {
            "x-header-name": "x-header-value"
        },
        "global_headers_remove": [
            "auth_id"
        ]
        ...
      }
    }
},

Using the global_headers_remove field it is possible to remove headers from all inbound requests before they are passed to your service.

Modifying body data

Sometimes, you may also be exposing an older API, or one that uses a legacy structure for input data, or, you are actually creating a new API schema and models that are cleaner, that you want to apply to your existing API without modifying it (it may have many legacy clients that cannot be upgraded).

As of Tyk 1.5 it is possible to modify inbound JSON requests and as of v2.2 XML requests using a golang template. Tyk will unmarshal the data into a data structure, and then make that data available to the template in dot-notation. Here is an example to illustrate:

JSON

Assume your inbound date structure is as follows:

{
    "value1": "value-1",
    "value2": "value-2",
    "value_list": [
        "one",
        "two",
        "three"
    ]
}

and you need to transform it into to a different format, you could use a golang template hat looks like this:

{
    "value1": "{{.value2}}",
    "value2": "{{.value1}}",
    "transformed_list": [
        {{range $index, $element := .value_list}}
            {{if $index}}
            , "{{$element}}"
            {{else}}
                 "{{$element}}"
            {{end}}
        {{end}}
    ]
}

To transform it. This template would produce the following output:

{
    "value1": "value-1",
    "value2": "value-2",
    "transformed_list": [
        "one",
        "two",
        "three"
    ]
}

XML

With an XML document it is a little different as XML cannot be easily decoded into strict structures as JSON, so the syntax isa a little different, to enable an inbound XML data type, simply set the

For this XML:

<?xml version="1.0" encoding="utf-8"?>
<servers version="1">
    <server>
        <serverName>Shanghai_VPN</serverName>
        <serverIP>127.0.0.1</serverIP>
    </server>
    <server>
        <serverName>Beijing_VPN</serverName>
        <serverIP>127.0.0.2</serverIP>
    </server>
</servers>

And this Template:

{
{{range $x, $s := .servers.server}}    "{{$s.serverName}}": "{{$s.serverIP}}"{{if not $x}},{{end}}
{{end}}
}

You get this output:

{
    "Shanghai_VPN": "127.0.0.1",
    "Beijing_VPN": "127.0.0.2"

}

It is not within the scope of this documentation to explain how golang templates work this is better done better elsewhere. Golang templates are powerful and versatile, and using them to transform your inbound data means any output format can be created, be it XML or a custom standard that your API understands.

It is also possible to insert keyspace meta data into a body transform (requests only), you can do this by calling the ._tyk_meta.KEYNAME namespace, e.g.:

{
    "value1": "value-1",
    "value2": "value-2",
    "transformed_list": [
        "one",
        "two",
        "three"
    ],
    "user-id": "._tyk_meta.uid"
}

Context variables

As of version 2.2 Tyk also allows context variables to be injected into the body using the ._tyk_context. namespace, unlike the context exposed to the URL Rewriter and header injector, the body transform can fully iterate through list indeces, so for example calling _tyk_context.path_parts[0] in a template will expose the first entry in the path_parts list.

The context variables that are available are:

  • request_data: If the inbound request contained any query data or form data, it will be available in this object as a key:[]value map
  • path_parts: The components of the path, split on /, it will be available in this object as a key:[]value map
  • token: The inbound raw token (if bearer tokens are being used) of this user
  • path: The path that is being requested
  • remote_addr: The IP address of the connecting client

Setting up transforms in your API definition:

"extended_paths": {
    "ignored": [],
    "white_list": [],
    "black_list": [],
    "cache": ["get"],
    "transform": [
        {
            "path": "widgets/{id}",
            "method": "POST",
            "template_data": {
                "template_mode": "file",
                "template_source": "./templates/transform_test.tmpl"
            }
        }
    ],
    "transform_response": [
        {
            "path": "widgets/{id}",
            "method": "POST",
            "template_data": {
                "template_mode": "file",
                "template_source": "./templates/transform_test.tmpl"
            }
        }
    ],
    "transform_headers": []
}

Tyk will load and evaluate the template on start, if you modify the template, you will need to restart Tyk in order for the changes to take effect.

The field representations are:

  • path: The path to match
  • method : The method to match
  • template_data.input_type: either xml or json, this represents the type of data the parser should expect
  • enable_session: This makes session metadata available to the transform template (see the section on session data below)
  • template_source: Either a file path, or a base64 encoded representation of your template
  • template_mode: Set to blob for a base64 template and file for a file path in the template source

Fixing response headers that leak upstream server data

[As of v2.1]

To ensure headers such as Location and Link reflect the outward facade of your API Gateway and also align with the expected response location to be terminated at the gateway, not the hidden upstream proxy, a new middleware has been added called header_transform:

"response_processors": [
    {
        "name": "header_transform",
        "options": {
            "rev_proxy_header_cleanup": {
                "headers": ["Link", "Location"],
                "target_host": "http://TykHost:TykPort"
            }
        }
    }
]

In this configuration, you set the headers to target and the target host to replace the values with, so in the above case, the Link and Location headers will be modified from the server-generated response, with the protocol, domain and port of the value set in target_host.

(This is not supported in the dashboard UI yet.)

A note on response modifying middleware

In order for responses to be processed by Tyk as they return via the proxy, the response middleware must be loaded for the API definition. Unlike inbound middleware, these processors are loaded only as required and must therefore be registered in advance. To do so is very simply, just add them to your API definition as follows:

"response_processors": [
    {
      "name": "header_injector",
      "options": {
        "add_headers": {"name": "value"},
        "remove_headers": ["name"]
      }
    },
    {
      "name": "response_body_transform",
      "options": {}
    }
]

For the header injector response middleware, it is possible to set global response injection headers, this means that they do not need to be set on a version-level, any headers set up in the options section of the response headers object will be applied to all replies.