Tyk Sync

Tyk Sync is a command line tool and library to manage and synchronise a Tyk installation with your version control system (VCS).

Note

Tyk Sync works with APIs and Policies. It does not work with Keys. See Move Keys between environments for details.

Features

Update APIs and Policies on remote Tyk Dashboards
Update APIs on remote Tyk Community Edition Gateways
Publish APIS/Policies to remote Tyk Dashboards
Publish APIs to remote Tyk Community Edition Gateways
Synchronise a Tyk Dashboard’s APIs and Policies with your VCS (one-way, definitions are written to the Dashboard)
Synchronise a Tyk Community Edition Gateway APIs with those stored in a VCS (one-way, definitions are written to the Gateway)
Dump Policies and APIs in a transportable format from a Dashboard to a directory
Support for importing, converting and publishing Swagger/OpenAPI JSON files (OpenAPI 2.0 and 3.0 are supported) to Tyk.
Specialised support for Git. But since API and policy definitions can be read directly from the file system, it will integrate with any VCS.

Sync

Tyk Sync tries to be clever about what APIs and Policies to update and which to create, it will actually base all ID matching on the API ID and the masked Policy ID, so it can identify the same object across installations. Tyk has a tendency to generate fresh IDs for all new Objects, so Tyk Sync gets around this by using portable IDs and ensuring the necessary portable IDs are set when using the dump command.

This means that Tyk Sync can be used to back-up your most important API Gateway configurations as code, and to deploy those configurations to any target and ensure that API IDs and Policy IDs will remain consistent, ensuring that any dependent tokens continue to have access to your services.

Prerequisites:

Tyk Sync was built using Go 1.10. The minimum Go version required to install is 1.7.
In order for policy ID matching to work correctly, your Dashboard must have allow_explicit_policy_id: true and enable_duplicate_slugs: true and your Gateway must have policies.allow_explicit_policy_id: true.
It is assumed you have a Tyk CE or Tyk Pro installation.

Installation

Currently the application is available via Go, Docker and Packagecloud. To install via Go you must have Go installed and run:

go get -u github.com/TykTechnologies/tyk-sync

This should make the tyk-sync command available to your console.

See our Tyk Sync Repo for more info.

Docker:

To install a particular version of tyk-sync via docker image please run the command bellow with the appropriate version you want to use. All available versions could be found on the Tyk Sync Docker Hub page here: https://hub.docker.com/r/tykio/tyk-sync/tags

docker pull tykio/tyk-sync:{version_id}

To run tyk-sync as a one-off command and display usage options please do:

docker run -it --rm tykio/tyk-sync:{version_id} help

Then the docker image tyk-sync can be used in the following way:

docker run -it --rm tykio/tyk-sync:{version_id} [flags]
docker run -it --rm tykio/tyk-sync:{version_id} [command]

As per the examples below tyk-sync will need access to the host file sytem to read and write files. You can use docker bind mounts to map files in the container to files on your host machine.

Usage

Usage:
  tyk-sync [flags]
  tyk-sync [command]

Available Commands:
  dump        Dump will extract policies and APIs from a target (Tyk Dashboard)
  help        Help about any command
  publish     publish API definitions from a Git repo or file system to a Tyk Gateway or Dashboard
  sync        Synchronise a github repo or file system with a Tyk Gateway
  update      Update a Tyk Dashboard or Gateway with APIs and policies
  version     This command will show the current Tyk-Sync version

Flags:
  -h, --help   help for tyk-sync

Use "tyk-sync [command] --help" for more information about a command.

Dump Command

Dump will extract policies and APIs from a target (your Dashboard) and place them in a directory of your choosing. It will also generate a spec file that can be used for syncing.


Usage:
  tyk-sync dump [flags]
Flags:
  -b, --branch string      Branch to use (defaults to refs/heads/master) (default "refs/heads/master")
  -d, --dashboard string   Fully qualified Tyk Dashboard target URL
  -h, --help               help for dump
  -k, --key string         Key file location for auth (optional)
  -s, --secret string      Your API secret
  -t, --target string      Target directory for files
      --policies           Specific policies ID selection (optional)
      --apis               Specific api_id's selection (optional)

Publish Command

Publish API definitions from a Git repo to a Tyk Gateway or Dashboard. This will not update any existing APIs, and if it detects a collision, the command will stop.

Usage:
  tyk-sync publish [flags]
Flags:
  -b, --branch string      Branch to use (defaults to refs/heads/master) (default "refs/heads/master")
  -d, --dashboard string   Fully qualified Tyk Dashboard target URL
  -g, --gateway string     Fully qualified Tyk Gateway target URL
  -h, --help               help for publish
  -k, --key string         Key file location for auth (optional)
  -p, --path string        Source directory for definition files (optional)
  -s, --secret string      Your API secret
      --test               Use test publisher, output results to stdio
      --policies           Specific policies ID selection (optional)
      --apis               Specific api_id's selection (optional)

Sync Command

Sync will synchronise an API Gateway with the contents of a Github repository. The sync is one way - from the repo to the Gateway, the command will not write back to the repo. Sync will delete any objects in the Dashboard or Gateway that it cannot find in the github repo, and update those that it can find and create those that are missing.

Usage:
tyk-sync sync [flags]
Flags:
-b, --branch string      Branch to use (defaults to refs/heads/master) (default "refs/heads/master")
-d, --dashboard string   Fully qualified Tyk Dashboard target URL
-g, --gateway string     Fully qualified Tyk Gateway target URL
-h, --help               help for sync
-k, --key string         Key file location for auth (optional)
-o, --org string         org ID override
-p, --path string        Source directory for definition files (optional)
-s, --secret string      Your API secret
    --test               Use test publisher, output results to stdio
    --policies           Specific policies ID selection (optional)
    --apis               Specific api_id's selection (optional)

Update Command

Update will attempt to identify matching APIs or Policies in the target, and update those APIs. It does not create new ones. Use tyk-sync publish or tyk-git sync for new content.

Usage:
tyk-sync update [flags]
Flags:
-b, --branch string      Branch to use (defaults to refs/heads/master) (default "refs/heads/master")
-d, --dashboard string   Fully qualified Tyk Dashboard target URL
-g, --gateway string     Fully qualified Tyk Gateway target URL
-h, --help               help for update
-k, --key string         Key file location for auth (optional)
-p, --path string        Source directory for definition files (optional)
-s, --secret string      Your API secret
    --test               Use test publisher, output results to stdio
    --policies           Specific policies ID selection (optional)
    --apis               Specific api_id's selection (optional)

Example: Transfer from one Tyk Dashboard to another

First, you need to extract the data from our Tyk Dashboard. Here you dump into ./tmp. Let’s assume this is a git-enabled directory

tyk-sync dump -d="http://localhost:3000" -s="b2d420ca5302442b6f20100f76de7d83" -t="./tmp"
Extracting APIs and Policies from http://localhost:3000
> Fetching policies
--> Identified 1 policies
--> Fetching and cleaning policy objects
> Fetching APIs
--> Fetched 3 APIs
> Creating spec file in: tmp/.tyk.json
Done.

If running tyk-sync in docker the command above would read

docker run --rm --mount type=bind,source="$(pwd)",target=/opt/tyk-sync/tmp \
 tykio/tyk-sync:v1.2 \
 dump \
 -d="http://host.docker.internal:3000" \
 -s="b2d420ca5302442b6f20100f76de7d83" \
 -t="./tmp"

Next, let’s push those changes back to the Git repo on the branch my-test-branch:

cd tmp
git add .
git commit -m "My dashboard dump"
git push -u origin my-test-branch

Now to restore this data directly from GitHub:

tyk-sync sync -d="http://localhost:3010" -s="b2d420ca5302442b6f20100f76de7d83" -b="refs/heads/my-test-branch" https://github.com/myname/my-test.git
Using publisher: Dashboard Publisher
Fetched 3 definitions
Fetched 1 policies
Processing APIs...
Deleting: 0
Updating: 3
Creating: 0
SYNC Updating: 598ec94f9695f201730d835b
SYNC Updating: 598ec9589695f201730d835c
SYNC Updating: 5990cfee9695f201730d836e
Processing Policies...
Deleting policies: 0
Updating policies: 1
Creating policies: 0
SYNC Updating Policy: Test policy 1
--> Found policy using explicit ID, substituting remote ID for update

If running tyk-sync in docker the command above would read

docker run --rm \
  --mount type=bind,source="$(pwd)",target=/opt/tyk-sync/tmp \
 tykio/tyk-sync:v1.2 \
  sync \
  -d="http://localhost:3010" \
  -s="b2d420ca5302442b6f20100f76de7d83" \
  -b="refs/heads/my-test-branch" https://github.com/myname/my-test.git

The command provides output to identify which actions have been taken. If using a Tyk Gateway, the Gateway will be automatically hot-reloaded.

Example: Dump a specific API from one Tyk Dashboard

First, we need to identify the api_id that we want to dump, in this case ac35df594b574c9c7a3806286611d211. When we have that, we are going to execute the dump command specifying the api_id in the tags.

tyk-sync dump -d="http://localhost:3000" -s="b2d420ca5302442b6f20100f76de7d83" -t="./tmp" --apis="ac35df594b574c9c7a3806286611d211"
Extracting APIs and Policies from http://localhost:3000
> Fetching policies
--> Identified 0 policies
--> Fetching and cleaning policy objects
> Fetching APIs
--> Fetched 1 APIs
> Creating spec file in: tmp/.tyk.json
Done.

If you want to specify more than one API, the values need to be comma-separated. For example --apis="ac35df594b574c9c7a3806286611d211,30e7b4001ea94fb970c324bad1a171c3".

The same behaviour applies to policies.

Example: Check the currently installed version of Tyk Sync

To check the current Tyk Sync version, we need to run the version command:

tyk-sync version
v1.2

Return to Top