Configure Custom Plugins in Tyk Cloud

​

Introduction

​

Setup Control Plane

1. You need to enable Plugins on a Control Plane and on a Cloud Data Plane.
2. You need to enter Provider details to enable you to store and access your plugins. For this version of Tyk Cloud, we are supporting Amazon AWS S3. If you haven’t got an AWS S3 account, go to https://aws.amazon.com/s3/ and set one up. You will need the following details to configure SW3 within your Control Plane:
   • Your AWS Key ID
   • Your AWS Secret
   • Your AWS Region
   For this release of Tyk Cloud, you need to enter your AWS Region manually. You also need to consider that uploading a custom plugin bundle to Tyk Cloud results in a new bucket being created for each bundle uploaded. It also requires that Tyk Cloud has permissions in the form of an AWS IAM policy to have create rights on AWS.
3. Create an AWS IAM policy that grants Tyk Cloud the permissions it needs to create and manage the S3 bucket, for example:
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": [
                   "s3:CreateBucket",
                   "s3:ListBucket",
                   "s3:GetBucketLocation",
                   "s3:DeleteBucket"
               ],
               "Resource": "arn:aws:s3:::mserv-plugin-*"
           },
           {
               "Effect": "Allow",
               "Action": "s3:ListAllMyBuckets",
               "Resource": "*"
           },
           {
               "Sid": "VisualEditor1",
               "Effect": "Allow",
               "Action": [
                   "s3:PutObject",
                   "s3:GetObject",
                   "s3:DeleteObject"
               ],
               "Resource": "arn:aws:s3:::mserv-plugin-*/*"
           }
       ]
   }
   

   See all 31 lines
   See the AWS IAM documentation for more information on creating IAM policies.

​

Set Up Your Plugin Development Environment

1. A local Tyk Gateway installation to create your Python plugin bundle. We recommend installing our Self-Managed version on Ubuntu Bionic 18.04.
2. A currently stable Python 3.x version installed.
3. The build tools, installed with apt-get install -y build-essential.
4. Our required modules, installed with:

apt install python3 python3-dev python3-pip
pip3 install protobuf grpcio

​

Create a Python Code Bundle

• You need to create the Python code bundle on your locally installed Gateway (not an Tyk Cloud Cloud Data Plane stack).
• You will create 2 files, a manifest file (manifest.json) and the python file (middleware.py)
• You then create a zipped bundle via our Tyk CLI tool that is built in to your local Gateway instance.

mkdir ~/my-tyk-plugin
cd ~/my-tyk-plugin

{
  "custom_middleware": {
    "auth_check": {
      "name": "MyAuthMiddleware"
    },
    "pre": [
      {
        "name": "MyAuthMiddleware"
      }
    ],
    "driver": "python"
  },
  "file_list": [
    "middleware.py"
  ]
}

• You import decorators from the Tyk module that gives us the Hook decorator, and we import Tyk Python API helpers
• You implement a middleware function and register it as a hook. The input includes the request object, the session object, the API meta data and its specification. The hook checks the authorization header for a specified value. In this tutorial we have called it Authorization.

from tyk.decorators import *
from gateway import TykGateway as tyk

@Hook
def MyAuthMiddleware(request, session, metadata, spec):
    auth = request.get_header('Authorization')
    if not auth:
        auth = request.object.params.get('authorization', None)

    if auth == '47a0c79c427728b3df4af62b9228c8ae':
        session.rate = 1000.0
        session.per = 1.0
        metadata["token"] = auth
    return request, session, metadata

@Hook
def MyPostMiddleware(request, session, spec):
    tyk.log("This is my post middleware", "info")
    request.object.set_headers["x-tyk-request"] = "something"
    return request, session

• You create a bundle to cater for a number of plugins connected to the one API, and using a bundle makes this more manageable.
• To bundle your plugin we run the following command in your working directory where your manifest.json and plugin code is.

docker run \
  --rm \
  -v $(pwd):/cloudplugin \
  --entrypoint "/bin/sh" -it \
  -w "/cloudplugin" \
  tykio/tyk-gateway:v3.1.2 \
  -c '/opt/tyk-gateway/tyk bundle build -y'

• A plugin bundle is a packaged version of the plugin, it may also contain a cryptographic signature of its contents. The -y flag tells the Tyk CLI tool to skip the signing process in order to simplify this tutorial. For more information on the Tyk CLI tool, see here.
• You should now have a bundle.zip file in the plugin working directory.
• Next you will configure uploading your plugin bundle file to your Amazon S3 bucket.

​

Uploading your Bundle

1. You need to install the mserv binary according to your local environment from the following repo - https://github.com/TykTechnologies/mserv/releases. Linux and MacOS are supported.
2. From your Control Plane you need the following settings.

• Your Tyk Cloud Control Plane Ingress File Server Endpoint (1)
• Your File Server API Key (2)

./mservctl.macos.amd64

./mservctl.linux.amd64

$ mservctl help
mservctl is a CLI application that enables listing and operating middleware in an Mserv instance.
Use a config file (by default at $HOME/.mservctl.yaml) in order to configure the Mserv to use with the CLI.
Alternatively pass the values with command line arguments, e.g.:
$ mservctl list -e https://remote.mserv:8989
Set TYK_MSERV_LOGLEVEL="debug" environment variable to see raw API requests and responses.
Usage:
  mservctl [command]
Available Commands:
  delete      Deletes a middleware from mserv
  fetch       Fetches a middleware record from mserv
  help        Help about any command
  list        List middleware in mserv
  push        Pushes a middleware to mserv
Flags:
      --config string     config file (default is $HOME/.mservctl.yaml)
  -e, --endpoint string   mserv endpoint
  -h, --help              help for mservctl
  -t, --token string      mserv security token
Use "mservctl [command] --help" for more information about a command.

1. Create a file (we’ll call it python-demo.mservctl.yaml)
2. Copy your Control Plane File Server endpoint URL and use it for your endpoint flag. Remember to prepend it with https://.
3. Copy your File Server API Key and use it for your token flag

endpoint: https://agreeable-native-mgw.usw2.ara-staging.tyk.technology/mserv
token: eyJvcmciOiI1ZWIyOGUwY2M3ZDc4YzAwMDFlZGQ4ZmYiLCJpZCI6ImVmMTZiNGM3Y2QwMDQ3Y2JhMTAxNWIyOTUzZGRkOWRmIiwiaCI6Im11cm11cjEyOCJ9

1. We are going to use the MacOS binary here, just substitute the binary name for the Linx version if using that OS. Note we have our YAML config file in the same directory as our bundle.zip file. Run the following mserv push command:

./mservctl.macos.amd64 --config ~/my-tyk-plugin/python-demo.mservctl.yaml push ~/my-tyk-plugin/bundle.zip

2. You should get confirmation that your middleware has been uploaded to your S3 bucket.

INFO[0000] Using config file:/Users/marksouthee/my-tyk-plugin/python-demo.mservctl.yaml  app=mservctl
Middleware uploaded successfully, ID: 9c9ecec1-8f98-4c3f-88cd-ca3c27599e6b

3. You will notice that the middleware uploaded has been given an ID. We are going to use that ID with an API that allows you to specify specific middlware. You can also check the contents of the middleware you have just uploaded using the mservctl list command. Run:

./mservctl.macos.amd64 --config ~/my-tyk-plugin/python-demo.mservctl.yaml list

4. You will see the list of middleware you have pushed to your S3 Bucket

INFO[0000] Using config file:/Users/marksouthee/my-tyk-plugin/python-demo.mservctl.yaml  app=mservctl
  ID                                    ACTIVE  SERVE ONLY  LAST UPDATE

  9c9ecec1-8f98-4c3f-88cd-ca3c27599e6b  true    false       2020-05-20T15:06:55.901Z

5. If you use the -f flag with the list command, you will see the functions within your middleware listed:

./mservctl.macos.amd64 --config ~/my-tyk-plugin/python-demo.mservctl.yaml list -f

6. As you can see, the 2 middleware hooks specified within your manifest.json are returned:

INFO[0000] Using config file:/Users/marksouthee/my-tyk-plugin/python-demo.mservctl.yaml  app=mservctl
  ID                                    ACTIVE  SERVE ONLY  LAST UPDATE               FUNCTION          TYPE

  9c9ecec1-8f98-4c3f-88cd-ca3c27599e6b  true    false       2020-05-20T15:06:55.901Z
  MyPostMiddleware  Post
  MyAuthMiddleware  CustomKeyCheck

​

Test Middleware

• A Postman API Client from https://www.postman.com/product/api-client/
• Your mserv middleware ID
• The auth value token from your middleware.py code

1. From your Control Plane in Tyk Cloud, click the Ingress > Dashboard link

2. From the Dashboard screen, click APIs from the System Management menu

3. Click Add New API
4. From the API Designer, enter the following in the Core Settings tab:
   • From the API Settings section, give your API a name. We’ll name this example “test”
   • Scroll down to the Authentication section and select Custom authentication (Python, CoProcess and JSVM plugins) from the drop-down menu
   • Select the Allow query parameter as well as header option
5. From the Advanced Settings tab enter the following:
   • In the Plugin Options, enter the Plugin Bundle ID as returned by mservctl. In our example 9c9ecec1-8f98-4c3f-88cd-ca3c27599e6b
   • To propagate your API to all your Cloud Data Plane Tyk Gateways connected to your Control Plane, you need to add the tag edge in the API Segment Tags section
6. Click Save.

1. First, a quick test. Copy the URL of your Cloud Data Plane (Note the “edge” tag in the tags column) and paste it in a browser tab. You should get a 404 page not found error.
2. Then add the “test” endpoint to the URL in your browser tab, so in our example uptight-paddle-gw.usw2.ara.app/test/. You should now see a 403 “error: “forbidden”. This is because your API has Authentication enabled and you haven’t provided the credentials yet.
3. Open up your Postman client:
   • Paste your Gateway URL with the API appended to the request - (uptight-paddle-gw.usw2.ara.app/test/)
   • Click Send. You’ll see the 403 “error: “forbidden response” again
   • In the Headers section in Postman, select Authorization from the Key column. Add some random text in the Value field and click Send. You should again see the 403 error.
   • Now replace the random text with the auth value from your Python code. In our example 47a0c79c427728b3df4af62b9228c8ae and click Send again.
   • You should now see the HTTPB in test page

4. As a further test of your plugin, you can add get to your API request in Postman. So in our example uptight-paddle-gw.usw2.ara.app/test/get. Click Send. This will return all the get requests, including headers. You should see the x-tyk-request: "something" which is the post middleware hook you set up in the Python code.

​

Serving Plugin Bundles To Hybrid Gateways

"enable_bundle_downloader": true,
"bundle_base_url": "your-control-plane-file-server-url/mserv/api/mw/bundle?api_key=your_fileserver_api_key"

"bundle_base_url": "https://solid-month-mgw.aws-use1.cloud-ara.tyk.io/mserv/api/mw/bundle?api_key=abcd1234"