Home
Deployment and Operations
API Management
Product Stack
Developer Support
APIM Best Practice

Create a Request Transformation Plugin with Java

Last updated: 7 minutes read.

This tutorial will guide you through the creation of a gRPC-based Java plugin for Tyk. Our plugin will inject a header into the request before it gets proxied upstream. For additional information about gRPC, check the official documentation here.

The sample code that we’ll use implements a request transformation plugin using Java and uses the proper gRPC bindings generated from our Protocol Buffers definition files.

Requirements

Create the Plugin

Setting up the Java Project

We will use the Gradle build tool to generate the initial files for our project:

cd ~
mkdir tyk-plugin
cd tyk-plugin
gradle init

We now have a tyk-plugin directory containing the basic skeleton of our application.

Add the following to build.gradle

buildscript {
  repositories {
    jcenter()
  }
  dependencies {
    classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.1'
  }
}

plugins {
  id "com.google.protobuf" version "0.8.1"
  id "java"
  id "application"
  id "idea"
}

protobuf {
  protoc {
    artifact = "com.google.protobuf:protoc:3.3.0"
  }
  plugins {
    grpc {
      artifact = 'io.grpc:protoc-gen-grpc-java:1.5.0'
    }
  }
  generateProtoTasks {
    all()*.plugins {
      grpc {}
    }
  }
  generatedFilesBaseDir = "$projectDir/src/generated"
}

sourceCompatibility = 1.8
targetCompatibility = 1.8

mainClassName = "com.testorg.testplugin.PluginServer"

repositories {
  mavenCentral()
}

dependencies {
  compile 'io.grpc:grpc-all:1.5.0'
}

idea {
  module {
    sourceDirs += file("${projectDir}/src/generated/main/java");
    sourceDirs += file("${projectDir}/src/generated/main/grpc");
  }
}

Create the Directory for the Server Class

cd ~/tyk-plugin
mkdir -p src/main/java/com/testorg/testplugin

Install the gRPC Tools

We need to download the Tyk Protocol Buffers definition files, these files contains the data structures used by Tyk. See Data Structures for more information:

cd ~/tyk-plugin
git clone https://github.com/TykTechnologies/tyk
mv tyk/coprocess/proto src/main/proto

Generate the Bindings

To generate the Protocol Buffers bindings we use the Gradle build task:

gradle build

If you need to customise any setting related to the bindings generation step, check the build.gradle file.

Implement Server

We need to implement two classes: one class will contain the request dispatcher logic and the actual middleware implementation. The other one will implement the gRPC server using our own dispatcher.

From the ~/tyk-plugin/src/main/java/com/testorg/testplugin directory, create a file named PluginDispatcher.java with the following code:

package com.testorg.testplugin;

import coprocess.DispatcherGrpc;
import coprocess.CoprocessObject;

public class PluginDispatcher extends DispatcherGrpc.DispatcherImplBase {

  @Override
  public void dispatch(CoprocessObject.Object request,
        io.grpc.stub.StreamObserver<CoprocessObject.Object> responseObserver) {
    CoprocessObject.Object modifiedRequest = null;

    switch (request.getHookName()) {
      case "MyPreMiddleware":
        modifiedRequest = MyPreHook(request);
      default:
      // Do nothing, the hook name isn't implemented!
    }

    // Return the modified request (if the transformation was done):
    if (modifiedRequest != null) {
      responseObserver.onNext(modifiedRequest);
    };

    responseObserver.onCompleted();
  }

  CoprocessObject.Object MyPreHook(CoprocessObject.Object request) {
    CoprocessObject.Object.Builder builder = request.toBuilder();
    builder.getRequestBuilder().putSetHeaders("customheader", "customvalue");
    return builder.build();
  }
}

In the same directory, create a file named PluginServer.java with the following code. This is the server implementation:

package com.testorg.testplugin;

import coprocess.DispatcherGrpc;

import io.grpc.Server;
import io.grpc.ServerBuilder;
import io.grpc.stub.StreamObserver;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;

public class PluginServer {

  private static final Logger logger = Logger.getLogger(PluginServer.class.getName());
  static Server server;
  static int port = 5555;

  public static void main(String[] args) throws IOException, InterruptedException {
    System.out.println("Initializing gRPC server.");

    // Our dispatcher is instantiated and attached to the server:
    server = ServerBuilder.forPort(port)
            .addService(new PluginDispatcher())
            .build()
            .start();

    blockUntilShutdown();

  }

  static void blockUntilShutdown() throws InterruptedException {
      if (server != null) {
          server.awaitTermination();
      }
  }
}

To run the gRPC server we can use the following command:

cd ~/tyk-plugin
gradle runServer

The gRPC server will listen on port 5555 (as defined in Server.java). In the next steps we’ll setup the plugin bundle and modify Tyk to connect to our gRPC server.

Bundle the Plugin

We need to create a manifest file within the tyk-plugin directory. This file contains information about our plugin and how we expect it to interact with the API that will load it. This file should be named manifest.json and needs to contain the following:

{
  "custom_middleware": {
    "driver": "grpc",
    "pre": [{
        "name": "MyPreMiddleware"
    }]
  }
}
  • The custom_middleware block contains the middleware settings like the plugin driver we want to use (driver) and the hooks that our plugin will expose. We use the pre hook for this tutorial. For other hooks see here.
  • The name field references the name of the function that we implemented in our plugin code - MyPreMiddleware. This will be handled by our dispatcher gRPC method in PluginServer.java.

To bundle our plugin run the following command in the tyk-plugin directory. Check your tyk-cli install path first:

/opt/tyk-gateway/utils/tyk-cli bundle build -y

For Tyk 2.8 use:

/opt/tyk-gateway/bin/tyk bundle build -y

A plugin bundle is a packaged version of the plugin. It may also contain a cryptographic signature of its contents. The -y flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial.

For more information on the Tyk CLI tool, see here.

You should now have a bundle.zip file in the tyk-plugin directory.

Publish the Plugin

To publish the plugin, copy or upload bundle.zip to a local web server like Nginx, or Apache or storage like Amazon S3. For this tutorial we’ll assume you have a web server listening on localhost and accessible through http://localhost.

Configure Tyk

You will need to modify the Tyk global configuration file tyk.conf to use gRPC plugins. The following block should be present in this file:

"coprocess_options": {
    "enable_coprocess": true,
    "coprocess_grpc_server": "tcp://localhost:5555"
},
"enable_bundle_downloader": true,
"bundle_base_url": "http://localhost/bundles/",
"public_key_path": ""

tyk.conf Options

  • enable_coprocess: This enables the plugin.
  • coprocess_grpc_server: This is the URL of our gRPC server.
  • enable_bundle_downloader: This enables the bundle downloader.
  • bundle_base_url: This is a base URL that will be used to download the bundle. You should replace the bundle_base_url with the appropriate URL of the web server that’s serving your plugin bundles. For now HTTP and HTTPS are supported but we plan to add more options in the future (like pulling directly from S3 buckets).
  • public_key_path: Modify public_key_path in case you want to enforce the cryptographic check of the plugin bundle signatures. If the public_key_path isn’t set, the verification process will be skipped and unsigned plugin bundles will be loaded normally.

Configure an API Definition

There are two important parameters that we need to add or modify in the API definition. The first one is custom_middleware_bundle which must match the name of the plugin bundle file. If we keep this with the default name that the Tyk CLI tool uses, it will be bundle.zip:

"custom_middleware_bundle": "bundle.zip"

Assuming the bundle_base_url is http://localhost/bundles/, Tyk will use the following URL to download our file:

http://localhost/bundles/bundle.zip

The second parameter is specific to this tutorial, and should be used in combination with use_keyless to allow an API to authenticate against our plugin:

"use_keyless": false,
"enable_coprocess_auth": true

enable_coprocess_auth will instruct the Tyk gateway to authenticate this API using the associated custom authentication function that’s implemented by our plugin.

Configuration via the Tyk Dashboard

To attach the plugin to an API, from the Advanced Options tab in the API Designer enter bundle.zip in the Plugin Bundle ID field.

Plugin Options

We also need to modify the authentication mechanism that’s used by the API. From the Core Settings tab in the API Designer select Use Custom Authentication (Python, CoProcess, and JSVM plugins) from the Target Details - Authentication Mode drop-down list.

Advanced Options

Testing the Plugin

At this point we have our test HTTP server ready to serve the plugin bundle and the configuration with all the required parameters. The final step is to start or restart the Tyk Gateway (this may vary depending on how you set up Tyk):

service tyk-gateway start

A simple CURL request will be enough for testing our custom authentication middleware.

This request will trigger an authentication error:

curl http://localhost:8080/my-api/my-path -H 'Authorization: badtoken'

This will trigger a successful authentication. We’re using the token that’s specified in our server implementation (see line 57 in Server.cs):

curl http://localhost:8080/my-api/my-path -H 'Authorization: abc123'

We also have a GitHub repository that includes tests and authentication middleware.

What’s Next?

In this tutorial we learned how Tyk gRPC plugins work. For a production-level setup we suggest the following:

  • Configure an appropriate web server and path to serve your plugin bundles.
Return to Top
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.
   Read More