Recently, we had a new user tell us they were looking for a gateway that could handle 10 million API requests per second…on the low end. They mentioned they were a cloud-first shop running everything out of AWS. This led to the ultimate question of “what performance could they expect while running Tyk in AWS?”
The best answer to this question, which nobody ever wants to hear, is that “it depends”. It depends on your setup, on your environment, on what type of CPU/Memory/Storage resources you will provide, on which features you want enabled, on what your payload is, on your application attributes… the list goes on and on. With Tyk being designed as a cloud native architecture from its inception, there are very few limitations to how it can scale both horizontally and vertically.
But the question around what kind of performance could they expect did get us thinking and we realised that it’s been about a year since we last published any kind of benchmarking results. In that exercise, our colleague, Ahmet Soormally, walked us through some very conservative benchmark testing where we showed Tyk’s ability to handle 6400 RPS with extremely low latency on a vanilla 2 core commodity server with just 4GB of RAM. Not only is Tyk extremely affordable, but it’s super fast as well!
The intent of that benchmarking exercise was not to compare ourselves against our competition. It was definitely not intended to make any crazy false claims of performance where we hid the environmentals or created some black box testing environment. It was also not done to fuel any haters with the “yeah buts”. You know: “yeah, but your app is not real. Yeah, but you used the most expensive virtual server on the market. Yeah, but you tweaked the workload. Yeah but… you cheated!”
Rather, at the time, the intent was to put some caveats out there about what we were doing with the benchmark and then make it known that Tyk is a very flexible gateway; one that can be tailored to meet any budget and any performance need. Depending on what your use case is, there are many ways to meet your API Management business requirement using Tyk.
Ahmet’s overview from last year touched on sizing, configuration, dependencies, installation and some proper “how to’s”. So rather than just answer the question with “yes, we are fast and affordable”, he wanted to actually show you how he did his tests and what his thought process was at the time.
Fast forward a year and we wanted to take a similar approach. We wanted to answer the same question, around what type of performance you can expect from Tyk, but this time with a specific focus on using AWS as the underlying infrastructure. With the goal of staying true to our open source roots by providing transparency into our approach, we have provided a step-by-step guide that walks through our performance testing process on AWS.
See below for a quick summary and recap of our results. And just like last time, we’ve also provided a very thorough step by step guide.
Benchmark summary before any gateway
75,000 Requests per second for 5 minutes
| RUN # | 99% Latency | Throughput |
| 1 | 0.4 ms | 74,961 req / sec |
| 2 | 0.4 ms | 74,904 req / sec |
| 3 | 0.4 ms | 74,900 req / sec |
| Average of 3 runs | 0.4 ms | 74,921 req / sec |
Benchmark summary results – Tyk as a transparent proxy
75,000 Requests per second for 5 minutes
| RUN # | 99% Latency | Throughput |
| 1 | 1.0 ms | 71,556 req / sec |
| 2 | 1.0 ms | 71,535 req / sec |
| 3 | 1.0 ms | 71,547 req / sec |
| Average of 3 runs | 1.0 ms | 71,546 req / sec |
| Delta of first test | 0.6 ms | 4.5% loss in throughput |
Benchmark summary – Tyk with full load (rate limit, authentication, analytics)
75,000 Requests per second for 5 minutes
| RUN # | 99% Latency | Throughput |
| 1 | 1.1 ms | 69,133 req / sec |
| 2 | 1.2 ms | 69,083 req / sec |
| 3 | 1.2 ms | 69,278 req / sec |
| Average of 3 runs | 1.17 ms | 69,165 req / sec |
| Delta of first test | 0.77 ms | 7.7% loss in throughput |
After doing our tests without a gateway, as a transparent proxy, and with multiple features enabled, we have shown that adding the Tyk gateway only added 0.77 ms while reverse proxying over 69,000 requests per second. Here is what we achieved for every API request:
- Performed authentication to validate the authentication token
- Performed authorisation to check the requested resource against the permissions of the authentication token
- Added Rate limiting to make sure the key didn’t query more than 100,000 requests per second
- Gathered analytics on each request & response trip for us to further analyse and track for future analysis
Moving forward, we will continue to do more benchmarking tests for performance with Tyk. In these tests, we’ll look to add more RPS to see just how far we can scale Tyk. We believe the main bottleneck will be the CPU usage of both the Tyk node(s) and the Redis clusters. As long as we can throw more hardware at the problem, we can have Tyk handle as much RPS as our platform needs. See our performance benchmarks.
As previously stated, budgets will vary and not everyone will be able to use large EC2 instances so we know that the answer is not to always throw hardware at the solution. However, as Ahmet showed last year and we validate here, Tyk proves itself as a flexible and highly performant gateway for a variety of budgets. As an example, based on testing of other instance sizes, we see that smaller EC2 instances will actually yield a much higher RPS / latency per dollar, thus driving down cost and increasing efficiency.
In future blogs, we will show what happens when we run our Tyk Pro edition in different set ups. We will also factor in other deployment methods such as Kubernetes. As containers and microservices continue to take hold, we will want to show how Tyk runs on K8s for our prospects that need to scale and manage infrastructure across the globe. We’ll also show the flexibility of Tyk to prevent vendor lock-in and to show the capability to have multiple clusters in different clouds, with Tyk gateways servicing our platform in each of them. If you’re interested in receiving these benchmarks, sign up for our newsletter via the form below.
We love it when prospects ask us what type of performance they can expect with Tyk, especially when they need 10M RPS! But we also know that any guidance we provide on performance should be taken with a grain of salt as everyone’s use case is different and everyone’s environment is unique.
Ultimately, we’d love to know what you, the Tyk community thinks! Think you’re able to break Tyk, or at least give it a good go? Why not replicate our process and tell us how your results compared with a blog post, or give us a heads up on the Tyk Community Forum. We’d also be happy to answer any questions you have or address any ‘gotchas’ you think you may have spotted.
