Implementing an API gateway can work wonders for your API security and governance, but it can also add a latency overhead. If you’re worried about API latency in relation to your gateway, read on to discover common causes of latency and how to reduce them. We’ve rounded up the best strategies to conquer API gateway latency problems, so you can achieve the blistering performance your enterprise deserves.
What is API gateway latency?
API gateway latency is the amount of time it takes an API gateway to process a request, compose a response to that request and send the response to the client. This metric is measured in milliseconds.
Another metric worth noting is API gateway integration latency. This is the amount of time it takes an API gateway to send a request to the backend and receive a response from it. It is also measured in milliseconds. When providing an API gateway latency figure, that figure will include API gateway integration latency within in.
Causes of high API gateway latency
For some enterprises, every millisecond counts – that’s why we’ve designed Tyk to be ultra performant! It’s one of the many benefits of Tyk. Whichever API gateway you use, if you’re running into latency issues, some of these common causes might be to blame.
Network issues
High traffic and connectivity issues can result in API gateway latency problems. If you’re keen to reduce API latency, investigating your network stability and reliability is a good starting point.
Inadequate server resources
If the gateway is calling a server with inadequate resources, you’re likely to run into latency issues, with requests taking longer to process than you would like. Likewise, if the gateway is calling a server in a different region, the extra milliseconds can quickly start to add up.
Unoptimized code
If your code isn’t optimized, you can run into performance and API gateway latency issues. Inefficiencies can lead to slowdowns and potentially to gateway timeouts. One of many reasons why optimizing your code is crucial.
Measuring API gateway latency
If you’re worried about latency, it’s time to start measuring and monitoring. What is API latency in the context of your enterprise? Unless you’ve taken the time to benchmark your latency, you won’t know how big a problem you have or whether any troubleshooting measures you implement are effective.
There are various ways you can approach API latency monitoring.
Tyk API Gateway latency monitoring
The Tyk API Gateway provides a handy example of different ways in which you can monitor latency. One approach is to use Tyk Dashboard, where you can view your average latency over time on a graph, alongside other metrics.
You can also use third-party tools such as Grafana and Prometheus to view and monitor latency. When used with Tyk, you can configure these tools to create alerts if latency goes beyond a threshold of your choosing.
Using CloudWatch metrics
Amazon’s CloudWatch metrics are a helpful feature of CloudWatch. If you’re using AWS, you can use these metrics to monitor your gateway latency.
Using third-party tools
As with every aspect of API management, there are a host of tools out there to help you. At the free, open source end of the spectrum is OpenTelemetry, which can provide you with sophisticated insights into many aspects of your systems.
Prometheus is another helpful, free, open source tool that records a range of metrics, while Grafana OSS is particularly handy as a data visualization and dashboarding tool.
Why the emphasis on open source? Because our industry-leading open source API Gateway here at Tyk has shown just how powerful open source tools can be in securing, processing and governing APIs for global enterprises.
Troubleshooting high latency
If your API latency monitoring indicates you need to troubleshoot high gateway latency, there are various steps you can take. As mentioned above, looking for network issues, inadequate server resources and unoptimized code can often solve common latency problems.
If you still need to reduce API latency after ticking those items off your troubleshooting list, the following could help:
- Look into the sizing and configuration of your gateway and its dependencies. A lightweight, highly performant gateway without computationally demanding and memory intensive dependencies will serve you best.
- Undertake benchmarking and experiment with rate limiting to understand how avoiding overloading your CPU can impact your baseline and introduced latency levels.
To dive deeper into fine-tuning your API gateway to reduce latency, check out this detailed example.
In addition to troubleshooting your current latency issues, use alerting to flag up any future API gateway latency problems. Doing so means you can troubleshoot proactively and jump on problems as soon as they arise – hopefully before your users even notice.
Performance issue workarounds
As well as the troubleshooting ideas mentioned above, there are various workarounds you can use to help address and avoid gateway performance issues. Let’s look at a few of these now.
Co-locating resources
Is your gateway regularly calling servers in far-off regions? Doing so can result in higher latency, so consider co-locating resources as an effective workaround.
Modifying authentication
Do you assign identity and access management roles that are verified in the backend as part of your authentication measures, for example by using AWS Identity and Access Management (IAM) roles? The way you do so can impact latency. Using a role-based invocation means the API gateway will call the Secure Token Service, thus adding latency. You can work around this by using a caller-identity or resource policy-based invocation instead (which won’t call the Secure Token Service).
Disabling cache encryption
Have you enabled encryption for your request caching? Doing so will add latency, as a result of the encryption and decryption of cache entries. Turning off cache encryption is a swift workaround for this. Just be sure that doing so is in line with your API security approach.
Alternative approaches to address high latency
Need further options to reduce API latency in relation to your gateway? These alternative approaches could provide the solution you need to address your latency issues.
Hosting functions on EC2
Amazon’s Elastic Compute Cloud (EC2) is designed to make it easier for developers to undertake web-scale computing. It erases any latency resulting from using an API gateway by removing the gateway entirely. Instead, you can handle requests with API wrappers and webservers.
Containerizing serverless functions
Microservices and containers, along with microservice gateway access patterns, go hand-in-hand when it comes to building distributed applications. Containers can also be handy for delivering efficiencies within a serverless environment – including as a strategy to reduce API latency.
Comparing serverless hosts
If the serverless route appeals to you, there are various solutions you can implement. AWS Lambda, Microsoft Azure and Google Cloud Compute are all options to investigate if this is the approach you want to take. The advantage is that you can call resources directly and take the API gateway for microservices out of the loop entirely, thus cutting out the associated latency.
Building your own web server
Prefer to take matters into your own hands? If so, building your own webserver might be the ideal way forward. You can use NGINX or Apache to achieve what you need, providing you with an ecosystem to tweak and fine-tune however you see fit. This means you have full control over what you change and by how much, enabling you to find the ideal latency scenario to meet your business needs.
Conclusion
We’ve provided a range of options above, ensuring you have plenty of flexibility in your approach to reducing API gateway latency. Using the strategies we’ve outlined, it should be fairly simple to benchmark your performance, then implement the changes you need to conquer your API gateway latency problems and optimize your API ecosystem.
If the idea of using OpenTelemetry particularly appeals, it’s well worth checking out our tutorial on how to reduce latency issues and optimize your APIs using OpenTelemetry and Tyk.
There is, of course, a huge range of other variables in the way you manage your API ecosystem. If you’re currently working out what good looks like for your use case, this article on API management architectural and deployment patterns is an excellent starting point.
Remember, too, that the Tyk team is always here to help. Our in-house experts are happy to share their advice and guidance, as are API professionals around the globe via the Tyk Community.
