If you’re looking for more control and security for your organizations API portfolio, your most likely solution is including an API management platform in your tech stack. Tyk’s API management platform is one of the most performant and extensive solutions out there covering all major areas of concern, plus some. Here at Tyk we employ a modular and open-source approach to our main components which allow for users to expand and add additional functionality they may need easily. Tyk is written in Go, a language that has allowed us to create a gateway and supporting components, including our “batteries-included” plugins, to handle high-throughput and scalability. But we didn’t stop there since we also include runtimes which support JavaScript, Python, LuaJIT, amongst many others.
A key factor in managing your APIs is the ability to drill deep into API analytics allowing you to observe how your customers are using your APIs and monitoring Key Performance Indicators like the performance of your APIs. Through these insights you can monitor the growth and health of your enterprise due to the major dependency of organization success, from a technology perspective, usually relying on dependable and well-maintained APIs.
With Tyk, when focusing on analytics, we have many options to support the needs of ever-changing organizations. Because of Tyk’s open platform and modular approach, you are able to choose your analytics provider that fits your needs whether it be an open-source or managed service, including building your own on ElasticSearch or Prometheus. All of this is made possible by Tyk Pump, the asynchronous component which allows for the forwarding of analytics data to your desired provider.
Moesif API analytics is one of the services supported by Tyk Pump and is an API observability and analytics service designed specifically for API platforms. With the help of our friends over at Moesif, let’s dig a bit further into the subject of Moesif and Tyk!
How does the Moesif Tyk plugin work?
With the Moesif Tyk plugin, your API logs are sent to Moesif to provide analytics and reports on usage. Moesif also collects information such as the authenticated user (AliasId or OAuthId) so you’re able to drill into activation funnels and track metrics like active users. Moesif also processed payload information so you’re able to understand utilization of specific payload keys, etc.
What is API observability and analytics?
Before API observability and analytics existed, most engineers developing APIs would leverage a monitoring service to synthetically probe their API for uptime status. While this was great to have a quick smoke test on API uptime and SLA metrics it required that you knew what to probe for, such as a specific test pattern or sequence. In addition, traditional API monitoring can only measure what the probe sees, meaning it’s unsuitable to answer arbitrary questions around user behavior which is required for advanced tasks like API security and detecting threats to API product analytics. Moesif is an API analytics tool that provides deep understanding of who is using your APIs and how they are used.
API Product Metrics and Business Insights
Once you have the Moesif and Tyk setup configured, you’ll start seeing API analytics data show up in Moesif. A powerful API analytics platform like Moesif can provide deep insight on not only performance and uptime, but also the value your APIs create and where to focus your efforts. Here are a few scenarios where Moesif can augment the upstream APIs you’ve configured with Tyk:
Understanding customer API usage
Understanding which customers are using your API the most is the first step to leveraging API analytics. APIs being transactional by design are naturally a medium to transfer value, with each API call being a measurable unit of value. As an example, a location API may have their users pay dollars to extract location data. The more location queries (i.e. API calls) they perform, the more that user gains from using your API.
The Moesif Tyk integration automatically maps a Tyk Token Alias to a user in Moesif. With a Moesif SDK, you can store additional customer demographics to break down API usage by customer email, company industry, and more such as in the below chart:
Drilling into API behavior
With customer demographics stored in Moesif, you can begin to drill down your API data by any field like URI parameters, HTTP headers, body fields, and more. In the below example, we are able to see which endpoints each customer is using:
Because Moesif also analyzes your payload data such as JSON or XML, you can understand how each entity is used and the data traveling over your API. In the below report, we are showing the “label” field within the response body broken down by the customer so we can see which customers experienced “Out of Stock” the most. This can provide valuable insights to business users and leadership on areas of the business that need improvement.
Creating conversion funnels
APIs calls are transactional in nature which usually represent some transaction like making a purchase or sending an SMS. By linking API activity to individual customers, you are able to measure these “customer conversion events” and understand where customers drop off in your customer journey. That conversion event can be as simple as making a single API call, or can be as complicated as making 100 purchases that succeeded within a set time period. With funnel analysis, you’re able to slice and dice these customer journeys and conversion events to understand what impact that conversion rate.
In the below report, we created a funnel analysis composing of three steps.
- The first step is a customer signing into your web app.
- Moving from the first to second step shows the drop off for customers who actually made their first payment transaction through your platform. The time it takes to get to this second step is called “Time to First Hello World” or TTFHW.
- Moving from the second to third step shows the percentage of user who end up making over 100 payment transactions. This could be your “aha” moment or when someone receives full value out of your API.
As seen in this report, you need to track what happens before that very first API call (such as sign up) and link it back to API traffic. The missing piece is adding a browser tracking SDK such as via Segment or moesif-browser-js to track user actions like “Sign Up.” As long as you use the same “Alias” within Moesif as the userId within the Moesif client integration, Moesif will stitch together everything automatically.
API Observability
Besides product and business analytics, using Moesif and Tyk together can enable your engineering team to investigate and fix API issues rapidly with high-cardinality, high-dimensional API logs. The Moesif Tyk integration instruments your API automatically to log common parameters out of the box including URI, HTTP headers, latency, status code, and more. Once you set up Tyk, we recommend first taking a quick look at any 4xx errors which can be done under the “Live API Log”
Tracking API performance is critical to ensure you know which upstream services need optimizations. The great thing about an API gateway like Tyk is that it provides a single point to analyze these metrics like latency and functional correctness from the view of the customer.
When it comes to breaking down latency by service or customer, we recommend looking at the 90th percentile latency. The reason is that while a consistently high latency may be tolerable such as for a large query, having a wildly different latency introduces unpredictability for your API users. One query might be super fast, but the next could be extremely slow stressing their infrastructure in new ways they never expected. You can bring up a report on 90th percentile latency in a few clicks with Moesif:
GDPR Compliance and Infrastructure Cost
Storing a bunch of API logs has its own risks of potentially having PII or exposing you to compliance risk from GDPR, CCPA, and other regulation. Since Moesif and Tyk links every API call to a user entity, this reduces your burden considerably. Handling GDPR compliance requires having mechanisms to delete all data associated with a user. In addition, you need a way to stop collecting future data.
The Moesif backend for Tyk Pump supports data suppression out of the box from Moesif’s sampling engine automatically. No code change is needed to modify these rules. To stop collecting data on a user, it’s as simple as going to any profile and clicking “suppress data collection”.
This engine also enables sampling of key customers such as your largest ones to reduce your API analytics cost. When a customer is sampled, the Moesif Tyk integration will log a random sample of API calls for that user. Yet, Moesif can still extrapolate the original values such as API usage so you have accurate reporting.
Closing thoughts
Building and scaling a new API platform is no easy task. With tools like Tyk and Moesif, you have many of the components needed to serve APIs at scale with speed and confidence. With Tyk’s open and modular platform, you’re able to enhance your Tyk experience with a variety of third party tools like Moesif and more.
