Meeting your product objectives can be tough. But leveraging API observability to understand your product adoption, engagement and retention rates can help. That’s why the Tyk team recently caught up with Derric Gilling, CEO of leading API analytics and monetisation platform, Moesif.
In this blog, we’ll look at:
- The crucial role of customer value metrics.
- Adoption metrics and how to get the best from them.
- How effectively your API meets customer needs.
- Tips for fostering transparency and reducing support burdens with the right tooling.
The Moesif team spend their days working with API-first businesses, focusing on product-led growth, driving developer adoption and different usage-based pricing models for APIs.
Of late, those conversations have revealed a trend of software-as-a-service (SaaS) giving way to APIs. Tasks previously done manually and through user interfaces can now be achieved through automation, with a lot of the value being delivered through APIs. So, how do you measure that value through your APIs?
Watch the full webinar video above, or read on for our top takeaways.
Infrastructure health metrics
When we think about observability, different infrastructure metrics come to mind: how you track the health of your infrastructure, your APIs, your servers and the like. But while things like memory usage, performance and latency are great for tracking engineering concerns, they don’t focus on the value that customers are getting out of your platform.
Customer value metrics will vary by business. At Sinch, for example, it’s all about SMS volumes – how many text messages are being sent and what the delivery times are. At Stripe, it’s about how the company’s payments APIs are supporting their customers to drive more revenue. Segment, meanwhile, is focused on unique user numbers and how it is helping businesses drive user growth and adoption.
In each of these examples, there are three key areas to focus on from a product standpoint: adoption, engagement and retention.
Product adoption
Over the past couple of decades, there’s been a significant shift in how developers are adopting APIs and developer platforms. Twenty years ago, it was all about on-premises solutions. A decade later, SaaS platforms were on the rise, with teams adopting Asana, Jira and the like, even when other teams within the business were using different solutions.
Fast-forward to the present day and many individual developers have the authority to choose the APIs with which they want to integrate. There are a lot of different platforms out there and it’s no longer just about team decisions. It’s about individual developers solving business problems based on what they like to use and what they can use successfully.
That said, developers still need to jump through company hoops: legal, security, product priorities, finance and the like. So, how can you help them adopt your APIs as quickly as possible?
This is where the idea of a developer-first, self service approach comes in. Instead of lengthy procurement cycles, the focus is on click-through terms of service and fast integration. The developer can get up and running fast, simply by using a manager’s credit card.
The result of this is a wider funnel. More developers can try out your API, paying a token amount to get up and running as quickly as possible to test out their proof of concept (POC) or minimum viable product (MVP). That way, they can realise the value of your product, fast.
Measuring product adoption metrics
With this developer-first approach to product adoption, it’s time to look at two key metrics for API adoption:
- Time to first hello world – this is something as simple as a first API call through Postman, downloading a Postman collection or using curl. It’s about the developer exploring the API to see if it will solve their use case.
- Time to first working app – this is the time it takes the developer to create a POC or MVP. Something they can show to their colleagues, manager or possibly potential users or customers, if it’s an external API.
Naturally, you’ll lose people as they head down the adoption funnel. You’ll need to get them through visiting your website and docs pages, requesting a demo and signing up for the service, first. Then they move onto the adoption and time to first hello world. After that, you’re likely to lose developers who have issues with your documentation, guides and tutorials or who simply realise that your product doesn’t solve their use case.
Those who make it to building something they can show and demo to other people provide you with adoption metrics you can measure in terms of what’s happening within your API.
Measuring your API adoption funnel
How long does it take for developers who adopt your product to get from sign up to first hello world? What percentage of those who sign up make it that far? And how many make it as far as the first working app? These are ways you can measure your adoption funnel.
Product engagement
It’s essential to ensure your metrics are aligned to customer value. This is both industry-dependent and business-specific. For an ecommerce platform, for example, it might be about measuring checkout conversion rates, while a SaaS platform could be focused on how many different integrations or extensions are being used. It’s about defining the right service level agreements (SLAs) from a business standpoint – but those SLAs aren’t necessarily about uptime. For a logistics firm, they might be about how long it takes for an order to be fulfilled. For a data API, it might be the percentage of queries that are actually found.
What’s key here is to break down what a self-service user will see, versus what the next layer of enterprise sales will see. We talked about self-service developers paying a token amount as part of product adoption but that’s not necessarily the end game. What follows is meeting enterprise users’ requirements and expanding usage, including adopting additional APIs.
Consider the example of a developer who has signed up at a token rate to get started using an SMS API. If you’re a telco company, you could also have voice, data, video and other products that might fit that developer’s use cases. Now you’re ready to run with product-led growth.
This is where the product-led growth flywheel kicks in. The developer journey is about self-service usage, that initial ‘aha!’ moment and then increasing usage – essentially, time to first hello world and time to first working app. At that point, the developer will see the value of your API product. Then they’ll show their manager that value and start digging in further. At this point, we can look at a range of other indicators.
Measuring product engagement metrics
Measuring increasing usage – for example by percentage growth in the number of API calls – gives your sales team and your customer service team the ability to identify users who are growing their engagement with your product. Your team can then engage with those users to check if your product is solving their problem in the right way and to identify new use cases. You can show all your other API products capabilities, encourage further adoption and accelerate their uses. All of which fuels that product-led growth flywheel.
By measuring engagement in this way, you can start scoring your accounts red, yellow and green. You can look at what customers are doing with your APIs: their transaction volumes, whether they’re adding new seats or integrations, what they’re using in terms of the platform and so on.
Just remember: not all traffic is good. Your customers aren’t purchasing requests per second. A sudden spike from a small amount of traffic to millions of requests per second doesn’t mean you’re delivering value. Instead, you need to measure in the right way.
Using an ecommerce example, not every query will return an item; something could be out of stock. As such, you can apply weighting so that queries that return items are higher weight than those that return out of stock responses. This means you can look at daily active users who are actually seeing items that are available – i.e. those who are getting value.
Retention
The other key area that observability can help with from a product standpoint is retention. This is how you know that those who adopt and engage with your product are sticking with it, rather than leaving your platform because they can’t solve their use case or are having issues.
Some level of drop-off from those who adopt your product is inevitable. But when you measure retention, it’s important that this level of drop-off evens out. That means you end up with stable retention. Knowing that gives you the confidence to invest in driving adoption, growing sales, marketing and so on.
Bear in mind, too, that you have the potential to reengage those who drop off. This means putting some work into understanding who you’re trying to reengage with, as different users will have different use cases and take different paths in terms of their adoption, engagement and retention of your product. You can send out automated emails or reach out via Slack, for example. Just remember that plenty of developers might not want to chat – so using a passive engagement model, where you send out a documentation link, a guide or a video might be more effective than a message proposing a call.
Also important is to understand what a ‘healthy’ drop-off rate looks like for your product. This is about understanding typical drop-off rates and applying the right filtering criteria to each of the stages at which users drop off. The more enriched your knowledge of your users, the more nuanced those filters can be.
Using observability to support self-service scaling
Scaling self-service is hard. You’ll face issues with onboarding, changing business needs, billing issues and surprises and more. One of the best ways to overcome the challenges of scaling is to deliver a great self-service experience.
You can do this by providing your customers with the same analytics observability that you have. It means your customers don’t have to ask whether they’re going to exceed their quota and that they’re not going to receive a surprise enormous bill. Instead, by providing your customers with the same metrics you have, you can provide them with transparency and also reduce the load on your customer support/account management team.
How you provide those metrics is up to you. You could provide an app dashboard. You could send emails when usage spikes occur. You could reach out with a handy docs link when 400 error messages hit a certain level. There are plenty of touchpoints around APIs and the integration process. It’s up to you to provide the tooling to ensure your users are on the right path.
It’s also up to you how you handle these situations. If a customer hits their quota, for example, do you immediately apply an overage fee or do you block the user? There are plenty of different strategies that can work, just be sure your users are aware of your approach, so they’re not unpleasantly surprised.
Finding the right observability tools
Measuring these different metrics and looking at different layers of data often means using more than one tool, but it’s important to avoid tool explosion and end up with dozens of different tools.
Generally, finding two or three core tools that drive the business is a solid approach. You might have one that focuses on the metrics that site reliability engineers and DevOps need, for example, then another for product and business metrics.
What’s key is ensuring that your different tools have integrations that can tie everything together. Those integrations can save you a lot of time building out things manually.
On the subject of using the right tools to leverage API observability to meet your product objectives, why not check out Moesif and Tyk as the perfect pairing?
