Adopting observability practices – making the case for technical teams 

Are you a developer or an engineer who wants to adopt observability practices in your team? Doing so when you don’t hold the decision-making power or the company purse strings can feel like a challenge. What you can do, though, is become an advocate for observability, showing how to do it and why it’s important. In a recent Tyk webinar, Microsoft Software Engineer Hope Oluwalolope explained how. 

In this blog post, based on Hope’s insights, we’ll cover:

  • How to grow your knowledge of observability and become an advocate for it
  • Why metrics, traces and logs are so important for solving developer pain points
  • How you can adopt the ‘understand, proof, pitch’ approach to advocate for observability practices 
  • Why it’s important to align your observability advocacy with management’s priorities 

The problem of life without observability 

We’ve all been there. That issue or outage where you spent hours trying to work out what the problem was. You inherited the code; the person who wrote it now works elsewhere. And before you can even think about how to fix the problem, you’ve got to work out what and where the problem is. 

This is life without observability. It’s why advocating to your management team for a better life – one with observability practices – is such a worthy goal. 

Learning about observability 

The first step in becoming an observability advocate is to grow your knowledge. There are plenty of resources out there, from events like KubeCon to on-demand learning such as Tyk’s API observability fundamentals programme. The OpenTelemetry community is also an amazing place to connect and grow your knowledge. Its End User Special Interest Group will even help you adopt and implement OpenTelemetry in your community. 

Even if you’ve already started on your observability journey, you can still advocate for new practices. More metrics, distributed tracing… whatever the next step in your journey is, by learning about it you can help advocate for it. 

Software developer pain points 

Are you constantly in bug fix mode and facing the direct effects of inadequate tools and practices? Is your system a mysterious black box, with customers reporting issues that you spend hours trying to understand? Are you relying on print statements to understand your code? 

You are not alone! But how can you get management to buy into adopting observability practices – which would help address all the above pain points – when your managers have no idea what observability is? 

Bringing observability practices to your team

Education is the starting point. You can explain the advantages of adequate telemetry data to your managers:

  • Metrics help you identify when there is a problem
  • Traces show easily where that problem is
  • Logs provide context on what’s causing the problem 

Of course, telemetry data alone isn’t enough. You’ll need to advocate for the use of visualisation dashboards too, to help you view and analyse the data effectively. 

If your managers are concerned about vendor lock-in, it’s time to extol the benefits of OpenTelemetry in allowing seamless integration with almost any observability tool. 

Another key selling point is, of course, that observability can help you identify issues before they impact your customers, rather than after. You can get ahead of the game, meaning there’s potential for happier customers, a lower churn rate and greater API impact. Explain all of this and you should certainly have management’s attention. 

Of course, once you understand observability, you can begin incorporating it into your day-to-day processes. When you’re writing code and developing a feature, you’ll already be thinking about observability. You can ponder which metrics you could gather right now to help understand how a feature is being used. 

Understand, proof, pitch

In advocating for the adoption of observability in your organisation, you can use the ‘understand, proof, plan’ approach. This can help you move from your current situation to your ideal observability scenario. 

Understand 

Start by seeking to understand three key points that will ultimately help build your case for adopting observability practices:

  • Your management/team – what interests them, what language they are using, what their focus is on and what they’re worried about
  • Your current system landscape – what your code is made of, how your systems are interacting with each other and what’s part of your core structure 
  • Observability practices that can help your team – which areas of observability will best serve your team 

Remember: when you can demonstrate understanding of your team’s or management’s pain points, you’ll be more likely to convince them that the solution you are proposing will help address those pain points. 

Proof 

Next, it’s time for a proof of concept. Yes, it means extra work for you but the time you commit now could really help you in the long run. 

Don’t over-complicate your proof of concept. You could use just one service or a few services. What about choosing the service that is giving you the most issues? 

Doing a proof of concept allows you to prove your understanding of what’s needed. It also makes it easier to demonstrate to management how your proposed observability solution is going to help. 

Pitch

When it comes to pitching your idea, start with your teammates. After all, they’ll be feeling the same pain points and frustrations that you are experiencing. Show them your proof of concept and support them to visualise your solution and suddenly you’ll have other people also keen to make the case to management. 

At that point, when you’re pitching observability to management, it’s important to be clear on the detailed changes and steps you’re proposing. Be specific on the areas of observability you’re focused on, including the exact telemetry data that’s relevant, and what it will deliver – traces, for example, will enable greater system understanding. 

Specifying the exact telemetry data you want to collect, how you’re going to do it and which tools you want to use will ensure a clear understanding and thus less pushback. It’s also essential to show what’s in it for management and to explain that in the context of the current business focus. Is management keen to drive up quality? Then show what observability can do to help. Are they focused on customer retention? Then demonstrate how observability can contribute to improved customer confidence. You need to articulate the value of observability in the context of your own organisation. 

Of course, you’ll also need to factor in cost. If you’re just starting out on your observability journey then you’ll likely want to minimise costs. There are several open source tools that can help with this, particularly if you’re not collecting too much data initially. Then, as you refine your observability practices, you can refine your tooling to suit your evolving needs. 

Equally important when you’re pitching observability to management is to resist the urge to do everything at once. Pushing for metrics, traces, logs, OpenTelemetry and more at the outset is too much. Instead, pick one area to begin with, based on your most pressing priorities and make the case for that change. If you want distributed tracing, for example, why not focus on tracing first and reap the rewards of that? You can start small and then expand your observability practices over time. 

Finally, as part of your pitch, it can help to share all the resources that you found so helpful when learning about observability in the first place – or perhaps to summarise the key learnings in a succinct article. Doing so reduces the learning curve and thus helps clear the path to adoption. 

Keep all this in mind and you’ll have an excellent chance of advocating successfully for the adoption of observability practices. 

If you’re ready to dive deeper into the detail, why not explore how you can use telemetry data collected by Tyk API gateway to set your service level objectives