This blog post summarizes a session from LEAP 2.0: The API governance conference, featuring key takeaways and insights. Explore the full on-demand videos, slides, and more here.
API governance has evolved significantly in recent years, with well-established and continually evolving patterns and practices. Event governance, however, hasn’t yet followed the same path. Events and event management have always been present, certainly, but they haven’t quite matured as APIs have.
It’s a problem that Ahmet Soormally, Head of Research and Development at Tyk, and Swen Helge-Huber, Senior Director within the Office of the CTO at Solace, have been collaborating on over the past year or more. The duo recently shared their insights at the Tyk LEAP 2.0 API governance conference, leading an industry discussion on the world of event driven architectures and the governance challenges that enterprises face. Key highlights, which we discuss in depth below, include:
- Why API governance is now so prominent
- What good API governance looks like
- Why we need to focus on events and event APIs
- How to bring API governance and event API governance together
Read on to benefit from expert insights on these issues.
Why is API governance becoming so relevant?
API governance is an established space, yet it has become more prominent of late. There are various reasons for this, with API sprawl being chief among them. Clearly, the idea of everyone in an organization being able to build reusable services, which other business teams can use, has its advantages. But it also means you end up with myriad services to manage. This is why you need API governance – to manage this evolving API landscape.
That’s not to say that there’s a single version of what “good” looks like when it comes to an enterprise architecture. While the idea of a well laid out, well-planned infrastructure with clear contracts between everything and clear quality standards is laudable, it doesn’t account for the realities of legacy. So, aspiring to that as your governance goal is likely setting yourself up to fail.
Instead, it’s important to think about what is realistic – what you can achieve that embraces both legacy and modernization while also supporting control and compliance through clear guidelines, minimum standards and robust security.
Of course, it’s also essential to put this in an organizational context. Organizations are places of flux and transformation, and this comes with uncertainty and complexity. You can either fight that or (far better) embrace it, enable it and make good things happen.
API governance, in this context, also needs to enable a great developer experience. That means things like ensuring minimum standards across APIs, adding documentation, delivering “light touch” lifecycle management, prioritizing security and providing federation or a marketplace for APIs.
Where does event API governance come in?
Twenty-five years of working with REST APIs has provided plenty of established processes, learnings and successes. But on the event-driven architecture side, things are just emerging. The OpenAPI Specification is now 15 years old, whereas the AsyncAPI standard is only around eight years old.
There are three key reasons why now is the time to embrace the AsyncAPI standard and event API governance:
- First, let’s learn from the challenges of governing REST APIs and not make the same mistakes again.
- Second, let’s aim for consistency across all API styles: REST, AsyncAPI, GraphQL and others.
- And third, let’s deal with the fact that event sprawl is actually worse than API sprawl.
What is event sprawl?
Event sprawl stems from the fact that event-based communications are often treated as an infrastructure concern. If services talk to each other via REST APIs and, at some stage, you need to decouple things, it becomes a very infrastructure-centric technical discussion:
“Hey, I need access to a broker so I can publish this event.”
“Hey, I need a queue so I can consume that event.”
It becomes a middleware inside out approach. You end up with a middleware team thinking about resource allocation and security on a very bit-by-bit basis – on a discrete event basis. It’s about which topic, queue or access controls you need, with everything very technically driven. You lose the interlock with what happened, why it’s happening, what the application team is doing, what payloads are being pumped through your brokers and so on. Instead, the focus is on discrete events, with everything loosely coupled and complex, and a disconnect between the infrastructure, the application and the business purpose.
The result is a tangle of technical infrastructure where it’s hard to see which applications are talking or sending data to each other and why. This is event sprawl in action.
In the REST world, you have very clearly defined patterns. You have resource URLs and HTTP verbs that are aligned with your create, read, update and delete operations. It’s simple, intuitive to understand and applicable to lots of situations.
If you decompose that into events, you end up with multiple commands, plus events that are sent out in response to those commands. It’s not just a question of (for example) an order being updated. You could pump out an event when an order is picked, when it’s shipped, delivered, cancelled… The ability to listen in to a specific part of that lifecycle gives you plenty of power but exploring that and seeing how each of the topics relates to each other isn’t easy.
That’s why it’s necessary to think about events from a conceptual standpoint. You can get basic information on your discrete events and catalog them, of course, perhaps via the event distribution platform that you flow them through. You could also use a traditional approach of different services to consume and send events. But there is a better way…
Using an event API provides a way to capture the fact that events are related and put them together into consumable units. It brings order to your event sprawl. It also creates reusable components that developers can easily understand and access, reducing the cognitive load on your development team.
Naturally, this raises the question of how you govern your event APIs. There are plenty of similarities between event governance capabilities and API governance capabilities, but also some fundamental differences. So, while there are learnings from how API management evolved and what API management does well, it’s crucial to take into account the ways in which events and event APIs differ from traditional APIs. Gartner has a good point to make in this respect:
“Software engineering leaders are tempted to use API management tools for event-drive systems, but these tools lack key features to manage events. Instead, they should equip their teams with event management tools and help them apply API management principles to event, schema and topic design.”
Event API governance tools
Tools are a key part of the equation when it comes to addressing event API governance.
The Solace Event Portal, for example, brings together approaches to address event sprawl. It enables you to deal with discrete events, taking the middleware inside out approach we discussed above and putting it into a governance framework. It also empowers you to combine events into event APIs, to bring order to them and publish them into places where developers already are.
Tyk Streams, meanwhile, enables you to configure and manage your event-driven APIs, easily connecting to event brokers and streaming platforms such as Kafka, while exposing, managing and monetizing your APIs securely.
People and processes
While tooling is important, governance is also about people and processes. You have to engage the teams that developed the functionalities you’re working with: Stream-aligned app teams, platform teams, broker operators, middleware teams and DevOps teams. You also have to engage enabling teams: Governance teams, data architecture teams, enterprise architect teams and the like. How are they working together, what information is currently captured and what tools and processes do they already have in place? Understanding this is essential before you can consider how best to hook into these existing processes and automations.
Often, this process starts with engaging with API teams, with the center of excellence, API product managers and so on. The fact that APIs are often outward-oriented means these teams have better visibility of the business capabilities and value that building APIs provides. They understand the business value of reusable functionality, as well as how to bring APIs to market as products, both for internal and external use.
This engagement leads to an understanding of what API governance capabilities are already in place, which provides the power to leverage what has already been built. It can also shed light on how to fit into existing software development lifecycles, API management and governance, CI/CD tooling and so on.
In practice, the result of doing so can feed event API management into API governance. Let’s say you have a PubSub plus event portal. You can manage your discrete events, package them into event APIs and get them onto the stage where all the other stuff is:
This is where API governance, unified API management, API marketplaces, developer portals and so on come in. This means you can add your Async APIs to your lineup of REST and OpenAPIs. Doing so provides developers with access to everything they need to build applications with responsive, real-time functionality, without the need for complex interaction with OpenAPIs to constantly retrieve updates and hope that something has changed.
As well as this developer experience focus to governance, there is also the need to enforce consistent security across all APIs:
As shown above, this is where a layer that enforces credentials and access control is invaluable. It enables you to take different platforms and different APIs and apply a consistent, uniform means of authenticating and authorizing access.
Wrap up
Governance is organizational change. It’s more than just putting a tool in place and telling people to use it. Governance must be inclusive, bringing different API styles together, including new API styles, into a unified developer experience. It must also be an enabling functionality, rather than a controlling or prohibitive one that people then want to circumvent.
In terms of event governance, it’s essential to follow specific principles. Those are very similar to API governance, but with crucial differences. However, your event governance also needs to plug into your overall API governance in order that developers have the tools available to them to build what they need. All of which leads to an API federation or marketplace approach. That’s something we explore further in this blog on balancing governance and agility in federated API management, so why not explore that post next?
