Tyk tips limit breaking changes

Using Evolutionary API Design to Limit Breaking Changes

Product teams don’t want to make life difficult for API consumers. However, in speaking with organisations around the world, one thing is certain: your API design will require a breaking change in the future. Why? It is best summarised in this quote:

 

“A design isn’t finished until someone is using it.” – Brenda Laurel

 

Until we have developers using our API to solve problems, the design isn’t finished. This means that our API design must be put into the hands of the developers that will use it. Yet, once we do, our API design is locked-in. Or is it?

Good Design Starts with Listening

Our approach should be to present a tangible design to drive constructive discussion and feedback with our audience. We must be willing to listen, learn, and adjust our API designs accordingly – anything short of this will serve your needs more than the needs of your current and future API consumers.

 

“If you think good design is expensive, you should look at the cost of bad design.” – Dr. Ralf Speth

 

To limit breaking changes to our API design, teams are opting to apply a design first approach to API design. While some may think this goes against the principles of agile development, in fact design first complements agile processes nicely.

Yet, API design first doesn’t solve the problems we face day-to-day: wrong assumptions, misunderstanding the needs of our developers and their users, and the desire for a deeper understanding of how our APIs will be used.

What we need is a way to combine API design first process with the flexibility to make design changes after they go out the door. We need to apply an evolutionary approach to our design process.

Evolutionary API design can be integrated into your existing API life cycle through three steps: clearly separate published areas of your API from unpublished ones, seek feedback early, and manage expectations through an API stability contract. Let’s look at each of these in turn to see how they can help us avoid introducing breaking changes to our API design.

Step 1: Keep Supported and Unsupported Operations Separate

If you are releasing an API, it is tempting to just release everything you have built. The decision usually ends with, “Who knows – they may want to use it someday?”

The problem is that every operation you release must be supported, even if it was originally built to drive an internal app. While I’m a huge fan of APIs that enable you to do everything available, often these unpublished operations make assumptions or take shortcuts. They don’t go through the same rigor as those you plan to publish and support. Eventually, you must make breaking changes to the API to support the needs of customers who become dependent on these unpublished operations.

Limit the API surface area you release with each version by separating supported areas from unsupported ones. This can be done different prefixes: /v1/{resource} for supported operations vs. /private/{resource} for unsupported operations. By using this approach, operations intended for internal use to power a web dashboard or internal process are placed into a separate namespace. If customers integrate with your unsupported operations, it is at their own risk. Offer support specifically for the supported operations only.

If demand from customers indicate value in offering a supported version of a private operation, this can be reviewed, the design improved, and then promoted to the list of supported operations. Meanwhile, unsupported operations may change over time and may include breaking changes that are absorbed only by the internal product team.

Step 2: Get Feedback Early and Often

Feedback is critical to avoiding breaking changes. As an aside, I’ve seen teams build some well-designed APIs. Some of these well-designed APIs immediately move from v1 to v2 because they didn’t ask for feedback and must make breaking changes. That is because this well-designed API is making assumptions about how people intend to use it – and those assumptions were wrong.

Use preview releases to gather feedback from API consumers on new features or enhancements. These releases allow developers to try out new releases, verify naming choices, and ensure documentation clarity. This period is typically a few weeks to provide enough time for developers to try it out and provide feedback alongside their other backlog items.

If you want to limit the preview release feedback, consider using feature toggles for specific users to prevent impacting all customers at once.

Just bear in mind that you must be clear to establish expectations that the API may change between preview releases and the final supported version – make no guarantee of long-term support for these preview releases.

Step 3: Establish an API Stability Contract

There may be times when you need to push some new API capabilities into production and allow for experimentation before you commit to supporting them. In this case, a pre-release isn’t enough as the assumption is that you will release the API at some point in the future – perhaps with some changes to the design.

An API Stability Contract is a method of establishing expectations between you, the API provider, and your consumers. In an API Stability Contract, you define a number of stages that the operations for an API capability may be assigned. These stages typically mirror an API lifecycle, from experimental operations to supported and (perhaps eventually) retirement.

The current stage of each operation or set of operations may be captured as tags in your OpenAPI Specification. Your API reference docs may choose to hide or render operations based on the tagged stage.

Below is a recommended starting point, though you may find that you need to customise these names and expectations to meet your needs:

  1. Experimental – This is an early release for experimentation and feedback. There is no guarantee that it will ever be supported. The design may change or it may be removed completely in a future release
  2. Pre-Release – The design has been pre-released for feedback and will be supported in the future. However, the design is not frozen and therefore may change in the future
  3. Supported – In production and supported. The design is frozen – breaking changes will require a new API version to be released
  4. Deprecated – Still supported, but will soon be sunset. Consider the use of the Sunset header RFC to programmatically communicate when it will be sunset
  5. Retired – No longer available or supported

As you can see, applying an API Stability Contract gives providers the freedom to make adjustments to the design of emerging capabilities without requiring a breaking change to their supported set of capabilities.

Wrap-Up

Breaking changes can be difficult for API consumers to absorb. Yet API providers need to evolve their API design as they gain a better understanding of how it will be used. Establishing some clear expectations, seeking feedback, and introducing an API Stability Contract are three ways to avoid breaking changes. These step introduce a more evolutionary design approach by seeking input early and often before releasing the final version of your API design. By adopting this approach, you reduce the risk of breaking changes in the future.


swipe

  1. Big, up-front design prior to applying agile processes for implementation (aka “water-scrum-fall” approach)
  2. Moving so fast you accept you won’t get it right and therefore you’ll need to break your consumers in the future (aka the old Facebook approach)
  3. Realisation that you’ve solved this once already and already know what you need (aka the arrogant approach)

Yet, we sometimes miss the mark and need to make a breaking change to our API design. Not ideal, but it happens – even if you apply an API design first approach.