API design is an important discipline that can contribute to better understanding and faster integration for developers. Even with a great design, if it doesn’t solve the problems of your stakeholders, your API will miss the mark.
To avoid this problem, I recommend a design-first approach. But, how do organisations take a design-first approach while remaining agile? Let’s take a deep-dive into the API design first approach, review how it fits with an agile process, and discuss some common practices to avoid reverting to a waterfall-based approach with our API design process.
What is API design-first?
An API design-first approach starts by identifying the capabilities to be delivered, then moves toward an API design to meet those capabilities. All of this occurs before we write a line of code for our API. This is often called an outside-in approach to API design.
Does this sound like the waterfall process to you? It might be, but only if you don’t elicit feedback from your stakeholders – those who are your target API consumers.
Anti-patterns in API design
Unfortunately, I see some anti-patterns when consulting with teams on their API design process:
- Starting bottom-up from the database or code, without considering how consumers will use your API
- Breaking the API design after a general release, without proper notification and change management – and without considering the impact to your consumers
- Completely missing the purpose of an API by designing and implementing in isolation
Let’s find out how we can do better with our API design process while remaining agile.
How to avoid a waterfall approach with API design
Before we look at API design first, let’s briefly review some highlights from the principles of the Agile Manifesto:
- Our highest priority should be to satisfy the customer (i.e. our stakeholders, the API consumers)
- We should welcome changing requirements, even late in development
- We must strive to deliver working software frequently
- Business people and developers must work together daily
- Working software is the primary measure of progress (i.e. our API design and implementation)
- Be attentive to technical excellence and good design, as this enhances agility (i.e. no wild-west coding sprints)
- Seek the simple by maximizing the amount of work not done (i.e. lines of code or “busy work” isn’t the key metric)
There are many more principles to the Agile Manifesto, but I have highlighted these and embellished them to call out the focus of an agile API design approach.
Remember: Agile doesn’t mean going fast without applying thoughtful design and consumer feedback.
Keeping these principles in mind, we have the opportunity to remain agile while communicating early and often with our stakeholders on our API design. An API design-first process enables us to move quickly, thoughtfully, and with the agility to make changes early in the process rather than once the API has been coded and delivered. It is the complete opposite to a waterfall-based approach to API design. Let’s find out what this looks like in practice.
How does API design-first fit with Agile?
APIs are powerful. They define a clear contract for developers. But no one ever said that you can’t share your API design before you implement the code behind it. In fact, it is often best if we don’t immediately implement our API. Instead, we can take a more agile approach to our API design process:
- Discover the capabilities that need to be delivered. This step focuses on behavior and desired outcomes before we focus too much on the data
- Model and design our API to meet the desired behaviour, applying a resource-based approach and using the power of HTTP
- Document the first-pass API design enough to provide understanding and provide some code examples to demonstrate common use cases
- Seek feedback from stakeholders to validate that the API design meets stakeholder needs
- Update the API design by incorporating feedback as appropriate, repeating steps 1 through 4 as often as needed to refine your API design
- Produce a mock version of the API from your OpenAPI spec for your stakeholders to review. There are plenty of tools that offer this capability
- Release the implemented API as a preview release. It is during this preview release that you can gain additional feedback and make necessary changes before officially releasing your API into production
- Promote the release to production once the preview release meets the needs of stakeholders
Keep in mind that this process may be used incrementally to deliver read-only endpoints, followed by endpoints that manipulate data or support more of the workflow process.
Take advantage of a preview release for additional feedback and early integration by early adopters. Use the preview release as time to work out trouble spots, improve documentation, and expand code examples that demonstrate API usage.
Repeat this process as improvements are identified as part of future releases. By focusing on constant communication and feedback from stakeholders, you avoid the need to redesign and reimplement the API under time constraints.
Remember: While it isn’t cheap to make changes once you are in a preview release, it is still easier than after your API has been officially released.
Using API Gateways with API design-first
One often forgotten tool in an API design-first process is the API gateway. Here are some tips and tricks I have witnessed by high-performing API teams:
- Stand up multiple gateway instances, one per environment (e.g. preview, prod). A separate preview environment clearly marks APIs as early, changing, and not for production use
- Use role-based access to endpoints to allow specific clients to access preview APIs while preventing production usage
- Apply strict rate limits to emerging apis to differentiate from production APIs
- Remove APIs from your preview gateway after they have moved to production, to prevent people from abusing the preview releases or accidentally using them for production code
By using the gateway as a defensive layer, we prevent inappropriate usage, allow for flexibility to make changes throughout our sprints, and clearly separate official releases from preview releases.
Conclusion
As we have discovered, API design-first doesn’t mean you have to use a waterfall process. Being agile means constantly communicating with your stakeholders, both internal and external. Use this opportunity to communicate via documentation, mock implementations, and example code to gather feedback quickly – before you code. This will help minimise the changes required while working closely with stakeholders to gain insight and manage their expectations of your incremental releases.
