A design that initially looks good may not be the best design to easily solve the problems at hand. Too often, our initial assumptions about our API and microservice designs are incorrect as they encounter the real world. So, how can we avoid designing an API that doesn’t meet the needs of our consumers?
One way of validating your API and microservice designs is to use some design techniques to surface issues before we commit to our design. Just as we use test-driven development to turn requirements into test cases that we code against, these design techniques help us to design APIs and microservices that meet the needs of our stakeholders. Let’s examine five techniques that can help you validate your API and service designs before, during, and after the design process.
Event storming
Overview: Event Storming is a technique developed by Alberto Brandolini to drive analysis and understanding of a problem domain. It requires that subject matter experts, developers, and other roles work together. The process isn’t meant to drive brainstorming, but rather create a shared understanding of how one or more systems work in cooperation to solve everyday problems. All work is done using sticky notes on a large surface area.
The process begins by capturing the domain events that occur, e.g. “Invoice was paid”. When identifying the initial set of events, it is beneficial to start near the end of the process. Then, work along the process to identify other surrounding events. This means asking questions such as “What happened before that?”, e.g. “Invoice was sent”. Once all events have been identified, you should be able to walk the events from the beginning to ensure you haven’t missed anything, reorganizing the positions of the events as you go.
Then, the group asks what caused each event to happen. Those are usually commands (e.g. user intentions), system events, external systems, or simply the passage of time. The board is then expanded to include read models, external systems, policies, commands, and UI components. Most importantly, teams identify hotspots where things are unknown and need further research or decisions to be made.
Example: Below is a photo of the early stages of an event storming session that I facilitated:
The group used the process throughout their design effort, to identify bounded areas that could be realized through internal systems, off-the-shelf solutions, and highly-cohesive APIs. They often referred back to the board often whenever a question arose, to determine what was previously discussed and how newly-gained information might impact their previous decisions and assumptions.
Usefulness: This technique is best used when you have a product or area of the business that needs to be understood better. I’ve seen this technique used to gain a breadth-wise understanding of how processes work across a company, as well as to further deep-dive into a specific area. Often, event storming is a great way to find boundaries where there are clear areas of concern that demand an API and perhaps two or more microservices.
More Info: Check out Brandolini’s book on LeanPub. As of this writing, he is still working to complete the book. However, there is plenty there to get you and your team started.
API modeling
Overview: API modeling is a technique that helps to bridge your requirements to your API design. It starts with a list of the personas that will be consuming the API, directly or indirectly. The next step is to map out the desired outcomes of each persona, along with the steps to get there. I often recommend the Job Story approach from the Jobs-to-be-Done (JTBD) framework as a way to identify the outcomes:
When ______, I want to _____, So I can _____.
Once the model is established, you then layer on the HTTP design based on the design style you prefer: REST, GraphQL, gRPC, etc. The advantage of this approach is that the API is designed to produce outcomes, not just move data over HTTP from a database.
Below is a diagram summarizing how the process works:
Example: The modeling output, captured from a recent training session, often looks like this:
Usefulness: This process helps to identify the workflow that the API will need to support, along with some candidate resources that may become part of your API design.
More Info: If you are interested in more details into how API modeling works and how to go from modeling to design, pick up a copy of my book, “A Practical Approach to API Design”.
API context mapping
Overview: This is a technique that I derived from context mapping, used in domain-driven design (DDD), along with some principles of customer journey mapping. The diagram helps to identify dependencies both upstream and downstream, to demonstrate how the API fits into the larger ecosystem. Without context mapping, teams may design their API in isolation. This may cause them to miss critical details that force API consumers to work around gaps in the overall workflow.
As an example, I have witnessed a team that was tasked with designing an endpoint to fetch a resource instance by ID – but failed to realize that there is no way to obtain that ID at all! API consumers were forced to work around this design flaw by directly accessing a database to obtain the ID, or perhaps avoiding the API entirely since it didn’t meet their needs.
Example: Below is an example of an eCommerce workflow that demonstrates how each step maps to the next. If a team has been assigned to each resource, they would understand better how their API fits into the expected workflow:
Usefulness: Context mapping is useful for teams producing microservices, as well as large organizations where many smaller teams design a portion of an overall workflow. By mapping your dependencies upstream and downstream, you will better understand how your API’s inputs and outputs will be used. I recommend API context mapping as a checkpoint to ensure that APIs being designed by multiple teams will work together to achieve the desired outcome.
More Info: API context mapping is rooted in understanding the customer journey. I recommend understanding the concept of customer journey maps by reading this article first. Consider how API consumers have a similar journey, especially when integrating APIs and microservices designed across multiple teams.
Sequence diagramming
Overview: Those familiar with UML likely recall sequence diagrams. A sequence diagram shows interactions between objects, services, or systems across a period of time to realize a scenario.
Example: Below is an example of a front-end API calling into several microservices:
The diagram demonstrates how a user’s interactions with a front-end interface calls into various APIs, which in turn require orchestrating calls between one or more microservices.
Usefulness: Sequence diagrams are a useful technique to determine how your API endpoints will be used to realize their intended use cases. I often recommend using sequence diagrams in the process of API modeling and design as a method for validating your API design by diagramming the various endpoints required to realize your use cases or user stories. They are also a great way to demonstrate microservice orchestration as you begin to decompose your APIs into services, helping you to detect incorrect microservice boundaries when it is still easy to adjust them.
More Info: Mike Amundsen’s slides from his recent talk, “Using Web Sequence Diagrams with your APIs” is a great demonstration of combining sequence diagrams with an iterative design process to design your APIs.
Readme-driven documentation
Overview: Before you code your API or microservice, consider writing Readme-style documentation to demonstrate how one or more of your API endpoints will be used to execute a specific workflow. Readme-style docs provide the API design with an outside-in view of how your APIs or services will be used. If you cannot easily document the common use cases in a Readme-style format, then it should signal that your design is more complex than necessary. In the process of writing this documentation, you may realize that some necessary API endpoints are missing.
Most developers look for a quick start guide or tl;dr approach to using a CLI or helper library. Using this design technique helps you to write that documentation, while validating your API design meets your consumers’ needs.
Example: Below is an example from the AWS CLI documentation that shows some common use cases for interacting with S3:
As you can see, the documentation doesn’t have to be a large task. Simply capturing common use cases and how to solve them is sufficient.
Usefulness: Readme-driven design is useful to communicate how you think your API will solve consumer problems. By sharing a high-level API design and a simple Readme demonstrating some common use cases, you will gain an outside-in perspective on your API design and also help stakeholders provide valuable feedback on your API design beyond your endpoint reference documentation.
More Info: Tom Preston-Werner’s post on “Readme Driven Development” is a great primer on the subject. Keith Casey’s post on the “The Three Styles of API Documentation” provides a deeper discussion on how documentation helps with design and production support.
Which technique should you use?
Not every technique is the best fit for you. If you try to use them all, you will be stuck in “analysis paralysis” and never deliver your API. Below are some recommendations for when you may wish to use each technique:
- Use event storming when you need a shared understanding of the processes that drive an area of a product or business
- Use API modeling when you need help identifying API resources or when you need to map workflow processes into API designs
- Apply API context mapping, Readme-driven documentation, and/or sequence diagrams to ensure your API design supports end-to-end integration with upstream/downstream dependencies
Conclusion
As you can see, there is no single way to design an API. The techniques I’ve presented are the common ones that I have used with teams to help them through the API design process. You may wish to try some or all of them to determine which technique, or combination of techniques, work best for you.
