When building an API, a good test strategy ensures that your API both works correctly and meets the promises of its definition. Yet, testing your API isn’t the same as the way we test our code and our user interface. We’ve gathered some tips to help you start and sustain your API testing approach.
Tip 1: Test your code and your contract
Those familiar with test-driven development (TDD) are familiar with testing your code using an approach called unit testing. Unit tests are the fastest tests to implement and execute. These tests focus on whether our code is working properly and are an essential building block for your API test strategy. They are used to ensure that each part of your API is operating as expected.
You may also use your unit tests to reproduce a discovered bug, verify the bug has been fixed, and to prevent regression of the bug as the code continues to change and evolve. Over time, your unit tests become a representation of the hard work and wisdom earned supporting a production API.
Moving beyond testing your code is testing your API contract using functional testing. Functional testing, sometimes referred to as contract testing, is used to verify that each API endpoint meets the expected behavior and honors the API’s defined contract for the consumer. This contract is typically defined using the OpenAPI Specification (fka Swagger), API Blueprint, or RAML. Implementing contract testing ensures that your team has implemented the API fully and to spec.
Tip 2: Test your customer outcomes
It is important to verify that your API not only functions properly and to its specification, but that it also produces the desired outcomes. Acceptance testing verifies that we are meeting the needs of our end users. Sometimes called solution-oriented testing, acceptance testing ensures that the API as a whole supports the intended use cases that it was designed to solve.
Acceptance testing executes a series of pre-determined paths that best represent the kinds of interactions your end-users will have with your API, verifying that a specific end state is achieved. When pressed for time to build tests for your API, start with acceptance testing first. Then begin to layer in contract and unit testing to supplement your tests, or to isolate specific bugs identified during acceptance testing.
Tip 3: Start API testing before you start coding
As many developers know, building an API can be a blocking task for QA teams building a test suite for the API. You can’t test an API before it is implemented, can you? Yes, you can. In a recent article titled “Moving to API design-first in an Agile world”, we discussed an approach for designing your API before the implementation. To reiterate, here is a summary of the steps:
- Discover the capabilities that need to be delivered
- Model and design our API to meet the desired behaviour
- Document the API design with feedback from stakeholders
- Product a mock version of the API
- Implement and test the API and release into production when ready
The first four steps provide enough detail about the API to-be-built that you can proceed with building your API test suite before, or in tandem, with your API implementation. This can shorten the time-to-market while providing valuable design feedback before and during the early stages of the development process, when the cost of fixing bugs is much lower.
Tip 4: Test your API separate from your UI
Some teams suggest that building dedicated tests for your API is wasteful. They attempt to make the case that their UI tests cover the API, since the UI calls the API. However, this is not the case. Instead, the UI only tests the API only as far as the UI exercises the API. This means that if the UI is performing client-side validation of user input, then you would never test the API’s ability to handle bad data. While some may say that this level of testing is sufficient, they may be forgetting the recommendation of OWASP regarding trusting input data: Don’t. From the OWASP “Dont Trust User Input” page:
A user or client will not always submit data your application will expect. By building robust applications that do not trust user input by default, you ensure the application will be able to handle unexpected data gracefully. Examples of user input include: form data, client information such as user-agent strings, cookies, referer, etc. Anything that is submitted in an HTTP request should be considered user input.
To put it another way, our goal is to ensure that our API is able to handle a multitude of good and bad values that may be submitted outside of a specific user interface. If we depend only upon our UI, which likely tries to prevent bad user input, then our API isn’t fully tested and therefore may not be sufficiently protected from malicious attacks.
Tip 5: Testing doesn’t stop once deployed
Testing doesn’t stop once we have deployed our API. Runtime testing, often through API monitoring and reporting tools, ensures that your API is performing properly in production.
Why is it important to monitor APIs? APIs can and often do provide the primary interface for applications, thereby playing the role of a dependency. It is critical for your API to be available and meeting the service level agreements (SLAs) that the company has agreed to (and could yield a negative financial result if missed). Runtime testing ensures that we detect when recent deployments result in an increased number of errors – something our automated tests may have missed.
Security testing aims to answer the question, “Does the API protect against attacks and PII leaks?” Security is a process, not a product, and a continual one at that. Even with the ever-changing security landscape, we can still lay down an evolving set of best practices, and test against them. Don’t forget to develop healthy practices for monitoring your API surface area for malicious attacks.
Tip 6: Maximise your testing tools
Agreeing on an approach and tooling for API testing is key to the success of the programme’s supportability and enabling future development. There are a wide range of tools available to aid in acceptance testing, both open source and proprietary, that support or can easily be adapted to API testing:
Additionally, there are a variety of libraries that can help convert your OpenAPI or Blueprint definitions into a beginning set of functional tests that verify that your API meets its contract.
Tip 7: Make API testing essential
Too often, teams choose to take shortcuts when time is short and this typically involves poor or no API testing. Like documentation, testing is often seen as a nice-to-have in the development process. However, we should view testing and documentation as essential steps to calling your API “done” and ready to deploy. Otherwise, we are creating opportunities for bugs to creep into our user experience. Worse, it could open your company up to malicious attacks through your API.
Wrap-up
By incorporating these tips for API testing, we can have greater confidence that our API is correct (i.e. meets its contract), meets the desired outcomes of our users, is operating properly, and is less likely to be the subject of a security breach. By making API testing an essential part of your overall development process, we can depend more-and-more on our tests to signal when bugs have been introduced – before they are deployed into production.
For information going beyond Tips for testing to Tips for building, you can read more on Tips for building an effective enterprise API programme.
