Some APIs need to offer an operation to convert a particular file format to another, e.g. converting a TIFF to a PNG. This doesn’t fit the typical JSON-based request common with REST-based APIs. This pattern offers options that build upon HTTP while preventing the need to BASE64 encode binary content within a JSON request. Let’s take a look at a few approaches to this pattern to support file uploads in a REST-based API. As well as covering how to send a file via a REST API, we’ve included a note below on the need to secure APIs from upload vulnerabilities.
This is a multi-part series on API design guidance, where we take a look at tricks and hidden troubles in API designs and how to avoid them. While these articles may not be exhaustive, they will serve to identify common patterns and anti-patterns in API design. The full series includes:
How to send image files in REST API
If your file upload API issues relate to images, there is a simple solution – direct file uploads. You can use this to solve the problem of how to send an image file in a REST API. It’s a neat solution that doesn’t add too much overhead and is the simplest of our REST API file upload solutions to implement, covering most use cases.
Direct file upload
The way to do it is by using the HTTP Content-Type header on the request to set the proper content. Rather than JSON, the client may push a JPG, PNG, PDF, TIFF, or other binary content to the API directly using the PUT method:
PUT /profile/1234/image HTTP/1.1
Content-Type: image/jpeg
Content-Length: 284
raw image content
This is the most straightforward method of accepting uploaded content and is recommended for most cases. That said, the value of the direct file upload will depend on precisely what it is you need to achieve.
How to pass multipart file in REST API
Not all RESTful API file upload needs can be met by direct file uploads. If you need to implement a REST API upload for multiple files, for example, or handle a REST upload file with metadata, then you will need a multipart solution. We’ve provided a couple of different options below, so that you can select the method that best meets your needs. As such, if you’re wondering how to pass a multipart file in a REST API, read on. Naturally, we abide by REST API file upload best practices in each instance, so that you can pass multipart files with minimal hassle and maximum efficiency.
Option #2: multipart HTTP request
If the API must support uploading multiple files at once or supporting associated metadata in the same request, the HTTP multipart content approach is recommended. HTTP multipart content is a formal specification that allows requests to upload multiple images or a combination of images and JSON metadata:
POST /profile/1234/images HTTP/1.1
Content-Type: multipart/form-data; boundary=MultipartBoundry
Accept-Encoding: gzip, deflate
--MultipartBoundry
Content-Disposition: form-data; name="image"; filename="12348024_1150631324960893_344096225642532672_n.jpg"
Content-Type: image/jpeg
rawimagecontent.goes.here
--MultipartBoundry
Content-Disposition: form-data; name="category"
my-category
--MultipartBoundry
Content-Disposition: form-data; name="location"
23,-75
--MultipartBoundry--
As you can see, composing requests by the consumer and parsing them on the server may not be as trivial as a typical HTTP request with a single content type. However, there are usually helper libraries that ensure that both the request and response may be reliably parsed and validated with minimal effort.
Option #3: two-step metadata + upload
If the typical API client interaction needs multiple steps, this option is useful. Below is an example of submitting image meta data first using the POST method and a 201 Created response:
POST /profile/1234/image HTTP/1.1
Content-Type: application/json
Content-Length: 284
{ "tags": [...], ... }
The response includes a new resource representation for the metadata submitted at creation, along with a hypermedia link of where to upload the image:
HTTP/1.1 201 Created
Location: /profile/1234/image
{
"id": ...,
"tags": [...],
"_links": [
{"rel":"self", "url":"/profile/1234/image" },
{"rel":"imageUpload", "url":"/profile/1234/image/file" },
...
]
}
This is a more RESTful way to approach the process, as hypermedia links help to drive the workflow as a multi-step process:
PUT /profile/1234/image/file HTTP/1.1
Content-Type: image/jpeg
Content-Length: 284
raw image content
However, the multi-step process prevents a single transactional wrapper around the file upload process and may also be more complex than your target audience would prefer.
Is there a maximum upload size for a REST API?
Maximum upload size is essentially unbounded for a REST API – it’s actually up to the consumer. Let’s use Tyk as an example of this.
With Tyk, the maximum throughput that can be sent in one month ranges from 100GB on our Starter plan to 10TB on our Scale plan. That’s how much data you can send in requests. The consumer can then average that down to work out their own maximum size for a single request. They might calculate, for example, that their maximum upload size should be 10KB. Of course, the higher the amount of throughput, the more likely you are to see longer wait times and performance impacts, so it’s important to take that into consideration.
Avoid Base64 encoding whenever possible
Historically, protocols such as XML-RPC and SOAP encouraged the use of BASE64 encoding of binary data to force-fit the content inside of an XML-based request payload. Since REST-based APIs build directly upon HTTP, there is no longer a need to fall back to this technique. Avoid encoding binary content when possible, as it requires more CPU and memory usage on both the server and client, complicates integration efforts by developers, and re-creates many of the existing HTTP capabilities offered by available HTTP-related RFCs.
Failing to secure APIs from upload vulnerabilities
Not every file uploaded will be what you expect. Some file uploads may be trying to inject exploits that allow malicious parties to access internal or user resources. Always validate the content properly before making it available, especially for profile image uploads or theme customisation files such as CSS/HTML/JS.
Submitting a URL for the file’s location is recommended by some API designers, but has a severe security drawback – the potential for server side request forgery (SSRF). SSRF enables a malicious attacker to perform network scans, view internal resources, and other exploits. Use caution if you decide to support external URLs to be used to fetch content. For more on this exploit, check out the article, “What is the Server Side Request Forgery Vulnerability & How to Prevent It?”.
Be sure to apply the proper OWASP file upload guidelines for file uploads to prevent introducing vulnerabilities for what should be a simple file conversion service.
Wrap up
While many APIs offer JSON or XML-based content types, the HTTP specification provides affordances for supporting multiple content types within the same HTTP API. By using content negotiation and an easy-to-understand design, your API can support file uploads easily.
If you’ve still got no APIs at all, check out our step-by-step guide to creating them.
