API Design Guidance: File Upload

Some APIs need to offer an operation to convert a particular file format to another, e.g. converting a TIFF to a PNG. This doesn’t fit the typical JSON-based request common with REST-based APIs. This pattern offers options that build upon HTTP while preventing the need to BASE64 encode binary content within a JSON request. Let’s take a look at a few approaches to this pattern to support file uploads in a REST-based API.

This is a multi-part series on API design guidance, where we take a look at tricks and hidden troubles in API designs and how to avoid them. While these articles may not be exhaustive, they will serve to identify common patterns and anti-patterns in API design.

Option #1: Direct file upload

The simplest option for this pattern is to use the HTTP Content-Type header on the request to set the proper content. Rather than JSON, the client may push a JPG, PNG, PDF, TIFF, or other binary content to the API directly using the PUT method:

PUT /profile/1234/image HTTP/1.1
Content-Type: image/jpeg
Content-Length: 284

raw image content

This is the most straightforward method of accepting uploaded content and is recommended for most cases.

Option #2: Multipart HTTP request

If the API must support uploading multiple files at once or supporting associated metadata in the same request, the HTTP multipart content approach is recommended. HTTP multipart content is a formal specification that allows requests to upload multiple images or a combination of images and JSON metadata:

POST /profile/1234/images HTTP/1.1
Content-Type: multipart/form-data; boundary=MultipartBoundry
Accept-Encoding: gzip, deflate

--MultipartBoundry
Content-Disposition: form-data; name="image"; filename="12348024_1150631324960893_344096225642532672_n.jpg"
Content-Type: image/jpeg

rawimagecontent.goes.here
--MultipartBoundry
Content-Disposition: form-data; name="category"

my-category
--MultipartBoundry
Content-Disposition: form-data; name="location"

23,-75
--MultipartBoundry--

As you can see, composing requests by the consumer and parsing them on the server may not be as trivial as a typical HTTP request with a single content type. However, there are usually helper libraries that ensure that both the request and response may be reliably parsed and validated with minimal effort.

Option #3: Two-step metadata + upload

If the typical API client interaction needs multiple steps, this option is useful. Below is an example of submitting image meta data first using the POST method and a 201 Created response:

POST /profile/1234/image HTTP/1.1
Content-Type: application/json
Content-Length: 284

{ "tags": [...], ... }

The response includes a new resource representation for the metadata submitted at creation, along with a hypermedia link of where to upload the image:

HTTP/1.1 201 Created
Location: /profile/1234/image

{
   "id": ...,
   "tags": [...],
   "_links": [
     {"rel":"self", "url":"/profile/1234/image" },
     {"rel":"imageUpload", "url":"/profile/1234/image/file" },
     ...
   ]
}

This is a more RESTful way to approach the process, as hypermedia links help to drive the workflow as a multi-step process:

PUT /profile/1234/image/file HTTP/1.1
Content-Type: image/jpeg
Content-Length: 284

raw image content

However, the multi-step process prevents a single transactional wrapper around the upload process and may also be more complex than your target audience would prefer.

Avoid Base64 Encoding Whenever Possible

Historically, protocols such as XML-RPC and SOAP encouraged the use of BASE64 encoding of binary data to force-fit the content inside of an XML-based request payload. Since REST-based APIs build directly upon HTTP, there is no longer a need to fall back to this technique. Avoid encoding binary content when possible, as it requires more CPU and memory usage on both the server and client, complicates integration efforts by developers, and re-creates many of the existing HTTP capabilities offered by available HTTP-related RFCs.

Failing to Secure APIs From Upload Vulnerabilities

Not every file uploaded will be what you expect. Some file uploads may be trying to inject exploits that allow malicious parties to access internal or user resources. Always validate the content properly before making it available, especially for profile image uploads or theme customisation files such as CSS/HTML/JS.

Submitting a URL for the file’s location is recommended by some API designers, but has a severe security drawback – the potential for server side request forgery (SSRF). SSRF enables a malicious attacker to perform network scans, view internal resources, and other exploits. Use caution if you decide to support external URLs to be used to fetch content. For more on this exploit, check out the article, “What is the Server Side Request Forgery Vulnerability & How to Prevent It?”.

Be sure to apply the proper OWASP file upload guidelines for file uploads to prevent introducing vulnerabilities for what should be a simple file conversion service.

Wrap-Up

While many APIs offer JSON or XML-based content types, the HTTP specification provides affordances for supporting multiple content types within the same HTTP API. By using content negotiation and an easy-to-understand design, your API can support file uploads easily.