A fair amount of software development involves interacting with APIs. I want to present some issues that I considered for a tagging API that I recently designed. I intend to only present the thinking process by discussing what capabilities each decision enabled or excluded. I do not intend to present guidelines or best practices, I am no expert on API design.

Although I say REST APIs, we can call them HTTP APIs here. For this article, I only care that they access some resources over HTTP without necessarily conforming to all the REST architectural constraints.

Why flexibility matters

Over-specified HTTP APIs with RPC-like names have been chided often for good reasons. They often leak implementation details of the system, even up to function names, and can tempt their consumers to share in such coupling. We observe this whenever a client requests a new API to do something like an existing capability, but just a little different.

Compare

POST /books/book-3/rename HTTP 1.1
Content-Type: application/json

{"new_name": "Book name"}


POST /books/book-3/add-tags HTTP 1.1
Content-Type: application/json

{"new_tags": ["#new_tag"]}

to

PATCH /books/book-3 HTTP 1.1
Content-Type: application/json

{"name": "Book name", "tags": []}

The other extreme – APIs so general that they can do only little (assuming they don't go off to some hidden business) – is rarer in HTTP. It is more common in library code. A good example in Go syntax is a function like the following.

type Runner interface {
    Run(func() error)
}

All Runner.Run can do is call its argument and handle the error. It is maximally flexible: we can wrap any function into one that returns an error and do whatever we want. But the use of such APIs is best limited to small scopes according to the rule of least power because they do not communicate much, and programming is about communication.

I prefer having APIs that are neither over-specified nor generalised into a semantic black hole.

Tagging system case study

Suppose we have a group study application where readers can discuss authors and books that they're reading together. Our system doesn't manage books, only group discussions. Readers want to organise their reading into folders like they would on their computers. They own the books, we help them organise readings with other readers. How can we give them folders?

Design requirements

We want to support these operations:

1. create, delete, or rename a folder;

2. move readings into/out of a folder; and

3. list readings by folders.

Let us look try out a few design options for these operations and see what they give us. Here are some properties I have in mind.

1. The mutating operations listed above should be idempotent.

2. We should have only as many URL schemas as needed, and no more.

3. If an operation is a special case of a more general operation, we should support the more general operation in a way that doesn't obscure the special case.

4. The resulting design should be resource-oriented.

We can start from (4) and list what resources we expect to have. Let us assume only books, discussions, and folders are the resources we care about. Books and discussions can be grouped in folders.

A first pass: over-specified API

To support the operations required, we may have API "endpoints" as follows.

1. POST /folders/ for creating folders; DELETE /folders/:id for deleting a folder; and POST /folders/:id/rename (or some variation of this) for renaming a folder.

2. POST /folders/:id/readings/ with the reading in the body, to add a reading to a folder. DELETE /folders/:id/readings/:id to delete a reading from a folder.

3. GET /folders/:id/readings to list readings in a folder.

There can be other schematic variations of these. For example, in (2), we can use POST /folders/add-reading with the folder ID and the reading ID in the body instead. I'm sure you've seen similar APIs before.

These get the job done, but let's see how well they meet the desired properties.

Idempotency

Generally, the POST method is not cacheable. Idempotency is not assumed, so we cannot get the full benefit where an HTTP client can safely retry it. Idempotency with POST is an implementation detail hidden away from the client (as in how webhook producers do not care about the changes made by the consumers, only about a success status code).

With our POST /folders/ API, the client most likely cares about the result and its metadata (like creation time). If we try to make it idempotent, we have to precisely define its semantics. Do we create a new folder always, or only if the request time is within a short duration of the existing folder's creation time? Do we signal a duplicate (in which case it's not cacheable)? Whatever we do here, we cannot change the fact that POST is not idempotent by definition.

Our DELETE and GET requests can easily be idempotent, but POST /folders/:id/readings/ has the same problems as POST /folders/.

Minimum URL schemas

We can see that we introduced six URL paths, one for each operation. This is indeed the maximum we can expect. It seems reasonable, but it is somewhat suboptimal to have our URL paths grow linearly with the system. That means we may have as many URLs as there are operations supported in the system, revealing its underlying complexity.

Special-casing of operations

If you squint a bit, you'll find that some of the operations we created URLs for are special cases of a more general operation. If you squint harder, you'll notice that the previous property (minimum URL schemas) is a special case of this one!

Take adding and deleting readings from folders. These two can be represented by one action – setting the folder for a reading, or setting the readings in a folder, depending on how the relationships go. We could have one idempotent operation here.

Furthermore, GET /folders/:id/readings assumes that a reading has been assigned a folder. It precludes the existence of readings without folders, which is a special case that we may want to exist. This can be designed away by always having a default folder, but we do not need to overfit the API. Also, listing readings by a single folder can be thought of as a special case of listing readings by various folders. The API should offer enough flexibility for change. Under this design, if we tried to support readings without folders or listing by multiple folders, we'd need new APIs for each case.

Resource orientation

We made a good try for resource orientation, except with POST /folders/:id/rename. While rename (as an operation) can be a resource, it doesn't fit well in our domain. It's a phantom resource – it appears only in the input of our API and has no material representation within the domain, just as phantom types do not appear within the body of a type definition or a function. Phantom resources should serve as a hint that some other resource can be better represented.

Trying harder for some REST

Let us look at the operations in a different shape. For creating folders, we'll make the operation idempotent and cacheable. We'll collapse the folder setting operations into a general one, and we'll remove the phantom resource in the rename operation. We'll have the following operations.

1. PUT /folders/:id to create folders; DELETE /folders/:id for deleting a folder, and PATCH /folders/:id for changing folder properties like the name.

2. PUT /readings/:id with the folder assignment in the body, for setting a reading's folder.

3. GET /readings?id=:folder_id for listing readings by folder.

We are down to five URLs that support the six required operations. Actually, we have addressed the shortcomings of the previous design and can now support even more operations without adding new URLs. Let's see how that works.

Idempotency

The semantics of the PUT method include idempotency. It is meant to replace the representation of a resource entirely. Replacing a resource with another should produce the same result, however many times we make the replacement. Similarly, deleting a resource multiple times should have the same effect as deleting it once. Thus, creating and deleting folders is now idempotent.

The PATCH method is technically idempotent with regard to the update being made. But interleaved PATCH requests can result in different representations after each one, so the same PATCH request repeated may put the resource in entirely different states. That is usually good enough for me. However, I'll reuse PUT /folders/:id to update a folder just to fully satisfy this property. That brings us to four URLs satisfying the six listed operations.

Special-casing of operations

We've generalised creating and renaming folders using a single API. That API can also handle changes to other folder properties in the future, where appropriate.

We've also collapsed assigning and unassigning readings to folders into a single API. This API can handle placing a reading in multiple folders if we choose to support that; all that will change is the request body. Now we have fewer URLs than the supported operations, only as many URLs as we need.

Finally, we parameterised the folder listing using a query parameter. This lets us list readings by many folders if we want to. We can also define more combinations of query parameters if necessary.

For queries, query parameters are more general than path parameters.

Applicability

Like every design guideline, this too must be contextualised. You may have constraints that limit the application of this thought pattern. If your team favours a different style, that's a valid constraint, however undesirable you find it. It is in the same class as your team choosing one programming language over others regardless of the technical merits.

Enforced segregation of operations is another good reason to not apply this thought pattern. Sometimes things are not quite the same, even if they look similar. Two API paths may have different operational requirements such as logs and traces distinguished by their URLs, for example.

In some cases, updates to different parts of a resource may require several rules. Such logic separation indicates that the updates are several operations which you may reify as resources. This isn't an exception to the pattern, rather it's an application of general resource-oriented design under REST.

In general, use common sense and consider the operations in their full context. Try to not over-specify or over-generalise. A middle ground that lets you evolve the system with more ease and conceptual cohesion is the goal.

If you have thoughts on this topic, please share them in the comments. I can learn from your feedback. Also share this article in your circles to help my visibility, and consider following my blog too. I wish you better REST 🙃.