3 Ways to Make Your API Responses Flexible
Most APIs deliver their responses the exact same way to every client. That’s by design; developers need to know what to expect.
There are plenty of circumstances where what’s returned is not quite what a developer wants, though. Some APIs provide flexibility in the types and amount of data in their responses to help mitigate this.
For example, Netflix realized that to support the many devices that stream its videos, it needed a more flexible approach to its API. In 2012, Netflix’s Daniel Jacobson shared how they created a unique endpoint for every device the company supports. There is a translation layer that queries for the exact data needed by a client. This made sense because they are supporting a “small group of known developers” mostly inside Netflix.
What about when you need to plan for what Daniel calls the “one-size-fits-all REST API?” Here are three different ways you can help developers declare the data they want from your API.
Include/Exclude Certain Fields
An API with a single approach to responses is likely to give developers much more or less data than they need. There are downsides to each situation, for both the developer and the API provider. If an API responds with more data than a developer needs, both sides have paid a tiny latency penalty in the time and bandwidth required to transfer data nobody cares about at the moment. Similarly, if a important field is excluded from the response, a developer may have to make a second request to fill in the missing data.
A bit too much or too little data may seem trivial, but in aggregate they can have a huge impact. An extra API call, at scale, could double the server costs of both sides. Here are how other APIs have approached the issue.
Foursquare
The location recommendations app is built on its own API. We can see the fingerprint of the primary use case within their approach to API responses. Foursquare has compact and complete versions of most objects the API returns. A venue request, for example, always has a name and location, but only shows the date created with a complete venue object response.
The API decides for you which length of response you’ll receive based on the type of data you are requesting. A nice modification would be to let developers override which object they receive.
GitHub
GitHub takes a similar approach as Foursquare with its summary and detailed variants:
When you fetch a list of resources, the response includes a subset of the attributes for that resource. This is the "summary" representation of the resource. (Some attributes are computationally expensive for the API to provide. For performance reasons, the summary representation excludes those attributes. To obtain those attributes, fetch the "detailed" representation.)
In this case, GitHub recommends two requests when you really need the additional data, and that request only gets the details of a single repository. The source control API has identified some fields as “computationally expensive,” which goes beyond latency based on the size of the data. GitHub may have decided that developers don’t need detailed views of multiple repos at a time, or that they don’t want to encourage those use cases.
Asana
Asana’s input/output options let developers override what fields will be included in a response. While Asana always sends the ID field, developers need to specify all the other fields they want when using output options to declare fields.
Google Plus
The other examples in this section make some decisions for the developer about which data is returned, then give the developers other ways to access additional data. Google Plus, on the other hand, defaults to a complete representation, then lets developers name their fields for partial responses.
Expand Certain Fields
Another approach to controlling the amount of data is to allow developers to expand single fields into larger objects. For example, Asana’s input/output options mentioned in the previous section also support expansions. Rather than simply asking for a project ID to be included, developers can request the full representation of a project be expanded within the results.
Similar to included and excluded fields, developers use expansions to get all the data they need in a single API request.
Eventbrite
The responses returned by Eventbrite are basic representations of the objects requested. Even the site’s core object, an event, is only about a dozen fields. An event has a name, description, URL, and a handful of status and timestamp fields. That’s it. No venue or format. No organizer or categories.
Eventbrite has many models that refer to each other, and often you’ll want to fetch related data along with the primary model you’re querying - for example, you’ll want to fetch an organizer along with each event you get back.
The way of doing this in the Eventbrite API is called ‘expansions’: you can specify a list of relationships to additionally fetch with every call.
Eventbrite’s expansions keep responses lean while giving developers the flexibility to request additional data. Ticket options for events would at least double the weight of the response, so Eventbrite is able to save time and bandwidth by requiring it be specifically requested.
The Eventbrite API essentially merges the idea of optional fields with expansions. Some fields, such as categories, are minimally expanded (three versions of the name and a list of subcategories). However, this approach allows for specificity when making requests. For example, when listing a user’s orders (events where they’ve registered), you can choose to expand only the event’s venue by using nested expansions.
HipChat
Just as there is a lot of data within events, chat rooms can have even more. Not only do you have the messages themselves, but each participant has their own wealth of potential data. HipChat provides title expansion options, allowing developers to choose between minimal data and expanded versions.
The API uses Hypermedia links as a way to signal that a field can be expanded. HipChat’s API response may include a “links” property, like so:
"links": {
"self": "https://api.hipchat.com/v2/{resource}/{id}"
}
To expand the resource, you can make the same API call and add expand={resource} to the query string. Now a complete representation of that resource is included within the response.
Alternatively, the developer can make a second call using the self link directly. Here, the expansion saves the additional request.
Go All In With GraphQL
For the ultimate flexibility, consider providing a GraphQL endpoint for developers. Still a nascent technology, it’s being adopted by some big names like Facebook, GitHub, and Intuit. With GraphQL, developers request the data they want based on a pre-defined schema. As the QL in the name implies, it uses a SQL-like query language—though it’s probably more useful to think of queries as filtering results.
Even with the simple example above, from the excellent GraphQL.org guides, you can probably see how it provides much more flexibility than even the most versatile of the earlier RESTful APIs. With that many different options in GraphQL results comes additional complexity, both on the provider and consumer sides.
GraphQL came out of Facebook’s transition from HTML-based mobile apps to native mobile apps. The engineers were frustrated by the process of fitting the company’s graph-oriented data into the resource-focused approach of most APIs. So, in 2012 they built GraphQL, which they made public in 2015.
While Facebook saw results in the frontend of its app, there are backend advantages as well, something that encouraged GitHub to adopt GraphQL:
The REST API is responsible for over 60% of the requests made to our database tier. This is partly because, by its nature, hypermedia navigation requires a client to repeatedly communicate with a server so that it can get all the information it needs. Our responses were bloated and filled with all sorts of *_url hints in the JSON responses to help people continue to navigate through the API to get what they needed. Despite all the information we provided, we heard from integrators that our REST API also wasn’t very flexible. It sometimes required two or three separate calls to assemble a complete view of a resource. It seemed like our responses simultaneously sent too much data and didn’t include data that consumers needed.
To see whether GraphQL works for your API, you could approach it from a single, simple use case. That’s what GitHub did, first implementing emoji reactions on comments as a proof of concept.
Regardless of how you implement flexible responses, it’s important to find out how developers would use your API. That means talking to the consumers, whether internal or external, to understand how they want to use the API. Choose an approach that makes sense for your and your users' needs, and follow these examples of others to build your own flexible API.
Comments powered by Disqus