The Backwards Compatibility Problem Statement

As usual, let’s conceptually define ‘backwards compatibility’ before we start.

Backwards compatibility is a feature of the entire API system to be stable in time. It means the following: the code which developers have written using your API, continues working functionally correctly for a long period of time. There are two important questions to this definition, and two explanations:

1. What does ‘functionally correctly’ mean?
   It means that the code continues to serve its function, e.g. solve some users’ problem. It doesn’t mean it continues working indistinguishably: for example, if you’re maintaining a UI library, changing functionally insignificant design details like shadow depth or border stoke type is backwards compatible, whereas changing visual components size is not.
2. What does ‘a long period of time’ mean?
   From our point of view, backwards compatibility maintenance period should be reconciled with subject area applications lifetime. Platform LTS periods are a decent guidance in the most cases. Since apps will be rewritten anyway when the platform maintenance period ends, it is reasonable to expect developers to move to the new API version also. In mainstream subject areas (e.g. desktop and mobile operating systems) this period lasts several years.

From the definition becomes obvious why backwards compatibility needs to be maintained (including taking necessary measures at the API design stage). An outage, full or partial, caused by the API vendor, is an extremely uncomfortable situation for every developer, if not a disaster — especially if they pay money for the API usage.

But let’s take a look at the problem from another angle: why the maintaining backwards compatibility problem exists at all? Why would anyone want to break at? This question, though it looks quite trivial, is much more complicated than the previous one.

We could say the we break backwards compatibility to introduce new features to the API. But that would be deceiving: new features are called ‘new’ just because they cannot affect existing implementations which are not using them. We must admit there are several associated problems, which lead to the aspiration to rewrite our code, the code of the API itself, and ship new major version:

• the code eventually becomes outdated; making changes, even introducing totally new functionality, is impractical;
• the old interfaces aren’t suited to encompass new features; we would love to extend existing entities with new properties, but simply couldn’t;
• finally, with years passing since the initial release, we understood more about the subject area and API usage best practices, and we would implement many things differently.

These arguments could be summarized frankly as ‘the API developers don’t want to support the old code’. But this explanation is still incomplete: even if you’re not going to rewrite the API code to add new functionality, or you’re not going to add it at all, you still have to ship new API versions, minor and major alike.

NB: in this chapter we don’t make any difference between minor versions and patches: ‘minor version’ means any backwards compatible API release.

Let us remind that an API is a bridge, a meaning of connecting different programmable contexts. No matter how strong our desire to keep the bridge intact is, for our capabilities are limited: we could lock the bridge, but we cannot command the rifts and the canyon itself. That’s the source of the problems: we can’t guarantee our own code wouldn’t change, so at some point we will have to ask the clients to change their code.

Apart from our own aspirations to change the API architecture, three other tectonic processes are happening at the same time: user agents, subject areas, and underlying platforms erosion.

Consumer applications fragmentation

When you shipped the very first API version, and first clients started to use it, the situation was perfect. There was only one version, and all clients were using just it. When this perfection ends, two scenarios are possible.

1. If the platform allows for fetching code on-demand, like the good old Web does, and you weren’t too lazy to implement that code-on-demand (in a form of a platform SDK — for example, JS API), then the evolution of your API is more or less under your control. Maintaining backwards compatibility effectively means keeping the client library backwards compatible. As for client-server interaction, you’re free.
   It doesn’t mean that you can’t break backwards compatibility. You still can make a mess with cache control headers or just overlook a bug in the code. Besides, even code-on-demand systems don’t get updated instantly. The author of this book faced the situation, when users were deliberately keeping a browser tab open for weeks to get rid of updates. But still, you usually don’t have to support more than two API versions — the last one and the penuiltimate one. Furthermore, you may try to rewrite previous major version of the library, implementing it upon the actual version API.
2. If code-on-demand isn’t supported or is prohibited by the platform, as in modern mobile operating systems, then the situation becomes more severe. Each client effectively borrows a snapshot of the code, working with your API, frozen at the moment of compilation. Client application updates are scattered over time at much more extent than Web application updates. The most painful thing is that some clients will never be up to date, because of one of three reasons:

• developers simply don’t want to update the app, its development stopped;
• users don’t wont to get updates (sometimes because users think that developers ‘spoiled’ the app in new versions);
• users can’t get updates because their devices are no longer supported.

In modern times these three categories combined could easily constitute tens of percent of auditory. It implies that cutting the support of any API version might be remarkable — especially if developers’ apps continue supporting a more broad spectrum of platforms than the API does.

You could have never issued any SDK, providing just the server-side API, for example in a form of HTTP endpoints. You might think, given your API is less competitive on the market because of a lack of SDKs, that the backwards compatibility problem is mitigated. That’s not true: if you don’t provide an SDK, then developers will either adopt an unofficial one (if someone bothers to make it), or just write a framework of themselves — independently. ‘Your framework — your problems’ strategy, fortunately or not, works badly: if developers write poor quality code upon your API, then your API is of a poor quality itself. Definitely in the view of developers, possibly in the view of end users, if the API performance within the app is visible to them.

Certainly, if you provide a stateless API which doesn’t need client SDKs (or they might be auto-generated from the spec), those problems will be much less noticeable, but not fully avoidable, unless you never issue any new API version. Otherwise you still had to deal with some distribution of users by API and SDK versions.

Subject area evolution

The other side of the canyon is the underlying functionality you’re exposing via the API. It’s, of course, not static and somehow evolves:

• new functionality emerges;
• older functionality shuts down;
• interfaces change.

As usual, the API provides an abstraction to much more granular subject area. In case of our coffee machine API example one might reasonably expect new models to pop up, which are to be supported by the platform. New models tend to provide new APIs, and it’s hard to guarantee they might be included preserving the same high-level API. And anyway, the code needs to be altered, which might lead to incompatibility, albeit unintentional.

Let us also stress that low-level API vendors are not always as resolute regarding maintaining backwards compatibility (actually, any software they provide) as (we hope so) you are. You should be prepared that keeping your API in an operational state, e.g. writing and supporting facades to the shifting subject area landscape, will be your problem, and rather a sudden one.

Platform drift

Finally, there is a third side to a story — the ‘canyon’ you’re crossing over with a bridge of your API. Developers write code which is executed in some environment you can’t control, and it’s evolving. New versions of operating system, browsers, protocols, language SDKs emerge. New standards are being developed, new arrangements made, some of them being backwards incompatible, and nothing could be done about that.

Older platform versions lead to the fragmentation, just like older apps versions do, because developers (including the API developers) are struggling with supporting older platforms, and users are struggling with platform updates — and often can’t update at all, since newer platform versions require newer devices.

The nastiest thing here is that not only incremental progress in a form of new platforms and protocols demands changing the API, but also a vulgar fashion does. Several years ago realistic 3d icons were popular, but since then the public taste changed in a favor of flat and abstract ones. UI components developers had to follow the fashion, rebuilding their libraries, either shipping new icons or replacing old ones. Similarly, right now ‘night mode’ support is introduced everywhere, demanding changes in a broad range of APIs.

Backwards compatibility policy

To summarize the above:

• you will have to deploy new API versions because of apps, platforms, and subject area evolution; different areas are evolving with different rate, but never a zero one;
• that will lead to fragmenting the API versions usage over different platforms and apps;
• you have to make decisions critically important to your API’s sustainability in the customers view.

Let’s briefly describe these decisions and key factors of making them.

1. How often new major API versions should be developed?
   That’s primarily a product question. New major API version is to be released when the critical mass of functionality is achieved — a critical mass of features which couldn’t be introduced in the previous API versions, or introducing them is too expensive. On stable markets such a situation occurs once in several years, usually. On emerging markets new API major versions might be shipped more frequently, only depending on your capabilities of supporting the zoo of previous versions. However, we should note that deploying a new version before the previous one was stabilized (which commonly takes from several months up to a year) is always a troubling sign to developers, meaning they’re risking dealing with the unfinished platform glitches permanently.
2. How many major versions should be supported at a time?
   As for major versions, we gave theoretical advice earlier: ideally, the major API version lifecycle should be a bit longer than the platform’s one. In stable niches like desktop operating systems it constitutes 5 to 10 years. In new and emerging ones it is less, but still measures in years. Practically speaking you should look at the size of auditory which continues using older versions.
3. How many minor versions (within one major version) should be supported at a time?
   As for minor versions, there are two options:

• if you provide server-side APIs and compiled SDKs only, you may basically do not expose minor versions at all, just the actual one: the server-side API is totally within your control, and you may fix any problem efficiently;
• if you provide code-on-demand SDKs, it is considered a good form to provide an access to previous minor versions of SDK for a period of time sufficient enough for developers to test their application and fix some issues if necessary. Since full rewriting isn’t necessary, it’s fine to align with apps release cycle duration in your industry, which is usually several months in worst cases.

We will address these questions in more details in the next chapters. Additionally, in the Section III we will also discuss, how to communicate to customers about new versions releases and older versions support discontinuance, and how to stimulate them to adopt new API versions.

This is the draft for the future ‘HTTP API’ section of the book; the work continues at Github. I’d appreciate if you share it on reddit, for I personally can’t do that.