Des Rivières, J. Evolving Java-based APIs. 2007

This document is about how to evolve Java-based APIs while maintaining compatibility with existing client code.

Yann-Gaël Guéhéneuc, 2014/02/06

This document is very interesting because it tells all about Java-based API evolution in one document (okay, in three separate pages) without wasting space but going straight to the point. It starts by defining some basic concepts, such as Component, Component API, and Client and different levels of compatibility between an old and a new version of the Component API wrt. Clients:

• Contract compatibility: API changes must not invalidate formerly legal Client binaries;
• Source code compatibility: API changes must not prevent Client code to compile against the new release of the Component;
• Binary code compatibility: pre-existing Client binaries must link and run with the new release of the Component without recompiling;
• Data compatibility: Component new implementation must contain readers for current and all past versions of the data format.

The document emphasises that source code compatibility is not necessary, because it argues that most API changes can be caught by the compiler and corrected by the developers. It is correct if the documentation regarding the transition between old and new API is sufficiently explicit for developers to easily find how to change their code.

Then, the document put forward the assumptions that “every aspect of the API matters to some Client” and that, therefore, the API must be compatible in principle and in practice: in principle, “all pre-existing Clients must still be legal according to the contracts spelled out in the [new] Component API specification”; in practice, “[p]re-existing Client binaries must link and run with the new release of the Component without recompiling”.

Contract compatibility is the highest form of compatibility and the one that must be sought when evolving a Component. Contract compatibility depends on the role played by the Component and the Client and what kind of changes occurred between the two versions of the API. The roles are “caller” and “implementor”:

• a Component can offer an API and its implementation, it is the implementor; the Client access the API, it is the caller;
• a Component can offer a “callback” API, it is the caller; the Client must implement the API, it is the implementaor.

and the question is “For roles played by Clients, would the […] API change render invalid a hypothetical Client making legal usage of the existing API?” The document then provides a table (augmented here with the addition of API methods to assess the contract compatibility of a change:

Then, the document goes on enumerating, for each possible types of changes to API-related constituents of a Component, which ones are binary compatible and which ones are not, see the original document for exhaustive lists. The API-related constituents include:

• API packages;
• API interfaces (methods, fields, and type members);
• API classes (methods and constructors, fields, type members).

The document also discusses annotations on API constituents, turning non-generic types into generic ones, and Java reflection. Regarding reflection, it emphasises that “[n]o additional provision are made for clients that access the API using Java reflection”. It also explains that most reflection APIs are safe, except Class.getDeclaredXXX(…) methods, because they are “dependent on the exact location of [a constituent] and include non-public [constituents] as well as public ones”.

Finally, the document includes several suggestions to design better API or make it easier to change them:

• Clients must only use published (not internal), API (even if other constituents are public);
• “[O]bsolete API elements are notoriously difficult to get rid of”;
• “[A]ll gender changes involving classes, enums, interfaces, and annotation types break binary compatibility […]”;
• “[A] pre-existing Client subclass of an existing implementation might still provide a pre-existing implementation [of a newly added method, by sheer luck]”;
• “API interfaces that Clients should implement should not include fields”, which is not always possible because enumeration constants used in switch statements;
• “[A]dding type parameters to a previously unparameterised type retains compatibility”;
• “API classes should always explicitly declare at least one constructor”;
• “Client code can use the values() method to determine the ordinal positions of enums constants”;
• “Adding and deleting checked exceptions declared as thrown by an API method does not break binary compatibility; however, it breaks contract compatibility (and source code compatibility)”;
• “Java compiler always inline the value[s] of constant fields […] this does not meet the objective of changing the API field's value”;
• “As long as the erasure looks like the corresponding declaration prior to generification, the change is binary compatible with existing code”;
• “[G]enerifying an existing API is something that should be considered from the perspective of the API as a whole rather than piecemeal on a method-by-method or class-by-class basis”;
• “[T]ag all stored data with its format version number”;
• “Deprecate and forward” and-or “Start over in a new package” and-or "COM style" and-or “Mak[e] obsolete hook method final”.

Also, the document provides the simplest, clearest explanation of Java 1.5+ type erasure mechanism, that I reproduce here for the sake of beauty:

A raw type is a use of a generic type without the normal type arguments. For example, “List” in the declaration statement “List x = null;” is a raw type since List is a generic type declared “public interface List<E> …” in JDK 1.5. Contrast this to a normal use of List which looks like “List<String> x = null;” or “List<?> x = null;” where a type augument (“String”) or wildcard is specified.

The term erasure is suggestive. Imagine going through your code and literally erasing the type parameters from the generic type declaration (e.g., erasing the “<E>” in “public interface List<E> …”) to get a non-generic type declaration, and replacing all occurrence of the deleted type variable with Object. For type parameters with type bounds (e.g., “<E extends T1 & T2 & T3 & …>”), the leftmost type bound (“T1”), rather than Object, is substituted for the type variable. The resulting declaration is known as the erasure.“