Compatibility levels and compatibility codes
Last updated at 2:26 pm UTC on 11 January 2006
from Stephan Rudlof
Contents
History
Introduction
If you are a package maintainer and don't like to think about definitions and these somewhat mathematical things, you should nevertheless
- read the Motivation in Compatibility levels in praxis, and
- take a look into the tables thereafter and especially their last column with questions to the maintainer:
- Are these questions clear?
- Could you answer them?
- Do you have better formulations?
- Any suggestions?
It would be nice to get Feedback to the suggested compatibility levels...
Compatibility levels in praxis
Motivation
As a package provider you want to
- have clients use the newest versions fulfilling their needs;
- avoid that people use an older version, since they fear, the newest one could be incompatible with their used older one, though it is not;
- have your package in widespread use, therefore you want to make life simple for users of it, and
- give them precise hints about the compatibility of your package,
- make upgrading to the newest version as easy as possible.
As a package client maintaining a package requiring other ones you want to
- express your special compatibility needs;
- avoid using an older version, just because you don't know enough about the compatibility of the used one;
- automatically upgrade to a newer version, if it is compatible with the used one;
- have as much compatibility as possible with other packages needing a newer version, since they are needing an extended API (not breaking your code, of course);
- have fixes, which cannot break your code, but
- not have fixes (in very rare cases), which can break your code, since you have worked around them (somewhat evil, though!).
As a package installer you want to
- have compatible packages;
- be able to install some new package needing some other package in an advanced version, though some installed package has been written for an older version of the other package, if possible;
- have fixes of installed packages just removing bugs, if possible;
- have automatically upgrading to improved versions of installed packages.
Conclusion: there is a need for some compatibility measure, which is provided by the so called 'compatibility codes' (CCs).
In the following tables 6 possible compatibility levels (CLs) are described: a change in the lower ranked compatibility levels, doesn't affect the compatibility level of the higher ranked ones.
Finding the right compatibility level should be easy, if a package maintainer is able to answer the questions in the last columns.
Expressing compatibility levels from a package maintainer POV
A maintainer wants to express the compatibility of a package to its previous version for others using it. In the following table a proposal for such a classification is given.
Expressing compatibility table
Each row describes a change between a new package and its former version, under the assumption, that the change is less than the ones described in the higher ranks (leading to equality of CC codes there, see From compatibility levels (CLs) to compatibility codes (CCs)).
CL rank | name of package CC number | description | semantics | question to the maintainer publishing a new version |
1. | break | surely incompatible; changed interface semantics, highest rank | this package version is expected to be incompatible with all software written for older or newer ones having another CC number here (at this position), it would be a wonder if any would work | "Do you think this version surely breaks code which has worked with the previous version?" |
2. | major | incompatible; changed interface semantics | this package version is expected to be incompatible with software written for older or newer ones having another CC number here (at this position), but some may work (with same higher ranked (1.) number, of course) | "Do you think this version breaks code which has worked with the previous version, but some may work?" |
3. | minor | downwards compatible; with extensions (e.g. extended interface) and/or improvements of old code (e.g. made faster), but without changes of previous interface semantics | the package has changed, and there are new features, but it is expected to work with software which has worked with packages having lower numbers here (same higher ranked (1., 2.) numbers) | "Do you think this version should work with code which has worked with the previous version?" |
4. | bugfix | downwards compatible; only bug fixes (e.g. better error handling) and/or refactoring (not changing functionality) in old code | 3. and just bug fixes and/or refactoring in old code (same higher ranked numbers) | 3. and: "You may have added code for new features: but have you just made bug fixes and/or refactoring (without changing any algorithm) in any older code?" |
5. | stable | surely compatible | no changes in old code - forbids fixes and refactoring! -, but with extensions (same higher ranked numbers) | "You may have added code for new features: do you think nothing can break, since you have not made any changes in the old code at all? So this version surely works with all code which has worked with the previous version." |
6. | static | no code changes at all, lowest rank | no changed semantics, but e.g. improved class comments (same higher ranked numbers) | "Do you have made no code changes at all (e.g. just changes in documentation), so nothing can break?" |
Note: the names in column 'name of package CC number' have to be interpreted from a compatibility and not a feature POV, which are different!
Subset property ordering
The ordering of these CLs is called subset property ordering:
- a change in the lower ranked compatibility levels should not affect the compatibility of the higher ones;
- package versions belonging to the same lower ranked compatibility level are a subset of package versions belonging to one of the corresponding higher ranked ones (each one includes all its lower ranked ones).
Feedback to the suggested compatibility levels
I would like to get feedback here, especially if this page (before the More technical stuff) is understandable: it is difficult for me to look with the eyes of these poor souls looking at this the first time... Any suggestions for improving the understandability of this page, or just questions to help me understand the difficulties for understanding it, are welcome!
Compatibility codes (CCs)
From compatibility levels (CLs) to compatibility codes (CCs)
The compatibility level expresses compatibility of one version relative to the previous one. A maintainer had to choose one out of six levels from the compatibility level table shown in Expressing compatibility levels from a package maintainer POV.
The CCs will be computed incrementally from the CLs for each version, started with the first (given CC (1, 1, 1, ...) as a start tuple, the first version does not have a CL). Note: there is one CC per version.
A CC is a measurement for the compatibility of one version to all other versions!
This includes successor versions: a comparison of their CC (after they exist, of course) to the given CC gives their compatibility to the given version.
It is easy to read the compatibility level information given by a maintainer if you have two CCs: the one from the version you are interested in and another from its previous version. Then the compatibility level given by the maintainer is just the highest ranked number in the CC tupel which has been increased (the lower ranked ones (in the newer version) will just be setted to number one (zero would be an alternative)).
It is important not to 'compare apples with bananas': a CC is valid only for the environment the persons have had in mind, which have stated the CLs!
E.g. two versions stated to be compatible in a Squeak3.7 environment do not have to be compatible in Squeak3.8.
Automatically upgrading
The main idea behind this (why these CC tuple?) is to express the Subset property ordering (see also Caps): a change in the lower ranked compatibility levels, should not affect the compatibility of the higher ones. If the maintainer hints are correct, then this ordering allows automatically upgrading: given a requirement as dependency to some version by a CC tuple down to some compatibility level (truncating the lower ranked ones, getting a shorter tupel as a full CC) allows increases in the last position (lowest given rank) and all positions behind (not given lower ranks), but no change in positions before (higher ranks).
For example: given a dep from some package P with a CC of
(4, 2)
would allow package versions with CCs
(4, 2, 1, 1, 1, 1),
(4, 2, 7, 2, 3, 2),
(4, 3, 1, 1, 1, 1),
(4, 3, 7, 5, 6, 3),
(4, 9, 2, 2, 1, 1);
but not
(5, 1, 1, 1, 1, 1) (too high),
(4, 1, 1, 1, 1, 1) (too low).
The truncated CC tuples for expressing dependencies are appearing as parts of Caps.
Caps
Caps (from 'capabilities') are the combination of a unique package name and a full or truncated CC tupel.
Semantically they denote sets of package versions: each position more in the CC tupel part denotes a subset of versions related to all shorter tuples, which are equal to the longer one in all given positions (compared from the beginning). Here the Subset property ordering is visible again.
Expressing requirements from a client POV
Assumed there is a package requiring another package. The package maintainer has to define the requirements to the other package then: in the following table a proposal for how to express them is given from a client POV.
Expressing requirements table
Example: A requirement from Caps P_1.2.3 means, that packages with Caps
P_a.b.c.d.e.f with
a = 1 (fixed), b = 2 (fixed), c >= 3 (variable)
d, e, f arbitrary
are needed/allowed. All others are not sufficient! So this expresses requirements from Caps having only incremental changes in the lowest rank with fixed higher ones.
CL rank | Caps | semantics | attitude | question to the maintainer requiring another package |
-. | (P) | package exists, no stable API; initial version sufficient, do not need any API; same as 1. Caps (P_1) | reasonable, if sure, that general (general) functionality will not decrease | "Do you need the general properties of the package (not using its API), and its very first published version is sufficient? Note: could make sense, if you think that needed functionality never decreases." |
1. | (P_a) | package exists, provides general features; some versions sufficient, do not need any API | reasonable, if sure, that needed (general) functionality will not decrease | "Do you need the general properties of the package (not using its API), from some version upwards? Note: could make sense, if you think that needed functionality never decreases." |
2. | (P_a.b) | package exists, providing some API, but it may change | risky, hoping the best | "Do you need some API from some version upwards, but it may break your code by an upgrade? Note: risky!" |
3. | (P_a.b.c) | frozen API, new code allowed for extending API and improving current functionality | reasonable, if serious package maintainer | "Do you need a stable API, and have no problems with extending it or improving the functionality of the package (e.g. being faster)? Note: suggested!" |
4. | (P_a.b.c.d) | frozen API, new code allowed for extending API, but code changes of current functionality limited to bug fixes and refactoring | reasonable in rare cases, if not so serious package maintainer: - inhibits improvements, and
- inhibits development of other packages needing extended API, if there just is an 'improvements in used code' release in between
| "Do you need a stable API, and have no problems with extending it, but do not want to have code changes for improving the functionality of the package (e.g. being faster)? Note: may be reasonable in rare cases, if you do not trust bigger changes made in code used by you. Note: inhibits development." |
5. | (P_a.b.c.d.e) | no fixes are allowed | paranoid:- inhibits development of other packages needing extended API, if there just is a bugfix release in between
| "Do you want no fixes of the package in used code (allowing only new extension)? Note: strongly inhibits development. Note: this forbids fixes! Note: may be reasonable in rare cases, if you have worked around a bug and do not want its fix." |
6. | (P_a.b.c.d.e.f) | any code changes forbidden | paranoid, do not trusting package maintainer at all (but why using this package then?):- no fixes,
- no improvements,
- no extensions!
| "Do you want no code changes at all?" |
In practice the client package maintainer chooses a serving package version working with his package and the wished compatibility level (one out of six, this determines the length of the corresponding Caps). The working serving package version may just be the currently installed one (just working). Technical note: the wished compatibility level will be encoded as dependency from the set of suited serving package versions, this set will be expressed via the corresponding Caps.
Expressing conflicts from a client POV
Not expressing conflicts is the best!
At first it has to be said: avoiding conflicts by far is the best!
Why? In short:
- expressing conflicts introduces dependency rules stating the exception from the normal behaviour;
- there are better ways to go, which should be preferred.
Bug example
Assumed there is a bug detected in a new package version: then the best is to inform the serving package maintainers. This serves the following purposes:
- The package maintainers know the bug.
- Somebody has shown to the package maintainers, that he/she is interested in fixing the bug.
- If the package maintainers know the bug and see the interest in fixing it, they may be motivated to fix it.
- If the package maintainers have fixed the bug, they can release a fixed version of the package.
- If there is a release of the fixed version, the CC dependency resolver automatically sees this corrected version!
- If the CC dependency resolver sees the corrected version, the installer can automatically install it (if its policy allows).
The normally expected behavior will be reestablished after an intermediate state of having unnormal behavior (the buggy package version).
But there may be problems, e.g. the serving package maintainers could be on holidays (so not able to fix the bug). Then stating a conflict in the dependency rules is the solution.
This may not be a temporary solution for the bug fixing time needed: the bug fix version gets a new CC and may be installed automatically, but the older buggy version may be required by other packages (leading to a conflict). So it's necessary to keep the conflict information as long as the buggy version is available.
Expressing conflicts table
Example: A conflict with Caps P_1.2.3 means, that packages with Caps
P_a.b.c.d.e.f with
a = 1 (fixed), b = 2 (fixed), c >= 3 (variable)
d, e, f arbitrary
are forbidden; all others are not forbidden (e.g. b >=3): so this expresses conflicts to Caps having only incremental changes in the lowest rank with fixed higher ones.
CL rank | Caps | semantics | question to the maintainer wanting to introduce a conflict with another package |
-. | (P) | general conflict (with initial and all other versions); same as 1. Caps (P_1) | "Do you want to express an ever lasting conflict with all versions of the package? Note: reasonable, if your package in principle conflicts with the other one." |
1. | (P_a) | general conflict from some version upwards; older versions not conflicting | "Do you want to express a conflict with all package versions from some version upwards? Note: reasonable, if - your package in principle conflicts with the other one starting with some version, and
- you think that conflicting behavior stays.
" |
2. | (P_a.b) | conflict with packages providing some changeable API | "Do you want to express a conflict to versions having a changed API from some version upwards until the next higher version? Note: unlikely case on its own; when will a surely code breaking API (higher ranked version change) remove some conflict?" |
3. | (P_a.b.c) | conflict with packages with frozen API, new code allowed for extending API and improving current functionality | "Do you want to express a conflict to versions without changes in the used API but changes in its functionality (e.g. being faster) from some version upwards until the next higher ranked version change? Note: unlikely case on its own; when will a possibly code breaking API (higher ranked version change) remove some conflict?" |
4. | (P_a.b.c.d) | conflict with packages with frozen API, new code allowed for extending API, but code changes of current functionality limited to bug fixes and refactoring | "Do you want to express a conflict to versions without a change in the used API and without improvements and fixes in code supporting this API from some version upwards until the next higher version? Note: forbids fixes! Note: could make sense - in very rare cases - together with additional conflicts for all higher ranks 1. - 3. (otherwise this would just hold until the next higher ranked version change...), if you have made workarounds for bugs, which could or will be breaked by fixes (evil scenario, it's better to also fix the workarounds, of course)." |
5. | (P_a.b.c.d.e) | conflict with packages, where no fixes are allowed | "Do you want to express a conflict to versions, where no bug fixes have been allowed, from some version upwards until the next higher version? Note: reasonable, if you have detected a bug in a certain code version and expect it to be removed in the next one (e.g. you have reported the bug and the package maintainer is willingly to fix it)." |
6. | (P_a.b.c.d.e.f) | conflict with packages, where any code changes are forbidden | "Do you want to express a conflict to versions without any code changes, from some version upwards until the next higher version? Note: very unlikely case on its own; when will a new version without any bug fixes remove any conflict? OK, if you don't like the new docu and generate conflicts to all higher ranks, too (for forbidding any changes)... (this is a joke!)" |
Notes:
- If there are conflicts, since the CL given by the maintainer of the serving package has been stated wrong, the right solution is to fix the CL (followed by recomputing the CCs of all successor versions), and not to introduce conflicts. As a start the maintainer should be informed.
- Mostly used cases should be -., 1. and 5..
- A conflict with some lower versions as the most actual one would be modeled as a requirement.
- Protecting a bug workaround by introducing conflicts with 1. - 4. is evil!
- Forbidding upgrades of a package (with some longer CC) by only defining conflicts needs many (in case of full CC 6) conflict Caps (and should be avoided of course). Requirements are better there.
More technical stuff
Definition
Compatibility codes (CCs) are expressing the compatibility of one version of a package to other ones in a dense form.
- They are tuples of numbers (positive integers), where each number corresponds to one compatibility level (CL).
- CCs are unique in a package, no CC occurs more than once in one package.
- Each version corresponds to one CC (1-to-1 relationship).
- The compatibility levels are ordered, for two CCs there is: as more numbers from left to right are equal between them, as more compatible their corresponding versions.
- CCs can be computed incrementally, if
- for each version someone has classified the result of a compatibility comparison with its previous one (starting with the first), and
- the first CC assigned to the first version of a package is a tupel with some numbers, typically all equal to 1 (e.g. (1, 1, 1)).
Example
Note: This is a very simple example for illustrating CCs: since it is so simple, it is not suited as an example for automatically upgrading.
There is a package P existing in some versions, say P_1, P_2, P_3 (as lower the number as older).
There are two compatibility levels 'incompatible' and 'compatible', which correspond to the 1. and 2. position in the CC (from left to right).
We are assigning the start CC = (1, 1) to the first version P_1 (nothing to be compatible to, so nothing more to do).
Now if the versions P_2 and P_3 come into play, CCs will be assigned with the following meanings (each table to read from top to down):
version | CC | meaning |
P_1 | (1,1) | start value |
P_2 | (1,2) | P_2 is compatible to P_1 |
P_3 | (1,3) | P_3 is compatible to P_2 |
version | CC | meaning |
P_1 | (1,1) | start value |
P_2 | (1,2) | P_2 is compatible to P_1 |
P_3 | (2,2) | P_3 is incompatible to P_2 |
version | CC | meaning |
P_1 | (1,1) | start value |
P_2 | (2,1) | P_2 is incompatible to P_1 |
P_3 | (2,2) | P_3 is compatible to P_2 |
version | CC | meaning |
P_1 | (1,1) | start value |
P_2 | (2,1) | P_2 is incompatible to P_1 |
P_3 | (3,1) | P_3 is incompatible to P_2 |
For an CC_v = (x, y) assigned to some P_v, the following version P_v+1 gets a CC_v+1 with the following meaning: if it is
- CC_v+1 = (x, y+1), then P_v+1 is compatible to version P_v,
- CC_v+1 = (x+1, y), then P_v+1 is incompatible to version P_v.
Sets of versions
Now it is possible to get sets of versions via their CCs: we could say we want to have all versions P_v with CC_v = (x, y), for fixed x, arbitrary y. In our example this would mean to select all compatible versions with first CC number x. Each fixed first CC number together with a free second one denotes a set of compatible versions.
It makes no sense here to fix y while having a free x: this would mean a set of versions mutually incompatible (under the assumption that the compatibility not increases if we go farer away from another version).
CC (x, y) select | meaning |
(free, free) | all versions |
(fixed, free) | set of mutually compatible versions, each element is incompatible to all versions in other sets (other fixed x) in this compatibility level |
(fixed, fixed) | one specific version |
(free, fixed) | nonsense: a set of versions mutually incompatible (with very high probability) |
This shows another thing: it makes sense to fix from left to right without gaps to describe more and more smaller sets of versions all having the same compatibility level!
Even the CC (free, free) select variant makes sense: this just means to select all versions of a package, which could mean some general property we want to have in the system (e.g. a WebBrowser, where all versions are sufficient, and we just want to express, that we need this WebBrowser).
This is just a very simple example: in reality more such compatibility levels are useful.
Suggested compatibility levels
The suggested compatibility levels
- lead to CC tupel with size 6;
- have the following semantics if used for expressing dependencies (requirements and conflicts), which is more advanced than the semantics in Example:
- all CC numbers before the last one are fixed, whereas
- the last CC number - corresponding to the lowest CL rank - will be interpreted as a lower bound (inclusive) for a variable CC number there.