Why Packages Universes Exist
Last updated at 12:41 pm UTC on 17 January 2006
I'd like to describe one aspect of package distribution systems that not everyone seems to get. It will take a little while to step through the argument, but I think it is worth it is because getting this wrong appears to cause a big mess. Here goes.
One essential property of a package distribution system is that users may select package names from a list of available packages that they want, hit "Go", and have the system automatically install and remove packages until it ends up in a state consistent with what the user
selected. The user should not be forced to think hard about this, and the user should have some reasonable level of expectation that all the install packages successfully work together.
This is different from a catalog of everything. A catalogue of everything is a very useful tool, but it absolutely lacks the property that a user can auto-install packages with much expectation that things will work. For most packages, it is true that you can grab them from a catalog of everything and just modify them a little to get them working. However, these little things will add up when people install a lot of packages. It is much better if the auto-install tool gets things right most of the time, and since this reliable auto-install is achievable, I see no reason to aim for less.
Now, if we are serious about this reliability of auto-installs, there are two implications.
Implication #1: The packages must be configured in relation to each other. Every installed system has its quirks no matter how many standards it obeys, and Squeak does not pretend to follow any standards at all. As Squeak is modified over the years, methods move from class to class, classes appear and disappear, different approaches are used to install items into menus, and so on. If we want a package system that lets users Click-and-Forget, then all of this fiddly configuration has to be done in advance, and all of the packages a user installs needs to have fiddles that are consistent with each other.
Implication #2: The packages must be tested in relation to each other. We should try to make packages be independent from each other, but it is fundamentally impossible to achieve it perfectly. After all, the point of assembling multiple packages together is typically that they will influence each other and thus make a better whole! Thus there is some amount of unovoidable interaction between packages. If you want a really stable system, you are going to want to use packages that not only have been tested by themselves, but which have been tested together with each other.
Both of these implications lead to the same ultimate conclusion: it does not make sense–for a good package system–to try to load packages unless they have been configured and tested together. Therefore, if we limit our attentions to good package systems, the packages of the world are divided into several distinct sets. Packages from the same one of these sets have all been configured consistently with each other and have been tested with each other loaded. Because of this, packages from the same set are good candidates for the package-installing tool to select to install. On the other hand, packages in different sets are not particularly likely to work together, and they are poor choices to automatically install.
We could call these sets "universes" of packages. People operating within one set of packages should not see – in the package tool – packages from other sets. Packages from separate universes have not been tested together and have not been configured consistently with each other. If a user is looking at packages in universe A, then a good package tool should not offer to install the risky packages from universe B. This is the point of my post:
Admit that package universes exist!
This is not only a restriction. Universes also provide a useful place to attach the differing policies that different users and developersdesire. Different universes can have different policies about what goes in, about how things are named, about what updates are allowed, and about how updates are approved. And on the flip side of that coin, you can boil down all these choices for users into a choice from one of the existing universes of packages; once a user chooses a universe to work within then all of their tools can do the right thing with no further intervention from the user.
The choice of universes should be a central part of the discussions about our processes. There are many ways to do it, but I suggest, as a start, that the following two universes would be interesting ones for our near future:
- 3.7 finalized – 3.7, with all packages that work as of gamma, and with no new package updates in the future other than important bugfixes. This is a stable development platform, and it's also a fine basis for doing EToys in a SuperSwiki.
- 3.8 – the development stream. Stuff goes in with only a little bit of testing.
Also, let's leave the door open for people to create localized universes such as this one:
- 3.7final + MyCoolApp — the union of the frozen 3.7 plus all the updates that my organization is working on.
Nesting works just fine even when unplanned: a new universe can always be created by grafting new stuff onto an older universe. I suspect that it is still a good idea to remember that this nesting is happening, however.
That's all for now. Let me leave by noting that it has proven dangerous in practice to ignore the existence of these universes. RedHat users often complain about the incompatibility between their packages, but when they are pressed, that they usually admit to installing random RPM's off the Internet that weren't associated with any particular version of RedHat. This is a classic case of pulling packages from different universes. Debian folks, on the other hand, have no such complaints. The main difference between these userbases in not the format of the packages. The difference is that the Debian tools admit the existence of multiple universes even within the multiverse known as "Debian". Both the user tools and the client tools support users in doing the right thing even when the users aren't thinking about them: the user tools encourage you to point within one universe, and the server tools encourage you to build new and separate universes instead of simply tossing packages out into the multiverse.
Lex Spoon
Of course "packages universes" as you call it exist. But I would rather take the more common keyword which you mention right in your first sentence package distribution and people are with you. Basically we would like to group packages and label the groups. But it is not obvious to me why I should want only to use packages from one group. [[Hannes Hirzel]]
Because a universe is a very special kind of group: it is a set of packages that are known to work together (for some level of "known"). Packages from different universes do NOT work together, by definition. Now yes, that means that the present page is tautological. See the other page, Package Universes, for why I think it is worthwhile explicitly track the universes instead of just treating them as theoretical constructs. Lex Spoon
