DeltaStreams
Last updated at 10:56 am UTC on 16 August 2017
Delta streams was a proposal for a replacement of Squeak change sets and the update stream.
2008
Delta streams aims to be a compelling replacement to change sets and the update stream. Delta streams (not yet implemented) are analagous, to the update stream, and deltas are analagous to change sets. Deltas improve on change sets in several different ways:
The main motives for this project:
- Much improved ChangeSet functionality, should be able to fully replace them. A nice goal in itself.
- Making it easy to move fixes and enhancements to the base image AND to packages between branches and forks of Squeak.
- Making it trivial to publish, discover and subscribe to streams of Deltas.
- Making it trivial to publish, install and uninstall (revert) Deltas.
Thus it needs:
- A clean and fairly standalone implementation that should be easily adoptable by all Squeak forks/dialects. It will initially depend on SystemEditor for applying changes and the SystemChangeNotifier mechanism that was added to Squeak in... 3.7? (not sure)
- A smart but not TOO smart way to deal with merging and conflicts.
- A good robust distribution mechanism.
- Some kind of feedback system, probably added on top.
Since it is meant to be able to move changes between radically different Squeak versions and forks, DeltaStreams runs in a lot of squeak releases. It is distributed as a .sar file for each supported squeak. It is not available on universes at this time.
Using Delta Streams
Features
- The alluring fabled marvellous "revert" function! Deltas are designed to enable this in full.
- Ability to be used as a true log.
- Hopefully able to be used as a convenient medium for "cherry picking" between radically different Squeak images.
- The revert function also could enable "patch queues" like in Mercurial or Quilt. This is the concept of "stacking" Deltas and then being able to apply/revert/reapply those whole stacks at will.
- Deltas vs. Change sets
Code
The code is currently developed in different Squeak images (3.8, 3.9, 3.10 - we try to keep it working broadly).
Releases are available on SqueakMap as complete installable SAR files.
The main parts in these SAR files are also available separately:
- Our Monticello packages are developed at SqueakSource: http://www.squeaksource.com/DeltaStreams:
- The SystemEditor package with our ongoing fixes/enhancements (not a fork). We are periodically synching it with the upstream (Colin's) repository for SystemEditor.
- The ToolBuilder package with our ongoing fixes/enhancements (not a fork). This is used to build the UI in release 0.1, but will probably/possibly be dropped in future releases.
- The PlusTools package with our ongoing fixes/enhancements (not a fork). This is also used to build the UI in release 0.1, but will probably/possibly be dropped in future releases.
- Our Extensions to SystemChangeNotifier is (slightly ironically) available on SM as a ChangeSet package :).
Related work
The following is not yet commented by Göran (will ASAP):
(N.B. the stuff below, expect for some use cases, was added by Andy Tween)
Use cases
- saving/undoing method recategorizations
- Göran: All developer actions are logged and revertable. Well, except for oddball doits of course.
- undoing RB actions
- Göran: As long as the actions generate proper notifications through SystemChangeNotifier.
- save Delta to disk
- Göran: We aren't bothering with that yet, but sure. They are designed to be trivially serializable though (only carries its own state and no direct references to classes for example).
- browse Delta on disk
- Göran: A Delta is first deserialized into the image, this just creates it as a domain object. Then you send apply to it to actually affect the image. So browsing a Delta is trivially done by making a tool on top of the domain model. We intend to "port" the dual change sorter tool (kinda) ASAP.
- manually add items to a Delta (e.g. for exporting code to another image)
- Göran: Yes, there is already some code for doing this and it will be simple - the base design of the model makes it very easy to do.
- manually edit a Delta's contents (is this a good idea? Why not just copy items into a new, empty, Delta)
- Göran: Not sure what "manually" means but a Delta is a clean manipulable domain object, just mess with it.
- add a comment to a Delta (Allows Delta to be used like a 'blog')
- Göran: The idea is to let it have a Dictionary with "simple" keyed fields (Strings, Integers, Boolean).
- Use a Delta as a change.log i.e. created as disk file; appends to file on every change
- Göran: The model of a Delta is already exactly that and you just need to write a backend for it. The most trivial way to do this is to use Magma. :)
- Browse/load from a 'change.log' Delta after crash.
- Modify some stuff in an image to allow a Monticello package to load (e.g. to load a 3.10 package into 3.8)
- Göran: Not sure I understand.
- Andy: For example, to port a package from 3.10 to 3.8, it may only load if some methods, outside the package, that are present in 3.10, but missing in 3.8 are loaded into the 3.8 image before the package is loaded. Rather than creating an additional changeset, or adding those methods to the package, a delta(stream) could be created with those methods in it. The delta(stream) is then a 3.8 specific patch to enable the 3.10 package to load and run.
- Modify some stuff in an image after a Monticello package is loaded to enable it to run
- Göran: Not sure I understand.
- Andy: same as above, but the delta(stream) patch is applied after loading the package. Perhaps it removes some methods that the package has added, that are only appropriate in 3.10, not in 3.8. The idea of the patching is to allow code to be more easily ported between versions and forks.
- Remove overrides, loose methods, preamble etc from a Monticello package and put them in a Delta patch
- Göran: Yes, various "converters" to and from Deltas will be useful to have.
- Andy: To enable a single package that can be loaded into any version/fork. i.e. MyPackage+3.10delta , MyPackage+CroquetDelta, MyPackage+MinimalMorphicDelta
Bootstrapping
- decide on Delta file format
- create a minimal Delta loader
- manually create a Delta file containing the rest of the Delta system (system.delta)
- use Loader to load the Delta system from system.delta
- develop the Delta system, using only Delta files for persistence/merging/loading etc.? Eat your own dog food :)
? does this mean that squeaksource / monticello can't be used ?
Göran: I don't think bootstrapping needs to be so complicated. Currently DeltaStreams consists of a changeset tailored per Squeak version with some small base fixes - and the rest is an MC snapshot (+ SystemEditor). There is no reason to not use existing facilities to load DeltaStreams support into an image.
Andy: ok. I was thinking of how DeltaStreams could be used during development (of DeltaStreams), rather than how the support would finally be distributed once complete.
Delta File Format
- Human readable?
- encoding , (utf8? BOM or no BOM?) (put encoding in header?)
Göran: I can easily imaging several formats but don't want to confuse things. Currently file format is the least of our worries in coding this. :) One thing is certain - we will never hardwire a specific file format into the code.
Andy: This ties in with the bootstrapping idea of using DeltaStreams during development of DeltaStreams. If they aren't used for development, and it is done with Monticello and changesets, then you are right, it doesn't need to be decided upon until later.
Method source
Chunk format (don't like. Would like method source to be unescaped)
What about terminating each method source with a marker specified in the method entry's header
e.g.
AddMethod class="Object" selector="xxx:yyy:" category="xxxx" timestamp="0101001" upTo="/n%"
xxx: a yyy: b
"this is my method"
^42
%
AddMethod class="Object" selector="%" category="xxxx" timestamp="0101010" upTo="/n%%"
% anObject
"this upTo has been chosen to be enough $% chars so that it does not appear anywhere in the method source"
^super
% anObject
%%
AddMethod class="Object" selector="yourself" category="xxxx" timestamp="0101010" upTo="%"
yourself
"we don't care about the Lf in the upTo terminator in this example"
^self%
AddMethod... etc.
Woot Woot