links to this page:    
View this PageEdit this PageUploads to this PageHistory of this PageTop of the SwikiRecent ChangesSearch the SwikiHelp Guide
Modular Squeak
Last updated at 11:40 am UTC on 9 May 2007
Note: This work has largely been moved to the Modules page.

We are assembling a number of relatively independent changes into a gestalt that we hope will enhance the modularity of Squeak and its fitness for a number of network applications:

We foresee the possibility of an extended universe of Squeak worlds, distributed over the internet as web pages, downloaded and internalized via SmartReferenceStreams, and swappable in a reasonable memory footprint by ImageSegments. The Squeak kernel could remain small – 5MB or less, yet it would have access to the entire universe of possible extensions. Users could send projects to each other, or store segments back to servers in export format, in the manner of multimedia swikis.

We plan to add a simple nested global address space to Projects and then to release a number of applets, along the lines of the Play-With-Me demos, in this framework. While it may not be the most general possible name space solution, it should nonetheless provide an architecture capable of hosting numerous reasonably modular networked extensions to the Squeak computing environment.

Image Segments

ImageSegments are created by a primitive operation that very quickly identifies what objects are reachable for a given set of roots, and by no other path. The primitive also copies the objects into a WordArray in the same form as they exist in the Squeak image itself, and copies into an Array all of the pointers from the segment into the rest of Squeak (ie the parts that are reachable from the outside). Naturally there is an inverse operation that will recreate the entire structure in a twinkle from the segment.

If you put these two operations together, you get a high-performance deepCopy operation. If you save the imageSegments on a local file, you get application swapping, and if you send that file far away and read it into another image, you have export/import of Squeak worlds. They are fast. They are compact. AND, they are platform-independent. Imagine a DLL that you can use on Windows or a Mac or a bare chip!

On modern PCs, Squeak can discover and extract a 1-MB image segment in less than a second, and can reconstitute it in about 200 milliseconds. The extraction process is accomplished by first marking all the "roots" of the segment, and then using the garbage collector's trace routine to discover all the objects that are only pointed to by those roots (they remain unmarked, being in the "shadow" of the marked roots).

More is required to put this technology into practice. For instance to swap out a segment, you must replace all its roots by ImageSegmentRootStubs. These are special objects that will retrieve their segment if any message is sent to them, and even if they are classes and a message is sent to one of their instances. Also, if you export an image segment, special work must be done when importing it to reconcile the outpointers with the state of the receiving system. That work comprises much of the intelligence of SmartReferenceStreams. [see additional information in class comments to ImageSegment, ImageSegmentRootStub and SmartReferenceStream]

For our first real test of ImageSegments, we have implemented swapping of Morphic Projects. Thus, if you have an up-to-date .changes file with a lot of (morphic) projects in it, you can open the preferences panel and set #projectsSentToDisk to true. Thereafter, every time you change projects, it will bring in the new project and send out a couple of others. Soon all your projects will be on disk, and your kernel image will be smaller. To write all projects initially in one long pause (not necessary) do this:

Project storeAllInSegments.

We have been testing this on a 70MB version of an image that Alan uses for a lot of his presentations. After segmentation the image is only 18MB, and there is a companion folder of segments ranging from 20K up to 8MB, and totalling 52MB (natch).

Key to the effectiveness of this technique is that there be no pointers from "the outside" into the morphic world that comprises the project. Anything pointed to directly or indirectly other than by the roots of the world will be excluded from the segment and will thus not get swapped out.

Note that if you start using imageSegments, a folder will appear on your disk named imageName_segs. If you move or rename your image, this folder must accompany it and remain named correspondingly.

ImageSegments and project swapping are still in the experimental stage

Isolated Projects

When you accept a method, the entire system feels the change, except projects that are "isolated". In an "isolated" project, changes are revoked when you leave the project. When you enter another project, that project's changes are installed. To make a project be isolated, choose "isolate changes of this project" from the "changes..." section of the screen menu. You can use an isolated project for making a dangerous change to a system, and you can get out if it crashes. An imported application can have the separate environment it wants. An isolated project shares methods with all subprojects inside it, unless they are isolated themselves. Each isolated project is the head of a tree of projects it shares all methods with.

You can also 'assert' all changes ever made in the current project to take effect everywhere. You are first informed if there are any direct method conflicts with other projects. The conflicts are presented in a ChangeList Browser. [Later: A project may be 'frozen'. Asserts do not apply to it after that. (Great for demos.) You should be informed when an assert was blocked in a frozen project]

Class definitions are global. If you add an instance variable, it happens in all projects. All versions of the methods are recompiled, in all projects. If you remove an inst var that is in use in another isolated project, it will become an Undeclared global. It is best not to remove an inst var when it is being used in another isolated project.

Senders and Implementors do not see other versions of a method in other isolated projects. When you ask for versions of a method, you will not get the history in other isolated projects. Moving methods and classes between changeSets, and merging changeSets has no effect on which methods are in force. But, when you look at a changeSet from a different isolated project, the methods will contain the 'wrong' code. A changeSet is just a list of method names, and has no code of its own.


The following section is outdated in 3.3a but may contain information useful for documenting the Modules
Environments offer the ability to partition Squeak's global name space. Each one operates pretty much the same way that the Smalltalk systemDictionary is used in a non-partitioned Squeak.

Each class has a direct-access environment in which it is compiled. Its environment slot points to an instance of this class, and it is there where the bindings of global variables are sought. The compiler looks up these bindings using normal dictionary protocol (at:, etc). If a binding is not found, then the name is looked up in the environment from which that one inherits, if any. In this way a class may be compiled in a context that consists of several nested name spaces, and direct reference may be made to any of the objects resident in those spaces.

Various methods may need to refer to objects that reside in environnments that are not a part of their direct-access environment. For these references, a simple global reference,
may not be used, and instead the construct,
Envt Thing
must be used. In this case Envt is a global reference to another environment, and the global name, Thing, is sent as a message to that environment.

Obviously, such a foreign reference cannot be resolved unless the environment in question implements a method of that name. This is how environmental variables are exported.

Each environment has its own unique class. With this structure, each environment can have its own instance-specific messages to provide access to its exported symbols. Note that this mechanism provides much faster runtime access than the Dictionary at: protocol. Also note that inheritance provides a trivial implementation of nested name scope by the same token.

In the early stages of installing partitioned environments in Squeak, interpreted access will be provided in several ways. To begin with, environments will intercept the doesNotUnderstand: message and, if the message begins with a capital letter, it will look up the corresponding name using #at:, and return the value if found. A refinement to this feature will be to compile an export method on the spot, so that subsequent accesses to that variable run much faster.

Note that there is no Environmental access pattern analogous to 'Envt Thing := newThing'. If an implementor wishes to store into environmental variables, he must do so by defining, eg, a SetThingTo: method and using a call to that method in his code. We may choose to only allow one certain pattern of access to be compiled in any subclass of Environment to enforce some understandable style of coding.

In the Squeak 2.8alpha release, execution of 'Environment reorganizeEverything' will convert Squeak into a partitioned organization, based on the major divisions in the SystemOrganization. We are experimenting now with the ability to swap out inactive environments as segments to reduce execution footprint.

Modular Import of Squeak Applets

We are also in the process of adding an environment slot to Projects, thus providing a project layer for global variables. This, together with the Isolated Changes mechanism, should enable a project to capture the classes, methods and active objects needed for any Squeak application. We are right now testing this capability with the following goal: It should be possible to browse the web, click on a proxy object which will then cause a project to be downloaded, complete with classes, methods, changes to kernel classes, and a running morphic world. This project will come to life in the Squeak image, will be swapped out if inactive, and will be brought back in by demand fault when it is next activated. Most importantly, if the user should choose to delete that project, the hose Squeak image will be left in a state that is indistinguishable from its state before the project was imported.

Back to Squeak Central project list