Squeak
  links to this page:    
View this PageEdit this PageUploads to this PageHistory of this PageTop of the SwikiRecent ChangesSearch the SwikiHelp Guide
Squeak VM Plugins
Last updated at 10:26 am UTC on 11 May 2017

2017


Types of plugins

Check text below and update....

2006

Built-in vs. external plugins

(See also VMMaker for important details about how to create VMs and Named Primitives for more info about using the SmartSyntaxInterpreterPlugin classes)
Plugins can be either built-in or external. Builtin plugins are linked directly with the VM executable so that only one executable file has to be redistributed. This is the standard form of distribution for many of the commonly used plugins (such as file support or BitBlt). Alternatively, plugins can be generated as stand-alone files. These files are in platform-specific shared library format (that is a DLL under Windows, a DSO on Unix, or a CodeFragment on Macs) and can be redistributed as single files.
There are reasons for both approaches. Builtin plugins reduce the number of files to redistribute and prevent an (accidental) use of an outdated external plugin. External plugins allow for easier upgrading since they can be replaced individually. Since both approaches are useful depending on the circumstances the lookup scheme for pluggable primitives (see below) is defined so that it can deal with both variants.
Translating a plugin as builtin or standalone is simply a decision of the person compiling the virtual machine. For compiling a plugin as builtin it must be added to the list of builtin plugins in Interpreter>>translate:doInlining:. Then, the entire VM must be regenerated and recompiled. For a standalone plugin, the code can be generated by executing 'MyPlugin translate'.

Lookup of primitives

The rules for looking up pluggable primitives are a little more complicated, due to the requirements for having both, builtin plugins as well as external plugins. The general rules are:
  1. The VM searches for a shared library (DLL,DSO,CF) with the given plugin name
  2. If any such library is found, the VM calls several initializers (see below)
  3. If no shared library is found, OR the initialisers fail, the VM tries to find a builtin version of the plugin (based on the existance of an appropriate "setInterpreter" function, see below on plugin initialization)
  4. If a builtin plugin is found the VM calls the initialisers
Once a plugin with the appropriate name has been found and is successfully initialised (either as a shared library or as a builtin) this plugin is used until it is either unloaded or Squeak terminates.
After the plugin was found, the primitive is looked up in the plugin and (if found) executed.

Plugin initialisation

Plugins have several initialiser functions, each of which is called at a certain point during startup of the plugin. The very first initialiser called when a plugin is loaded is
       char* getModuleName(void);
This initializer function must return the 'compiled name' of the plugin. Note that for all subclasses of InterpreterPlugin the function is automatically generated and no further action needs to be taken. [Some of the pre-2.8 plugins will not implement this entry point. Although this is possibly dangerous it is ignored to have full backward compatibility]
The 'compiled plugin name' is important for preventing accidental misspellings of plugin names on systems that have case-insensitive lookups for shared library names. Assume, for example, two named primitives defined as
 MyClass>>primitiveOne
       
 MyClass>>primitiveTwo
Given that a shared library (such as 'MyModule.dll') exists, the VM could attempt to load a different module for each named primitive (this is because on several systems there is no difference between 'mymodule.dll', 'myModule.dll', and, 'MyModule.dll'). The compiled name of the module prevents this from happening, e.g., either of primitiveOne or primitiveTwo would fail unless there really are two different plugins, one reporting 'myModule' and the other one reporting 'MyModule' as the compiled name.
The second initialiser being called when a plugin is loaded is
       int setInterpreter(struct VirtualMachine *proxy);
installing a proxy to a set of well-known functions exported by the virtual machine. Again, this function is automatically generated for subclasses of InterpreterPlugin, but if implemented manually, it must check if the major numbers of the expected and the provided proxy match and if the minor number of the provided proxy is equal or higher than the one expected. If not, setInterpreter() must return false (that is zero) to indicate that it can not interact with the VM in question.
The third initialiser is the optional
       int initialiseModule(void);
providing a way to perform module initialisation if needed (such as allocating resources or installing hooks) before any primitives are called. If provided, this function must return true (e.g., non-zero) to report a successful initialisation of the module.
If any of the above described functions fail, the module will be silently unloaded (without calling any shutdown function), since it is assumed that failure in any of the initialisers will be handled by the function itself.

Plugin shutdown

If necessary, a plugin can implement the function
       int shutdownModule(void);
which will be called when either the plugin is explicitly unloaded (by Smalltalk>>unloadModule:) or when Squeak terminates. This provides a way of gracefully freeing critical resources, uninstalling system hooks, etc.

More on builtin plugins

Name lookup of builtin functions can be complicated on systems that do not understand the concept of a shared library (or implement it too poorly to be of use). Since this is a problem on many bare-hardware ports and since Squeak is relying strongly on named primitive support, the mechanisms for builtin named primitive lookup have been redesigned to not require any OS specific facilities. The following describes some of the implications.

Naming conventions

All builtin plugins have to follow strict naming conventions. A 'primitiveBar' in the builtin plugin named 'Foo' will be looked up under the name "Foo_primitiveBar" (rather than just "primitiveBar"). This is to distinguish primitives having the same name but residing in different plugins (the C compiler would complain about multiply defined symbols here).
This issue is of no importance for subclasses of InterpreterPlugin (the naming convention is automatically provided) but can be interesting for primitives that are directly written in C. In this case an exported primitive should be declared as
#ifdef STANDALONE /* plugin is compiled standalone, e.g., as shared library */
EXPORT(int) primitiveBar(void)
#else /* plugin is compiled builtin */
EXPORT(int) Foo_primitiveBar(void)
#endif
assuming that STANDALONE is defined for standalone builds. Note that in the case of a builtin plugin completely written in C the list of exported primitives must be modified manually (see below).

List of exports

Since the mechanism for looking up builtin named primitives must not rely on any OS specific functions a list of exported symbols is generated during the translation phase of the virtual machine. This list is contained in the file "sqNamedPrims.h" and contains the necessary declarations for finding the known builtin primitives (and other exported symbols).

Extending the list of exports

Because "sqNamedPrims.h" is automatically generated, it should never be touched manually. However, sometimes certain symbols need to be exported that are not included in the list. An example is a specific platform dependent primitive that is implemented on no other platform (so, for instance, the Win32 VM exports several debugging primitives that have no counterpart on any other platform). Since these exports need to show up in the list of exports a separate file can be modified. The exports can be listed in the file "platform.exports" which consists of exported names such as
XFN(Foo_primitiveBar)
XFN(exportedName2)
XFN(exportedName3)
and will be automatically included by sqNamedPrims.h so that these exports can be found by the common lookup mechanism. It is also possible to add 'data' to the list of 'exported functions' - the above defines a generic entry point into the VM and it depends on the client of any such entry point to know what the associated meaning is. As an example, a VM might export something like
XFN(stWindow)
to give plugins access to the main window (for whatever reason).

Finding exports in other modules

Sometimes it is necessary, to find certain entry points in different modules. As an example, the Balloon engine (both 2D and 3D) rely on BitBlt for performing the final rasterization and color conversion of a single rendered scan line. Since both, Balloon and BitBlt are plugins, it is necessary for the Balloon engine to lookup exported functions such as "loadBitBltFrom" or "copyBits".
Another example is the current (Squeak 2.7) implementation of the Unix sockets. The Unix socket implementation requires the installation of a callback hook in the VM so that changes are monitored regularly. This requires the Unix support code to look up a specific entry point in the VM (e.g., something like "SetSocketPollHook").
To deal with these cases, an extra function has been added to the InterpreterProxy:
       int ioLoadFunctionFrom(char *functionName, char *moduleName);
which can be used to look up entry points from one module in another module. For the above examples, this means that the Balloon engine can implement initialiseModule as follows:
BalloonEnginePlugin>>initialiseModule
       "Initialize the Balloon engine"
       loadBBFn := interpreterProxy 
               ioLoadFunction: 'loadBitBltFrom' 
               From: 'BitBltPlugin'.
       copyBitsFn := interpreterProxy
               ioLoadFunction: 'copyBits'
               From: 'BitBltPlugin'.
       ^(copyBitsFn ~= 0 and:[loadBBFn ~= 0])
The above will look up the two required functions and store them for later use.
The Unix socket implementation illustrates a more interesting problem. The code in the plugin that checks to see if anything interesting has happened needs to be called on a regular basis; so somehow the main event loop in the core VM needs to know to call aioPoll if and only if the Socket plugin is loaded and running.
The first version of the pluginised system used an explicit callback to set a function pointer:-
since 'SetSocketPollHook' is not an automatically generated export, the first step is to define it in "platform.exports"
XFN(SetSocketPollHook)
Then, the Unix specific support code could request the entry by
int socketInit(void) { /* socketInit() is called from initialiseModule */
       int setHookFn;

       setHookFn = interpreterProxy->ioLoadFunctionFrom("SetSocketPollHook", NULL);
       if(!setHookFn) {
               /* no hook found. can't continue */
               return 0;
       }
       /* install my socket polling function */
       ((void (*) (SocketPollFunction)setHookFn)(mySocketPollFunction);
       return 1;
}
Note that the NULL module specifies an export residing within the VM itself.

It was later pointed out that the unix shared library system can handle this directly by simply referring to the coreVM global variable "socketPollFunction" from the plugin code. The original exemplar code above is retained here for illustration and may be of use to porters onto other platforms.

Module unload notification

If a module (such as the Balloon engine) depends on entry points in another module, it should implement the (exported) function #moduleUnloaded: aModuleName. This function is called whenever a module is unloaded and allows the dependent module to release any dangling pointers to the unloaded module.
BalloonEnginePlugin>>moduleUnloaded: aModuleName
       "The module with the given name was just unloaded.
       Make sure we have no dangling references."
       self export: true.
       self var: #aModuleName type: 'char *'.
       (aModuleName strcmp: 'BitBltPlugin') = 0 ifTrue:[
               "BitBlt just shut down. How nasty."
               loadBBFn := 0.
               copyBitsFn := 0.
       ].
Of course, the dependent module needs to dynamically reload the entry point when they are needed.

Preloading modules

It can be of interest to 'preload' modules of which are known to be needed in the immediate future. Since the process of loading usually requires a (relatively time consuming) search on the hard disk for an appropriately named shared library and the initialisation may be quite expensive as well it is sometimes a good idea to couple the loading process of one plugin with that of another (e.g., the sound plugin will most likely require the mixer functions in the immediate future and any delay in finding the mixer could result in noticable distortions of the result).
For this purpose it is allowed to use ioLoadFunctionFrom with a NULL function pointer, e.g., the initialisation code of the sound plugin could look like
int soundInit(void) { /* soundInit() is called from initialiseModule */
       interpreterProxy->ioLoadFunctionFrom(NULL, "SoundMixPlugin");
       return 1;
}
which will request the SoundMixPlugin to be loaded. Three things are to be noted about the above:
  1. As of now (Squeak2.8alpha) there is no SoundMixPlugin (it is an example, see?!)
  2. The return value of ioLoadFunctionFrom will indicate if the module has been loaded successfully. A non-zero return value means the module has been successfully loaded. This can be used to make the initialisation of one module dependent on the successful load of another module.
  3. WARNING: Care must be taken that there are no circular requests for plugins to be loaded. In the above example a request of the SoundMixPlugin to load the sound plugin would result in infinite recursion. [NOTE: We should probably do something about this but I'm not quite sure how and what]

VM Implementor issues

To simplify porting efforts, a generic (cross-platform) implementation for named primitives is provided. It is generated with the support code in the file 'sqNamedPrims.c'. Note that ALL functions formerly located in 'sqXYZExternalPrims.c' are now part of sqNamedPrims.c and ONLY the following three functions must be implemented in a platform specific way:
/* ioLoadModule:
       Load a module from disk.
       WARNING: this always loads a new module. Don't even attempt to find a loaded one.
       WARNING: never primitiveFail() within, just return 0
*/
int ioLoadModule(char *pluginName);

/* ioFindExternalFunctionIn:
       Find the function with the given name in the moduleHandle.
       WARNING: never primitiveFail() within, just return 0.
*/
int ioFindExternalFunctionIn(char *lookupName, int moduleHandle);

/* ioFreeModule:
 Free the module with the associated handle.
 WARNING: never primitiveFail() within, just return 0.
*/
int ioFreeModule(int moduleHandle);

All of the three functions can be implemented to return zero on platforms that do not implement the concept of shared libraries. Stubbing out these functions will leave all builtin plugins fully functional.

Any VM should call ioShutdownAllModules(void) when Squeak exits. This ensures that any resources allocated by the plugins will be free'd upon exit. It should even be called if Squeak terminates abnormally (given that there's still a way of calling ioShutdownAllModules).

Platform VM conversion issues

You will need to edit many of your platform specific source files to make them work cleanly with the plugin architecture.

sq{plat}Window.c

Remove references to sqFileInit() and joystickInit(). Use the ANSI C atexit() or your platform's equivalent to ensure that ioShutdownAllModules() is called on exit.

sq{plat}Directory.c

Actually, this aplies pretty much to all othe plaptform files - after the #include of sq.h, add #include "FilePlugin" (or "AsynchFilePlugin" or "MIDIPlugin" as appropriate. sq.h no longer includes the definition of the SQFile struct or most of the other stuff it once contained.
Add an
 extern struct VirtualMachine  interpreterProxy;
so you have access to the vm proxy functions.
Change any uses of
 success(flag)
to
 interpreterProxy->success(flag)
along with any uses of signalSemaphoreWithIndex() and so on.
During the original work, a few places were found where the vm's successFlag global was used directly. That would need to be changed from something like
 if( successFlag ) {
to
 if( !interpreterProxy-failed()) {

If your code (ab)uses a function in the main vm, use a technique like that described above for the unix socket poll function to get a function pointer. Remember to add the function name to your platform specific 'platform.exports' file - see the Mac sources for an example usage.
If all else fails, emailTim RowledgeorAndreas Raab for help.