Ragnar Hojland Espinosa and the Wonderful Land Of Pluginia
Last updated at 5:45 am UTC on 21 August 2008
Here are some notes for personal use I wrote regarding writing plugins around 2003, and that I later posted hoping they could help someone else to have an easier time in figuring how everything worked together. Hopefully they aren't all that outdated by now. They are about creating a pluginized version of ncurses, which is a library for character output on terminal displays.
Building more complex plugins can be easily taken once you have figured out how to do the basic cases presented here. Be aware though that personal experience has gently taught me that sometimes it is better to implement in smalltalk whatever you are trying to pluginize, rather than the plugin. So if you are having trouble having to create and handle listeners, threads and non thread safe libraries, pointers and complex structures that must be passed around from C to smalltalk and viceversa, just do it in squeak. It's far more fun, productive and easier to debug.
Setup
- 0. we assume we are under linux, have ncurses installed
- 1. untar the sources of the vm (something like Squeak-3.6-beta11.src.tar.gz). Frow now on we will call the top level dir sq instead of Squeak-3.6.betall/
- 2. we create the dir in which we are going to compile our stuff (dir is o from now on). so: cd sq; mkdir o
- 3. inisqueak wherever you want, and use squeakmap to install vmmaker.
Creating the skeleton for the plugin
We open a browser and scroll down to VMMaker. There we can see a VMMaker-SmartSyntaxPlugin category and some Plugin classes in it. The ncurses plugin we are going to create must be a subclass of one of these. I remember TestInterpreter, but SmartSyntax is new. Which one to try.. lets go for SmartSyntax and see if the name is as misleading as TestInterpeter (which, no, its not a class to test the interpreter). With all those letters in the name it must surely rock.
- 1. Create class CursesPlugin (lets drop that N, looks ugly), subclass of SmartSyntaxInterpreterPlugin and set it to the new category 'Curses-Plugin'. In CursesPlugin we are going to create the primitives, that is, the lower level methods most closely binded to the C ncurses library, and that we are later going to use in our yet to be created Curses class which is going to be the public interface.
- 2. Create instance method initialiseModule and shutdownModule. These I think, as their names imply, are going to be called when we install and remove the plugin. self cCode: 'function()' inSmalltalk: [true] looks like it means "for this method, call function() in C, and do [true] in squeak". We'll see if we can figure out what the self export: true really is for later..
initialiseModule
self export: true.
^ self cCode: 'ncurses_init()' inSmalltalk: [true]
shutdownModule
self export: true.
^ self cCode: 'ncurses_shutdown()' inSmalltalk: [true]
We don't have to implement these two methods if we don't want to. Or we cas use cCode: 'true' inSmalltalk: [true] as a placeholder for the future.
- 3. Create class method moduleName
moduleName
^ 'CursesPlugin'
First test: compiling something
Nothing like testing from time to time how you are doing. We are going to tell squeak to generate the glue for our plugin, and attempt to compile that. Its not going to do much, but at least we'll see something. Besides, it'll serve as a good excuse to go for a beer while it compiles :)
- 1. From the world menu, open, and then open vmmaker. You may want to save the config so you don't have to set it each time. Set the fields correctly:
- platforms code: that, obvious as it may be, is the dir called platforms in the sources tree
- platform name: unix
- path to generated source: its the src dir in the sources tree.. I'm still sort of uneasy about that "clean out" button
- 2. In plugins not built we should see somewhere a CursesPlugin. We drag that with the mouse to external plugins
- 3. Click on generate external plugins. After a few seconds on the bottow of the vmmaker window we will see a CursesPlugin generated as CursesPlugin. Guess we can change the filenames somehow then..
- 4. We go out of squeak for a bit. Having a look in dir sq/src/plugins, we see a CursesPlugin dir which contains a CursesPlugin.c file that squeak has generated for us. If we have a look in the file, we can see our future C function ncurses_init is being called from initialiseModule, and how it returns a int.
- 5. we cd to sq/o and do a ../platforms/unix/config/configure and a make plugins (now its time to go for that beer).
Adding a really simple primitive
So it compiled, right, but can we see something? Nooo. Time to add some code then. We are going to add code to initialize and close the ncurses library, and output a little beep too. First we'll go with the beep.
- 0. An Oop is some kind of internal pointer in squeak that we are going to use to pass values from C to squeak and viceversa. Taken note of that:
- 1. In CursesPlugin, create instance primitiveCursesBeep. It is going to call a C function named ncurses_beep, which returns 0 if error and 1 if all the beeping went ok. We will later fill ncurses_beep().
primitiveCursesBeep
| result |
"bind a squeak variable to a C variable"
self var: #result declareC: 'int result'.
self primitive: 'primitiveCursesBeep'.
self cCode: 'result = ncurses_beep()'.
"cast the value we get back into a boolean"
^ result asOop: Boolean.
It will complain of an unused result. Ignore it for now and accept the code.
- 2. Go to the vmmaker, regenerate the external plugins (that'd be our CursesPlugin) and recompile. If you have a look at sq/src/plugins/CursesPlugin/CursesPlugin.c you'll see that squeak has generated some glue code at the primitiveCursesBeep fuction.
- 3. Now we have to fill somewhere the code for the ncurses_beep(), which also requires we perform some initialization and cleanup code for ncurses. That means we need platform specific code, our C code, compiled in. To tell vmmaker to take that into account, we create a class method of CursesPlugin
requiresPlatformFiles
^ true
Try to re-generate the plugin via vmmaker. It won't let you now. So we have to create our file.
- 3.1. mkdir sq/platforms/unix/plugins/CursesPlugin
- 3.2. we create the file sqCursesPlugin.c Yes, the sq is a special prefix to automatically find the file for the plugin. After we've done this, we regenerate the plugin and compile it via make plugins.
#include <ncurses.h>
int ncurses_init()
{
return initscr() ? 1 : 0; // the library call actually returns something else...
}
int ncurses_shutdown()
{
return endwin() == OK ? 1 : 0;
}
int ncurses_beep()
{
return beep() == OK ? 1 : 0;
}
- 3.3. Did we see ncurses calls in that code? Yes. Now we have to tell squeak to link the plugin with the ncurses library. Now, the best way I've found to do this is creating the file sq/platforms/unix/plugins/CursesPlugin/Makefile.inc with:
PLIBS=$PLIBS -lncurses
- 3.4. We need to have the Makefile for the plugin pick up Makefile.inc, so we go to sq/o and do the configure and make ritual. Now our CursesPlugin.so will be linked to libncurses.so
- 4. We create now a class method of CursesPlugin which is going to be responsible for calling the C code. If the primitive isn't there, we raise an error.
primCursesBeep
<primitive: 'primitiveCursesBeep' module: 'CursesPlugin'>
self primitiveFailed
- 5. Now we can go ahead and actually test the thing (provided the CursesPlugin.so we generated can be accessed by squeak ... the -plugins command line option may be useful). Just do a Curses primCursesBeep in a workspace and hear it beep. True, true, we can't actually see anything yet, but we are making progress :)
Is it there, or not?
- 1. After that test, we are going to ask squeak to print a list of loaded modules, just for fun. In a workspace:
Smalltalk listLoadedModules do: [ :each | Transcript show: each ; cr. ]
- 2. And now we can remove the module, and if you want, you can print the list of modules and ensure it's gone.
Smalltalk unloadModule: 'CursesPlugin'.
Some tweakings
- 1. We are going to add a class method to CursesPlugin so we later dont get errors. This will make it look for the include file when compiling the plugin, and stick it right before the generated code.
hasHeaderFile
^ true
- 2. Now we create the file sq/platforms/Cross/plugins/CursesPlugin/CursesPlugin.h (and the directory too, as its most likely missing) with a simple line in it:
#include <ncurses.h>
- 3. We go to vmmaker, and tell it to clean out the source.. if you attempt to compile the plugin right now, make will probabily complain of a missing plugins.int. To solve that you have to generate all in vmmaker, and you are set.
Primitives with parameters
It looks easy. It is easy. For now. :) We choose to pluginize the
int scrl (int)
function, as it looks the simplest.
- 1. Create the primitive. The diferences between this one and the beep one are basically:
- The method takes an argument (aNumberOfLines)
- In primitive, we now have a parameters: #(...) part, where you specify the kind of parameter the primitive is going to be called with. Want more than one? Easy: self primitive: 'primitiveFoo' parameters: #(SmallInteger SmallInteger Boolean)
- We can use the aNumberOfLines variable directly in the cCode.
primitiveCursesScrl: aNumberOfLines
| result |
self var: #result declareC: 'int result'.
self primitive: 'primitiveCursesScrl' parameters: #(SmallInteger).
self cCode: 'result = ncurses_scrl (aNumberOfLines)'.
^ result asOop: Boolean.
- 2. The C side is plain C. In your sqCursesPlugin.c, add:
int ncurses_scrl (int numlines)
{
return scrl (numlines) == OK ? 1 : 0;
}
- 3. Generate the boring side. Remember to add the parameter here, or you'll bang your head wondering why your code doesn't work.
primCursesScrl: aNumberOfLines
<primitive: 'primitiveCursesScrl' module: 'CursesPlugin'>
self primitiveFailed
I tended to forget wether in primitiveFoo: aParm1 parm2: aParm2 you used parm2 or aParm2 when calling the function, but then i realized that if you used parm2, you would have no way to refer to the first aParm1
Our first friendly speed bump, SmallInteger
Next in the list of functions to pluginize is
int echochar (chtype ch)
where chtypeis an unsigned long because ch can contain both the ascii character to print ORed with some other flags. Seeing the problem yet? No? Did you know SmallInteger is shorter than 32 bits? Now thats a problem.
We can get around it by passing the value as an Oop instead of a SmallInteger. Remember to write the usual primCursesEchoChar: aChar too. The C side would be as usual (int ncurses_echochar (unsigned long ch))
primitiveCursesEchoChar: aChar
| longchar result |
self var: #longchar declareC: 'unsigned long longchar'.
self var: #result declareC: 'int result'.
self primitive: 'primitiveCursesEchoChar' parameters: #(Oop).
longchar _ self positive32BitValueOf: aChar.
self cCode: 'result = ncurses_echochar (longchar)'.
^ result asOop: Boolean.
and to test, in a Workspace:
CursesPlugin primCursesEchoChar: 'a' asCharacter asInteger.
CursesPlugin primCursesEchoChar: $b asInteger.
Yeah, as comfortable to use as this mouse with a broken left button that I have, but remember this is just the raw glue. Nobody really likes glue. But you can use it to put together nice things.
Dealing with pointers, and SmallInteger strikes back
Lets do something more interesting. Pointers. We have our new candidate to pluginization
int wrefresh (WINDOW *window);
but, wait.. what argument are we going to call it with? We first need to get some valid window pointer to test it with. Lets do a ncurses_getstdscr() which returns the stdscr pointer. But what to return? Because SmallInteger is 31 bits..
primitiveCursesStdscr
| result |
self var: #result declareC: 'WINDOW *result'.
self primitive: 'primitiveCursesStdscr'.
self cCode: 'result = ncurses_stdscr()'
^ result asOop: Unsigned.
That Unsigned over there will make squeak return the value via a positive32BitValue macro in C. SmallInteger is finally defeated. Lets deal with wfrefresh now:
primitiveCursesWRefresh: aWindow
| w result |
self var: #w declareC: 'WINDOW *w'.
self var: #result declareC: 'int result'.
self primitive: 'primitiveCursesWRefresh' parameters: #(Oop).
w _ self cCoerce: (self positive32BitValueOf: aWindow) to 'WINDOW*'.
w = nil ifTrue: [^0].
self cCode: 'result = ncurses_wrefresh(w)'.
^ result asOop: Boolean.
To test, we cant do much. If we get a reasonable integer, and if it doesn't crash, we'll assume we are doing good.
Transcript show: (CursesPlugin primCursesStdscr); cr.
CursesPlugin primWRefresh: (CursesPlugin primCursesStdscr).
Yeah :)
Hints
- If you clean out, you don't have to recompile the whole thing.. just go into the CursesPlugin dir and make
- When you add header files (for example by hasHeaderFile) it looks like you have to clean out and at least do a make plugins. Otherwise they wont get picked up and youll get missing .h file errors.
- If you are getting primitive faileds but everything looks ok, make sure you are using the right number of parameters when testing.. that is, Transcript show: (CursesPlugin primCursesNewWin); cr. isn't ok.. the size and position paramenters for primCursesNewWin are missing.
- If you get an error message while generating the plugin that looks like "TSendNodes are not indexable" you should check the parameters: in the primitive line. Most likely you wrote parameters: (#String String) instead of #(String String)
- primitiveFoo is instance method, primFoo is class method, because plugin vars.. explain
- If you are getting crashes early in a primitive and you suspect pointers, and you have a handle _ self cCoerce: (self positive32BitValueOf: aHandle)... make sure you arent using aHandle in self cCode: 'foo (...)', but handle (doh:)
Comments anyone?
You don't need to implement a moduleName method if you want the plugin name to be the same as the class name. Several plugin classes have strange names and use the moduleName to give the actual plugin a less strange name. Of course this can cause confusion later. Never let it be said that we cannot use the simplifying power of Smalltalk to cause confusion. Tim Rowledge