We want to get rid of the operating system under Squeak. For this we know we will need to implement lots of low level things, we are not afraid, but we want to take a little different approach: Implement the bare minimum as native code (a mix of assembly and C), and then do everything else in Squeak. The people originally in the group only knew low level PC stuff, so we started with this, however there is already people participating that want to give Apple/PowerPC a try!
There are two incarnations of SqueakNOS, one was pretty much abandoned in 2001, and had support for 640x480x1 or 640x480x4, keyboard, mouse, serial port and a pseudo file system over serial port, although some of this features never saw the light. In May 2006 we restarted SqueakNOS, and a new version was released in May 16th, with support for 1024x768x32, and bare IRQ handling.
The SqueakNOS development is planned in different phases and small steps, as of May 20th, the first 2 are accomplished and the 3rd is almost ready. There is a lot of people willing to contribute, and there's certainly space for everybody, however we believe the first 3 phases must be ready so it's easy to contribute.
Get latest .iso image (containing everything you need to boot and recompile) from SourceForge's mirrors.
We've put up a mailing list to talk SqueakNOS, join in, or just drop an email there.
erm... we used to have a page with rss feeds and everything at people.squeakfoundation.org, but it doesn't exist anymore. Hopefully we'll get it back somewhere :-(
Phase 1: Make it boot
Make it boot and start intepreting the .iamge with text output for tracing and debugging.[Done]
The old SqueakNOS used a custom made mutant boot loader, borned from mixing LILO, the Linux kernel boot loader, and our dirty fingers. We had quite a few problems with it, and it was really complicated to put the needed environment to compile it. So we chose to replace it for something better and more modern: GRUB.
We first thought on GRUB because it, of course, handles all initialization, switches to flat 4G mode can load huge kerneles in high memory, and also has support for modules loading (module and modulenounzip GRUB commands). We wanted to detach the native part and the Squeak .image, which were glued in a single file in the old SqueakNOS, making it easier for Squeakers to touch the .image without having to set up the compiling environment. [done]
We needed to implement the switch to graphics mode before starting jumping to the interprenter. In the old SqueakNOS we did it from real mode (that's why we had so pour video support). But this time GRUB gave us control already in protected mode, so we couldn't just call the right INT. Luckily, GRUB already has the roots for supporting graphic video mode selection, and although it is not usuable in the standard GRUB distro, we could solve the problem quite easily, by copying the testvbe command into the setvbe command, and removing erverything after the command switches the video mode. We latter found out that the video memory address changes from box to box, so we also hacked the vbeprobe command to show the video memory address. [Done]
We also learned that GRUB supports part of the multiboot protocol and is able to load ELF binaries, this led us reduce the amount of assembly code to barely around 20 line. It's not that we are afraid of assembly, but if multiboot was supported in some other platform (Apple/PowerPC), porting SqueakNOS' kernel should be really easy. [Never tried]
At this point we had GRUB loading the kernel and loading the .image all in memory, we had some glue code to make the .image look like a file so Squeak's intepreter could start interpreting it. We coded console text output support, hacked some debugging messages in the interpreter loop, some stack traces here and there and figured out what Plugins needed to be compiled for Squeak to start up: we finally managed to make the interpreter loop run loose. [Done]
Phase 2: Make it breath
Add graphics mode support, and native IRQ setup and handling[Done]
After the nice suprise that GRUB had graphic mode switching almost ready to be used, we jumped into making it work for us, and we chose a graphic mode that is really nice AND maps 1 to 1 from Squeak's representation to video RAM: 1024x768x32 bits is our choice although any 32 bits mode would suit the needs. Hacking a ioShowDisplay() that works was easy: it's no more than a bounded memcpy(). With this we got some grphic output in SqueakNOS' screen!. [Done]
With the old SqueakNOS we had some problems at this point, mainly because Squeak does not refresh the screen upon entering, so this time we solved it really fast, by doing a "self currentWorld fullRepaintNeeded" after a small Delay, and voila! Squeak's full screen was shown! [Done]
Now we were facing the most delicate part: making IRQs properly work. Subtasks: set a void handler for every IRQ and enable IRQs (see that it doesn't halt). Add an infinite loop handler for the keyboard IRQ (see that it hangs on a keypress). Set a visible handler for the timer interrupt (see that it draws something on the screen... you can still see it in the upper right corner of SqueakNOS). Finally change all void handlers for code that will trigger a Squeak Semaphore and exit, also add the primitive to register the Semaphores (see that it doesn't hang, then see that the Semaphore gets triggered from inside Squeak). At last, loop waiting on the semaphore for keyboard's IRQ, read keyboard's port, and signla IRQ end all from inside Squeak. [Done]
SqueakNOS IRQ Handling
Squeak VM lets you signal Squeak Semaphores from the native world by calling signalSemaphoreWithIndex(). We want to serve IRQs from Squeak, using interpreted code, not native code. We seriously think that with hardware close to 1000 times faster than 20 years go we should be able to do it without any problems. Of course we cannot set the native IRQs handlers to jump to Squeak code, so our idea is to have a different Semaphore for every IRQ and have a Squeak Process with highIOPriority blocking on the Semaphore.
The code inside Squeak, taken from InterruptRequestDispatcher>>installOn: looks like:
There is an interesting detail in all this: the software interrupt ending (IRET) and the hardware IRQ ending (outb(0x20,0x20)) are detached in SqueakNOS, were they are almost always done at the same time in every other code we saw. This gives a really desirable result: The IRET lets the software continue, going back to the interpreter and letting the Process waiting on the Semaphore be rescheduled, however, the hardware part (Interrupt Controller) still thinks the IRQ has not been served yet, and will wait until the Squeak side of the handler (shown above) signals the end of the interrupt (aComputer interruptController signalEndOfInterrupt: interruptNumber.
We seriously believe that with computers close to 1000 times faster than 20 years ago this should be ok, however we are not sure yet if it may bring any problems latter, and we are open, although reluctant, to the possibility of coding some glue code, or even complete "device drivers" natively (or hopefully using Exupery). Probably for sound and video I/O, we'll see.
We were successfully using the very same scheme in the old SqueakNOS, so we just ported it to the new code base.
So, in short, IRQ handling [Done]
Phase 3: Give it awareness
Make Keyboard and Mouse work. This will let us code SqueakNOS from within SqueakNOS.[Done]
If we do this, we'll want to be able to save the changes, so we'll need some persistency. Our idea (taken, again, from the old SqueakNOS), is to implement serial port support[Done], and some type of File System over it.[Kind of Done]
Phase 4: Help it grow
Start codding a few HardwareDevices as model, specially some Network card support[Not started yet]
Code PCI Bus handler. [Done]
Code Network support.
AMD Lance PCNet-32 PCI Network Card (vmware). [It's working, could be improved]
Code CMOS support (hardware clock for example).[Parts Done]
Phase 5: Let it be
What to do
Ufff! There are tons of things to do. You can pick any to do :-)
Some in particular no order:
Add better video mode selectiong support. This mainly involves solving how GRUB can pass SqueakNOS' kernel information on what video mode is selected. Not too complicated, but as we already have video support, we won't stop to do it for now. We think it'll be better to do it for GRUB 2 than for GRUB Legacy, we'll see.
Test what Plugins can be compiled natively without any platform support. We have now LargeIntegerPlugin, Balloon support, and some other. We know there are quite a few that could be included just by drag and drop in VMMaker.
Reimplement PCKeyboard, remove everything not needed because the Sensor now is an EventSensor. Rework all the hardware part, looking at the right documentation. Move ps2 mouse support to a separate class (PS2Mouse).
Fix the Mouse cursor problems.
Somebody should check how the basic primitives have been implemented (nos/sqGluc.c) and say if they are Ok. We know some of them are not (ioRelinquishProcessorForMicroseconds() for example), but we need some VM expert to review them.
See how spoon and SqueakNOS can take advantage from each other (specially SqueakNOS from spoon).
See how Exupery can help SqueakNOS. Although we'd like to avoid native code as much as possible, we understand it may be necesary.
Saving the image from within Squeak (no native Hard Disk support). We think the easiest way is to copy the hole .image from RAM to a ByteArray using a primitive (primitiveReadMemory is alreay there), and then save this ByteArray using the Squeak Hard Disk "driver". It should work, not sure.
Related to the prebious: Smalltalk startUp is executed after saving, and we are not sure how this will affect HardwareDevices that are reinitialized without rebooting the system. I think we'll need a slightly different startUp scheme.
We may need to add primitives for external memory handling (a la heap), specially because some HardwareDevices may need to allocate locked memory (is this possible to lock a Squeak object so the GarbageCollector doesn't move it?). Even if it's possible to lock objects, some hardware needed memory to be located below the 640k barrier.
Although we doubt it, speed may be a problem for some HardwareDevices. In this case we'll need to add some native code support.
Too much power consumption (bad for laptops), because we have not implemented sqInt ioRelinquishProcessorForMicroseconds(sqInt mSecs) (it's a noop right now). We tried putting a hlt there, but it makes everything impossibly slow, and the keyboard and mouse don't work as expected. Our theory is that we need to increse the timer interrupt frequency (right now 18.2 hertz), and then, the hlt instruction should be enough, however, we don't really understand why the mouse and keyboard are not responding.
Apparently the interpreter loop doesn't check for process rescheduling as often as we'd like it to (for example, in VMWare, the mouse freezes when Squeak is opening a System Window). We are not sure how it really works, but as IRQ handling depends on Process scheduling, we may find some problems there