BitSyntax
Last updated at 9:55 am UTC on 7 October 2020
Introduction
This library extends Squeak with an extensible embedded domain-specific language (EDSL) for serializing and deserializing binary data into Squeak objects.
It is heavily inspired by Erlang's binaries, bitstrings, and binary pattern-matching. The Erlang documentation provides a good introduction to these features:
The EDSL implementation, however, is closely related to parser combinators.
See also my Racket bit-syntax implementation.
Tony Garnock-Jones, 2020
Packages
Available at https://squeaksource.com/BitSyntax.html.
- BitSyntax-Core - the implementation itself
- BitSyntax-Examples - examples, some of which are quite sophisticated, and all of which are drawn from real software
- BitSyntax-Help - the manual
The help text is quite extensive! Load all three packages (you can omit the examples if you like) and run
HelpBrowser openOn: BitSyntaxHelp
(or just look in the table of contents of the help system)
Example
Classes include a class-side method, bitSyntaxSpec, which produces either
- a single BitSyntaxCodec, or
- an ordered collection of BitSyntaxCodecs.
A BitSyntaxCodec includes a BitSyntaxSpecification along with two selectors, and is used by a BitSyntaxCompiler to generate methods named after the selectors for (de)serializing binary data according to the specification.
Input is provided to deserialization methods as anything that yields a PositionableStream when send readStream; for example, a ReadStream itself, or a ByteArray. If deserialization succeeds, a non-nil value is returned. If it fails, nil is returned.
Output is sent directly to a WriteStream by serialization methods.
Imagine a binary structure containing a 16-bit status code followed by an ASCII message prefixed with a 32-byte message length. We might want to deserialize such structure into instances of the following class:
Object subclass: #BitSyntaxTrivialExample
instanceVariableNames: 'statusCode statusMessage'
classVariableNames: ''
poolDictionaries: ''
category: 'BitSyntax-Help'
We specify the mapping between instance variables and the binary structure by implementing BitSyntaxTrivialExample bitSyntaxSpec:
bitSyntaxSpec
^ BitSyntaxCodec spec:
(2 bytesLE >> #statusCode),
(4 bytesLE
storeTemp: #messageLength
expr: 'statusMessage size'),
('messageLength' bytes ascii >> #statusMessage)
If we then run
BitSyntaxCompiler updateClass: BitSyntaxTrivialExample
the compiler will generate two methods, loadFrom: and saveTo:, on BitSyntaxTrivialExample's instance-side, as well as loadFrom: on its class-side. (See also BitSyntaxCompiler updateAllClasses.)
The following examples illustrate loading and saving data:
example1
^ BitSyntaxTrivialExample loadFrom: #[200 0 2 0 0 0 79 107]
example2
| inst data |
inst := BitSyntaxTrivialExample new
statusCode: 200;
statusMessage: 'Ok'.
data := ByteArray streamContents: [:w | inst saveTo: w].
^ data
The class-side method is:
loadFrom: s
"Autogenerated by BitSyntaxCompiler"
^ self new loadFrom: s
The instance-side methods generated are:
loadFrom: s__arg__
"Autogenerated by BitSyntaxCompiler"
| s__ messageLength |
s__ := s__arg__ readStream.
^ (((statusCode := s__ nextExactBitsLittleEndian: 16)
ifNotNil: [messageLength := s__ nextExactBitsLittleEndian: 32])
ifNotNil: [statusMessage := (s__ nextExactBits: (messageLength) *8)
ifNotNil: [:temp1__ | temp1__ asString]])
ifNotNil: [self]
saveTo: s__arg__
"Autogenerated by BitSyntaxCompiler"
| s__ temp1__ messageLength |
s__ := s__arg__.
s__ nextExactBitsLittleEndian: 16 put: statusCode.
messageLength := statusMessage size.
s__ nextExactBitsLittleEndian: 32 put: messageLength.
(temp1__ := ((statusMessage) asByteArray)) size = ((messageLength)* 8 // 8)
ifFalse: [BitSyntaxBitsSpecification badLength].
s__ nextPutAll: temp1__
