links to this page:    
View this PageEdit this PageUploads to this PageHistory of this PageTop of the SwikiRecent ChangesSearch the SwikiHelp Guide
Last updated at 9:55 am UTC on 7 October 2020


This library extends Squeak with an extensible embedded domain-specific language (EDSL) for serializing and deserializing binary data into Squeak objects.

It is heavily inspired by Erlang's binaries, bitstrings, and binary pattern-matching. The Erlang documentation provides a good introduction to these features:

The EDSL implementation, however, is closely related to parser combinators.

See also my Racket bit-syntax implementation.

Tony Garnock-Jones, 2020


Available at https://squeaksource.com/BitSyntax.html.

The help text is quite extensive! Load all three packages (you can omit the examples if you like) and run

HelpBrowser openOn: BitSyntaxHelp

(or just look in the table of contents of the help system)


Classes include a class-side method, bitSyntaxSpec, which produces either

A BitSyntaxCodec includes a BitSyntaxSpecification along with two selectors, and is used by a BitSyntaxCompiler to generate methods named after the selectors for (de)serializing binary data according to the specification.

Input is provided to deserialization methods as anything that yields a PositionableStream when send readStream; for example, a ReadStream itself, or a ByteArray. If deserialization succeeds, a non-nil value is returned. If it fails, nil is returned.

Output is sent directly to a WriteStream by serialization methods.

Imagine a binary structure containing a 16-bit status code followed by an ASCII message prefixed with a 32-byte message length. We might want to deserialize such structure into instances of the following class:

	Object subclass: #BitSyntaxTrivialExample
		instanceVariableNames: 'statusCode statusMessage'
		classVariableNames: ''
		poolDictionaries: ''
		category: 'BitSyntax-Help'

We specify the mapping between instance variables and the binary structure by implementing BitSyntaxTrivialExample bitSyntaxSpec:

		^ BitSyntaxCodec spec:
			(2 bytesLE >> #statusCode),
			(4 bytesLE
				storeTemp: #messageLength
				expr: 'statusMessage size'),
			('messageLength' bytes ascii >> #statusMessage)

If we then run

BitSyntaxCompiler updateClass: BitSyntaxTrivialExample

the compiler will generate two methods, loadFrom: and saveTo:, on BitSyntaxTrivialExample's instance-side, as well as loadFrom: on its class-side. (See also BitSyntaxCompiler updateAllClasses.)

The following examples illustrate loading and saving data:

		^ BitSyntaxTrivialExample loadFrom: #[200 0 2 0 0 0 79 107]

		| inst data |
		inst := BitSyntaxTrivialExample new
			statusCode: 200;
			statusMessage: 'Ok'.
		data := ByteArray streamContents: [:w | inst saveTo: w].
		^ data

The class-side method is:

	loadFrom: s
		"Autogenerated by BitSyntaxCompiler"
		^ self new loadFrom: s

The instance-side methods generated are:

	loadFrom: s__arg__
		"Autogenerated by BitSyntaxCompiler"
		| s__ messageLength |
		s__ := s__arg__ readStream.
		^ (((statusCode := s__ nextExactBitsLittleEndian: 16)
			ifNotNil: [messageLength := s__ nextExactBitsLittleEndian: 32])
			ifNotNil: [statusMessage := (s__ nextExactBits: (messageLength) *8)
					ifNotNil: [:temp1__ | temp1__ asString]])
			ifNotNil: [self]

	saveTo: s__arg__
		"Autogenerated by BitSyntaxCompiler"
		| s__ temp1__ messageLength |
		s__ := s__arg__.
		s__ nextExactBitsLittleEndian: 16 put: statusCode.
		messageLength := statusMessage size.
		s__ nextExactBitsLittleEndian: 32 put: messageLength.
		(temp1__ := ((statusMessage) asByteArray)) size = ((messageLength)* 8 // 8)
			ifFalse: [BitSyntaxBitsSpecification badLength].
		s__ nextPutAll: temp1__