Squeak sound architecture
Last updated at 6:07 am UTC on 28 July 2022
The main classes in Squeak's Sound Architecture are
Currently, you must read the code for these classes to understand how they work.
This page might become a better introduction to Squeak's sound architecture, if anyone both bored and knowledgeable on the subject wanders by. (Alternatively, if such a page exists, then go ahead and point this page in that direction!)
Here are some basics – but you'll still need to look in the image under 'Sound-Synthesis' for real comprehension.
AbstractSound subclasses:
- FMSound - Squeak's implementation of Frequency Modulation sound. Used in synthesis.
- LoopedSampledSound - This is used in synthesis. The sampled data will be pitched shifted to the appropriate note. Some portion of the samples are designated to be looped to provide sustained notes. YOu may generate these in Squeak - see the RecordingControlsMorph.
- SampledSound - this is your basic wav-type sound, generally to be played or mixed as-is, not used in further synthesis.
MIDI classes:
Others:
- SoundPlayer - This is the single point of exit for sampled (not midi) data. All methods are on the class side. Useful code for sound hackers: SoundPlayer shutDown to kill runaway sounds.
- SoundInputStream - records constantly and buffers its input, allowing you to read it when convenient.
- SoundBuffer - This is a souped up array which contains sample data. For stereo data, it contains 2 monuaral samples in each "slice" - so there is only one SoundBuffer for a stereo sound.
Feel free to expand/correct.
I have a few questions (I'll post if I figure them out, perhaps a newer image would help me)-
- Q: Is there a way to stream outgoing sound? The Midi Synth must do something like this, but I haven't figured out where, yet. There is a class named SoundInputStream - basically what I want is SoundOutputStream, something that would buffer a second or two worth of samples, and play them while I periodically write more to the buffer. (There are a couple reasons for this, one being that the sound playing mechanism has 50-100 ms of jitter when starting sounds, and this would solve that problem)
A: Write it!
- Q: Can you save a sampled sound in wav or other common formats? I see the code to load wave/AIFF, but not to save.
A: Write it!
- Q/RFE: I was considering working with large sounds in Squeak. Lets say 16 tracks, 30 minutes each, at 44khz/16 bits (mono) which might happen in a mixing setting. Thats 2,540,160,000 bytes! Ideally we could use sampled sounds and hope that the OS's virtual memory would transparently handle this, but this doesn't work very well in practice. I think the solution would be to have a new type of SampledSound whose samples sit on disk, but can get them transparently when asked. Obviously there is increased latency, but thats much better than trying to lock 2 gigs of virtual memory. The SoundOutputStream I mentioned above would come in handy here.
A: Make a suggestion!
- Q/RFE: Eventually, it would be nice to support 24-bit or more sampled sounds. If you do a lot of manipulation, error can build up to noticeable levels in a 16 bit sound. Example: Decrease the volume of a clip by 90% and then boost it to its previous level - it will sound awful because you've effectively lost many bits of resolution.
A: Make a suggestion!
- Q: SequentialSound walks an internal list of sounds, playing them in order. It checks each sound to see if it has already played. This is fine if your sounds are unique objects, but I want to play the same sound over a few times. Should I pass in copies of each sound, or modify SequentialSound to reset each sound in the list before it starts playing it? (SqueakPages 11/20/2002)
Ideas for Squeak Sound Architecture
