How to create sampled instruments
Last updated at 4:03 pm UTC on 12 October 2007
(DISCLAIMER: I am a very novice novice with respect to computer music. But since I finally got sampled instruments to work in Squeak, I thought that I'd write down the process that I used. It's really VERY exciting to build one's own instruments, and the quality for even small samples is amazingly nice. So, I'm offering this tutorial. Please do correct my mistakes below and feel free to add on to the tutorial, e.g., identify good sound recording tools for Windows and other platforms. Thank you! – Mark Guzdial)
A sampled instrument is a voice/sound that Squeak can use for playing tracks of MIDI files (and for other uses) which is based off a recorded sound, not a generated waveform. Most of the instruments that Squeak uses (e.g., oboe1, bass1, etc.) are generated via FM Synthesis. A sampled instrument takes up more memory, but often has a richer and more complex sound.
Typically, you generate more than one sample for a given instrument, at different pitches. Squeak can play your sample at whatever pitch is necessary for the given piece, but the farther it has to shift your pitch, the more degradation there's going to be. Providing more samples (if possible) makes for better sound quality.
A key concept in a sampled instrument is the notion of a "loop." Your recording is only going to be so long (oh, say, a quarter second). A particular use of that instrument might require a sustained note for a second or more. A sampled instrument does this by playing a prelude (as you would strike the note), then repeating a portion of the recording for as long as necessary. Getting the loop points (start and end) just right is important for a good sound without clicks or oddities. Note: As long as you have loop points, the sampled instrument will work. It just might not sound good. I'm learning that sampling is an art – I think it's going to take me a while to learn to take really good samples and pick really good loop points. But the rewards in sound (even with bad loop points) are really nice.
- BUILD YOUR SAMPLE. The first step to building a sampled instrument is to record your sample. Now, you can do this with Squeak. Choose a new morph, under the Widgets submenu: RecordingControlsMorph. Record your sound, then click the Show button. You're now in the Wave Editor on your sample. There's lots that you can do here (under the "<>" menu), from generating a Fast Fourier Transform (FFT) to checking out the sound's envelope. The process of using the Wave Editor to define loop points is actually quite simple – see John Maloney's discussion below on how to do it.
I've started using D-SoundPro on a Mac (http://ourworld.compuserve.com/homepages/sdaino/). It allows you to record sounds, edit them, do all kinds of funky manipulations of them (add echo, reverb, etc), and most important for this purpose, define loop points. I've done by own loop points with limited success. D-SoundPro does have a nice interface for modifying loop points and making sure that the start and end match well. I've had better luck with D-SoundPro's "AutoLoop" option, which picks its own loop points. Sometimes the algorithm fails, but when it does (and I pick my own loop points), I usually find that I've got a yucky sounding sample. I'm relying on the AutoLoop alot right now, but the author of D-SoundPro suggests that the X-Fading Autoloop command results in a more pleasing sample.
A particularly nice feature of D-SoundPro is that it provides you a virtual keyboard that you can use for testing your loop points. When you press a key, it uses the loop to let you hear how the sample will sound when shifted to a given pitch.
- SAVING THE SOUND. The pre-made code in Squeak to create a sampled intrument expects recent AIF format. (Older tools, like SndSampler, use an older AIFF format that did not save loop points.) John Maloney used a tool called TransferStation that output AIF's with loop points stuffed in a markers section of the AIF header. D-SoundPro also saves loop points in that format.
- FORMAT THE DIRECTORY STRUCTURE. Create a directory to hold all your sampled instruments, say "Notes" or "Orchestra". Create a subdirectory for each of your instruments, with a name of instrumentName,' f', e.g., "guitar f" and "piano f". Put all your looped AIF files in the instrument subdirectory. Surprise to me was the name of the looped AIF files doesn't matter! John's code just reads them all in and creates the sampled instruments.
You certainly could write your own code to load in your sounds! But I've been using John's code since it's already there. :-)
NOTE: ONLY LOOPED AIF FILES SHOULD BE IN THE INSTRUMENT DIRECTORY! Any other kind of AIF file, or any other kind of file, will generate an error.
Surprisingly, though, stereo or not, sampling rate, etc. doesn't seem to make much difference. I've put in both stereo and mono samples, and 22050 and 44100 sampling rate samples, and all have worked okay. There are probably better and worse options here, but I haven't found them yet.
- LOAD IN THE INSTRUMENT.
SampledInstrument readSimpleInstrument: 'piano' fromDirectory: 'Guz G3:Squeak2.3:notes' is the command that I use to read in the "piano f" subdirectory of samples. (Note the lack of a final path delimiter on the all-the-instruments directory name.)
- USE THE INSTRUMENT. Easiest way to use the new instrument is from the ScorePlayerMorph. Choose a MIDI file from a filelist, and select the "play midi" option. A ScorePlayerMorph opens up. For each track, there is a pop-up menu of possible instruments for playing that track. The default instrument is "oboe1". Click on the pop-up, and you should find your instrument with "-f" added to it, e.g., "piano-f". Play the piece, and your sampled instrument will be used for the given track.
I've created a zip file with four instruments in it: http://guzdial.cc.gatech.edu/squeakers/samplenotes.zip 20071012: bad zip file
- A guitar, which is actually the demo samples from VSamp (a Macintosh sampler synthesizer), which I added loop points to via D-SoundPro.
- A piano, which has only two samples that I sampled myself. Doesn't sound great, but it does sound like a piano (at most pitches :-)
- A snare (drum), which shows that a single sample works, but not always great :-)
- And Mark saying "La" at three different pitches. Consider this a new "random/wierd" sound. ("Fur Elise" in La's is pretty funny :-)
Response from John Maloney
Here's how to use Squeak's editor to find loop points. You first need identify where the loop ends. I usually choose a point just before the sound level starts to tail off in the decay phase. Place the cursor there (typically at zero crossing; the cursor will turn blue), then push the "Set Loop End" button.
You next need to identify the approximate fundamental frequency. (Yes, it should be possible to guess this from the FFT, but that hasn't yet been attempted.) By eye, identify what appears to be one complete cycle of the wave, and place the cursor one wave before the loop endpoint. This needn't be sample-accurate. Now hit the "One Cycle" button. Now Squeak knows the approximate pitch of the sound and it won't get confused as it searches for loops.
You can now invoke the "choose loop start" menu command. This will search backwards from the loop end point for possible loops starts. Squeak performs an auto-correlation error calculation for each loop point and ranks them in a menu. (This calculation is somewhat expensive, but is only done the first time you ask for this menu after changing the loop end point.) At this point, you can select a possible loop, using both ranking and loop duration as a guide, and test it using the "play" button. The "play loop" button will play just the loop by itself, but it sometimes sounds a bit rougher than playing the loop in context via the "play" button.
Incidentally, the code that reads in sampled instruments is based on the idea that the set of samples is stored as a set of AIFF files in a folder labelled by the name of the instrument. Thus, the individual file names don't matter. But, as you found, they'd better be AIFF files! Two folder naming conventions that I use are: "-f" and "-p" distinguish loud and soft versions of the same instrument. "s.xxx" indicates a solo instrument, versus a section. For example, "s.violin-f" would be a single solo violin playing loud whereas "violin-p" would be an entire violin section playing softly.
Hope this helps!
From: Luciano Notarfrancesco
Organization: CORE SDI
To: Mark Guzdial
Subject: Creating sampled instruments
Hi, Mark and John!
I began to make sampled instruments. I found samples of the notes of a trumpet and a tenor saxophone; last night I spent some time creating some loops and it sounds great!!! In order to do this all in Squeak, I extended the John's AIFFFileReader to an AIFFFileReadWriter. I also added an 'edit sampled sound' option in the FileList menu for .aif and .aiff files (it opens a WaveEditor), and a 'save to AIFF file' item in the WaveEditor. I think this could be of interest to you, so I attached the change set.
http://guzdial.cc.gatech.edu/squeakers/AIFFReaderWriter.st 20071012: broken link