So, let’s talk envelopes. First what they are, and what they can do to the quality of a sound.

Demo here: http://www.benfarrell.com/labs/examples/envelopes-12-17/, but read on for how it all works!

An envelope is a real simple concept actually. Take any signal, like a sound. Maybe that sound plays at a constant volume. When you apply an “envelope” to that sound, you are changing the volume of that sound while it’s played. It might go up and down, back up again, whatever.

How it goes up and down, and the speed at which the volume is changed is up to the details of the envelope used.

You could create an envelope that takes 8 hours to complete. Maybe you want to go to sleep with some music, and then wake up in the morning with music. If you know it takes you 30 minutes to fall asleep, you’ll start the music playing at a loud volume. Over the next 30 minutes, you envelope your sound from loud to quiet, to off. In the morning, 30 minutes before you wake up, the envelope makes the music go from off, to quiet, to loud again. It wakes you up!

While this 8 hours illustrates how an envelope works, it’s a bit different when talking musical tones.

It’s not that different, however. We’re still talking about volume over time, but we’re talking milliseconds instead of hours or even minutes or seconds.

The character or personality of a musical tone can be changed greatly by altering the envelope. While we talk about this in terms of 1/1000th of a second, you don’t really notice the volume changing as you listen to the tone. Your ear doesn’t necessarily detect that the volume is going up and down.

Instead, the sound just has a different tonal quality! A piano for example wouldn’t sound as sharp if it didn’t go from no sound to loud that quickly. An accordion though, has a longer time as it goes from quiet to loud. And that quality – the “attack” (amongst other factors) create the personality of the tone.

Let’s talk specifics now. ADSR.

That’s Attack, Decay, Sustain, Release. The ADSR envelope is just one type of envelope, but it’s a popular one that’s been used in electronic music for decades.

• Attack is the period of time after the initial release, it’s typically the loudest part of the sound
• Decay is the phase while you’re going from the attack to the sustain – you’re “decaying”  the volume from this sharp initial phase to the normal volume phase
• Sustain is the normal phase of the sound.  It is typically less volume than the attack, and go on for an indefinite period of time, or for a specific amount of time
• Release is the draw down from the sustain period to no sound.  A fade out

the attack, decay, sustain, and release phases

I’ve talked about Audiolib.js in previous posts.  Audiolib is the Javascript library that enables you to make these dynamic sounds in Chrome and Firefox.

Audio programming isn’t easy though!  So while Audiolib helps out in awesome ways, it doesn’t have concepts of notes and music theory.  You have to tell it what frequencies to play, and if playing a chord, which individual frequencies make up the chord – which I explored and created my own helpers for.

The ADSR envelope is another example of something that Audiolib.js provides, however, it doesn’t provide any obvious usage for it.

There’s actually a VERY good reason for this.  While, we’re talking about envelopes on musical tones, the 8 hour sleepy-time envelope is another good usage example.  And that’s restricting envelopes to the volume of a sound.  There are tons more examples in audio synthesis that envelopes can be applied.  Like effects – you can apply a distortion effect to a sound (like a rock guitar).  But you can envelope the amount of distortion which is applied to the guitar.  This has nothing to do with volume, and everything to do with just how much of something is applied to something else.

So, I’d like to create a usage of our ADSR envelope that is limited to producing a musical tone – especially in the example of producing live sound by using a trigger (here it will be your computer/laptop keyboard).

Let’s start with our previous example where we extend the Audiolib.js Oscillator via a plugin.  We extended it to simply take a musical notation, like an “A” and set the correct frequency:

audioLib.generators('Note', function (sampleRate, notation, octave){
// extend Oscillator
for ( var prop in audioLib.generators.Oscillator.prototype) {
this[prop] = audioLib.generators.Oscillator.prototype[prop];
}
// do constructor routine for Note and Oscillator
var that = this;
 
// are we defining the octave separately? If so add it
if (octave) {
notation += octave;
}
that.frequency = Note.getFrequencyForNotation(notation);
that.waveTable = new Float32Array(1);
that.sampleRate = sampleRate;
that.waveShapes = that.waveShapes.slice(0);
}, {});

So let’s figure out how to work in an ADSR envelope. The usage for the Audiolib.js version is like so:

myEnvelope = audioLib.ADSREnvelope(sampleRate, attack, decay, sustain, release, sustainTime, releaseTime);

The parameters work like so:

1. Sample Rate:  The sample rate of the audio – I won’t go into it here, as it’s a basic setting for audiolib
2. attack – the amount of time (in milliseconds) that the attack phase takes to complete
3. decay – the amount of time (in milliseconds) that the decay phase takes to complete
4. sustain – the level of volume during the sustain phase (from 0 to 1)  - the default is 1
5. release – the amount of time it takes for the release phase to complete (in milliseconds)
6. sustainTime – the amount of time it takes for the sustain phase to complete (in milliseconds).  This param is pretty important though, because if you pass in null, the sustain period is indefinite.  Unless you call into the envelope with a trigger, it will continue being in the sustain phase forever
7. releaseTime – the amount of time between the release phase and the envelope looping around to the attack phase again

The envelope has 6 states of being (0-indexed inside the code):

1. Attack Phase
2. Decay Phase
3. Sustain Phase
4. Release Phase
5. Timed Sustain Phase
6. Timed Release Phase

To kick off our envelope, you trigger it:

myEnvelope.triggerGate(true);

Now we can start using our envelope.  The usage is a little weird to me, as the Audiolib.js library treats it like a “generator” which seems a bit complicated for what it does.  I just want a stream of numbers, but OK I’ll bite.  I’ll use it with the byte arrays and whatnot, as if it’s an Oscillator.

var buffer = new Float32Array(1);
myEnvelope.append(buffer, 1);

So, I’m just pulling one value at a time from the envelope, and putting it into my “buffer”.  But my buffer only has one value in it at any time.  Like I said, I feel like I’m being forced into using it in more complicated of a way than I need!  Maybe there’s something I’m missing.

Now, every time, I get create an audio data point in my sound, I can multiply the data point by my envelope.  Thus my envelope is applied!

this[this.waveShape]() * buffer[0];

To do this, I overrode the “getMix” function in the Audiolib Oscillator.  But I needed to do other things too.

Since I trigger the envelope with triggerGate, it will cycle through the attack and release to the sustain phase as the envelope is used, automatically.

It gets complicated at the release phase though.  After your computer/laptop keyboard is released, we need to enter the release phase.  But we’re still grabbing sound, because the release phase still produces sound as it fades.  So our Oscillator needs to track that it’s in the release phase (it knows by keeping track of the envelope.state, which is 3 or 5 here for release or timed release).

Then finally when it gets back to state 0, or the cycle begins again, we mark this note as “released”, so our buffer knows that it doesn’t need to pull from it anymore.  We have to be very careful of note pulling from more notes than we need, cause all this music stuff is hard work, and too many notes slows down your CPU and breaks the audio processing.

The above is if the sustain phase goes on as long as you hold the key!  What if it’s a timed sustain – then we need the logic in there to release the key when the envelope is done, rather than when our user releases the key.

Here’s our final Note.js code.  And here’s a controller for keeping track of keys being pressed.

It all comes together in my (Chrome only) demo:

http://www.benfarrell.com/labs/examples/envelopes-12-17/

The demo starts out by not using an envelope at all.  You’ll hear some clicky-ness when you press and release a key.  That’s because you’re hearing the transition between no sound and the abrupt start in the phase of the waveform.  It’s EXACTLY one of the reasons why envelopes are useful – to ease these transitions in and out.

When you turn on the envelope, you can start adjusting parameters to see how the different properties of attack, decay, sustain, and release affect the overall personality of the tone.

I just started checking out Audiolib.js this weekend.  Audiolib is a Javascript project to help you synthesize sounds and tones in JS.

Just recently, Google Chrome added the Web Audio API, while Firefox added the Audio Data API.  Both let you get low level with sound – you can add bytes to an audio buffer and manufacture sound in realtime.

The library is bundled with “sink.js” which handles the inconsistencies in the audio API between Firefox and Chrome to create a common API.  Once this is abstracted away, we can make some audio.

I’ve done the nitty gritty of writing bytes to the buffer in Flash, and from my limited playing Audiolib does a great job with this, and I’m ecstatic I don’t have to rewrite my Flash stuff!

Audiolib provides “generators” for you to work with.  A “generator” is something that makes a sound, and defines a common API for anything that makes a noise to write that sound to the audio buffer.

In the generators namespace/package, we have an Oscillator, White/Pink/Brown Noise, and a Sampler.  White noise is cool and all, but I jumped right to the Oscillator to make a real tone.  The Oscillator takes a sample rate and a frequency.

You can define it thusly:

dev = audioLib.AudioDevice(audioCallback /* callback for the buffer fills */, 2 /* channelCount */);
osc = audioLib.Oscillator(dev.sampleRate /* sampleRate */, 440 /* frequency */);

So, first we defined the device to use – we told it what to use for the audio callback, and how many channels there are (we have 2 channels…left and right).

The audio callback is a pretty standard thing in the world of audio. When the sound card is starting to run out of audio to play, it signals Javascript and says “Hey I need more audio! Fill me up with some bytes”. With our “audioCallback” function, we say “No problem sound card! I got your back! When you run low, please call our audio callback method – this will fill your audio buffer up”

Here’s an example of the audio callback:

function audioCallback(buffer, channelCount){
  osc.append(buffer, channelCount);
}

All that happens here, is that the buffer and channel count gets passed into the method, and we tell out Oscillator what this is, and this Oscillator generates the appropriate bytes and send them to the buffer.

But what is that generator…the Oscillator? Well, it creates a sound at a certain frequency. We pass in the frequency as we instantiate this “osc” object. When the audioCallback fires, it pulls this frequency from the osc object and plays a specific tone!

This is awesome! All the hard work is done for us – but one of the first things I wanted to tackle here was to make it easier for musicians to understand.

See, casual musicians don’t know that a middle “A” on a piano oscillates at 440hz – they just know that they hit a middle “A” to play the tone. I want something that makes sense for casual musicians, so my first small task with audiolib is to override Oscillator to take a notation vs a frequency.

Audiolib provides a plugin spec, but it tripped me up a little to understand how a “Generator” worked. See, they have a js/generation folder which contains the Oscillator. It’s all quite readable and self-documented.

However, the “append” method was not found in the Oscillator class at all! I was looking EVERYWHERE for it!

Eventually I found it in the “wrapper-end.js” script that’s outside of all the folders.

Basically, you create a generator like Oscillator, and you write it to the audiolib.generators namespace. By virtue of being in this namespace, the “wrapper-end” script comes along to wrap things up. It takes all the things in the audiolib.generators namespace and adds the generator base functions. Basically it makes all the generators extend the generator base class after the fact.

Kinda confusing and sneaky if you ask me! Oh well.

Regardless, the plugins are well documented, so I copied the sample generator plugin:

audioLib.generators('SemiClock', function (sampleRate){
	this.sampleRate = sampleRate;
}, {
	prevSample: 0.0,
	generate: function(sample){
		this.phase = + !this.phase;
	},
	getMix: function(){
		return this.phase;
	},
	phase: 0
});

Cool, so we’re naming a generator in the generator namespace, and defining some required methods, like “generate” and “getMix”.

Well, all I want to do is make a copy of all the objects in Oscillator and add a function and initialization routine to accept a musical notation and instantiate an oscillator with the correct frequency.

So here’s my take:

audioLib.generators('Note', function (sampleRate, notation, octave){
    // extend Oscillator
    for ( var prop in audioLib.generators.Oscillator.prototype) {
        this[prop] = audioLib.generators.Oscillator.prototype[prop];
    }
 
    // do constructor routine for Note and Oscillator
	var	self = this;
    self.octave = isNaN(octave) ? 4 : octave;
    self.setNotation(notation);
	self.waveTable	= new Float32Array(1);
	self.sampleRate = sampleRate;
	self.waveShapes	= self.waveShapes.slice(0);
}, {
    /* incremental tones as sharp notation */
    sharpNotations: ["A", "A#", "B", "C", "C#", "D", "D#", "E", "F", "F#", "G", "G#"],
 
    /* incremental tones as flat notation */
    flatNotations: ["A", "Bb", "B", "C", "Db", "D", "Eb", "E", "F", "Gb", "G", "Ab"],
 
    /**
     * notation setter
     * @param notation (octave is optional)
     */
    setNotation: function(nt) {
        this.notation = nt;
 
        // does notation include the octave?
        if ( !isNaN( parseInt(nt.charAt(nt.length -1)) )) {
            this.octave = parseInt(nt.charAt(nt.length -1));
            this.notation = nt.substr(0, nt.length-2);
        }
        this.frequency = this._getFrequencyForNotation(this.notation);
    },
 
    /**
     * turn a notation into a frequency
     * @param notation
     * @return frequency
     */
    _getFrequencyForNotation: function(nt) {
        var freq;
        var indx = this.sharpNotations.indexOf(nt);
 
        if (indx == -1) {
            indx = this.flatNotations.indexOf(nt);
        }
 
        if (indx != -1) {
            indx += (this.octave-4) * this.sharpNotations.length;
            freq = 440 * (Math.pow(2, indx/12));
        }
        return freq;
    }
});

Basically what I did here is extended the Oscillator class. I looped through all the properties of the Oscillator.prototype object and copying it onto the new class.

I then, went in to the Oscillator generator and copied the small initialization routine.

Then I injected my own methods! I replaced the frequency setting with a method to lookup the index of the musical notation in my preset arrays of notations. I also parsed out the octave (if it existed).

In the end, I have a “Note” class which is exactly the same as the Oscillator class – however, instead of passing in the frequency (ie 440hz), you pass in “A”, or “A4″ for the 4th octave.

To change the notation of the oscillator/note, simply call the setNotation again!

I’m definitely looking forward to exploring Audiolib.js more. It has a few concepts that I hadn’t gotten around to implementing in my own Flash framework yet, so I’m excited to see how it’s done.