Benjamin Farrell

Javascript Continuous Integration with Google Closure

admin — Mon, 14 May 2012 02:13:44 +0000

In less than a week, I’ll be speaking at cfObjective! Why am I psyched about this, when in fact I don’t use Adobe’s server side tech Coldfusion at all? Well – see, cfObjective features a whole track dedicated to Javascript…they dub it jsObjective. Looks like I’m up there with some fantastic speakers!

If you’re short on time, already saw my preso, or whatever… here’s some demos I showed:

Using Type Enforcement with Google Closure and JSDoc style comments

Using a Two-Pass Compilation Process to heavily compact your code but leave things like jQuery intact

Lastly, here’s my presentation slides

What is this that I’m speaking about? Well I’m talking about a workflow I adopted when working on some fairly large or fairly complex applications. I’m used to workflows where I can create a project with many files – all organized to work towards a company library. This works great when you compile everything down to a binary – but not so well with Javascript when we aren’t in the practice of doing so.

With Javascript, you write it and run it. While it can be pretty easy to tweak and then refresh the page to view your work, it’s not the best for my workflow because when I complete a project I might have 10, 30, 50, or even more Javascript files that are in use in my project.

Without any sort of compilation process, we’re talking about tons of separate file requests, a large bloated tag full of

By all accounts, I’ve done what needs to be done, right?

Not exactly.

First of all, I might be thrown off by Googling “dojo progress bar” and coming up with dijit.ProgressBar. That’s lesson the first!

Dojo is divided up into 3 separate projects: Dojo, Dijit, and Dojox. Dojo is the center of it all. In it’s (almost) 90kb core, it offers a lot of the same stuff something like jQuery would offer. The rest of the Dojo namespace has extra things which we can pull in as we need them (more on this in a bit).

Dijits, or “Dojo Widgets”, are the UI components to the Dojo team. ProgressBar, being a UI component, lives here in this namespace.

Dojox features are the more experimental features. What’s here might not be central to the Dojo vision, but pretty close – close enough to be shipped with the rest of Dojo.

What you should understand, is that all of these features put together weigh in at 5.3MB when you grab the zip. You might say that’s too large to include in your project. You’d be right.

Fortunately you don’t have to force this on your users all at once (nor should you). The 90kb Dojo core is all you need at the start, but as you need the other features of Dojo, you can pull them in realtime.

In other languages this might be an “include” or “import”. In Dojo, it’s a “require”. So now the next piece of this puzzle, is to require the progress bar:

Fortunately for us, we can require the progress bar and anything the progress bar depends on is pulled in automatically because the Progress bar has its own include directives in the component.

So now, we’ve written the HTML/Dojo markup to include the progress bar, and we’ve required the necessary classes to support it. But it still doesn’t work!

There’s one last piece to this puzzle – the script tag. If you recall, I did say I wasn’t an idiot – I did include Dojo.js here…something like this in my head tag:

If we were simply using Dojo programmatically, this is all we’d need for basic functionality of a component…but we aren’t! We’re using some nifty declarative HTML markup, so there’s one last step. We need to tell Dojo to scan the tags on the page and look for Dijit components. We do this by altering our script tag to be:

Telling our script to parse on load, means that when Dojo is loaded, it will scan and register all of our tags.

Fantastic! Here’s what we have so far:

Unfortunately, while our progress bar exists, and it technically works, it’s looking kinda silly. That’s because we haven’t bothered to load our themes in yet. Without a theme picked, we get a barebones component which kind of looks like a broken wireframe.

At this point, I’m going to go to my handy Dojo Theme Tester and pick the Dojo theme that suits me best for the immediate future. Claro looks good, so I’ll use that. Not only do I need to include the Claro CSS and the Document CSS, but I also need to assign my body tag with a Claro class so that my entire document uses this theme.

Now we’re rocking! We have a Dijit Progress bar functioning on our webpage. Even though the point of this post is to simply get a UI component onto the page, I really can’t leave it hanging at 0%. Let’s go ahead and crank it to 80%.

For this, we’ll need a bit of script. For the sake of ease, I’m going to ignore all good sense and just put it in action in my HTML script tag.

We’re going to change our Javascript slightly – instead of a simple “require” directive, we’re going to have it do something after the required thing has loaded.

The simplest form of this is to write the following:

There are a couple of cool variations to this require script. First of all, you’ll notice that we’re requiring an array of Dojo components. Typically, we’ll just want to listen for “dojo/ready” and perhaps a custom script and start our application from there.

Secondly, you’ll notice that we didn’t pass in any parameters through our function. Each index in our require list corresponds to each function parameter. So I could do the following:

    require(["dijit/ProgressBar"], function(ProgressBar) {

See, when you require something, the parameter that comes through in the function is the constructor for that component. We can instantiate this object from this parameter!

However, what we REALLY want to do is tweak the progress bar we already have. So we need to find the existing progress bar on the page. You’ll see two references in the Dojo docs to “byID”. One is dijit.byId, the other is dojo.byId. The rule of thumb here is that if you want a reference to the DOM element, you would use dojo.byId, and if you want a component reference use dijit.byId.

Unfortunately, this is allllll old news in Dojo 1.7! It’s very confusing – but this way of selection is in lots of basic documentation! Now, we use the “dom” object to find something in the DOM, and the “registry” object to find something in Dojo’s component registry.

We’re going to need to step up our game here, and require the “dom”, the “ProgressBar”, and the “ready” component. In this way, it’s kind of a lot to ask and checklist for basic setup, however once you get rolling, I PROMISE it gets easier.

Our new script tag reads:

Here, we require all these necessary components. We then hinge the usage of our script on when dojo is “ready” and has loaded all that we required. We get a reference to the HTML declared component through the registry. From there we can update the progress bar as we want!

A full working example is here:

So there you go! A functional progress bar.

It’s unfortunate we had to go through such a long journey and learn so much to get to such a basic state – and that, I think, is Dojo’s biggest weakness. If you choose to power through this initial learning and setup phase I really think you’ll find Dojo to be absolutely fantastic.

There will be the moments when you bang your head on the desk because you missed something really stupid or the Dojo docs weren’t clear enough, however once you get into the rhythm of writing the code, it’s a fun and productive toolkit to work with.

I hope this post and future ones will help with that!

The Seven Stages of Learning Dojo

admin — Tue, 21 Feb 2012 03:55:58 +0000

So, yah I’m learning Dojo, though from what I’ve heard this could easily apply to ExtJS. It’s a different mindset to get accustomed to an enterprise Javascript framework, and comes with a bit of a steep learning curve.

And, so it begins….

“Ooooooh look at all these awesome features on http://dojotoolkit.org! They have a hot themetester, full documentation for like every single part of their toolkit. And GLORIOUS quickstarts!”

1. Shock and Denial

“WTF, I’ve copied this code EXACTLY, but it spews tons of errors, and doesn’t even come close to working.”

“I’ve looked up all these error messages, but I can’t find a straightforward answer. Everyone references concepts I’m not familiar with….and they keep telling me to hop on an IRC channel to talk to the developers. Is IRC still a thing? Do they have a BBS as well? Maybe I can learn the language of my 14.4 kilo-baud modem and ask IT how to do stuff in Dojo by chriping and clicking at it”

“It’s gotta be me….I’m an idiot, it can’t be this difficult to get basic stuff working”

2. Pain and Guilt

“Screw, it – let’s man up and just read massive quantities of documents and tutorials. Hell, I’m going to buy a book from 2008 even thought it’s horribly out of date. I’m just going to sit through this initial pain to get a better understanding of the whole system.”

“Am I really this dumb?”

3. Anger and Bargaining

“Dojo SUCKS, I hate it! It doesn’t work. I need to be a freakin rocket scientist to understand this system”

“jQuery, will you take me back? Please? I just want some data stores, and some binding, and some well written and documented plugins.”

4. Depression, Reflection, and Loneliness

“So lonely….the only materials I see online are from 2008, and from SitePen who charges for training, why would they help and give it all away for free?”

“Maybe if I had studied harder in school, this wouldn’t be a problem”

5. The Upward Turn

“Going back to the beginning and reading a bunch of stuff seems to have paid off, I think I’m beginning to understand now”

“Where in the hell did they tell you about this parseOnLoad attribute in the script tag? It’s not in any of the quickstart examples, but it’s freaking mandatory to get anything to work”

“Well, no WONDER, this example didn’t work, it was written in Dojo 1.6 before the AMD Loader, and I’m mixing and matching my code between 1.6 and 1.7 – and 1.7 has REALLY changed the game”

“Ahhhhhh…..I need to nest the requires, cause my require doesn’t know what the ready object is before it loads the core stuff”

6. Acceptance and Hope

“Now that I understand the basics, I really like Dojo. It rocks pretty hard!”

“There are things I don’t yet understand, but now that I have the big picture, I can roll with anything Dojo can throw at me, but all in all, I’m very psyched about all these new toys!”

7. Reconstruction and Working Through

“Dude, did I really just wire up a data store to a div which I made into a repeater, all inside a freaking accordion? Damn….awesome”

“I made a build system, and moved all my layers somewhere else, so I’m completely relying on my own custom build….I only get like 2 errors, and none of them block functionality. This will get better, but for now, it’s awesome enough”

Fin

And here I am. Though, I think I cycled through steps 3, 4, and 5 a few times.

Why Dojo? From a Flash/Flexer’s Perspective

admin — Tue, 21 Feb 2012 02:56:59 +0000

Hey you guys… Is your Twitter stream bombarded with Ted Patrick tweets and retweets about Sencha? Do you get Sencha ads targeted at you when you visit websites, like EVERYWHERE you go?

I am. It’s kind of annoying since I’m not looking to learn Sencha. Truthfully, though, you can hardly blame the current Sencha evangelist. After Adobe abandoned Flex sending it off to Apache, Ted seems to be leading a HUUUUUUGE marketing push trying to turn former Flex devs into Sencha/ExtJS devs. Ted is leading webinars, reaching out to people, posting on G+ threads, the whole deal.

I can hardly blame him because:

a) I follow him on Twitter, so it’s mostly my own damn fault

b) This is the perfect time to grab a bunch of disillusioned and lost Flex devs, and Ted is doing it in a perfectly ethical (although a bit overzealous way).

I looked into going “beyond” jQuery because of a G+ post by Jesse Warden claiming that ExtJS is the perfect compliment to enterprise development. He made the very apt analogy that Flash is to jQuery as Flex is to ExtJS/Sencha. It really made me re-think things because it had been drilled into my brain that jQuery “won”, and given the tide of developers (84.8% of all websites using a library use jQuery) it seemed like there was no reason to use anything else.

Turns out, there really is a use for the lesser used libraries/frameworks/toolkits. We can see this because even if jQuery is and always will be your library of choice, as you build more advanced applications you might want some binding, templating, MVC, or other action. AngularJS by Google is getting more widely used. It gives you binding, declarative HTML markup for data, and other MVC goodies. There’s Backbone.JS, Underscore.JS, and tons of others. A lot of folks seem to be combining all of these separate libraries together to custom fit their project. And I’m sure that works fantastically – I was doing similar things for a time.

As these separate projects get more and more features, you may now have feature overlap between 3 different libraries you’ve used in your project. It seems a LITTLE wasteful at best. At worst, maybe your dev team does a very basic thing 3 different ways because binding is available 3 different ways given each of the libraries you’re using.

Wouldn’t it be better, or even just worth looking into, if you could find a cohesive library/framework that offered all of these things together? Maybe it could even do a little more and help you make a massive Javascript application that loaded in components and scripts AS YOU NEED THEM instead of all at once in the beginning.

It looked like Dojo and ExtJS were the most mature offerings in terms of what I describe. I took a look at ExtJS by Sencha first. At first glance, it looks pretty awesome. You’ve got designer and animator tools. They are a wee pricey, but considering my Flash background, spending $279 on Sencha Designer and $199 on Sencha Animator doesn’t seem obscene.

What I can’t get past is having to spend $329 for a Javascript library license! When I’m doing Flash/Flex development, I use the Flash Builder IDE for coding and sometimes Flash Professional if I’m doing animations or graphics. And that’s cool. I chose to buy into these tools. I didn’t have to pay to use Actionscript 3 though. My fellow developers who may not do Flash all the time can crack open any text editor and edit my code. They can check it back into SVN and contribute to the project. They can launch the Flex SDK command line and build my project. None of this costs a dime.

If and when they get more invested in Flash/Flex, they might then want to buy Flash Builder or even IntelliJ for the advanced featureset including debugging, profiling, etc.

Not so with ExtJS. I’m buying a license to use the code base. Kind icky in my mind. If I start a project in ExtJS/Sencha and I pay my $329 for the seat, but then I want some more help on the project occasionally, each developer will then need to pay out $329 just to use/edit the code no matter what tools they use. If you are starting a major project, and plan the ExtJS cost into the budget, then great. But if you’re a lone developer who wants to start something which blossoms into something bigger – the licensing can be a major roadblock.

Moreso, if I’m a consultant and I’m paid x dollars to develop a web application, I can easily choose and pay to use ExtJS. After two months, I make a deliverable. It might be the type of project where I do the major architecture and first phase, but then it gets shipped off internationally for maintenance mode or a lower cost second phase of the project. Well, if you use ExtJS, you’ve just upped maintenance mode by $329 for every developer that needs to touch the project. You’ve just cost your client lots of money.

This is why I discounted Sencha and ExtJS and moved on to exploring Dojo. Dojo seems to offer MOST of what Sencha does. They both have a rich set of components, offer lazy loading of scripts so that your massive enterprise application doesn’t load several hundred scripts (or a huge 2MB block of script) before the application starts up. Both have a build system – Dojo’s runs on Rhino or Node, and is built on Closure or Shrinksafe. ExtJS seems to be built off of a Yahoo builder.

In terms of scripting, both seem to nudge you in the direction of organizing your Javascript into classes which can be based on straight inheritance, or mixins.

Both seem to offer a visual tool for design. Dojo’s Maqetta is free and can be used for jQuery, YUI, or Dojo. I don’t see myself being very productive in it though – I can lay things out, but it wasn’t obvious to me if I could wire in data or do anything extensive. Though I haven’t used Ext Designer, I have to imagine it’s a better product than Maqetta. But then I seriously question how much I’d use a good tool for this anyway. It’s like when everyone complains that design mode in Flex really sucks – but what if it didn’t? Would you actually use it, or just code everything? I think we’d use it only like 5% of the time. But this is all personal preference – ExtJS wins here, but only if you feel like you need a good design tool, which I don’t.

Dojo seems to offer one thing that ExtJS doesn’t though – and that’s declarative HTML markup. This may be where my ExtJS ignorance comes in, so feel free to correct me if I’m wrong, but to mark up a component like you might do with Flex’s MXML, you write a JSON object describing the component or layout you’re introducing in ExtJS. This is pretty cool – but with Dojo, you get to write REAL HTML. I can tag existing divs or other elements as a Dojo component and they’ll act like Dojo components. Yes, Dojo does offer templating, but you can choose which to use…or even to use both.

The worst part of Dojo? They don’t have a MASSIVE marketing and tutorial push. There is tons of ExtJS and Sencha material out there to learn from. Yes, there is tons of Dojo material, but much of it is older and not up to date. The material on the Dojo site is pretty technical and not very “quickstart-ish”. What quickstart material they do have, don’t seem to be big picture enough to inform you what’s really going on. I always had bad luck copying and pasting code from tutorials and having it work without spending hours debugging. Most of the trouble with Dojo material is the switch to new loading routines that happened with the 1.7 release in December 2011. I feel like when they release 2.0 sometime this year, we’ll have some better effort from the Dojo Foundation in terms of educating people on how to use the toolkit

All in all, Dojo and Sencha seem eerily similar from everything I read. I feel so lucky that by learning Dojo, I’ll probably feel right at home in ExtJS as well. For me it really comes down to the licensing that drives me to Dojo. If that wasn’t an obstacle for me, there were more than a few times I’ve been frustrated by Dojo in the past couple weeks of learning it that I’d been open to abandoning ship if there was another choice. I get the feeling though that there are similar frustrations with understanding ExtJS (helped in large part by their tutorials).

That’s why I’m team Dojo right now! However, I’ll continue using jQuery for the smaller projects. Dojo could easily replace jQuery, but for the smaller stuff when something minimal is all you need, there’s no sense in going against the grain and using something that not many people have experience with.

I’ll be posting more technical stuff on Dojo, but for now, this is how I got where I am!

iOS Video is Cranky!

admin — Sat, 28 Jan 2012 19:45:06 +0000

I’m close to wrapping up a decent sized iOS project, and my part was video playback. Prior to this, I’ve done tons of work with Flash video.

I hate to be labeled biased, but I’ve never appreciated the level of detail put into the Flash API before working with both HTML5 and iOS. Flash has tons of quality of service queries you can make, you have explicit control of bitrate on multibitrate streams, you have complete seek accuracy in streams. Virtually anything you could ever want, you got it!

Compare that to the AVPlayerFramework API for iOS. You can listen to status messages and put an observer on the current playhead time. That’s pretty much it. And those status messages come in the form of ready to “play”, “failed”, “played to the end”, “failed to play to the end” and “unknown”.

The lack of features actually isn’t too much of a problem for streaming playback. Streaming content is handled like a champ by the framework. A bad network connection might cause the stream to stutter, freeze, and stall – but as the connection gets better the video is recovered quite well automatically.

One of the complications presented by Apple’s guidelines is that an application that streams must provide an audio only low-bitrate stream to compliment the higher bitrate video/audio streams. The AVPlayer framework does a great job doing what it needs to do switching the user to the best available stream. Unfortunately, if bandwidth gets really bad, the audio only stream is a valid option – and iOS WILL switch to it.

It presents an interesting conundrum. Video looks broken to the user, and especially your clients who are footing the bill. I’d almost rather have the video stammer and stutter than to switch to this audio only bitrate – at least then, most tech savvy users would know that they are having network connectivity issues.

The other problem is the ability to seek to exact positions in a stream using the AVPlayer in anything less than iOS 5. A stream is broken up into chunks. Those chunks are transport segment (.ts) files. Typically those chunks will be around 7-10 seconds each. Turns out that if you want to seek to a specific second of your stream, it better lie at the beginning of a segment, otherwise your seek accuracy is (at worst) 7-10 seconds off.

Past that, streams are pretty decent, like I said. What got frustrating though, was simple progressive video!

Progressive is fairly easy to play. You load, play and go. After loading, you need to wait for the item ready status, and then tell the content to play. By nature, progressive loads welllll….progressively. You can start playing back the video before the entire thing finishes loading.

The limited API of AVPlayer seems to present some problems here. Your video is only “ready” once. What happens if you are progressively playing, but suddenly your connection drops low, and you can’t quite finish playing the content because you’ve reached the end of whats been playing? I’ll tell you what happens….your video pauses. While paused, the rest of the video will finish loading. Unfortunately, there is no status message to inform you that the video is ready to play again. So there it sits. Paused.

Drastic shortcomings call for stupid hacks. Such as setting up a timer to fire off every few seconds and tell the content to play. If the content is currently playing, the call to play will do nothing. However, if it’s stalled and paused, we can keep the video moving along automatically.

Another shortcoming I ran across is an unfortunate reality with the AVQueuePlayer. The AVQueuePlayer is like the AVPlayer, but you can playlist things. I chose to use this type of player due to other requirements in my project. For my work, I wasn’t really using it as a queue. Instead, I’d simply replace the current item in the queue to play a new asset. Unfortunately this method of doing things didn’t seem very reliable. If an item failed in my queue, I could add further items and make them play, however I wouldn’t get any status events about the item. That means I’d never get told when it was ready to play, when it completed, when it failed. It was really a showstopper. Sometime, even if the item didn’t fail to play, I’d see that subsequent items wouldn’t produce status changes.

I ended up having to destroy the player and starting fresh each time – oh well. Even though I wasn’t using it as a queue before, now I REALLY wasn’t using it as a queue.

So there it is. I got through it, but video was a little cranky!

The ADSR Envelope with Audiolib.js

admin — Sat, 17 Dec 2011 21:01:17 +0000

So, let’s talk envelopes. First what they are, and what they can do to the quality of a sound.

Demo here: http://www.benfarrell.com/labs/examples/envelopes-12-17/, but read on for how it all works!

An envelope is a real simple concept actually. Take any signal, like a sound. Maybe that sound plays at a constant volume. When you apply an “envelope” to that sound, you are changing the volume of that sound while it’s played. It might go up and down, back up again, whatever.

How it goes up and down, and the speed at which the volume is changed is up to the details of the envelope used.

You could create an envelope that takes 8 hours to complete. Maybe you want to go to sleep with some music, and then wake up in the morning with music. If you know it takes you 30 minutes to fall asleep, you’ll start the music playing at a loud volume. Over the next 30 minutes, you envelope your sound from loud to quiet, to off. In the morning, 30 minutes before you wake up, the envelope makes the music go from off, to quiet, to loud again. It wakes you up!

While this 8 hours illustrates how an envelope works, it’s a bit different when talking musical tones.

It’s not that different, however. We’re still talking about volume over time, but we’re talking milliseconds instead of hours or even minutes or seconds.

The character or personality of a musical tone can be changed greatly by altering the envelope. While we talk about this in terms of 1/1000th of a second, you don’t really notice the volume changing as you listen to the tone. Your ear doesn’t necessarily detect that the volume is going up and down.

Instead, the sound just has a different tonal quality! A piano for example wouldn’t sound as sharp if it didn’t go from no sound to loud that quickly. An accordion though, has a longer time as it goes from quiet to loud. And that quality – the “attack” (amongst other factors) create the personality of the tone.

Let’s talk specifics now. ADSR.

That’s Attack, Decay, Sustain, Release. The ADSR envelope is just one type of envelope, but it’s a popular one that’s been used in electronic music for decades.

Attack is the period of time after the initial release, it’s typically the loudest part of the sound
Decay is the phase while you’re going from the attack to the sustain – you’re “decaying” the volume from this sharp initial phase to the normal volume phase
Sustain is the normal phase of the sound. It is typically less volume than the attack, and go on for an indefinite period of time, or for a specific amount of time
Release is the draw down from the sustain period to no sound. A fade out

the attack, decay, sustain, and release phases

I’ve talked about Audiolib.js in previous posts. Audiolib is the Javascript library that enables you to make these dynamic sounds in Chrome and Firefox.

Audio programming isn’t easy though! So while Audiolib helps out in awesome ways, it doesn’t have concepts of notes and music theory. You have to tell it what frequencies to play, and if playing a chord, which individual frequencies make up the chord – which I explored and created my own helpers for.

The ADSR envelope is another example of something that Audiolib.js provides, however, it doesn’t provide any obvious usage for it.

There’s actually a VERY good reason for this. While, we’re talking about envelopes on musical tones, the 8 hour sleepy-time envelope is another good usage example. And that’s restricting envelopes to the volume of a sound. There are tons more examples in audio synthesis that envelopes can be applied. Like effects – you can apply a distortion effect to a sound (like a rock guitar). But you can envelope the amount of distortion which is applied to the guitar. This has nothing to do with volume, and everything to do with just how much of something is applied to something else.

So, I’d like to create a usage of our ADSR envelope that is limited to producing a musical tone – especially in the example of producing live sound by using a trigger (here it will be your computer/laptop keyboard).

Let’s start with our previous example where we extend the Audiolib.js Oscillator via a plugin. We extended it to simply take a musical notation, like an “A” and set the correct frequency:

audioLib.generators('Note', function (sampleRate, notation, octave){
// extend Oscillator
for ( var prop in audioLib.generators.Oscillator.prototype) {
this[prop] = audioLib.generators.Oscillator.prototype[prop];
}
// do constructor routine for Note and Oscillator
var that = this;
 
// are we defining the octave separately? If so add it
if (octave) {
notation += octave;
}
that.frequency = Note.getFrequencyForNotation(notation);
that.waveTable = new Float32Array(1);
that.sampleRate = sampleRate;
that.waveShapes = that.waveShapes.slice(0);
}, {});

So let’s figure out how to work in an ADSR envelope. The usage for the Audiolib.js version is like so:

myEnvelope = audioLib.ADSREnvelope(sampleRate, attack, decay, sustain, release, sustainTime, releaseTime);

The parameters work like so:

Sample Rate: The sample rate of the audio – I won’t go into it here, as it’s a basic setting for audiolib
attack – the amount of time (in milliseconds) that the attack phase takes to complete
decay – the amount of time (in milliseconds) that the decay phase takes to complete
sustain – the level of volume during the sustain phase (from 0 to 1) - the default is 1
release – the amount of time it takes for the release phase to complete (in milliseconds)
sustainTime – the amount of time it takes for the sustain phase to complete (in milliseconds). This param is pretty important though, because if you pass in null, the sustain period is indefinite. Unless you call into the envelope with a trigger, it will continue being in the sustain phase forever
releaseTime – the amount of time between the release phase and the envelope looping around to the attack phase again

The envelope has 6 states of being (0-indexed inside the code):

Attack Phase
Decay Phase
Sustain Phase
Release Phase
Timed Sustain Phase
Timed Release Phase

To kick off our envelope, you trigger it:

myEnvelope.triggerGate(true);

Now we can start using our envelope. The usage is a little weird to me, as the Audiolib.js library treats it like a “generator” which seems a bit complicated for what it does. I just want a stream of numbers, but OK I’ll bite. I’ll use it with the byte arrays and whatnot, as if it’s an Oscillator.

var buffer = new Float32Array(1);
myEnvelope.append(buffer, 1);

So, I’m just pulling one value at a time from the envelope, and putting it into my “buffer”. But my buffer only has one value in it at any time. Like I said, I feel like I’m being forced into using it in more complicated of a way than I need! Maybe there’s something I’m missing.

Now, every time, I get create an audio data point in my sound, I can multiply the data point by my envelope. Thus my envelope is applied!

this[this.waveShape]() * buffer[0];

To do this, I overrode the “getMix” function in the Audiolib Oscillator. But I needed to do other things too.

Since I trigger the envelope with triggerGate, it will cycle through the attack and release to the sustain phase as the envelope is used, automatically.

It gets complicated at the release phase though. After your computer/laptop keyboard is released, we need to enter the release phase. But we’re still grabbing sound, because the release phase still produces sound as it fades. So our Oscillator needs to track that it’s in the release phase (it knows by keeping track of the envelope.state, which is 3 or 5 here for release or timed release).

Then finally when it gets back to state 0, or the cycle begins again, we mark this note as “released”, so our buffer knows that it doesn’t need to pull from it anymore. We have to be very careful of note pulling from more notes than we need, cause all this music stuff is hard work, and too many notes slows down your CPU and breaks the audio processing.

The above is if the sustain phase goes on as long as you hold the key! What if it’s a timed sustain – then we need the logic in there to release the key when the envelope is done, rather than when our user releases the key.

Here’s our final Note.js code. And here’s a controller for keeping track of keys being pressed.

It all comes together in my (Chrome only) demo:

http://www.benfarrell.com/labs/examples/envelopes-12-17/

The demo starts out by not using an envelope at all. You’ll hear some clicky-ness when you press and release a key. That’s because you’re hearing the transition between no sound and the abrupt start in the phase of the waveform. It’s EXACTLY one of the reasons why envelopes are useful – to ease these transitions in and out.

When you turn on the envelope, you can start adjusting parameters to see how the different properties of attack, decay, sustain, and release affect the overall personality of the tone.

Chords and Arpeggiators with Audiolib.js

admin — Tue, 13 Dec 2011 03:19:25 +0000

So, as you may have noticed with my recent flurry of blog posts I’ve been playing with Audiolib.js. I can’t thank the author Jussi Kalliokoski enough for getting this project going. When I did my Flash version, I spent many moons getting the audio buffer working correctly, then the oscillators, and I hadn’t even gotten started on effects so much yet.

But audiolib.js has all of this, which means I can focus on the usage with the API rather than making the low level audio stuff work.

For example, Chords and Arpeggiation!
Sorry! This only works in the latest version of Chrome

After a couple nights with the project, I extended out the standard Oscillator with a Note. I’ve since refactored a bit to do the minimum amount of music theory in the Note Oscillator object itself, and offload into a different object – but the net effect is that instead of instantiating a new Oscillator with a frequency, we can instantiate a new Note with a musical notation. For example with Oscillator, we’d say 440hz, with Note, we’d say “A” or “A4″ (if you don’t want to assume the 4th octave).

The obvious extension to this is groups of notes – or rather Chords. There are two challenges here. The first is to convert a chord notation to array of notes. For example, if I say “Cm”, or C Minor, I want an array such as ["C", "Eb", "G"]. We need to convert the C Minor notation into the implied triad of 3 notes using the minor scale. Also possible is a 6th, 7th, 9th, 11th, or 13th. In fact, now that I type, I realize I never did a 5th, which if I recall correctly, omits the middle note of a triad.

It all came out fairly well. I don’t think augmentation, sustain, or diminish work properly yet, but the chord structures are there. The simplest way to use this is to use ChordFactory instead of Chord. Simply call ChordFactory.createNotations(“C”, 4) to get a list of notes (strings) or ChordFactory.createNotes(“C”, 4) to get an array of Note Oscillators.

ChordConstants = {
   MAJOR_TRIAD: "maj",
   MINOR_TRIAD: "m",
   SEVENTH: "7",
   MINOR_SEVENTH: "m7",
   MAJOR_SEVENTH: "maj7",
   NINTH: "9",
   MINOR_NINTH: "m9",
   MAJOR_NINTH: "maj9",
   ELEVENTH: "11",
   THIRTEENTH: "13",
   SIXTH: "6",
   MINOR_SIXTH: "m6",
   SUSTAIN: "sus",
   AUGMENTED: "aug",
   DIMINISHED: "dim"
};
 
ChordFactory = {
    /**
     * create a list of notations from chord
     * @param chord notation
     * @param notation array (individual notes)
     */
    createNotations: function createNotations(notation, octave) {
        var chord = new Chord(notation, octave);
        return chord.getNotations();
    },
 
    /**
     * create an array of note oscillators using the audiolib framework
     * @param sampleRate
     * @param notation
     */
    createNotes: function createNotes(sampleRate, notation) {
        var chord = new Chord(notation);
        nts = chord.getNotations();
 
        var osc;
        var oscs = [];
        for (var nt in nts) {
            osc = audioLib.generators.Note(sampleRate, nts[nt])
            oscs.push(osc);
        }
        return oscs;
    }
}
 
function Chord(notation, octave) {
    var that = this;
 
    /** root note of chord */
    that._root = "C";
 
    /** octave of root */
    if (octave) {
        that._rootOctave = octave;
    } else {
        that._rootOctave = null;
    }
 
    /** chord notation */
    that._notation = notation ? notation : "Cmaj";
 
    /** notes in built chord */
    that._notes = [];
 
    /**
     * get notes from built chords
     *
     * @return notes
     */
    this.getNotations = function() {
        return this._notes;
    }
 
    /**
     * chord notation setter
     *
     * @param notation
     */
    this.setNotation = function(value) {
        this._notation = value;
        this.buildChord();
    }
 
    /**
     * chord notation getter
     *
     * @return notation
     */
    this.getNotation = function() {
        return this._notation;
    }
 
    /**
     * root note setter
     *
     * @param root
     */
    this.setRoot = function(value) {
        this._root = value;
        this.buildChord();
    }
 
    /**
     * root note getter
     *
     * @return root note
     */
    this.getRoot = function() {
        return this._root;
    }
 
    /**
     * root octave setter
     *
     * @param octave
     */
    this.setRootOctave = function(value) {
        this._rootOctave = value;
        this.buildChord();
    }
 
    /**
     * root octave getter
     *
     * @return root octave
     */
    this.getRootOctave = function() {
        return this._rootOctave;
    }
 
    /**
     * get notes in major triad
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.majorTriad = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,3);
    }
 
    /**
     * get notes in minor triad
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.minorTriad = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, false, false, rootOctave).slice(0,3);
    }
 
    /**
     * get notes in seventh chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.seventh = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,4);
    }
 
    /**
     * get notes in major seventh chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.majorSeventh = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,4);
    }
 
    /**
     * get notes in minor seventh chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.minorSeventh = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, false, false, rootOctave).slice(0,4);
    }
 
    /**
     * get notes in ninth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.ninth = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,5);
    }
 
    /**
     * get notes in major ninth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.majorNinth = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,5);
    }
 
    /**
     * get notes in minor ninth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.minorNinth = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, false, false, rootOctave).slice(0,5);
    }
 
    /**
     * get notes in eleventh chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.eleventh = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,6);
    }
 
    /**
     * get notes in major eleventh chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.majorEleventh = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,6);
    }
 
    /**
     * get notes in minor eleventh chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.minorEleventh = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, false, false, rootOctave).slice(0,6);
    }
 
    /**
     * get notes in thirteenth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.thirteenth = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,7);
    }
 
    /**
     * get notes in major thirteenth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.majorThirteenth = function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, true, false, rootOctave).slice(0,7);
    }
 
    /**
     * get notes in minor thirteenth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.minorThirteenth= function(root, rootOctave) {
        return this.getStandardNotesInChordMakeup(root, false, false, rootOctave).slice(0,7);
    }
 
 
    /**
     * get notes in sixth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.sixth = function(root, rootOctave) {
        var keySig = Note.notesInKeySignature(root, true, rootOctave);
        var keys = new Array();
        keys.push(keySig[0], keySig[2], keySig[4], keySig[5]);
        return keys;
    }
 
    /**
     * get notes in minor sixth chord
     *
     * @param root note
     * @param root octave
     * @return notes
     */
    this.minorSixth = function(root, rootOctave) {
        var keySig = Note.notesInKeySignature(root, false, rootOctave);
        var keys = new Array();
        keys.push(keySig[0], keySig[2], keySig[4], keySig[5]);
        return keys;
    }
 
    /**
     * sustain chord
     *
     * @param notes
     * @param direction to sustain
     * @return notes
     */
    this.sustain = function(notes, sus) {
        sus = (sus == undefined) ? 4 : sus;
        // grab the third in the chord
        var third = notes[1];
        var notations = Note.sharpNotations;
        var thirdIndex = Note.sharpNotations.indexOf(third);
        if (thirdIndex == -1) {
            notations = Note.flatNotations;
            thirdIndex = Note.flatNotations.indexOf(third);
        }
 
        if (sus==2) {
            // lower the third one half step
            if (thirdIndex-1 < 0) {
                notes[1] = notations[notations.length-1];
            } else {
                notes[1] = notations[thirdIndex-1];
            }
        } else { // assume sus == 4
            // raise the third one half step
            if (thirdIndex+1 >= notations.length) {
                notes[1] = notations[0];
            } else {
                notes[1] = notations[thirdIndex+1];
            }
        }
        return notes;
    }
 
    /**
     * augment chord
     *
     * @param notes
     * @return notes
     */
    this.augment = function(notes) {
        // grab the fifth in the chord
        var fifth = notes[2];
        var notations = Note.sharpNotations;
        var fifthIndex = Note.sharpNotations.indexOf(fifth);
        if (fifthIndex == -1) {
            notations = Note.flatNotations;
            fifthIndex = Note.flatNotations.indexOf(fifth);
        }
 
        // raise the fifth one half step
        if (fifthIndex+1 >= notations.length) {
            notes[2] = notations[0];
        } else {
            notes[2] = notations[fifthIndex+1];
        }
        return notes;
    }
 
    /**
     * get all standard notes in a chord, from triad to thirteenth
     *
     * @param root note
     * @param major key (true/false)
     * @param major chord (true/false)
     * @param root octave
     * @return notes array
     */
    this.getStandardNotesInChordMakeup = function(root, majorKey, majorChord, octave) {
        majorKey = (majorKey == undefined) ? true : majorKey;
        majorChord = (majorChord == undefined) ? false : majorChord;
 
        var majKeySig = Note.notesInKeySignature(root, true, octave);
        var minKeySig = Note.notesInKeySignature(root, false, octave);
 
        // grab the next octave if we need it
        var majKeySig2 = Note.notesInKeySignature(root, true, octave+1);
        var minKeySig2 = Note.notesInKeySignature(root, false, octave+1);
        var notes;
        if (majorKey && majorChord) {
            // C Major Seventh for example
            notes = [ majKeySig[0], majKeySig[2], majKeySig[4], majKeySig[6], majKeySig2[1], majKeySig2[3] ];
        } else if (!majorKey && majorChord) {
            // C Minor Seventh for example
            notes = [ minKeySig[0], minKeySig[2], minKeySig[4], minKeySig[6], minKeySig2[1], minKeySig2[3] ];
        } else if (majorKey && !majorChord) {
            // C Seventh for example
            notes = [ majKeySig[0], majKeySig[2], majKeySig[4], minKeySig[6], majKeySig2[1], minKeySig2[3] ];
        } else if (!majorKey && !majorChord) {
            // C Seventh for example
            notes = [ majKeySig[0], minKeySig[2], majKeySig[4], minKeySig[6], majKeySig2[1], minKeySig2[3] ];
        }
        return notes;
    }
 
    /**
     * convert notation to note list
     *
     * @param notation
     * @param use the octave in the notation
     * @return note list
     */
    this.notesFromChordNotation = function(notation, octave) {
        var root;
        var major = 0;
        var chordType;
 
        // find root
        if (notation.charAt(1) == "#" || notation.charAt(1) == "b") {
            root = notation.substring(0, 2);
            notation = notation.substring(2);
        } else {
            root = notation.substring(0, 1);
            notation = notation.substring(1);
        }
 
        // major or minor? (3 states - 1 is on, -1 is off, 0 is unspecified)
        if ( notation.substr(0, 3) == "maj") {
            major = 1;
            notation = notation.substring(3);
        } else if (notation.substr(0, 1) == "m") {
            major = -1;
            notation = notation.substring(1);
        }
 
        // set chord type
        if ( notation.charAt(0) == "6" ) {
            if (major == -1) {
                chordType = ChordConstants.MINOR_SIXTH;
            } else {
                chordType = ChordConstants.SIXTH;
            }
            notation = notation.substring(2);
        } else if ( notation.charAt(0) == "7" ) {
            if (major == 0) {
                chordType = ChordConstants.SEVENTH;
            } else if (major == 1) {
                chordType = ChordConstants.MAJOR_SEVENTH;
            } else if (major == -1) {
                chordType = ChordConstants.MINOR_SEVENTH;
            }
            notation = notation.substring(1);
        } else if ( notation.charAt(0) == "9" ) {
            if (major == 0) {
                chordType = ChordConstants.NINTH;
            } else if (major == 1) {
                chordType = ChordConstants.MAJOR_NINTH;
            } else if (major == -1) {
                chordType = ChordConstants.MINOR_NINTH;
            }
            notation = notation.substring(1);
        } else if ( notation.substr(0,2) == "11" ) {
            chordType = ChordConstants.ELEVENTH;
            notation = notation.substring(2);
        } else if ( notation.substr(0,2) == "13" ) {
            chordType = ChordConstants.THIRTEENTH;
            notation = notation.substring(2);
        } else {
            if (major == 1 || major == 0) {
                chordType = ChordConstants.MAJOR_TRIAD;
            } else {
                chordType = ChordConstants.MINOR_TRIAD;
            }
        }
        var notes = this.notesFromChordType(chordType, root, octave);
 
        // modify note set if needed
        var modifier = notation;
        switch ( modifier.substr(0,3) ) {
            case ChordConstants.AUGMENTED:
                notes = augment(notes);
                break;
 
            case ChordConstants.DIMINISHED:
                // to do
                break;
 
            case ChordConstants.SUSTAIN:
                var param = int(modifier.charAt(3));
                notes = sustain(notes, param);
                break;
        }
 
        return notes;
    }
 
    /**
     * get notes from chord types
     *
     * @param type
     * @param chord root
     * @return notes
     */
    this.notesFromChordType = function(type, root, rootOctave) {
        switch ( type ) {
            case ChordConstants.SIXTH:
                 return this.sixth(root, rootOctave);
 
            case ChordConstants.MINOR_SIXTH:
                 return this.minorSixth(root, rootOctave);
 
            case ChordConstants.SEVENTH:
                 return this.seventh(root, rootOctave);
 
            case ChordConstants.MINOR_SEVENTH:
                 return this.minorSeventh(root, rootOctave);
 
            case ChordConstants.MAJOR_SEVENTH:
                 return this.majorSeventh(root, rootOctave);
 
            case ChordConstants.NINTH:
                 return this.ninth(root, rootOctave);
 
            case ChordConstants.MINOR_NINTH:
                 return this.minorNinth(root, rootOctave);
 
            case ChordConstants.MAJOR_NINTH:
                 return this.majorNinth(root, rootOctave);
 
            case ChordConstants.ELEVENTH:
                 return this.eleventh(root, rootOctave);
 
            case ChordConstants.THIRTEENTH:
                 return this.thirteenth(root, rootOctave);
 
            case ChordConstants.MAJOR_TRIAD:
                 return this.majorTriad(root, rootOctave);
 
            case ChordConstants.MINOR_TRIAD:
                 return this.minorTriad(root, rootOctave);
 
            default:
                 return this.majorTriad(root, rootOctave);
        }
    }
 
    /**
     * build the chord given the parameters set in this class
     */
    this.buildChord = function() {
        this._notes = [];
        var notations = this.notesFromChordNotation(this._notation, this._rootOctave);
        for (var c = 0; c < notations.length; c++) {
            this._notes.push(notations[c]);
        }
    }
 
    // do a build based on initial params
    that.buildChord();
}

Namespacing will probably come when I think of a name for this project!

Anyway, we have some code that creates chord structures! Yay! Now, what about sending this to the buffer for mixing. The most basic example in audiolib.js is like so:

function audioCallback(buffer, channelCount){
    // Fill the buffer with the oscillator output.
    osc.append(buffer, channelCount);
}

This takes the Oscillator (or any audio generator for that matter) and generates some bytes to send to the audio buffer. Audiolib had some examples of mixing, but I didn’t care for them so much, as they seemed a little overly complicated with whatever idea of “leads” they had – I was a bit confused. In the end, I came up with something that worked a little better for me, but is probably virtually the same concept:

function audioCallback(buffer, channelCount){
    // get a list of generators from our keyboard controller object - it keeps tracks of keys pressed
    var gens = ctrl.pull();
    var bl = buffer.length;
    var gl = gens.length;
 
	// loop through each sample in the buffer
	for (current=0; current<bl; current+= channelCount){
		sample = 0;
                // here we're combining samples from our list of generators
                // and combining down into one sample
                for (i=0; i<gl; i++){
			gens[i].generate();
			sample += gens[i].getMix()*0.5;
		}
 
		// Here we write this sample to each channel we have (I'm just doing 2 channels)
		for (n=0; n<channelCount; n++){
			buffer[current + n] = sample;
		}
	}
}

That’s pretty much it! We’re constructing Oscillators from Chord notations, and then mixing the result for playback.

The last thing I wanted to try was an arpeggiator. An arpeggiator takes a chord structure, and instead of playing all the notes at the same time, only plays one note at a time in sequence.

I extended the Audiolib Oscillator again, making an Arpeggiator plugin. I needed to override the “generate” method. Every time “generate” is called, I’d step through. Once my step count reaches a threshold (the samplerate divided by some number), I’d change to the next frequency.

Here’s what I got:

audioLib.generators('Arpeggiator', function (sampleRate, arpeggiatorRate, notes, octave, autoReverse){
    // extend Oscillator
    for ( var prop in audioLib.generators.Oscillator.prototype) {
        this[prop] = audioLib.generators.Oscillator.prototype[prop];
    }
 
    // do constructor routine for Note and Oscillator
	var	that = this;
    if (autoReverse === undefined) {
        that.autoReverse = true;
    } else {
        that.autoReverse = autoReverse;
    }
    that.buildFrequencies(notes, octave);
	that.waveTable	= new Float32Array(1);
	that.sampleRate = sampleRate;
	that.waveShapes	= that.waveShapes.slice(0);
    that.arpStep = 0;
    that.arpIndex = 0;
    that.arpeggiatorRate = arpeggiatorRate;
 
    /**
     * override generate function
     */
    that.generate = function(){
		var	self	= this,
			f	= +self.frequency,
			pw	= self.pulseWidth,
			p	= self.phase;
		f += f * self.fm;
		self.phase	= (p + f / self.sampleRate / 2) % 1;
		p		= (self.phase + self.phaseOffset) % 1;
		self._p		= p < pw ? p / pw : (p-pw) / (1-pw);
 
        self.arpStep ++;
        if (self.arpStep > self.sampleRate * self.arpeggiatorRate) {
            self.arpStep = 0;
            self.arpIndex ++;
            if (self.arpIndex >= self.frequencies.length) {
                self.arpIndex = 0;
            }
            self.frequency = self.frequencies[self.arpIndex];
        }
	}
}, {
    buildFrequencies: function(notes, octave) {
        var self = this;
        self.frequencies = [];
        if (!(notes instanceof Array)) { // work with either a chord structure...
            notes = ChordFactory.createNotations(notes, octave);
        }
 
        for (var c = 0; c < notes.length; c++) { // ...or array of notations
            self.frequencies.push( Note.getFrequencyForNotation(notes[c]) )
        }
 
        if (self.autoReverse == true) {
            for (var c = notes.length-2; c > 0; c--) {
                self.frequencies.push( Note.getFrequencyForNotation(notes[c]) )
            }
        }
        self.frequency = self.frequencies[0];
    }
});

Lotsa code here tonight! I’d like to get started on some basic effects and envelopes next to make some non-boring tonal qualities to my playback. At that point, I hope to have a nice little useable API I can put up somewhere – especially since I don’t have to put lotsa code up!

Making an Audiolib.js Plugin

admin — Mon, 05 Dec 2011 04:00:01 +0000

I just started checking out Audiolib.js this weekend. Audiolib is a Javascript project to help you synthesize sounds and tones in JS.

Just recently, Google Chrome added the Web Audio API, while Firefox added the Audio Data API. Both let you get low level with sound – you can add bytes to an audio buffer and manufacture sound in realtime.

The library is bundled with “sink.js” which handles the inconsistencies in the audio API between Firefox and Chrome to create a common API. Once this is abstracted away, we can make some audio.

I’ve done the nitty gritty of writing bytes to the buffer in Flash, and from my limited playing Audiolib does a great job with this, and I’m ecstatic I don’t have to rewrite my Flash stuff!

Audiolib provides “generators” for you to work with. A “generator” is something that makes a sound, and defines a common API for anything that makes a noise to write that sound to the audio buffer.

In the generators namespace/package, we have an Oscillator, White/Pink/Brown Noise, and a Sampler. White noise is cool and all, but I jumped right to the Oscillator to make a real tone. The Oscillator takes a sample rate and a frequency.

You can define it thusly:

dev = audioLib.AudioDevice(audioCallback /* callback for the buffer fills */, 2 /* channelCount */);
osc = audioLib.Oscillator(dev.sampleRate /* sampleRate */, 440 /* frequency */);

So, first we defined the device to use – we told it what to use for the audio callback, and how many channels there are (we have 2 channels…left and right).

The audio callback is a pretty standard thing in the world of audio. When the sound card is starting to run out of audio to play, it signals Javascript and says “Hey I need more audio! Fill me up with some bytes”. With our “audioCallback” function, we say “No problem sound card! I got your back! When you run low, please call our audio callback method – this will fill your audio buffer up”

Here’s an example of the audio callback:

function audioCallback(buffer, channelCount){
  osc.append(buffer, channelCount);
}

All that happens here, is that the buffer and channel count gets passed into the method, and we tell out Oscillator what this is, and this Oscillator generates the appropriate bytes and send them to the buffer.

But what is that generator…the Oscillator? Well, it creates a sound at a certain frequency. We pass in the frequency as we instantiate this “osc” object. When the audioCallback fires, it pulls this frequency from the osc object and plays a specific tone!

This is awesome! All the hard work is done for us – but one of the first things I wanted to tackle here was to make it easier for musicians to understand.

See, casual musicians don’t know that a middle “A” on a piano oscillates at 440hz – they just know that they hit a middle “A” to play the tone. I want something that makes sense for casual musicians, so my first small task with audiolib is to override Oscillator to take a notation vs a frequency.

Audiolib provides a plugin spec, but it tripped me up a little to understand how a “Generator” worked. See, they have a js/generation folder which contains the Oscillator. It’s all quite readable and self-documented.

However, the “append” method was not found in the Oscillator class at all! I was looking EVERYWHERE for it!

Eventually I found it in the “wrapper-end.js” script that’s outside of all the folders.

Basically, you create a generator like Oscillator, and you write it to the audiolib.generators namespace. By virtue of being in this namespace, the “wrapper-end” script comes along to wrap things up. It takes all the things in the audiolib.generators namespace and adds the generator base functions. Basically it makes all the generators extend the generator base class after the fact.

Kinda confusing and sneaky if you ask me! Oh well.

Regardless, the plugins are well documented, so I copied the sample generator plugin:

audioLib.generators('SemiClock', function (sampleRate){
	this.sampleRate = sampleRate;
}, {
	prevSample: 0.0,
	generate: function(sample){
		this.phase = + !this.phase;
	},
	getMix: function(){
		return this.phase;
	},
	phase: 0
});

Cool, so we’re naming a generator in the generator namespace, and defining some required methods, like “generate” and “getMix”.

Well, all I want to do is make a copy of all the objects in Oscillator and add a function and initialization routine to accept a musical notation and instantiate an oscillator with the correct frequency.

So here’s my take:

audioLib.generators('Note', function (sampleRate, notation, octave){
    // extend Oscillator
    for ( var prop in audioLib.generators.Oscillator.prototype) {
        this[prop] = audioLib.generators.Oscillator.prototype[prop];
    }
 
    // do constructor routine for Note and Oscillator
	var	self = this;
    self.octave = isNaN(octave) ? 4 : octave;
    self.setNotation(notation);
	self.waveTable	= new Float32Array(1);
	self.sampleRate = sampleRate;
	self.waveShapes	= self.waveShapes.slice(0);
}, {
    /* incremental tones as sharp notation */
    sharpNotations: ["A", "A#", "B", "C", "C#", "D", "D#", "E", "F", "F#", "G", "G#"],
 
    /* incremental tones as flat notation */
    flatNotations: ["A", "Bb", "B", "C", "Db", "D", "Eb", "E", "F", "Gb", "G", "Ab"],
 
    /**
     * notation setter
     * @param notation (octave is optional)
     */
    setNotation: function(nt) {
        this.notation = nt;
 
        // does notation include the octave?
        if ( !isNaN( parseInt(nt.charAt(nt.length -1)) )) {
            this.octave = parseInt(nt.charAt(nt.length -1));
            this.notation = nt.substr(0, nt.length-2);
        }
        this.frequency = this._getFrequencyForNotation(this.notation);
    },
 
    /**
     * turn a notation into a frequency
     * @param notation
     * @return frequency
     */
    _getFrequencyForNotation: function(nt) {
        var freq;
        var indx = this.sharpNotations.indexOf(nt);
 
        if (indx == -1) {
            indx = this.flatNotations.indexOf(nt);
        }
 
        if (indx != -1) {
            indx += (this.octave-4) * this.sharpNotations.length;
            freq = 440 * (Math.pow(2, indx/12));
        }
        return freq;
    }
});

Basically what I did here is extended the Oscillator class. I looped through all the properties of the Oscillator.prototype object and copying it onto the new class.

I then, went in to the Oscillator generator and copied the small initialization routine.

Then I injected my own methods! I replaced the frequency setting with a method to lookup the index of the musical notation in my preset arrays of notations. I also parsed out the octave (if it existed).

In the end, I have a “Note” class which is exactly the same as the Oscillator class – however, instead of passing in the frequency (ie 440hz), you pass in “A”, or “A4″ for the 4th octave.

To change the notation of the oscillator/note, simply call the setNotation again!

I’m definitely looking forward to exploring Audiolib.js more. It has a few concepts that I hadn’t gotten around to implementing in my own Flash framework yet, so I’m excited to see how it’s done.