I got pretty deep into building a modular synthesis environment using it (https://github.com/rsimmons/plinth) before deciding that working within the constraints of the built-in nodes was ultimately futile.

Even building a well-behaved envelope generator (e.g. that handles retriggering correctly) is extremely tricky with what the API provides. How could such a basic use case have been overlooked? I made a library (https://github.com/rsimmons/fastidious-envelope-generator) to solve that problem, but it's silly to have to work around the API for basic use cases.

Ultimately we have to hold out for the AudioWorklet API (which itself seems potentially over-complicated) to finally get the ability to do "raw" output.

The MPU-401 also had a "dumb" a.k.a. "UART" mode. You had to do everything yourself... and therefore could do anything. It turned out that early PCs were fast enough -- especially because you could install raw interrupt service routines and DOS didn't get in the way. :)

As a sequencer/DAW creator, you really want the system to give you raw hardware buffers and zero latency -- or as close to that as it can -- and let you build what you need on top.

If a system is far from that, it's understandable and well-meaning to try to compensate with some pre-baked engine/framework. It might even meet some folks' needs. But....

I've been heavily into procedural audio for a year or two, and have had no big issues with using Web Audio. There are solid libraries that abstract it away (Tone.js and Tuna, e.g.), and since I outgrew them working directly with audio nodes and params has been fine too.

The big caveat is, when I first started I set myself the rule that I would not use script processor nodes. Obviously it would be nice to do everything manually, but for all the reasons in the article they're not good enough, so I set them aside, and everything's been smooth since.

So I feel like the answer to the articles headline is, today as of this moment the Web Audio API is made for anyone who doesn't need script nodes. If you can live within that constraint it'll suit you fine; if not it won't.

(Hopefully audio worklets will change this and it'll be for everyone, but I haven't followed them and don't know how they're shaping up.)

Then if you look into it:

    dictionary DynamicsCompressorOptions : AudioNodeOptions {
             float attack = 0.003;
             float knee = 30;
             float ratio = 12;
             float release = 0.25;
             float threshold = -24;

Check out a vintage compressor that has a dozen implementation as VST plugins:

http://media.uaudio.com/assetlibrary/t/e/teletronix_la2a_car...

https://wiki.mozilla.org/Audio_Data_API

1. Firefox always clicks when starting and stopping each tone. I think that's due to a longstanding Firefox bug and not the Web Audio API. I could mostly elminate the clicks by ramping the gain, but the threshold was different for each computer.

2. This was the deal-breaker. Every mobile device I tested had such terrible timing in JavaScript (off by tens of milliseconds) that it was impossible to produce reasonably correct-sounding Morse code faster than about 5-8 WPM.

I found these implementation problems more frustrating than the API itself. At this point I'm pretty sure the only way to reliably generate Morse code is to record and play audio samples of each character, which wastes bandwidth and can be done more easily without using the Web Audio API at all.

That said, this current pull request on emscripten is a fantastic step forward and I'm very excited to see it's completion: https://github.com/kripken/emscripten/pull/5367

I put some weekends into trying to build a higher-level abstraction framework of sorts for my own sound art projects on top of Web Audio, and it was full of headaches for similar reasons to those mentioned.

The thing that I put the most work into is mentioned here, the lack of proper native support for tightly (but prospectively dynamically) scripted events, with sample accuracy to prevent glitching.

Through digging and prior work I came to a de facto standard solution using two layers of timers, one in WebAudio (which support sample accuracy but gives you hook to e.g. cancel or reschedule events), and one using coarse but flexible JS timers. Fugly, but it worked. But why is this necessary...!?

There's a ton of potential here, and someone like myself looking to implement interactive "art" or play spaces is desperate for a robust cross-platform web solution, it'd truly be a game-changer...

...so far Web Audio isn't there. :/

Other areas I wrestled with: • buffer management, especially with CORS issues and having to write my own stream support (preloading then freeing buffers in series, to get seamless playback of large resources...) • lack of direction on memory management, particularly, what the application is obligated to do, to release resources and prevent memory leaks • the "disposable buffer" model makes perfect sense from an implementation view but could have easily been made a non-issue for clients. This isn't GL; do us some solids yo.

Will keep watching, and likely, wrestling...

One thing that really irks me at the moment is the huge variation in sound volume of the increasing plethora of videos in my social media feed. If there was some way we could use a real time WebAudio manipulation on the browser to equalise the volume on all these home made videos, so much the better. Not just volume up/down, but things like real time audio compression to make vocals stand out a little.

Add delay and reverb to talk tracks etc. for podcasts.

EQ filters to reduce white noise on outdoor videos etc. also would be better. People with hearing difficulties in particular ranges, or who suffer from tinnitus etc. would be able to reduce certain frequencies via parametric equalisation.

It would be intriguing to see a podcast service or SoundCloud etc. offer real time audio manipulation, or let you add post processing mastering effects on your audio productions before releasing them in the wild.

EDIT: I should also add that the teams behind the apis are quite responsive. You can make an impact in the direction of development simply by making your needs/desires known.

As someone mentioned elsewhere on this thread Android suffered from a crappy Audio/MIDI library. iOS's CoreMIDI was great, but not transportable outside of iOS/OSX. Web Audio API's MIDI control seemed a great way to go - just build a cross platform interface using Electron App and use the underlying WebAudio to fire off MIDI messages.

Unfortunately, at the time of developing the project, WebAudio's MIDI SYSEX spec was still too fluid or not completely defined, so I had trouble sending/reading SYSEX messages via the API, and thus shelved the project for another day.

Not really, the full range of human hearing is over 120db. Getting to 120db within 16 bits requires tricks like noise shaping. Otherwise, simple rounding at 16 bits gives about 80db and horrible sounding artifacts around quiet parts.

It's even more complicated in audio production, where 16 bits just doesn't provide enough room for post-production editing.

This is why the API is floating-point. Things like noise shaping need to be encapsulated within the API, or handled at the DAC if it's a high-quality one. (Edit) There's nothing wrong with consumer-grade DACs that are limited to about 80-90db of dynamic range; but the API shouldn't force that limitation on the entire world.

I think the whole point is that Javascript used to be slow, and using the CPU as a DSP to process samples prevents acceleration. Seems to me what is needed is like "audio shaders" equivalent to compute/pixel shaders, that you farm off to OpenAL-like API which can be compiled to run on native HW.

Even if you grant emscripten produces reasonable code, it's still bloated, and less efficient on mobile devices than leveraging OS level DSP capability.

  const audioContext = new AudioContext();
  const osc = audioContext.createOscillator();
  osc.frequency.value = 440;
  osc.connect(audioContext.destination);
  osc.start();

So the most important question is: why isn't this interface implemented in any browser yet?

That a BufferSourceNode cannot be abused to generate precision oscillators isn't very enlightening.

To be frank, graphics world had some type of standard (OpenGL) long time ago, next to DirectX. So WebGL had a good example. However in the audio world we haven't seen a cross platform quasi-standard spec covering Mac, Linux and Windows. So IMHO, non-web audio lacks also common standards for mixing, sound engineering, music-making. That's why web audio appears to lack a use case. IMHO, that smells opportunity.

I use Web Audio, in canvas-WebGL based games where music making is needed. I understand the issues - we definitely need more than "play" functionality.

If we had gone with the Audio Data API, it wouldn't have been satisfying, because the web platform's compute engine simply could not meet the requirement of reliably delivering audio samples on schedule. Fortunately, that is in the process of changing.

Given these constraints, the complexity of building a signal processing graph (with the signal path happening entirely in native code) is justified, if those signal processing units are actually useful. I don't think we've seen the evidence for that.

I'd personally be happy with a much simpler approach based on running wasm in a real-time thread, and removing (or at least deprecating) the in-built behavior. It's very hard to specify the behavior of something like DynamicsCompressorNode precisely enough that people can count on consistent behavior across browsers. To me, that's a sign perhaps it shouldn't be in the spec.

Disclaimer: I've worked on some of this stuff, and have been playing with a port of my DX7 emulator to emscripten. Opinions are my own and not that of my employer.

You just can't do that with the same level of tightness of rhythm on low hardware with web techs today. Flash was bad yet Flash also opened up insane possibilities on the web when it comes to multimedia applications that just can't be matched with Webtechs. ASM.js might fill the gap, but i haven't seen any equivalent yet.

Without using ScriptProcessorNode, there was no way of tuning the synthesizer because of the limitation that any loop in the audio graph has a 128 samples delay at least.

Maybe a more "compilation-oriented" handling of the audio graphs (at the user's choice) could help overcome this?

Video AND audio? They good you all covered with nice APIs!

Just audio? You're screwed!

I cannot think of one.

And on that note, what I think Web Audio tried to be was a drop-in kit for game engines. Getting the full functionality of Unreal into the browser motivated the requirement for audio processing. But the actual implementation was muddled from the start: basic audio playback remains challenging(try to stream a BGM loop instead of load+uncompress and discover to your woe that it's not going to loop gaplessly, even when the codec is designed to allow that.) and my hobby stab at an independent implementation ran out of gas when I tried to get their envelope model working. The spec has a lot of features but not enough detail, and my morale sank further when I looked at how Chrome did it(stateful pasta code). I got something half-working, put it aside and never came back.

OTOH I had also tried Mozilla's system. That was very simple, and I got a synth working in no time at all with decent performance and latency. Optimizing from that point would have been the way to do it, but something in browser vendor politics at that time led to it being dropped.

(If I could return to Cakewalk, I would. Wrote some of my best tracks with that little ISR of yours!)

Other proposals for audio APIs solved a wider set of use cases, while also making it possible to do procedural audio without depending on browser vendors to implement key features for you.

The effects are useful in one setting: hobbyist and toy usage, where you really don't have that many constraints and can play with whatever cool things are around. That said, I'm sure you'd actually get a lot more mileage out of a library of user-made script nodes, rather than whatever the browsers have built for you.

If you're trying to build something production-ready, or port an existing system to the web, most of the fun toys seem like just that: toys.

AudioWorklets don't look like they would improve things for me, but that's a topic for another blog post.

As a side note, for some common audio DSP tasks, you could presumably take better advantage of highly parallel processing by doing a fourier transform and working in the spectral domain. There has been research do do this on GPUs and it works. However, if you do this you'll have high latency, and it's not a hardware problem, it's inherent to the FFT algorithm, so it's kind of a dead end for many applications.

I hadn't heard that, but some of the "processor node" stuff does sound familiar.

What OS X also has, though, is proper low-level low-latency sound APIs. And that's why there are so many Mac (and iOS) music apps.

Please use your imagination and try to imagine one of infinitely many other streams that I could make at runtime that are not easily made with the built-in toy oscillators.

It also misses the point, because, as with Vulkan, you just want a stable, sane, fast low-level API to access the hardware because OpenGL immediate mode doesn't get you beyond kindergarten in todays computer graphics. In audio, that is a sample-level API. Everything else should be handled by the application!

You can still make a source/sink directed graph system with components like "oscillators". In a fricking library!

I used the same solution when I tried to perform realtime audio streaming from a daemon on an embedded device to a browser (which is probably even a more realistic use-case for a browser audio API than generatic sine waves). I basically stumbled over the same issues than the author: A deprecated ScriptProcessorNode and high-level APIs which don't help me (like the oscillator one).

In the end I opted for a very similar solution as the author: Whenenver I got enough samples through websocket (I encoded them simply as raw 16bit samples there) I created a BufferSource, copied all samples in there (with conversion to floating point), and enqueued the buffer for playback at the position where the last buffer finished.

I really didn't expect that to work well due to all the overhead of creating and copying buffers and due to the uncertainity whether the browser will switch between 2 buffers without missing samples. But surprisingly it worked and did the job. I included a buffering of 200ms, which means I only started playback 200ms to be able to receive more data in the background and have a little bit more time to append further buffers. I experimented a little bit with that number but can't remember how deep the lower limit was before getting dropouts regurarly. It definitely wasn't usable for low-latency playback.

If you provide a low-level "play" API, others can build stuff on top because it's just numbers. Sure, sometimes there's "expensive numbers" like MP3 decoders, FFTs, etc., but these can be added as needed.

I think the bigger issue is that non-experts sometimes get tasked with adding support for things.

The "audio device API that leaves the sample rate completely unspecified" example is, believe it or not, one I've seen before elsewhere. And yet, if you know the first thing about PCM samples, you know this is a mind-numbingly stupid mistake to make. Yet it's a mistake that a few people have made into shipping products, because they can't or won't reason about audio, and this did not stop them from being in charge of an audio API.

I'd rather have a comprehensive API that someone can dumb down than one that's so crippled as to be unusable beyond very basic functionality.

OpenAL: https://www.openal.org/