Show HN: SOUL Language 1.0 – A platform for writing and running audio code(soul-lang.github.io) |
Show HN: SOUL Language 1.0 – A platform for writing and running audio code(soul-lang.github.io) |
Think of all of the graphics technologies that were enabled by OpenGL/Direct3D/... and consider the equivalent for sound/music.
The trend is clearly going into the direction of application-specific processors and a lot of hardware already has dedicated DSP chips for audio. It is high time that a standard language is proposed to access them in a unified way just like OpenGL did.
Full disclosure: I worked with Jules and Cesare at ROLI/JUCE where Jules was my mentor and colleague. If anyone can pull this off, it’s them.
The PS 2 had a timid attempt with ES GL 1.0 + Cg, and some Nintendo models do support a subset, miniGL sytle, and that is about it.
Most studios kept writing engines with plugins for various 3D APIs.
> What is the licensing/business model?
> Our intention is to make SOUL entirely free and unencumbered for developers to use. All our public source code is permissively (ISC) licensed. We’re currently keeping some of our secret sauce closed-source, but the EULA allows use of it freely to encourage its adoption in 3rd party hardware and software. Ultimately, we plan to commercialise SOUL by licensing back-end drivers and other IP for use by vendors who are building SOUL-compatible hardware products.
How would it be possible to allay your concerns? I think we've tried to be as clear as possible with our intentions - how could we rule out such possibilities with our license agreement?
- you can compile Faust code to SOUL, and export it (as the .soul and .soulpatch files) from the Faust Web IDE https://faustide.grame.fr/ (or https://fausteditor.grame.fr for a simpler version)
- lower-level tools like "faust2soul" are part of the Faust distribution: https://github.com/grame-cncm/faust and https://github.com/grame-cncm/faust/tree/master-dev/architec...
- using Faust/SOUL on Bela and having SOUL as the intermediate language to JIT compile Faust code is certainly possible, but not the easiest way ! Bela developer Giulio experimented a more direct Faust JIT support on Bela (since Faust can directly generate LLVM IR code and JIT it) here: https://github.com/giuliomoro/bela-faust-jit, but this projet is a bit frozen by lack of time to improve it. Feel free to bring it to life again.
"Faust (Functional Audio Stream) is a functional programming language for sound synthesis and audio processing with a strong focus on the design of synthesizers, musical instruments, audio effects, etc. Faust targets high-performance signal processing applications and audio plug-ins for a variety of platforms and standards."
"The core component of Faust is its compiler. It allows to "translate" any Faust digital signal processing (DSP) specification to a wide range of non-domain specific languages such as C++, C, JAVA, JavaScript, LLVM bit code, WebAssembly, etc. In this regard, Faust can be seen as an alternative to C++ but is much simpler and intuitive to learn."
Faust also has a more advanced syntax. SOUL basically looks like C, while Faust is more functional and designed from the ground up to represent graphs, as such it can look pretty alien. In particular, partial application in Faust is great for graphs but it also means there are 100 ways of writing a program.
For those interested in audio-specific languages, I've had some success with Vult, a transpiled language which runs everywhere, compiles to Pure Data and runs on the Teensy. I used it to make some really powerful, extremely performant filters.
It's a language for writing the absolute lowest-level, close-to-the-metal, bit-twiddling realtime code which you'd then glue together using a higher-level language like C++, javascript, python, lua, etc.
* Can I use SOUL to create freestanding interactive performance environments a la Pure Data?
* What prospects are there for beefing up hardware to run more DSP and bring down latencies? Are there, or will there be open-hardware DSP projects which run SOUL, or open-source runtimes which achieve high performance on standard issue CPUs? If the company behind SOUL goes under (Roli, I think), what happens?
* Could I write a program in SOUL that does real time beat detection given an audio signal, say a steady drum beat (to make things simple)?
It would be very nice to include this, as it would open up some useful patterns for analog modelling and ML runtimes.
I have a few questions, I hope are not stupid. I'll confess to not having delved into the heart of SOUL source yet.
1) what are these "audio processors" that computers already have embedded in them? (DSP cores and associated extensions or an MCU controlling the audio-HW-codec state machine and providing a buffer for the computer or?)
2) Does SOUL engage in the audio sub-system hardware in any way or perform any firmware like activities? DMA engine configuration typically driving the audio engine I/O, right?
3) Do external audio DSP hardware like UAD provide, for example, pipe audio out and then back into the computer, typically? Could such hardware be addressed if it weren't proprietary?
- Is time/frequency analysis and its analogs a first class citizen? (STFT, wavelets, other real-time spectral/cepstral algorithms)
- How do I embed the SOUL runtime into an application? Can SOUL scripts be used like user scripts in a larger application?
We've tried to make embedding it as easy as possible - there's a DLL, and a simple COM interface and a few header-only C++ classes to load and JIT-compile a patch dynamically. We have an example project showing how to do this. FWIW Tracktion Waveform is doing exactly this, and only has a few hundred lines of glue code to enable patches in a full-blown DAW.
Why COM (and is it actually COM, or is it COM-inspired like VST3)? It's a bit of a pain to interface with it from hosting languages (C is much more straightforward, particularly when worrying about C++ exception safety and garbage collection).
I have a (probably dumb) question. I've looked through the docs, and I can't see how you would play a sample back (i.e. to create a simple sampler, which is one of the projects I'm most interested in creating). Am I missing something, or is that something that wouldn't be present at that level, and you'd need to roll your own?
EDIT: I've just read the code for the simple piano, and I see that it implements the sample playback itself, so I guess that answers my own question!
Thank you for addressing this. I was meaning to get into making VST plugins of my own, but currently in order to do that one has to jump quite a few hoops and often as the work grows in complexity so does the required expertise.
One of the supported features of the soul system is to export a JUCE project from a soulpatch - it cross compiles to a C++ project supporting all the standard plugin formats, so you can get AU support for OSX, and VST3 support on Windows out of that one project.
JUCE is one of Jules' previous projects and is pretty much the standard for building cross platform audio plugins used by commercial plugin vendors (https://juce.com/)
You can also browse the example code on the github: https://github.com/soul-lang/SOUL/tree/master/examples/patch...
Looks like an interesting project!
Would love to see an example with a gui ! My dream is to make a sample chopper I can use directly from maschine
Anyway to do that in a Docker container, maybe a ci/cd pipeline where I can just push my code and get an VST as a build artifact.
What's the latest on using SOUL with platforms like Faust and Bela?
Faust has also had SOUL support for a while. Not sure what's new over there but maybe Stephane will see this and comment :)
On licensing, how will it work for creators of open source hardware? Will they still need to pay a licensing fee if they are targeting a specific device with closed driver IP? Or could there be an exception there?
Do you also see yourselves supporting open low-level hardware (RISC-V, FPGA, ...?), and in that case would you consider opening your driver IP for those targets?
So you're saying it should be technically feasible to live code/hot reload faust.dsp on Bela with no glitching?
OSC is more something you'd want to use for a stand-alone program, which needn't be written as a patch. But yes, would be nice to add some OSC wrappers one day. Maybe it's the kind of thing that someone else will contribute before we get around to it.
Please view on desktop/tablet.’
I was curious to see some code, but I guess that has to wait.
EDIT: ah, I misunderstood you.. You mean you'd found the examples but the playground page didn't fit on a phone. Yeah, it's tricky to squeeze all the stuff needed into a small space! We're planning to do a huge redesign of the site in 2021 to turn it into a much more powerful developer portal, and we'll certainly look at other form-factors in doing that.
SOUL does have a JIT compiler and you can live-code it, and you can certainly use it to write programs that generate musical patterns etc.. So sure, the language could be used like that.
Our background and focus has always been more from the pro-audio side of things rather than performance, though, so the tools we've built so far aren't really targeted at performance use-cases, it's more about development of apps and plugins.
> What prospects are there for beefing up hardware to run more DSP and bring down latencies? Are there, or will there be open-hardware DSP projects which run SOUL, or open-source runtimes which achieve high performance on standard issue CPUs? If the company behind SOUL goes under (Roli, I think), what happens?
Right now we've not had the resources to get stuck into that side of things deeply yet - hoping to be able to do more on it this year. (And ROLI aren't going to go under, ha!)
> Could I write a program in SOUL that does real time beat detection given an audio signal, say a steady drum beat (to make things simple)?
Yep, totally the kind of thing we're expecting people to do with it.
Well that's the million dollar question. We're imagining things moving (hopefully) along similar lines to how GPUs have developed, with dedicated processors for audio processing. It's pretty clear though that the current model of distributing binaries for the CPU is not magically going to enable hardware vendors to sell audio accelerator cards, and hence why we feel a change of direction for the audio industry is required.
The way we see this being enabled would be a JIT based approach where the audio driver translates device independent code to run on their given hardware (and the 'soft' rendering on the CPU which is where we currently are for machines without dedicated audio accelerators).
The design of SOUL has this sort of support in mind, with per sample processing and the parallel structure of the DSP still visible within the language, which allows the driver to make threading decisions at the JIT stage, enabling parallelism.
The binary release includes a build of the soul command for Bela, which is a very cool low latency board + OS for audio (https://bela.io/) based on xenomai. It's probably as close as we get to open hardware for DSP, and it puts you in the sub millisecond latency range.
RISC-V is an easy one to support, as our JIT uses LLVM so it's just a case of adding a build flag for that.
FPGA is more interesting. I think there could be some really clever tricks we can do for soul programs which are highly parallel, but that kind of rocket-science stuff is probably going to have to wait until we have a couple of PhD students to work on it :)
A lot of people have been down this road before, and it doesn't end well for the users.
At least your lawyers are happy.
I want to script together many soul patches into a complex systems written in higher level languages. Integrating C++ dependencies is avoided because of the headache of FFI, C is quite straightforward. I suspect users like myself will need to wrap your COM API in its C-equivalent through manually written vtables regardless, which is its own minefield rather than a proper API.
I hope it doesn't require the COM runtime...
As does macOS/iOS driver model.
So then, update your faust code, re-run faust2soul and the soul command will see the updated patch and recompile.
Even though this already sounds great, hot swapping state would be the really big deal here. Do you think the Faust program itself might be able to save/load state somehow?
We've sketched out how this would work, and it's on the project backlog as something we want to support, so at some point we'll improve this area of the runtime.
I can vouch for that being a high priority feature for any interactive audio development, not just for live coding scenarios.
It isn’t that I don’t trust you or your intentions now, it’s that intentions (and requirements) change.
You asked about allaying fears. Personally for me this means actually take-to-the-bank licensing that I could get through a legal audit, even if hypothetical.
Talk of “secret sauce” and future closed source and IP licensing of future maybe-must-have magic... that’s way too much future risk for me.
An example of what I think is good licensing is JUCE (https://juce.com). Everything from “free” to “paid with support”, all open source… but the point is I always know exactly where I stand legally, what I can and cannot do... and what I can and cannot do in the FUTURE.
We're very keen to make sure that the end-user developers are totally unencumbered and license-free, because we want lots of people to adopt it.
All our licensing is more aimed at device and driver builders - think of it like e.g. openGL - the industry making GL cards and drivers is legal complex, but none of that stuff affects the coders writing GL apps.
Or what if want to write a Vulkan to Metal like MoltenVK?
What if I want to write an interpreter for your bitcode?
What if I want to port soul secret source on top of Apple's Accelerate framework? Would you sue me to oblivion?
And good point. I understand where you’re coming from, but... I didn’t understand it when I read the post and blurb(s).
I never want to agree to an EULA before starting to develop something outside of a professional setting. That's an immediate dealbreaker for me. But even if I did - I wouldn't be able to disassemble or reverse engineer your software (2.2)? Also - why are there so many clauses in the EULA telling me to obey really specific laws? Aren't these redundant to section 2?
- end-user developers. This is kind of like being a user of any other language, there's no EULA, nothing to sign, you just write SOUL code, test and debug it with whatever tools (which may themselves have a EULA, but probably nothing heavy)
- device and host developers: These are the people writing DAWs, plugins, hardware that can run SOUL code, audio device drivers etc. These are professionals, and licensing is a normal part of life when you're doing this kind of work.
What secret sauce is this? What parts of https://github.com/soul-lang/SOUL are closed source?
Real answer: Mainly the JIT engine, and the very complex rewriting algorithms that turn multi-threaded soul code into a form that can actually be executed.
We're certainly keen to open-source everything when possible, just been advised by our lawyers not to do that yet. C'est la vie.
But that doesn't stop you as a developer from freely writing a 3D game that runs on it.
Same here. Think of it as a device driver. Ideally we'll open-source everything eventually, but anyone who's ever been involved in real-world commercial software development will understand that things aren't always quite that idyllic when you're building a business around it.