YuE2 · Frontier Music with Symbolic Planning(map-yue2.github.io) |
YuE2 · Frontier Music with Symbolic Planning(map-yue2.github.io) |
Most of the existing music creation heavily relies on daw and audio plugins, there are two major approaches among many: 1. recording with acoustic instruments, then using plugins to process 2.using a lot of virtual instruments directly, which is more often used in scoring.
Some of the process is already heavily ai assisted, like the postprocessing and mastering. For scoring, more film scorers are using the Noteperformer which is directly note to audio but quality is far from Suno level. The very popular virtual singers(vocaloid, etc) evolved from audio concatenation to full neural network generations. But so far, none of these have encroached into the essential part of the music creation.
Things will get more interesting if this get to the next level, when every single part of the music creation process (in the digital chain) is analyzable and synthesizable by ai. YuE2 appears to me can do note level analyze but stil not synthesize. To be able to synthesize, you get Noteperformer on steroid. Then, the daw and ai music creator will become one. This is somehow unappealing but paradoxical situation: You, as a creator, now can vastly do more. The ai tools gives you back the creative process. The ai music on the internet will no longer be pure slops because lots of them will be finely crafted by human musicians, but at the same time it all becomes super pointless because ai's capability is much stronger and do all these things itself, the only difference is whether you choose to do it yourself or let ai automate some of the parts.
Therefore I'm not sure all the bad qualities of the ai music (slop, lacking soul, or whatever you name) is a sin or blessing. The condemnable part of the ai music is exactly what currently saves human musician.
There are other side effects: the current virtual instrument, audio plugin market will mostly collapse. AI based audio generation will be a hegemony which reduces the diversity. You know that limitation of the process is a major source of curiosity and creativity, and itself is fun part. Wait until we rediscover the appeal of limitation, like the pixel art genre. We will see more of chiptunes, concatenative-sampling punks, 2010-digital vaporwaves and so on. But now the most important part is, genuineness will be a requirement: you have to show the process - you fake it with ai, you lose. Music will be a much more performative art (as it always was in the past) and community centric. Music streaming in its currently state of affair will be abandoned by all but the most casual listeners.
Another trend will be gamifications and participatory music. There is still one part of music tech that is currently underexplored. The physical modeling. The synthesizer tech has hit a wall: it still struggle to generate natural and interesting sound, which is also why sample based virtual instrument still thrives. The physical modeling is promising but extremely compute intensive. AI accelerated physical modelling can help with this. This can be actually an antidote to the generative AI. Physical modeling can be enhanced by more visceral visualizations or hardware analogues (materialization of virtual process). There is joy of participating and watching the corporeal and physical process of making music. While elite artist can create very elaborate process(think of the eurorack and experimental synths) and visualizations and show them to the audience, the process can be simplified and gamified to promote collective creative performance, all without dumbing down, as the physical process itself is open ended, and we as human beings, have stornger connection to the physical sounds than the abstract sound of the previous gen synthesizers.
To talk about participatory music and community, we can get glimpse of it from the Vocaloid scene. These crowd has very interesting take on ai music which I feel can be very enlightening to those uninitiated. They are enamored by the new AI soundbanks (which is the progression from the old contatenative technology) while very opposed to (generative) ai music. How is it so? They see the music by the Vocaloid producers a token of love poured into the community. The quality of music is not deciding factor, it is whether the producer know their community and pay respect to the virtual idol they loved. Their idol is a collective creation, it is not directed by any single entity. Derivative work also plays central role in the scene. Diaglogue is weighted a lot more than pure output. This is quite different from some of the AI idols though many uninitiated tend to conflate the two together. The AI industry may see Vocaloid as the predecessors to the AI idols, and you see some of the singing synth engine and soundbank makers are also giving nods to the AI music industry (ace studio, traditionally a singing synth company, is dipping toes in prompt based generations, and looks like is behind YuE2), but the two have very different spiritual cores. The conflation of AI music and Virtual Singers has caused further damage to the scene. One example is some streaming platform categorizes all virtual singer songs into AI music, which completely denies the hard work of the producers, and monetization and visibility possibilities, despite that some virtual singer still uses concatenative technology and have absolutely zero AI involved. But just for the AI soundbanks many has seen their drawbacks, now many consider they are too realistic. They want to keep the essence of the mechanical sound of their beloved idols, so you see the latest Miku soundbank delibrately not pursuing the realism like many do, e.g. SynthV, and those keep their idol unique characters are more successful in retaining their audience.
btw. thanks for your perspective.
But listening to these samples, it's obvious that, while technically proficient, they have no voice. Funnily enough, neither does most of the human produced music slop that existed before gen AI.
All it exposes is that most people have no taste. This is as good as shuffling through popular songs on Spotify.
Mainstream is for suckers.
Properly mixed ofc.
https://www.discogs.com/artist/9699799-Neurojacker
Am I as good as Claudio PRC, Donato Dozzy, Ken Ishii or Takkyu Ishino? Not even close. But wanting to make more is reward enough for me to keep trying.
Content factories that use AI will hopefully force humans to up their game.
Personally I feel humans are capable of so much more than pop music has become over the last 50 to 60 years.
A lot of today's music is so overproduced and formulaic.
I would love to see an arms race between humans and AI where humans are forced to continually adapt and get better.
The only thing about AI generated music that makes me sad is, I worry musical tastes will fragment even more and none of us will recognize the music our neighbors, classmates, or colleagues are listening to because we actively tailor our music to our personal tastes.
I use Suno and I've already noticed this.
A friend sends me a song they generated and I want to tweak it a bit to my personal preference. When we all can create music to our personal taste, we lose a little bit of the joy of all listening to the same song or album release.
You can't call it “your writing” or “your music” if you’re using the same tool millions of others are using that boils your idea down to the same reduction as everyone else’s made with the exact same (stolen) ingredients.
Remember how people treated auto-tune?
Hopefully not, but there will be a lot of people trying, which is the problem. That alone will drown out a lot of human made music and muddy the waters.
No it'll be buried under slop. This was a huge issue before AI-generated art and I can only see it getting worse.
Someone is using AI to make songs for ads on YouTube. Maybe it's not a new thing, but I just started seeing them recently, and they're so fucking bad. They're these dull, rambling things telling a boring story badly, with equally bland AI animations to accompany it. They just go on, and on, and on, and suck the entire time.
With that said... ai generated music, the same can be said about software... but compilers are basically ai, there is no way a for loop on extremely high level abstraction that then open connections to some service, generates jtokens, validates, and apply math non stop from a couple of lines is anything but alien to the 99% of us (yes I know you can write assembly for all your programs and you are better than the compiler)
What if you say, if humans where not involved and a phisically non electric instrument where involved and done with only humans, it's AI assited? because again you don't understand ALL the relationships, how it's designed every single thing, all the things in a music studio to master a song, ALL the related mechanisms used in the promotion, distribution and live execution of those songs... and yes, again there is some of you who can build all those things from sand and pumped from the ground with a tool you designed bare petroleum and program, design and execute EVERY SINGLE THING required to get a record of music you like...
So if a group of 20 people spends 20 weeks working with AI tools to get you a song, then you don't care I guess right?
Did you know that Enya for this song recorded thousand something times voices to make a chorus until it had a particular form... does that mixing of thousands of executions and the tools to analyze and give it the form wanted qualify as AI?
If you move sliders and push buttons and you don't understand the math behind it, are you vibe mixing a song?
Yes, if it is a "give me a catchy song with lyrics that talk about big butts" and then publish... yes this is crap.
All of these projects are basically just destroying entire fields of human creativity.
It doesn't seem so bad at first when the quality is mediocre, but as soon as they get good, then any human composition will be completely devalued, despite the enormous amount of knowledge, skill and effort that goes into producing one.
A composer who writes a piece of music themselves, instead of being recognised and admired basically becomes the equivalent of someone who bakes their own sourdough instead of going to the supermarket: a curiosity, someone who does things the hard way for their own amusement.
Graphic artists and bloggers were the first to suffer this, but it feels like it's just going to happen to basically every type of creative expression that can be represented digitally. It's just a question of how many months until that happens.
I don't have a ton of time right now to really mess around with it, but I took 15 minutes to run it on my Mac M1 with an old melody I wrote back in high school for an RPG I was writing in VB6 (dating myself a bit here).
If anyone wants to hear what it sounds like:
The point of art isn’t to generate beautiful patterns. It’s to create meaning. And you can’t have that without human care and intent.
Furthermore, that care needs to be intrinsically interwoven into the patterns that comprise the work. That’s what craft is. The mastery to generate exquisite patterns, and the care to infuse them with meaning at every level.
Without care, in a Heideggerian sense, we are left with something which tastes sweet but is completely hollow, perhaps even illusional.
Meaning is listener’s own perception (author’s own intent is a suggestion at best - if I’d be paid a dollar every time I thought some artist meant something when they didn’t or meant something different, I’d sure have a good wad of cash), and when a listener learns about the purely mechanistic “authorship” they start to feel uncanny unless they find an outing in appreciating the mechanicity itself.
If anyone has tried Suno's Studio 2, I would love to hear your take on it. At $24-30/month, it's approaching Ableton Live 12 Suite rent-to-own territory, without the "own" part.
So many possibilities.
There has almost always been backlash against new technologies in music. Ultimately, musicians adopted those technologies and continued producing great works.
I recently dug some 20-year-old CDs out from storage that were full of riffs and chord progressions for songs I never got around to completing. I fed them to Suno as a sort of “what might these have sounded like if I’d finished them?” experiment. I found the results quite impressive and plan to publish them some time in the near future.
None of the tracks sound like marketable products to me, and I certainly don't think AI is going to generate the next Bohemian Rhapsody -- but it definitely could be a great thing for curing writer's block.
I imagine you can get pretty close to replicating an artist's sound if you specifically train on their output, but at that point it's the same question of whether you're satisfied with abstracting away the things that make writing music/software interesting to us (with music the answer's a lot more obvious since the point of writing music has always just been... to write music).
Previously: https://news.ycombinator.com/item?id=46641627
It's even more interesting that the “Contemporary Christian” music it generated was specifically Mormon music (specifically about the exile of Lehi and his family from Jerusalem into the wilderness, during which they reach the ocean, build a boat, and sail to the Americas), but in the style of contemporary Christian rock.
This is baffling to me. Idk what alignment means. Perhaps brainrot and endless mischaracterization of creativity is one of them
Similar to coding, where you are have a bunch of tools that can do the drudge work for you (the mundane things in mixing mastering for example), but still retain full creative freedom and expression.
I am not articulating this well, but basically transitioning from IDE->Coding harnesses what is the DAW-> ??
It would be useful to be able to generate backing vocals/hums/melismas and whatnot, that could be plugged into a DAW.
https://electricliterature.com/wp-content/uploads/2017/11/Tr...
"The Petty and the Small; Are overcome with gall ; When Genius, having faltered, fails to fall. Klapaucius too, I ween, Will turn the deepest green To hear such flawless verse from Trurl's machine."
Can't relate to this. Art doesn't come out as a left-brained problem solving exercise or identifying the right product for a market. It's about expression. You do it when you're feeling things.
https://www.bbc.com/news/world-latin-america-40090809
Or I suppose they could just put "baby shark" on a loop.
The playlist included "the sound of jet planes all night" and "the screams of rabbits being slaughtered". I don't think they really need AI for that, anything loud would probably do the trick after 51 days.
Prayer for external effects has long been mechanized [1]. But praying for your inner effect is a different thing. Similarly, production of songs for external consumption is being mechanized. But what you feel when you sing cannot be mechanized. Sing for yourself, play for yourself and for those immediately around you in the moment. It's as valuable as ever.
I think it will take many more years until an AI is able to compose like e.g. John Williams and play a score and sound like e.g. the London Symphony Orchestra. So there is room left for human contribution. On the other hand, the majority of humans even before AI was not able to value good vs. bad musical composition and performance, and instead satisfied with the commercial slop the market was flooded over the last thirty years. Studies of e.g. Spotify consumer behaviour demonstrated for many years before AI that more and more people consume anonymous playlists never caring who composed or played the music, just using music for its functional purpose like a commodity. It's depressing, but that's how society develops.
> a curiosity, someone who does things the hard way for their own amusement
That development already started long ago. In the eighties and nineties, music schools florished and children wanted to learn instruments and play in bands or orchestras. That completely changed over the last thirty years. Musical instruments will be a curiosity in a decade or two people are going to watch in a museum, wondering why anyone would have learned these skills in the past.
Unless you exclusively listen to old music color me skeptical
If you start living in some kind of idealist world where you say that you will drop the book upon the understanding of the process over the outcome, you inevitably have to start to deal with the definitions like this.
Fortunately there are still many talented (but struggling) musicians out there. I worry that the flood of passable AI generated content will finally finish them off.
are you kidding
And there are just too many albums to go through each week. I'm on Redacted and Orpheus though, pulling the Top 10s each week.
This is that.
Pre-ai:
You compose a song. It exists in your mind, or on paper, as a series of notes and chords, sometimes with words. This is not yet "a song", it's sheet music. To get to a finished product, you have to do a demo, and that involves a lot of choices, but those choices may still be only a temporary bridge to get to a later re-recording, with the demo acting only as a proof of concept that was necessary to get sign-off or funding from other members of your band, a record label, a producer, etc.
Back to the demo. You have to decide how to ARRANGE that music you've written. That means you have to decide which kinds of instruments/synthesizers play what parts. Maybe the bassline is played on a bass with a fuzz filter. Maybe it's clean. Maybe it's a slap bass. Got a guitar part? Is it acoustic or electric? Get it? Lots of choices to make.
If you have vocals, you have to decide who is going to be singing the vocals and in what style.
Now, with this new model: you can either hand-write the "sheet music" (in its proprietary format, but it's still a form of musical notation) or have an LLM do that. Either way, given the sheet music, this model can go and reliably produce new demos for you, so that you can try out different arrangements of the same exact musical structure (including lyrics). It just automates all the arrangement and renders it in one go, instead of you recording each part yourself.