AI in Music - Are We Ready (to Discuss It)?

[dubbmann]

Hi all,

First, to the mods, please move this to whatever forum you deem best for discussing the recent AI developments in music. There's been less attention to AI in music than the "AI in Art" news about ChatGPT, Stable Diffusion, and OpenAI's text-to-image tools, but it's coming to all aspects of music creation and music production both.

[FWIW, I track this because of professional reasons (ie, my day job, such as it is) but actually received my introduction to AI back in the 1980s at the feet of a true British eccentric named Donald Michie, who was one of Alan Turing's friends and colleagues at Bletchley Part during WW2. DM, as he was known to intimates, was a visionary professor and researcher who sadly didn't live long enough to see the astounding level of progress AI and machine learning has made specifically in the last 5 years.]

I think it's going to be massive, and something forum members/SoSers should be aware of if they're not already. I've seen ads for tools on YT that will create AI-generated vocals, even generating the lyrics if you want it to. In the manner of ... whoever you want. These AI models are trained using millions of samples but once trained can generate limitless permutations on whatever themes you ask it to. Those who've enjoyed the Peter Jackson master class on the Let It Be have already experienced the 'magic' of AI in that Jackson's boffins back in NZ used these same neural network tools to isolate the voices of John, Paul, George, and Ringo even in their quietest discussions.

I'm sure SoS's crack team is on this and look forward to a professional level exposition but for now I thought that starting a discussion thread might be a good start.

If folks are so inclined, post your thoughts and even questions to this thread and we can start a discussion. There is a huge amount of money be invested in AI related startups and a non-trivial percentage are going to focus on music creation/production as a target because of how much money is spent annually on it. I think within a year or at most two we will see these in the next generation of softsynths,plugins, and even studio hardware. It's going t get wild

Cheers
d

[RichardT]

I expect that in 10 years, possibly less, AIs will be able to compose and perform on a par with good musicians. I expect also that AI will be able to impersonate any singer.

What it will mean for real musicians? I think we’ll have to wait and see. People do like things made by people, so there may be hope.

[Drew Stephenson]

I've been keeping half an eye on this since I first read about Jukedeck a few years back and I'm intrigued about how the copyright angle will play out.
We'll come back to copyright in a bit, but from a music creation perspective I see quite a few ways AI (or, more accurately in a lot of cases, machine learning) will come into play.
1) Mix assistants. We're already seeing this from a few companies. I'd expect these to get smarter and quicker. To the point where you load in a series of audio files and it creates a more-than-acceptable basic mix.
Leading to...
2) Production assistants. Got the basics of a song together? Your AI production assistant will suggest (and generate) a set of additional tracks, according to your preferred genre, and then (along with (1)) will mix it for you. As they get smarter they will get better at adding variety in terms of key, tempo and arrangement changes.
3) Complete song creation. Again, we're already seeing this in basic form from some companies. Expect to see rapid improvements.

What does this mean for musicians, mixers and producers? Well I suspect those at the top of their game and industry will be largely unaffected for a while. Reputation, inertia, and an anti-AI push-back will keep business coming in. Plus it will take a while before things get genuinely in the same ball-park as the best human offerings anyway.
For those at the bottom of the pile, or just getting into the pastime, these things could be a massive accelerator to allow people to create release-ready content without having to go through a heavy learning curve for something they have no interest in. Think of the musicians you know who have no interest in the tech or technical side of things but just want to play and make music. That will get easier.
It will be very interesting to see if the educational side keeps up though. I suspect it won't for a while, so I can see a situation where you have a lot of people who are able to create content that is 90% of the way there, but because they've never learned the basics, they don't know how to get it the last 10%.
Obviously for a lot of people 90% will be close enough, but for those who do want their stuff to be that bit better, then they'll need to find someone else to help them or go back to basics and learn from scratch.
So that should keep the mid-level of professional mixers and educators busy for a while.
In the meantime there will be a lot of music that is currently produced by people (library music being the obvious stuff) that will be almost completely supplanted by AI generated content.
Will it be as good? Nope, probably not. Not for a while at any rate. But will be it be good enough? And cheap enough? Probably.
I've said it before on here, but music in itself just isn't as culturally important as it used to be. Gaming is now a bigger industry that either video or music and how important is the music in video games? Fairly important yes, but worth the increase in price to go from a 90%-there AI track to 100%-perfect human one? Probably only for AAA games.

So in the short / medium term I see AI enabling a huge new range of creators but causing major disruption to a lot of existing creators who are producing library or soundtrack music in particular.
It will also hugely increase the overall amount of music that exists. This will make the perennial problem of getting heard even harder.

Longer term? Who knows. Some of that will be driven by economic rather than technical pressures.

Which brings me back to copyright. In the UK and US for a work to be copyrightable it requires a human creator. No human creative element - no copyright, which is why the famous 'monkey selfie' pictures are actually all public domain.
But...
If something is created by a human with the assistance of AI, there's a good chance that could still be copyrighted as the human creative element would be recognised.
The fun stuff (for the lawyers) will start when cases start coming to court to define what the threshold is for that creative element.
If I write a melody, chord progression and set of lyrics, that an AI then adds some additional parts to it, we'd probably expect that to be copyrightable. There's a reasonable amount of human expression there.
But what if I take an AI written melody and chord progression and just add words over the top? And what if those lyrics were AI generated as well? Is there a credit for combining two AI-generated outputs?
And what if my input is limited to, as in Jukedeck, a few keywords? My typing, "Summer; up-tempo, feelgood, dance, pop" into an AI generator doesn't feel like I'm doing a great deal to deserve a copyright there...
But what about the coder who wrote the software that interprets those questions?
Well obviously they have the copyright on the code, but again we go back to the fact that an AI-created work can't have copyright.
So I expect some very interesting court cases where companies will start arguing that their AI is actually a lot less "I" than advertised and really it's the hard work of their coders in the background that deserves the copyright. Expect to find that a lot of these supposed AIs are actually just generative algorithms. But again, does that deserve a copyright?
Both of these outcomes may then leave them open to other legal cases if they're deemed to have misrepresented their tech to shareholders etc.

The other interesting copyright angle will be down to the afore-mentioned massive increase in amount of music. There being only 12 notes and a constrained set of relationships for those notes in most western music, expect a lot more copyright lawsuits. A lot, lot more.

In short, I predict a field day for the lawyers, a general muddling through for most people, and it being just as hard as ever to make a living as any kind of creative artist.

[Sam Inglis]

Funnily enough I've just written a leader column about this exact topic! But you'll have to wait until the March issue appears to read it.

In short, we live in interesting times.

[Wonks]

To paraphrase Pete Townsend, 'Hope I die before AI gets old'.

[RichardT]

Perhaps we are not too far away from Frank Herbert’s ‘Butlerian Jihad’….

[OneWorld]

Hi all,

First, to the mods, please move this to whatever forum you deem best for discussing the recent AI developments in music. There's been less attention to AI in music than the "AI in Art" news about ChatGPT, Stable Diffusion, and OpenAI's text-to-image tools, but it's coming to all aspects of music creation and music production both.

[FWIW, I track this because of professional reasons (ie, my day job, such as it is) but actually received my introduction to AI back in the 1980s at the feet of a true British eccentric named Donald Michie, who was one of Alan Turing's friends and colleagues at Bletchley Part during WW2. DM, as he was known to intimates, was a visionary professor and researcher who sadly didn't live long enough to see the astounding level of progress AI and machine learning has made specifically in the last 5 years.]

I think it's going to be massive, and something forum members/SoSers should be aware of if they're not already. I've seen ads for tools on YT that will create AI-generated vocals, even generating the lyrics if you want it to. In the manner of ... whoever you want. These AI models are trained using millions of samples but once trained can generate limitless permutations on whatever themes you ask it to. Those who've enjoyed the Peter Jackson master class on the Let It Be have already experienced the 'magic' of AI in that Jackson's boffins back in NZ used these same neural network tools to isolate the voices of John, Paul, George, and Ringo even in their quietest discussions.

I'm sure SoS's crack team is on this and look forward to a professional level exposition but for now I thought that starting a discussion thread might be a good start.

If folks are so inclined, post your thoughts and even questions to this thread and we can start a discussion. There is a huge amount of money be invested in AI related startups and a non-trivial percentage are going to focus on music creation/production as a target because of how much money is spent annually on it. I think within a year or at most two we will see these in the next generation of softsynths,plugins, and even studio hardware. It's going t get wild

Cheers
d

AI in Music - Are We Ready (to Discuss It)? Has anyone asked a clever (AI'ified) computer yet? surely if they intelligent then they should be part of the debate.

I too have skin in the game going way back to the late 80's early 90's having worked extensively in the area of computational linguistics. During that period I actually became more interested in natural language as opposed to computer/artificial language processing, and just how complex that are of work the brain does. Trying to get a computer to process unseen text is not a trivial exercise. At the lexicon level, that's easy, although even then, there is much ambiguity eg.....

Bank.....of the river
Bank....the noun
Bank.....verb - "I'll 'bank' on that

The syntax level is relatively easy

Deteminers - a, the, my, yours, their et al

But when we get to semantics, and metaphor etc - things start to get much more difficult - "It's raining cats and dogs" - "A man that jumps off a cliff often jumps to the wrong conclusion" and of course the famous anecdote where as the story goes, the US intelligence services, short of translators of Russia <> English created a machine translation programme and they tried it on a piece of Shakespeare -

"The flesh is willing but the spirit is weak"
and the computer came back with
"The meat's ok but the vodka is rubbish"

Or an early English <> French translator asked to translate the French for 'The hydraulic ram' and it came back with the French for 'The water goat'

Even just the other day, apparently the Taliban have reviewed laws regarding the apparel women can adopt, and the law was reviewed that women were allowed to 'bare arms' but the edict was understood as women were to be allowed to 'bear arms'

The list goes on an on. My point being that the trouble is, we are not quite sure what 'intelligence' is yet. And in the case of music, yes AI can relatively easily knock together a piece of music - but will it stand up to scrutiny when it comes to a subjective perspective. But in a more mundane setting, music is relatively easy to emulate artificially because there is a fixed lexicon and syntax - eg - scales, keys, notes, time signature. This is why for example there have been successes with AI in the field of medicine, because there is an agreed set of names for bones in the body, and the organs etc and what are likely to be the symptoms when there is an accident or infection - of course I am not claiming a computer can be a substitute for a medical professional.

Interestingly, some years ago a group of programmers sought to introduce a programme they had developed that would be applied in the legal profession as of course to a great extent the law is codified, and we employ lawyers to disambiguate the interpretation of the law. Unsurprisingly the legal profession were hostile to an notion of co-operating with the programmers, because of course, they lawyers didn't want to put themselves out of a job. It is said however, that AI is still looming in the legal background and lawyers better stand by their beds, change is coming.

Many claims are made for machine learning using semantic nets, pattern matching etc but for the main part, the 'successes' are more common in areas where a lot of repeating data is processed and possibly much to our dismay, for about 90% of the time, 90% of the people do 90% of the things they did the day before and will most likely do the next day too - we like routine. And that is what computers are very good at, processing torrents of data, over and over, looking for patterns, extraordinary events and applying rules accordingly.

Thing is, even the fastest of today's CPUs are digital of course and deal with things in a binary fashion - the CPU can have billions of transistors, and as they get down to 3nanometer levels, the CPU won't be doing anything differently, just doing things faster, and admittedly there is a cognitive function because decisions can me made as the data travels through these transistors.

It should be remembered though, that a transistor is simply an on/off switch, and those billions of transistors generate heat, the processing is inefficient when compared to the human brain, which can do infinitely more complex processing and the energy required is a tiny fraction of that required of a fast CPU. And this is because the brain is analogue, and whilst it seems wasteful, a 'firing' event happens all the while, it's basically like, don't bother firing up a HS2 train, because there is a train passing this point every second, so if there is any processing to be done, jump on the next train, it's here already. Yes there is a redundancy, with neurons firing all the time yet the most of the time they aren't carrying data, the brain is idling.

This however has one massive advantage over the digital computer and that is randomness - things happen just by serendipity. Can you imagine an AI computer dreaming? Can you imagine today's AI telling a joke - yes this is often tried but the results are a joke! (pardon the nup) That's not to say abandon all these efforts, but we need novel approaches. I just get the feeling AI has had its day, oir more to the point, it isn't doing anything special. But I feel we are on the cusp of new horizons.

And what's new on the horizon is the neuro-processor which is an analogue cpu, and it works much more like the brain. To my knowledge there have been encouraging results experienced by some Indian researchers, who have now caught the attention of Intel et al with their prototypes, which admittedly are quite crude but the principle has been proven, and neuro-processor uses a fraction of the energy than our silicon wafer CPUs use at present. And because of the random nature of the way they work, it is not beyond possibility that they can be 'creative' because creativity is often the result of extraordinary approaches to mundane events. How many discoveries, great tunes etc are the result of an 'accident'

[OneWorld]

The fun stuff (for the lawyers) will start when cases start coming to court to define what the threshold is for that creative element.

If I write a melody, chord progression and set of lyrics, that an AI then adds some additional parts to it, we'd probably expect that to be copyrightable. There's a reasonable amount of human expression there.
But what if I take an AI written melody and chord progression and just add words over the top? And what if those lyrics were AI generated as well? Is there a credit for combining two AI-generated outputs?

Thing is Drew, if 99% of the music making community are relying on AI to write a ditty - what are the chances of your ditty or my ditty (using the same AI) or 50 million or so others ever getting beyond a 100 or so listeners/viewers on YooToob? Yes if you got say 50 million views, then the lawyers will sniff the air and be like hyenas around a kill on the Serengeti - but if there's only a few meagre pickings, methinks they won't bother

Any lawyer will tell you, never sue a man of straw

[Arpangel]

I’ll discuss anything, the concept of AI is exactly the opposite to what I want, and this is where the "we can’t stop progress" brigade start waving banners.
Give me something that I can’t predict, something that does something so amazing that no one would have thought about it, give me totally random unpredictability, you’d think that was possible and beyond "given" the fact, that AI is an accumulative coming together of billions of bits of intelligence, sourced from the machine, and human worlds. We already have enough creativity and unpredictability in the world, they are called "human beings"
Is AI a good thing or not? my instinct says that, like anything to do with computers, it makes "average" and "easier" more attainable, and at the same time, creativity instantly becomes devalued, it’s so easy, anyone can do it, anyone, can be average, OK, good enough.
You can’t stop progress? give me a sledge hammer, I'll show you how.

[Kwackman]

To paraphrase Pete Townsend, 'Hope I die before AI gets old'.

Nominated for quote of the month.

[Drew Stephenson]

On the legal front there's an interesting write up of a case being raised on AI-generated art: https://www.technollama.co.uk/artists-f ... midjourney

And to respond to OneWorld, yes the lawyers will only be focused on the very top selling stuff, but the focus may shift from lawsuits aimed at individuals to ones aimed at companies.

The other area that might be more interesting (and dare I suggest useful) is in terms of recommendations. As more and more content is produced it would be nice to have a good tool to help me find the interesting stuff. Most of the current streaming services do this to an extent but it is a bit hit and miss with a focus on the more established acts. AI could act as a levelling tool here.
Or, of course, it could be used as a tool to drive even more listeners to an even smaller pool of content creators.

[Sam Spoons]

WRT the copyright issue, I remembered this from a few years ago https://www.hypebot.com/hypebot/2020/02 ... drive.html it doesn't seem to have made any impact though.

My understanding was that once a concept/melody/piece of writing was in the public domain it could no longer be patented or copyrighted. The article does go some way to explaining that but this, from the comments, goes a step further to explain

David Carson February 26, 2020 at 10:38 am
Nice try, but far from foolproof. If you come up with a melody that, unbeknownst to you, is on their hard drive, and I copy your melody without having heard the version that’s on their hard drive, I’ve probably infringed the copyright in your melody. A work does not have to be novel in order to be protected by copyright; it only needs to be “original” (i.e., independently created by an author — and more than one author can independently create the same work) and have a “modicum of creativity.”

[OneWorld]

To paraphrase Pete Townsend, 'Hope I die before AI gets old'.

Nominated for quote of the month.

"'Hope I die before AI gets old'" - the irony there is - he will. We should be careful what we prAIy for

[Drew Stephenson]

Yes it's an interesting experiment. It would be interesting to see a) how it stood up against the test of having a human creator, but also b) the concept of parallel creation is always a bit of grey area.
More work for the lawyers!

[OneWorld]

The other area that might be more interesting (and dare I suggest useful) is in terms of recommendations. As more and more content is produced it would be nice to have a good tool to help me find the interesting stuff. Most of the current streaming services do this to an extent but it is a bit hit and miss with a focus on the more established acts. AI could act as a levelling tool here.
Or, of course, it could be used as a tool to drive even more listeners to an even smaller pool of content creators.

I find the streaming companies much like the record company model - there are those that find favour with (or buy 10,000's of 'likes' in Indian Like Factories) or have record companies that buy popularity by buying publicity campaigns. That's why I favour YooToob, there are legions, of artists from every country, from whoever a country's 'top artists' are, to people like us, knocking together a ditty that when you listen/watch it, you conclude "that ain't half bad" Somehow I don't find the streaming services have the breadth of content that YT has, and anyway, YT is free, so I can sit here all day, clicking on this and clicking on that and every now and again coming across a gem of a tune.

Yes YT has algorithms too, but the thing is, with today's 'AI' (and I use the term loosely, 'Artificial' Intelligence is like 'Artificial Sex' I would imagine and call me old fashioned but I prefer the more organic analogue version) anyway, I digress.

Yes YT has algorithms, but the best AI can do is employ complex and sophisticated pattern matching algorithms by noting what music I have looked at in the past, and which boxes I have ticked, for example - Jazz, Brazilian, Guitar, Easy on the Eye Chicadees etc etc etc and the algorithm says to itself "Right I'll go and rummage around in the stock room to see what fits that criteria" so the AI tends to come up with more of the same. And once it has found a list of likely contenders, which ones appear at the top of the list? - of course those with a bigger/better standing in the Google Clickopoly.

Now for the main part that works, but it doesn't allow for serendipity, where on the odd occasion, we as unpredictable humans, decide on the occasion to stray off the path. We like the unexpected, we relish surprise and eccentricity, the bohemain indulgences. For example one day I decide to take a different route to work, and along the way I hear a busker play, and am drawn down another street to watch them play and say "Do you know, that tune you played has made my day, thank you and I toss a sheckle or 2 into their tin"

AI doesn't do that........yet!

I note that TikTok is introducing a new service, aimed at giving aspiring musicians their 5 minutes of exposure - be interesting to see how it turns out. I'm not a neophile, I don't favour something simply because it's new, but because it's new, I won't have heard it before, and neither will AI either.

[OneWorld]

.........the concept of parallel creation is always a bit of grey area.......

Best left to the 'grey' matter!

[jxnWHITE]

ya, been having this discussion for a while now as it relates to music (since the launch of LANDR), and suspected it's been in play for some time based on the likes of Max Martin, Diane Warren, etc. We see it as certainly possible, doable and likely, to the extent that it will satisfy the needs of a rather large % of the population.

"Band-in-a-Box" will become "Song-in-a-Box", film scores will be generated by directors during lunch break and traditional careers in music will be a thing of the past.

There's probably a real business opportunity for "Talent-in-a-Box" though, with formant algos to transform raw vocal tracks into your favorite singer. It demos with a free Bob Dylan model but something like Adele would command a nice price. Which might have talent shows morphing into some sort of musical Turing Test, (i.e. "(your country)'s Got Talent" to "(your country)'s Got Algos"?)

"AI" is not really creative though, or even self learning and can only generate (not create) an output based on accessible data+algorithm. Given the tried and true formulas of the business, one would expect "songs" to converge on the same old same old quite rapidly and lead almost as quickly to boredom and loss of relevance or interest.

Sure, you could program some sort of randomness into the algorithm to generate novelty, but "creativity" then becomes subject to good old Darwinian outcomes and likely to generate far more garbage than something that resonates with a human (with cash to make it worthwhile).

So much to look forward to.

(NB: This post was generated by ChatGPT)

[Sam Spoons]

"AI" is not really creative though, or even self learning and can only generate (not create) an output based on accessible data+algorithm. Given the tried and true formulas of the business, one would expect "songs" to converge on the same old same old quite rapidly and lead almost as quickly to boredom and loss of relevance or interest.
(NB: This post was generated by ChatGPT)

Stock, Aitken and Waterman have been doing exactly this for years with, at least in my case, a remarkably similar end result

[BillB]

Interesting:

https://www.theguardian.com/music/2023/ ... -nick-cave

Nick Cave describes an AI song 'in his style' as a grotesque mockery.

[Drew Stephenson]

I don't think he could be described as a neutral observer though!

[BillB]

I don't think he could be described as a neutral observer though!

True, although he will have a closer perspective than most as to what his songs should sound like

[Drew Stephenson]

Yes, and to be fair, a) this is just a discussion about what we think / feel about it, so there's actually no requirement for objectivity, and b) he's right - it is a terrible facsimile of Nick Cave lyrics.

Full disclosure, I'm a huge fan of his lyric writing.

[lg8316@gmail.com]

I have tried various AI music generators such as Aiva, Boomy, and Soundraw, but Soundful https://soundful.com/ was the one that impressed me the most. I upgraded to their premium subscription and while their licensing agreement is not entirely clear, I am able to download the STEMs, upload them into Ableton, and make enough adjustments so that I don't violate any licensing terms . The quality of their STEMs is exceptional

[awjoe]

"AI" is not really creative though, or even self learning and can only generate (not create) an output based on accessible data+algorithm.

Yes, but we *are* creative, and AI will be able to cruise on top of our creativity, doing the things that only AI can do. All we have to do is relinquish our hold on the top position of the hierarchy on this planet and continue to build and service the machines. (The machine overlord idea isn't that drastic, really, considering how close we are to it already. Some people would even argue that we're making a hash out of our stewardship of the planet anyway, so maybe it's time we stepped down.) Most people would be happy with that scenario if their existence was comfy enough. Anyway, in terms of music, it might look something like this: I write a song and then plug it into the musical version of Chatgpt or whatever it's called, enter my parameters for genre, instrumentation and sound and hit 'Do it'.

Don't like it? Then take my idea, add a tragic hero and a love story and some sidebar elements like climate crisis, gender confusion and space travel and write the sci-fi story. You're welcome.