Creating Stems From A Stereo Mix For Free

[James Perrett]

I was intrigued by the mention of Spleeter in this thread

https://www.soundonsound.com/forum/view ... 17&t=79577

Spleeter performs a very similar function to Music Rebalance in Izotope's RX by taking a stereo mix and separating it into vocals, drums, bass and other instruments. In fact they are so similar in function that it looks like they could well be based on the same code.

Spleeter was created by Deezer and made available as a Python library for free but it doesn't seem to have found its way out to the wider music world. This is probably down to the difficulty in using it. I have found various scripts for Reaper that allow you to use it but they involve installing a new version of Python and what appears to be a little jiggery pokery. Otherwise it is a command line program.

However, on my searches I also happened to find someone who had translated it to C++ and then created a VST plug-in. This can be found at

https://github.com/diracdeltas/vstSplee ... tag/v0.2.1

for 64 bit Windows and

https://github.com/diracdeltas/vstSplee ... es/tag/0.3

for Mac

I have had a quick try of the Windows version and it certainly works. The results are different to RX's Music Rebalance but it is hard to say which one is better - sometimes the Spleeter version wins out while sometimes the RX version is better. There are big problems with latency and timing but they can be easily fixed by manually moving things around. I'd probably render the results rather than try to use them in real time although the plug-in certainly works well in real time for me.

Hopefully this is useful to anyone else looking for an affordable way to split mixes.

[BWC]

I was also intrigued, and saved the link to Spleeter's GitHub page. I've now also saved the link to this VST. Thanks James!

[Eddy Deegan]

Spleeter performs a very similar function to Music Rebalance in Izotope's RX by taking a stereo mix and separating it into vocals, drums, bass and other instruments.

May I play Devil's advocate for a moment?

I realise that for certain situations this sort of capability can be extremely useful, especially for musical engineers working with otherwise difficult mixes, but part of me dislikes that a carefully packaged and mixed stereo track can be so readily picked apart.

I've not tried it, so I may be getting ahead of myself, but assuming that it's either pretty good at what it does or that it's the forerunner of tools that will do the job better it sounds like a resource that can be used for a lot of bad as well as good.

I'm thinking along the lines of lifting individual performances from one work and incorporating them into another, producing unauthorised remixes of a work for which no multi is available or extracting otherwise unusable samples from a finished mix.

I'm not trying to be precious, and as I said I am playing Devil's advocate here, but the connotations of such capability leaves me slightly uncomfortable. It seems quite ... invasive?

[evolvetek]

The idea of this being invasive to you / the industry is irrelevant. It is happening, and it behooves you to take advantage of it, not cower in fear of it.

Also, it is not "so easily" picked apart if you have solid ears. The AI for this, while having been available in an open source format for several years now, and several new companies formed as a result, refining, offering, but still quite early, and still quite rough in its current execution. I can think of 100 additional use cases, and supporting code modifications, off the top of my head that need to be fleshed out.

The best implementation of the AI separation that I have found is Accusonus' Regroover Pro, and other tools.

What we truly need is audio's version of the image world's AI Up-Resing.

/e

[Eddy Deegan]

The idea of this being invasive to you / the industry is irrelevant. It is happening, and it behooves you to take advantage of it, not cower in fear of it.

By definition it's not irrelevant because it's relevant to me (speaking mainly with a musician hat on as opposed to that of an engineer). I think it's also perfectly reasonable to dislike the idea that if I put together and release a track I've worked long and hard on that someone with dodgy intent can plunder the individual parts therein.

You may have a different opinion but in that particular scenario we would be in disagreement.

That said, I'm still playing Devil's advocate. I do dislike the concept personally but I get that I'm not 'everyone' of course.

[BWC]

Pretty much all tools can be put to use for good or evil purposes. Don't blame the tools! It's like my favorite tech support diagnosis, "Problem exists between keyboard and chair." I'm fascinated by the prospect of "de-mixing," and I have more old work with serious mix problems than I'd like to admit. I've seen some things about the de-mixing that Abbey Road can do, but not much, and I understand why they might be slow to reveal some of their tricks. The last software I was looking at was https://hitnmix.com/

[Eddy Deegan]

Pretty much all tools can be put to use for good or evil purposes.

Some can, but most? ... no, they really can't. Misused yes, but 'for evil' not so much. How does one use a compressor, expander, reverb, chorus, flanger, echo, EQ etc. for evil? What's your definition of 'tools'?

It's like my favorite tech support diagnosis, "Problem exists between keyboard and chair."

I am familiar with, and have cited, that many times myself. I see no correlation between that and the subject of 'un-mixing' at all.

I have more old work with serious mix problems than I'd like to admit.

That's a valid point (I do too!) but are you ever going to get as good as, yet alone better than, a result as if you'd done it right in the first place? Very likely not. Revisiting the work and doing it over from source is all but certain to give the best results.

A stereo master will contain the faults you embedded in it, particularly if you made bad EQ or other processing choices during mixdown and splitting the mix back out with a tool won't fix those. Further, having split it if you 'correct' them and remix you'll end up with something that can never be as good as if it had been done in the first place.

I get that for some things it's 'better than nothing' (hence my acknowledgement it could be useful in my original reply) but IMHO at best it's always going to be something of a hack and I think you're better off remixing the originals or, in the absence of stems, decent retakes of them.

There is quite possibly a remaining pool of use-cases where de-mixing could be the best option but I assert it's far smaller than advocates of such technology would care to admit.

[BWC]

Pretty much all tools can be put to use for good or evil purposes.

Some can, but most? ... no, they really can't. Misused yes, but 'for evil' not so much. How does one use a compressor, expander, reverb, chorus, flanger, echo, EQ etc. for evil? What's your definition of 'tools'?

Yes, most. Walk around a hardware store sometime and note how much of their inventory could be lethal. I was using the most general definition of "tools". I was similarly generalizing with "good" and "evil". There have been many uses of such effects that my ears have deemed EVIL ear poison!

It's like my favorite tech support diagnosis, "Problem exists between keyboard and chair."

I am familiar with, and have cited, that many times myself. I see no correlation between that and the subject of 'un-mixing' at all.

Really?!? You don't see it? The problem is with the people who get up to such "evil", not the "tools" they use. OK, it's intended target is human error, but it's still true when it comes to less than honorable humans' intentions.

I have more old work with serious mix problems than I'd like to admit.

That's a valid point (I do too!) but are you ever going to get as good as, yet alone better than, a result as if you'd done it right in the first place? Very likely not. Revisiting the work and doing it over from source is all but certain to give the best results.

A stereo master will contain the faults you embedded in it, particularly if you made bad EQ or other processing choices during mixdown and splitting the mix back out with a tool won't fix those. Further, having split it if you 'correct' them and remix you'll end up with something that can never be as good as if it had been done in the first place.

I get that for some things it's 'better than nothing' (hence my acknowledgement it could be useful in my original reply) but IMHO at best it's always going to be something of a hack and I think you're better off remixing the originals or, in the absence of stems, decent retakes of them.

There is quite possibly a remaining pool of use-cases where de-mixing could be the best option but I assert it's far smaller than advocates of such technology would care to admit.

The old work I was referring to is mostly a case of, source tapes long gone. I absolutely agree that it's always best to get it right from as close to source as possible, and my primary plan is to attempt to recreate the best of that work rather than try to rescue it, for exactly the reasons you've given. That, however, doesn't make me any less interested in the latest developments in de-mixing (or un-mixing, whatever the proper term), and I've seen enough in my time to know better than to say, "It'll never be as <?> as <?>!!"

[Zukan]

Thanks for the useful info James.

[Drew Stephenson]

I would expect one of the main use cases here is in audio-for-video where the original dialogue track can be removed and replaced with a new language over-dub.

[BWC]

I don't think I can predict what the main use cases will be when the tech matures enough for there to be many users. I think we all might be surprised by what emerges. Instead, I prefer to think about what all I might use it for, and watch and wait to see what the more clever people out there will do with it. Regardless of uses, I'm amazed by the mathematical accomplishment (even if it's still kinda, sorta) of what was previously thought impossible.

[James Perrett]

I'm already making quite a bit of use of Music Rebalance. There's a track on the new Vapors box set that simply wouldn't have been usable without it. I also have one unconventional client who creates his demos by singing with his guitar into his phone and then wants to use those demos as the basis for the final recording. The guitar often overpowers the vocals which carries over to the final product. He does most of the recording and mixing elsewhere but comes to me for the final mastering. The people doing the mixing do a great job but their hands are tied by the initial balance. Now that I have Music Rebalance I can bring out his vocals and reduce the acoustic guitar far more effectively than before.

And of course Eddy's arguments against it have a similar ring to the arguments against sampling 40 years ago. There are both good and bad ways to use it but I'm finding it very useful.

[Tomás Mulcahy]

James, this is brilliant, thank you. Yet another great free tool for my students.

@Eddie, going with your devil's advocate:
1. If you're gonna lift something from a track, you still have to pay for the copyright no matter what process you used to acquire it. As already said, don't blame the tool. Blame the user (who tries to get away with not paying for a copyright).

That said, it's not yet good enough to, say, extract a clean vocal that does not sound like it's been flushed down the toilet.

2. I've found it incredibly useful to, say, raise a vocal level on an otherwise good mix where the multitrack is lost. I cannot wait for this technology to improve.

[OneWorld]

The idea of this being invasive to you / the industry is irrelevant. It is happening, and it behooves you to take advantage of it, not cower in fear of it.

By definition it's not irrelevant because it's relevant to me (speaking mainly with a musician hat on as opposed to that of an engineer). I think it's also perfectly reasonable to dislike the idea that if I put together and release a track I've worked long and hard on that someone with dodgy intent can plunder the individual parts therein.

You may have a different opinion but in that particular scenario we would be in disagreement.

That said, I'm still playing Devil's advocate. I do dislike the concept personally but I get that I'm not 'everyone' of course.

"The idea of this being invasive to you / the industry is irrelevant. It is happening, and it behooves you to take advantage of it, not cower in fear of it."

That's telling you LOL

[OneWorld]

I just tried it on a tune of mine and error message came up and said "please enter some music" accompanied by the word 'turd' get some polish

[Eddy Deegan]

"The idea of this being invasive to you / the industry is irrelevant. It is happening, and it behooves you to take advantage of it, not cower in fear of it."

That's telling you LOL

It's opining at me, I'll give you that