Creating Stems From A Stereo Mix For Free

For everything after the recording stage: hardware/software and how you use it.

Moderator: Moderators

Creating Stems From A Stereo Mix For Free

Post by James Perrett »

I was intrigued by the mention of Spleeter in this thread

https://www.soundonsound.com/forum/view ... 17&t=79577

Spleeter performs a very similar function to Music Rebalance in Izotope's RX by taking a stereo mix and separating it into vocals, drums, bass and other instruments. In fact they are so similar in function that it looks like they could well be based on the same code.

Spleeter was created by Deezer and made available as a Python library for free but it doesn't seem to have found its way out to the wider music world. This is probably down to the difficulty in using it. I have found various scripts for Reaper that allow you to use it but they involve installing a new version of Python and what appears to be a little jiggery pokery. Otherwise it is a command line program.

However, on my searches I also happened to find someone who had translated it to C++ and then created a VST plug-in. This can be found at

https://github.com/diracdeltas/vstSplee ... tag/v0.2.1

for 64 bit Windows and

https://github.com/diracdeltas/vstSplee ... es/tag/0.3

for Mac

I have had a quick try of the Windows version and it certainly works. The results are different to RX's Music Rebalance but it is hard to say which one is better - sometimes the Spleeter version wins out while sometimes the RX version is better. There are big problems with latency and timing but they can be easily fixed by manually moving things around. I'd probably render the results rather than try to use them in real time although the plug-in certainly works well in real time for me.

Hopefully this is useful to anyone else looking for an affordable way to split mixes.
User avatar
James Perrett
Moderator
Posts: 11376 Joined: Mon Sep 10, 2001 12:00 am Location: The wilds of Hampshire
JRP Music - Audio Mastering and Restoration. JRP Music Facebook Page

Re: Creating Stems From A Stereo Mix For Free

Post by BWC »

I was also intrigued, and saved the link to Spleeter's GitHub page. I've now also saved the link to this VST. Thanks James!
BWC
Regular
Posts: 164 Joined: Thu Apr 29, 2021 9:12 am Location: Florida, US
BWC

Re: Creating Stems From A Stereo Mix For Free

Post by Eddy Deegan »

James Perrett wrote: Fri Sep 24, 2021 11:16 pm Spleeter performs a very similar function to Music Rebalance in Izotope's RX by taking a stereo mix and separating it into vocals, drums, bass and other instruments.

May I play Devil's advocate for a moment?

I realise that for certain situations this sort of capability can be extremely useful, especially for musical engineers working with otherwise difficult mixes, but part of me dislikes that a carefully packaged and mixed stereo track can be so readily picked apart.

I've not tried it, so I may be getting ahead of myself, but assuming that it's either pretty good at what it does or that it's the forerunner of tools that will do the job better it sounds like a resource that can be used for a lot of bad as well as good.

I'm thinking along the lines of lifting individual performances from one work and incorporating them into another, producing unauthorised remixes of a work for which no multi is available or extracting otherwise unusable samples from a finished mix.

I'm not trying to be precious, and as I said I am playing Devil's advocate here, but the connotations of such capability leaves me slightly uncomfortable. It seems quite ... invasive?
User avatar
Eddy Deegan
Moderator
Posts: 6611 Joined: Wed Sep 01, 2004 12:00 am Location: Brighton & Hove, UK
Some of my works | The SOS Forum Album projects
 

Re: Creating Stems From A Stereo Mix For Free

Post by evolvetek »

The idea of this being invasive to you / the industry is irrelevant. It is happening, and it behooves you to take advantage of it, not cower in fear of it.

Also, it is not "so easily" picked apart if you have solid ears. The AI for this, while having been available in an open source format for several years now, and several new companies formed as a result, refining, offering, but still quite early, and still quite rough in its current execution. I can think of 100 additional use cases, and supporting code modifications, off the top of my head that need to be fleshed out.

The best implementation of the AI separation that I have found is Accusonus' Regroover Pro, and other tools.

What we truly need is audio's version of the image world's AI Up-Resing.

/e
evolvetek
Posts: 1 Joined: Sat Sep 25, 2021 2:12 am

Re: Creating Stems From A Stereo Mix For Free

Post by Eddy Deegan »

evolvetek wrote: Sat Sep 25, 2021 2:22 am The idea of this being invasive to you / the industry is irrelevant. It is happening, and it behooves you to take advantage of it, not cower in fear of it.

By definition it's not irrelevant because it's relevant to me (speaking mainly with a musician hat on as opposed to that of an engineer). I think it's also perfectly reasonable to dislike the idea that if I put together and release a track I've worked long and hard on that someone with dodgy intent can plunder the individual parts therein.

You may have a different opinion but in that particular scenario we would be in disagreement.

That said, I'm still playing Devil's advocate. I do dislike the concept personally but I get that I'm not 'everyone' of course.
User avatar
Eddy Deegan
Moderator
Posts: 6611 Joined: Wed Sep 01, 2004 12:00 am Location: Brighton & Hove, UK
Some of my works | The SOS Forum Album projects
 

Re: Creating Stems From A Stereo Mix For Free

Post by BWC »

Pretty much all tools can be put to use for good or evil purposes. Don't blame the tools! It's like my favorite tech support diagnosis, "Problem exists between keyboard and chair." I'm fascinated by the prospect of "de-mixing," and I have more old work with serious mix problems than I'd like to admit. I've seen some things about the de-mixing that Abbey Road can do, but not much, and I understand why they might be slow to reveal some of their tricks. The last software I was looking at was https://hitnmix.com/
BWC
Regular
Posts: 164 Joined: Thu Apr 29, 2021 9:12 am Location: Florida, US
BWC

Re: Creating Stems From A Stereo Mix For Free

Post by Eddy Deegan »

BWC wrote: Sat Sep 25, 2021 3:50 am Pretty much all tools can be put to use for good or evil purposes.

Some can, but most? ... no, they really can't. Misused yes, but 'for evil' not so much. How does one use a compressor, expander, reverb, chorus, flanger, echo, EQ etc. for evil? What's your definition of 'tools'?

BWC wrote: Sat Sep 25, 2021 3:50 am It's like my favorite tech support diagnosis, "Problem exists between keyboard and chair."

I am familiar with, and have cited, that many times myself. I see no correlation between that and the subject of 'un-mixing' at all.

BWC wrote: Sat Sep 25, 2021 3:50 am I have more old work with serious mix problems than I'd like to admit.

That's a valid point (I do too!) but are you ever going to get as good as, yet alone better than, a result as if you'd done it right in the first place? Very likely not. Revisiting the work and doing it over from source is all but certain to give the best results.

A stereo master will contain the faults you embedded in it, particularly if you made bad EQ or other processing choices during mixdown and splitting the mix back out with a tool won't fix those. Further, having split it if you 'correct' them and remix you'll end up with something that can never be as good as if it had been done in the first place.

I get that for some things it's 'better than nothing' (hence my acknowledgement it could be useful in my original reply) but IMHO at best it's always going to be something of a hack and I think you're better off remixing the originals or, in the absence of stems, decent retakes of them.

There is quite possibly a remaining pool of use-cases where de-mixing could be the best option but I assert it's far smaller than advocates of such technology would care to admit.
User avatar
Eddy Deegan
Moderator
Posts: 6611 Joined: Wed Sep 01, 2004 12:00 am Location: Brighton & Hove, UK
Some of my works | The SOS Forum Album projects
 

Re: Creating Stems From A Stereo Mix For Free

Post by BWC »

Eddy Deegan wrote: Sat Sep 25, 2021 4:04 am
BWC wrote: Sat Sep 25, 2021 3:50 am Pretty much all tools can be put to use for good or evil purposes.

Some can, but most? ... no, they really can't. Misused yes, but 'for evil' not so much. How does one use a compressor, expander, reverb, chorus, flanger, echo, EQ etc. for evil? What's your definition of 'tools'?

Yes, most. Walk around a hardware store sometime and note how much of their inventory could be lethal. I was using the most general definition of "tools". I was similarly generalizing with "good" and "evil". There have been many uses of such effects that my ears have deemed EVIL ear poison!

Eddy Deegan wrote: Sat Sep 25, 2021 4:04 am
BWC wrote: Sat Sep 25, 2021 3:50 am It's like my favorite tech support diagnosis, "Problem exists between keyboard and chair."

I am familiar with, and have cited, that many times myself. I see no correlation between that and the subject of 'un-mixing' at all.

Really?!? You don't see it? The problem is with the people who get up to such "evil", not the "tools" they use. OK, it's intended target is human error, but it's still true when it comes to less than honorable humans' intentions.

Eddy Deegan wrote: Sat Sep 25, 2021 4:04 am
BWC wrote: Sat Sep 25, 2021 3:50 am I have more old work with serious mix problems than I'd like to admit.

That's a valid point (I do too!) but are you ever going to get as good as, yet alone better than, a result as if you'd done it right in the first place? Very likely not. Revisiting the work and doing it over from source is all but certain to give the best results.

A stereo master will contain the faults you embedded in it, particularly if you made bad EQ or other processing choices during mixdown and splitting the mix back out with a tool won't fix those. Further, having split it if you 'correct' them and remix you'll end up with something that can never be as good as if it had been done in the first place.

I get that for some things it's 'better than nothing' (hence my acknowledgement it could be useful in my original reply) but IMHO at best it's always going to be something of a hack and I think you're better off remixing the originals or, in the absence of stems, decent retakes of them.

There is quite possibly a remaining pool of use-cases where de-mixing could be the best option but I assert it's far smaller than advocates of such technology would care to admit.

The old work I was referring to is mostly a case of, source tapes long gone. I absolutely agree that it's always best to get it right from as close to source as possible, and my primary plan is to attempt to recreate the best of that work rather than try to rescue it, for exactly the reasons you've given. That, however, doesn't make me any less interested in the latest developments in de-mixing (or un-mixing, whatever the proper term), and I've seen enough in my time to know better than to say, "It'll never be as <?> as <?>!!"
BWC
Regular
Posts: 164 Joined: Thu Apr 29, 2021 9:12 am Location: Florida, US
BWC

Re: Creating Stems From A Stereo Mix For Free

Post by Zukan »

Thanks for the useful info James.
User avatar
Zukan
Moderator
Posts: 9522 Joined: Fri Sep 12, 2003 12:00 am
'Shaka. When the walls fell. Zukan...with his arms wide.'
Eddie Bazil
Samplecraze 
The Audio Production Hub

Re: Creating Stems From A Stereo Mix For Free

Post by blinddrew »

I would expect one of the main use cases here is in audio-for-video where the original dialogue track can be removed and replaced with a new language over-dub.
User avatar
blinddrew
Jedi Poster
Posts: 16518 Joined: Sun Jul 05, 2015 12:00 am Location: York
Ignore the post count, I have no idea what I'm doing...

Re: Creating Stems From A Stereo Mix For Free

Post by BWC »

I don't think I can predict what the main use cases will be when the tech matures enough for there to be many users. I think we all might be surprised by what emerges. Instead, I prefer to think about what all I might use it for, and watch and wait to see what the more clever people out there will do with it. Regardless of uses, I'm amazed by the mathematical accomplishment (even if it's still kinda, sorta) of what was previously thought impossible.
BWC
Regular
Posts: 164 Joined: Thu Apr 29, 2021 9:12 am Location: Florida, US
BWC

Re: Creating Stems From A Stereo Mix For Free

Post by James Perrett »

I'm already making quite a bit of use of Music Rebalance. There's a track on the new Vapors box set that simply wouldn't have been usable without it. I also have one unconventional client who creates his demos by singing with his guitar into his phone and then wants to use those demos as the basis for the final recording. The guitar often overpowers the vocals which carries over to the final product. He does most of the recording and mixing elsewhere but comes to me for the final mastering. The people doing the mixing do a great job but their hands are tied by the initial balance. Now that I have Music Rebalance I can bring out his vocals and reduce the acoustic guitar far more effectively than before.

And of course Eddy's arguments against it have a similar ring to the arguments against sampling 40 years ago. There are both good and bad ways to use it but I'm finding it very useful.
User avatar
James Perrett
Moderator
Posts: 11376 Joined: Mon Sep 10, 2001 12:00 am Location: The wilds of Hampshire
JRP Music - Audio Mastering and Restoration. JRP Music Facebook Page
Post Reply