Third time lucky? Software to change the voice's timbre...

[VOLOVIA]

I tried to start a discussion on this topic twice already and failed to raise a single comment, thus I try one more time and let it lie. This is surprising since I believe it represents a fascinating subject:

In an era of AI, machine learning, deep fakes, etc. but also very successful profiling of amps, cabs, effects, etc.

AND, vocal manipulation of the pitch including within a choir or embedded in a track (Melodyne etc.),.. (the holy grail of post-mixing editing)...

the actual changing of the vocal timbre has eluded all attempts. I have tried the various Waves plug-in, the built-in in Cubase and LogicPro X, and they all fail miserably. The formant change option in Logic for instance does nothing to change the timbre significantly and only at extreme settings you can hear it, but it becomes highly artificial and robotic. Waves' with its 'throat' parameters etc. promised much but delivers nothing but aliens.

So, where is the technology to modify the vocal harmonics convincingly to change the timbre (not just transpose or tune, which is done rather successfully already)? What are the technical obstacles stopping a well-tuned, sung vocal take be modified to sound closer to a soul singer let's say, or different gender? How far are we from being able to morph my vocal take (providing it is of professional standard) into something like Barry White, say?

I'd rather not discuss the 'moral' implications of such technology, or even waste time in sly comments such as "learn how to sing properly", and keep the discussion on the technological challenges. Maybe the technology is already out there, mature, but I am not aware of it. If so, links, please.

What would I use it for? Not much to change my own voice, which represents me for what it's worth, but I would love to have Stevie Wonder and Madonna "doing" my BVs...

[Wonks]

I think you are misusing the word ‘timbre’ here, as all the formant adjusting software I’ve used certainly noticeably affects the timbre of the voice.

What it doesn't do is affect the overall character of the voice and make you sound like someone significantly different.

What you are talking about is software that will morph your voice into sounding just like someone else’s voice. I doubt that software exists at the moment.

I’d guess that at best you’d either have to start with a voice very similar to the original you want to mimic, or you’d need a comprehensive recording of that voice saying every word in the dictionary, and each word in several different ways. You’d still need very complicated software that works out how to select the right word, the closest version of the target voice pronunciation and the ability to morph that sound to the way and speed you say it.

[Arpangel]

The human voice is so complex, to modify it, you’d have to do it in such a convincing way, because the brain is tuned to pick up the human voice more than anything, and it will recognise anything that doesn’t sound natural.
I can see a picture of someone from years ago, and not recognise them, but if I hear a voice, it’s instant recognition.
Pitch correction and transposition is still not very good, and regards tonal changes, I don’t know of anything at all, if you want a natural sounding, convincing change.

[Mixedup]

I suspect that the answer to what you want lies in a sort of AI sample instrument, since you'd need to be able to tell the system what sort of character you want at any given point — much like you tell a sample instrument what sort of playing style you want (can't just say 'classical', as you need to be much more specific...). The tech exists – as in the demo fakes from Adobe and other years ago, where they took Obama and others and created new speeches; there's no reason in principle that such a system couldn't learn two voices and use one to 'drive' the other. But singing styles and voices involves way more critical parameters than speech. It would take a LOT of learning of both the singer and the 'donor' voice. The reason they used Obama etc is that there are hours and hours of public speaking to use for the learning. Even with a prolific artist there's much less, and that's even before you start to consider the particular style/delivery. So I reckon you'd have to get session singers in for the learning sessions... Yamaha's Vocaloid is about half way there, I suppose. But it will not give you what you seek. I' sure there are research projects out there, and I'm sure it's coming... just not yet. Meanwhile, there are a gazillion great online session vocalists/musicians you could turn to if you're unhappy with your own voice, not to mention singers looking for musical collaborators...

[Mixedup]

To add one point... I think this is far more complex than you imagine. Consider mic modelling. There are a handful of companies doing this at all successfully, and none are perfect in all applications. Those that are best employ a specific mic — a known quantity — and impose characteristics on the result. Ie a one:many relationship. To do something like that successfully for any singer, it would be a many:many relationship, with all the additional complexity of there being many more variables for each 'one'.

[Drew Stephenson]

In contrast to the previous posters, I reckon we're probably not that far off.
Text to voice generators like Adobe's can work off about two minutes recorded speech to generate new content, but obviously they're not having to work with mumbled words, interesting accent changes and the tonal and dynamic ranges of typical singing.
I think this will be driven by the movie and games industries (foreign language over-dubbing being the obvious use), but music won't be far behind.

[James Perrett]

I already find it difficult to work out whether some of the voices I hear in our son's programming are real or fake. The speech synthesiser in Scratch is extremely good and I don't think it will be long before it can sing as well.

[VOLOVIA]

I already find it difficult to work out whether some of the voices I hear in our son's programming are real or fake. The speech synthesiser in Scratch is extremely good and I don't think it will be long before it can sing as well.

Hi guys, thanks for your input. As I mentioned before, I don't need to replace my voice. I think some people might even find it palatable! But yes, I have been bombarded on Facebook by a company selling 'voice-over software' and I gave it a try. Started with English and it sounded rather good. However, there is no mother like the mother tongue, thus I tried it in Italian, and still sounded very, very natural (TXT to Speech).

The next step will be modulating the voice according to notes... But this is NOT what I was looking for, a "machine that sings for me". I was talking about a 'filter box' that can alter simple parameters like the volume of the throat canal or the pipe length. Again, I remember a plug-in from Waves having the 'right' parameters but sounding useless, not better than the bog-standard Logic's "formant change" option. In my technical innocence, again, I would have thought that a sung phrase nowadays could be easily manipulated to sound like a different voice. I am obviously wrong. Thanks!

[Drew Stephenson]

I wonder if it's another candidate for the paradox of robotics, where things that are simple and menial for humans (cleaning a toilet, mimicking someone else's voice) are actually really difficult for robots?

[Wonks]

There's this Antares/Autotune product which is along the lines of what you were looking for, but I've no idea what it's like. But you can try it for free. https://www.antarestech.com/product/throat/

[Drongoloid]

I'd rather not discuss the 'moral' implications of such technology

I'd say the moral implications are the only important ones in this discussion. How nuclear fission works is of interest to but a handfull but it's implications are of interest to all of us.

[Drew Stephenson]

I'd rather not discuss the 'moral' implications of such technology

I'd say the moral implications are the only important ones in this discussion. How nuclear fission works is of interest to but a handfull but it's implications are of interest to all of us.

Well that would be a potentially interesting discussion, and one that we could start a thread on, but it feels a trifle rude to go directly against the wishes of the OP.

[VOLOVIA]

I'd rather not discuss the 'moral' implications of such technology

I'd say the moral implications are the only important ones in this discussion. How nuclear fission works is of interest to but a handfull but it's implications are of interest to all of us.

Well that would be a potentially interesting discussion, and one that we could start a thread on, but it feels a trifle rude to go directly against the wishes of the OP.

Feel free! morals/morality/philosophy is my 4th hobby of choice... glad to read opinions on the matter. Sure, the technology as we are allows any sung performance to be pitched near to ideal as one wishes, the so-called "anyone can sing in tune nowadays". Sure.
Singing is not just "notes-in-tune", but also requires delivery, call it passion/meaning if you liked. At this point the next step would be to turn the great guitar solo I played "perfectly" on a cheap studio guitar into a PSR-59-strat-Mesa- blah blah tone monster. This is kind of possible already, as you all know.
Now, do the same with a voice, in tune, full of expression, but sung with a 'thin voice'. Hey, presto, pre-set 24 Stevie Wonder Young Boy II turns the character of the voice, the harmonics, into a great sound. Like turning a studio flute tone into a solid-silver marvel.
Would this be ethical? Artistic? Worthwhile? Soulless?

[Forum Admin]

I'm sure iZotope will now have read your post and be working on their forthcoming VoiceMimic app! If Jon Culshaw and other great vocal impressionists can mimic, I'm sure AI will soon be able to.

Maybe iZotope DeadRingers is a better name?

I recall SOS reviewing the TC-Helicon VoicePrism hardware processor in 2001, which claimed to be capable of changing your sonic personality. SOS review here:
www.soundonsound.com/reviews/tc-helicon-voiceprism-plus

As an aside, has anyone tried Murf AI for voice-overs?

[James Perrett]

As an aside, has anyone tried Murf AI for voice-overs?

I had a quick listen to their demos and I'm not convinced that they're much better than the ones you could create for free in Scratch although they seem to offer a wider range of voices.

https://en.scratch-wiki.info/wiki/Text_ ... _Extension

Looks like the Scratch extension is powered by Amazon Polly

https://aws.amazon.com/polly/

which has both free and paid for tiers and a wider variety of voices.

[Eddy Deegan]

When it comes to creating music, two major aspects that contribute to the final result are equipment and performance. Engineering is also very important but that's more about the finesse of the final recording than the art itself (he said controversially).

I really like the fact that these days there are plugins and hardware within reach of the average consumer that are capable of generating the kinds of noises that in years gone by were only available to the wealthy.

When it comes to performance my view differs. There is no substitute for a talented player/drummer/vocalist and call me a luddite but I think that's the way it should be.

If there was a plugin that made anyone sound like Freddie Mercury (my favourite singer of all time) Marko Saaresto (a close second) or <insert singer of your choice here> then I think that would be a shame.

Music is so much more than technology. The human element is the magic ingredient and if it were to be achievable in software then although it might be incredibly good I think we'd lose something emotionally, and as a result wouldn't value that product as much as we do when we hear an artist whose work we love.

[RichardT]

That’s interesting Eddy - I agree, except that I’d say the key element is good musical judgement rather than virtuoso-level performing skills (although those would be a great help). I don’t see a problem with editing performances to make them better as long as the results are musical.

I do agree that completely artificial performances would be soulless. The point of music is to connect the artist with the listener and if there is nothing behind the music then something fundamental is lost.

I would very happy though if I had tools to generate, for example, convincing string backings that I could then edit, rather than going through the slog of starting from scratch!

[Drew Stephenson]

I'm going to disagree slightly with the last couple of posts but only in my suggestion that music is created for multiple reasons.
One of those reasons is the artistic drive of the creator, they have something they want to say and share with the world. And for this the passion and humanity of the performance is critical.
But other music is created as backing tracks, advertising jingles, lift music etc. Archetypal muzak if you like. And for that you just need something that fits in the ball park.

[VOLOVIA]

We are so fine-tuned to detect minute -anomalies- in the spoken/sung pattern that probably any variations from the norm will always sound artificial. I give you an example: after 35 of using English as my main language (left Italy in 1984), I still have a perceptible non-Anglo-cadenza that I find hard to shake off (yes, sometimes I need to: Italian accent does not evoke 'efficiency' or 'rationality', like a German accent, let's say does, and in my line of work, I need both in spades).

Anyhow, a few years back I was asked by a friend to record some short English poems to be played to non-native speakers in a class. As hard as I tried, I could not get rid of the pervasive 'Dolmio' flavour. As disheartened as I was, I decided to at least to lower the pitch in Logic to sound 'more studio professional'. I was shocked to hear that the perceived 'accent' had dropped dramatically. What the hell?? Then I started investigating two key factors that make me 'sound foreigner': Italian is -faster- (because of reasons based on linguistics) and, on average, higher-pitched than English (there are other important factors, such as inconsistency of accents, sometimes pronouncing the same word in a slightly 'Scottish' or 'American' way, for instance, which is a giveaway of non-nativity).
Thus, if you slow my voice down a bit, pitch it a 'note' down, as per magic 'half' of my accent disappears. Thus, I can imagine that a remodelling of the vocal tracts post-singing could be extremely difficult to achieve in a credible manner before start sounding robotic.

However, when in the late 90s I was working in Denmark Street as a manager of a -high-tech- floor, I stumbled upon an orange box either branded Roland or Boss that COULD change my vocal gender rather convincingly. I used to play jokes on the phone with colleagues pretending to be "Giselle". Never found anything that comes so close since... any ideas? It was a desktop orange box...

[N i g e l]

Rolands variphrase technology was from the 90s I think.
It allowed for formant analysis and repitching keeping the formants constant.
Examples of this are the VP9000 sampler & I have V-Vocal which came with my DAW.
V-Vocal I use to manipulate my vocals but I find solo vocals are very delicate and easy to "break" if your after a natural sound.

Another way to do gender changing is to use vocoder with both inputs the same but with the analysis and synthesis not patched directly 1:1