So Fake It’s Real

New vocal technology called Vocaloid:

Developed at Pompeu Fabra University in Spain and financed by the Yamaha Corporation, the software, which is due to be released to consumers in January, allows users to cast their own (or anyone else’s) songs in a disembodied but exceedingly life-like concert-quality voice. Just as a synthesizer might be programmed to play a series of notes like a violin one time and then like a tuba the next, a computer equipped with Vocaloid will be able to “sing” whatever combination of notes and words a user feeds it. The first generation of the software will be available for $200. But its arrival raises the prospect of a time when anyone with a laptop will be able to repurpose any singer’s voice or even bring long-gone virtuosos back to life. In an era when our most popular singers are marketed in every conceivable way – dolls, T-shirts, notebooks, make-up lines – the voice may become one more extension of a pop-star brand.

The human voice has proven the most difficult of all sounds to synthesize. Digital technology can produce something clear enough to convey meaning, but only in a clipped monotone that sounds more like a robot than a real live person. A convincing human voice, spoken or sung, with all its complex, flowing articulations and quivering uncertainties has been unattainable. Yamaha has not yet made Vocaloid available for scrutiny, but judging by some early samples and demonstrations, the company seem to have made that quantum leap.

You can think of the software as a kind of audio font: musical notation and lyrics can be translated into the chosen voice, then saved for replay, just as a word processor might translate a text into Helvetica or Times New Roman and print it out as many times as you like.

These fonts are made up of a database of phonemes, the basic sounds that make up any language. To create the database, technicians record a singer performing as many as 60 pages of scripted articulations (like “epp, pep, lep”). Assorted pitches and techniques like glissandos and legatos are also thrown in the mix; with all the combinations, the process takes a week of five-hour singing days. The resultant font is “reminiscent” of the singer’s voice, says Ed Stratton, the managing director of Zero-G Limited, a London-based company that has licensed the Vocaloid technology.

….Hit music producers like Dan (The Automator) Takemura (a creator of the Gorillaz, a band that appeared only in an animated form, but sold several million albums anyway) and the Matrix (the trio of Scott Spock, Graham Edwards and his wife, Lauren Christy, that produced the three No. 1 hits from Avril Lavigne’s last album) say they are likely at least to try recording with Vocaloid instead of backup singers. “As producers, you run into some artists and oh god, it’s so hard to get the right vocal,” Mr. Spock said. “It’s intriguing, this idea of `O.K., just give me all your vowels and all your consonants and I’ll see you later.’ ”

Mr. Takemura says he would want to use the software to create sounds that human voices could not. “The first producers to work with this are probably going to have a hit just based on the novelty factor,” he said. But, he warns, “it’s the imperfections in a voice, the happy accidents, the human-ness that are often what’s best in a song.”

….In the long term, Mr. Stratton is aware that the true killer application will be recognizable celebrity fonts — the Elton, say, or the Aretha. But so far, none of the world’s most famous voices have volunteered.

Michael Stipe of R.E.M. heard a Vocaloid version of “Amazing Grace” online, and he said he was impressed. (The Yamaha Corporation includes samples with a recent press release) But he wasn’t prepared to rush out and have a font created. “I would hate to think that 250 years from now Altria would use the Michael Stipe voice to sell organic soy to a Mars landing,” he said. “It’s intriguing in 2003. I’m not sure about 2303.”

….Elvis seems like an obvious candidate for vocal reanimation. Recently (and for the first time), his estate licensed a couple of his songs for dance-floor remixes; one of them became a No. 1 single in England. Licensing Elvis for Vocaloid would be a different matter, though, says Gary Hovey, vice-president of entertainment for Elvis Presley Enterprises. “If someone came to us and said, `We want Elvis to sing this new song,’ we’d have a lot to contemplate,” he said. “We tried to retain the integrity of his original song with the remixes. Now you’re talking about a whole new vocal performance of a song he never sang or knew? How do we know he’d want to sing it?”

“Believe me, that would go all the way to Lisa,” he added, referring to Elvis’s daughter, Lisa Marie Presley, who owns Elvis’s estate.

….Once a full palette of vocal fonts is available (or once Yamaha allows users to create their own), the possibilities become mind-boggling: a chorus of Billie Holiday, Louis Armstrong and Frank Sinatra; Marilyn Manson singing show tunes and Barbra Streisand covering Iron Maiden. And how long before a band takes the stage with no human at the mike, but boasting an amazing voice, regardless? [NY Times]

Pretty amazing to contemplate – we may get back to the question of can a machine have (a) soul? I think for a long time you will be able to “feel” the difference, just like you can with even the best computer generated animation.