Among creative applications for algorithms, writing lyrics and poetry has proved particularly challenging. The art of producing lyrics with a machine also differs greatly from other tasks in natural-language processing. The ultimate goal is to be creative rather than accurate, which can have a difficult-to-pin-down definition.
That didn’t stop two researchers at Google from trying to create an automated lyric-generation machine. They approached the task with two separate machine-learning models. They trained the first on the structure of a massive set of lyrics by extrapolating individual song lines into their parts of speech. They trained the other on words from 20 popular books, to build up a corpus of diverse vocabulary. Then the machine mashed it all together by pouring the vocabulary words into the structures of existing lyric lines.
The result, measured on a scale of word and line diversity (in other words, a low fraction of repeated words and lines), proved better than the output of a model trained purely on lyrics or purely on words. But the lyrics are still hilariously garbage. Behold:
i'm your big and brave and handsome romeo
you know my secret secret
you have my second estate
you suit your high origin
you have my cursed youth
you have my life
come on, uh
you remember the voice of the widow
i love the girl of the age
i have a regard for the whole
i have no doubt of the kind
i am sitting in the corner of the mantelpiece
The exercise proves how far we still are from replicating the ingenuity of human language. We still haven’t figured out how to imbue machines with a sophisticated understanding of what we’re saying—let alone an ability to write anything as masterful as “Shake It Off.”
Geoffrey Hinton tells us why he’s now scared of the tech he helped build
“I have suddenly switched my views on whether these things are going to be more intelligent than us.”
ChatGPT is going to change education, not destroy it
The narrative around cheating students doesn’t tell the whole story. Meet the teachers who think generative AI could actually make learning better.
Deep learning pioneer Geoffrey Hinton has quit Google
Hinton will be speaking at EmTech Digital on Wednesday.
We are hurtling toward a glitchy, spammy, scammy, AI-powered internet
Large language models are full of security vulnerabilities, yet they’re being embedded into tech products on a vast scale.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.