These awful AI song lyrics show us how hard language is for machines

Among creative applications for algorithms, writing lyrics and poetry has proved particularly challenging. The art of producing lyrics with a machine also differs greatly from other tasks in natural-language processing. The ultimate goal is to be creative rather than accurate, which can have a difficult-to-pin-down definition.

That didn’t stop two researchers at Google from trying to create an automated lyric-generation machine. They approached the task with two separate machine-learning models. They trained the first on the structure of a massive set of lyrics by extrapolating individual song lines into their parts of speech. They trained the other on words from 20 popular books, to build up a corpus of diverse vocabulary. Then the machine mashed it all together by pouring the vocabulary words into the structures of existing lyric lines.

The result, measured on a scale of word and line diversity (in other words, a low fraction of repeated words and lines), proved better than the output of a model trained purely on lyrics or purely on words. But the lyrics are still hilariously garbage. Behold:

i'm your big and brave and handsome romeo
you know my secret secret
you have my second estate
you suit your high origin
you have my cursed youth
you have my life

And another:

come on, uh
you remember the voice of the widow
i love the girl of the age
i have a regard for the whole
i have no doubt of the kind
i am sitting in the corner of the mantelpiece

The exercise proves how far we still are from replicating the ingenuity of human language. We still haven’t figured out how to imbue machines with a sophisticated understanding of what we’re saying—let alone an ability to write anything as masterful as “Shake It Off.”