Project Overview
This project attempts to analyze the complexity of pop song lyrics by quantifying the repetition of lyrics using tokenization and a lossless compression algorithm. 10,000 songs by the Billboard 100 Top Pop Artists are evaluated and used to compare the artists against each other.
Details
Consider the following songs. "Sherry" appears to repeat its lyrics more often than "Sherry, Darling". The complexity algorithm identifies this repetition by tokenizing the song and translating each unique word into a 16-bit integer. The song is reconstructed using these tokens and then compressed using the gzip compression algorithm. The ratio of the size of the compressed data to the uncompressed data (times 100) gives the complexity score (shown below the lyrics).
As expected "Sherry, Darling" has a markedly higher score than "Sherry".
| Sherry The Four Seasons | Sherry, Darling Bruce Springsteen |
|---|---|
|
Sherry Sherry baby Sherry Sherry baby Sherry baby (Sherry baby, Sherry baby) Sherry can you come out tonight? (Come come come out tonight) Sherry baby (Sherry baby) Sherry can you come out tonight? (Why don't you come out) (come out) to my twist party (Come out) Where the bright moon shines (Come out) We'll dance the night away I'm gonna make-a you mine Sherry baby (Sherry baby, Sherry baby) Sherry can you come out tonight (Come come come out tonight) (Come come come out tonight) You better ask your mama (Sherry baby) Tell her everything is all right (Why don't you come out) (come out) with your red dress on (Come out) Mmm you look so fine (Come out) Move it nice and easy Girl you'll make me lose my mind Sherry baby (Sherry baby, Sherry baby) Sherry can you come out tonight (Come come come out tonight) (Come come come out tonight) Sherry Sherry baby Sherry Sherry baby |
Your Mamma's yappin' in the back seat Tell her to push over and move them big feet Every Monday morning I gotta drive her down to the unemployment agency Well this morning I ain't fighting tell her I give up Tell her she wins if she'll just shut up But it's the last time that she's gonna be ridin' with me You can tell her there's a hot sun beatin' on the black top She keeps talkin' she'll be walkin' that last block She can take a subway back to the ghetto tonight Well I got some beer and the highway's free And I got you, and baby you've got me Hey, hey, hey what you say Sherry Darlin' Now there's girls melting on the beach And they're so fine but so out of reach 'Cause I'm stuck in traffic down here on fifty third street Now Sherry my love for you is real But I didn't count on this package deal And baby this car just ain't big enough for her and me So you can tell her there's a hot sun beatin' on the black top She keeps talkin' she'll be walkin' that last block She can take a subway back to the ghetto tonight Well I got some beer and the highway's free And I got you, and baby you've got me Hey, hey, hey what you say Sherry Darlin' Well let there be sunlight, let there be rain Let the brokenhearted love again Sherry we can run with our arms open wide before the tide To all the girls down at Sacred Heart And all you operators back in the park Say hey, hey, hey what you say Sherry Darlin' Ho, ho, ho, say, hey, hey, what you say Sherry Darlin', oh come on Say hey, hey, hey, what you say Sherry Darlin' |
Complexity Score: 47 | Complexity Score: 60 |