Nur's Blog

I love having nice desktop wallpaper, mainly because a lot of jobs I’ve worked in involves me being stuck in windowless labs, and having a picture of mountains and streams on your desktop is somewhat of a substitute to having a nice view (a sad one, but you have to make do).

But the same wallpaper can also get pretty boring, so that’s why many desktop managers let you rotate your wallpapers from a set of images. But then, that set gets boring too, what if you could get a fresh wallpaper from the internet every few minutes?

A simple randomized pull from Flickr, for instance, would give you a nice new photo every refresh interval. Flickr has a web service API that lets you search and download publicly accessible photos (the Creative Commons licensed ones all permit this, for example), and a simple Python script could give you a new wallpaper every time it ran. Flickr has a search parameter that lets you return photos by a metric called “interestingness“, which is an algorithm based (at least partly) on how much other Flickr users liked or commented on a particular photo. The interestingness algorithm is basically crowd-sourced photo curation, and it works quite well; the top photos really are pretty stunning. You need a Flickr API key to use this service, but it’s free to request for one.

Are we just happy with random pulling from the top interesting photos though? Well, we could be, but we’re not going to stop there. As I was playing some of my favorite tunes from my music playlist, I thought to myself, “Why don’t I set the desktop wallpaper based on the mood and imagery that the current music was creating?”

So what does that entail? First, finding the lyrics of the song that’s playing, and picking out the key words or themes from it. Then, feeding these key words into Flickr’s search parameters and then grabbing the most interesting photo of what Flickr users’ interpretation of what those words might mean photographically. The lyrics part is easy, there’s a Python module that searches a lyrics wiki and returns the words to a song if you give it a title and artist.

The hardest part it seems, is the the natural language processing required to comprehend the meaning of a song. Now this is completely subjective, and using a machine to algorithmically do this is going to give you a lot of literal keywords. Getting a machine to understand songs and poetry remains a challenge in machine learning (depending on the song or poem, it can be a challenge even for human beings). But now our silly wallpaper script is getting a bit involved, so can we cheat on the machine learning part? Turns out we can, because IBM’s Watson has several public web services you can use, one of which is a text summarizer called Alchemy Language. The service will take a blob of text and return the top keywords, entities described in the text, and even emotions associated with said words. The accuracy of the emotions part leaves a lot to be desired, but the top keywords are useful tools to search Flickr with, so I used that to feed it into the image search, with some randomization. Again, you need an API key to use Alchemy Language, but signing up for one is free, just like Flickr.

Putting it all together, I hacked up a Python script, glued together by bits of Bash, that will get the current playing song from the MPD daemon (which I use to play music with), and from there is a chain of feeding web services, and feeding the output into yet more web services, until I get a nice picture inspired by the song. The full source code to the script (which at this point is really hacky) can be found on its own Github repo. I call it Wall of Sound, because I enjoy terrible wordplay. You can try it yourself, but you’ll need the Flickr and Alchemy Language keys, plus some hacking to suit your music playing and operating system arrangements.

How well does it work? I can say the results are always interesting, if not immediately obvious what the connection between the music and the picture is. Here are a some sample runs:

The Beatles – Here Comes The Sun

The algorithm chose to use the phrase “cold lonely winter”, and it came up with this picture:

In the picture it does look like the sun is coming up, but that is just a fortuitous coincidence. It’s a beautiful picture though, and a great desktop wallpaper.

Gemma Hayes – Keep Running

“Keep Running” by Gemma Hayes is a great source of keywords; the song mentions several city names, and they are often chosen by the algorithm as important key words.

I got this picture when the script chose “Seoul”:

Aoife O’Donovan – Fire Engine

This is where the script failed to bring up a relevant picture. First, the Lyrics Wiki does not seem to have Aoife O’Donovan in their database, so whenever that happens the script just feeds the song title into Flickr, so we still have something to search with. In this case, “fire engine”.

However, Flickr returned this picture.

Not a fire engine, but still an awesome picture! So even when the script fails, it still succeeds!

Led Zeppelin – Stairway To Heaven

The script chose “gold” from “There’s a lady who’s sure all that glitters is gold”:

…and I got this. It seems appropriate somehow, just from that one word.

The current state of the script still needs a bit more error checking when searching Flickr (I’ll fix it when I have the time!), but it’s still mostly usable as it is.