From the lab

Background and problem

We were eager to try out this popular open source text-to-image AI algorithm called Stable Diffusion. We decided to explore how we could make this very technical thing into something that is enjoyable for the average person.

That’s when we came up with the idea of rethinking the concept of a music quiz and making it more image based. Using lyrics, artist name, chorus and song title as prompts for Stable Diffusion to generate images for a quiz where the player(s) guess the song.

Idea and solution

An AI-image-based music quiz based on Stable Diffusion. The base premise is that you select the number of songs you want in the quiz, start up the game, and watch as the song title, group name, or chorus lyrics are transformed into images on the screen (with the help of the AI algorithm Stable Diffusion).

Implementation and results

We decided to use a pretty new machine learning model for creating images from natural language prompts. So basically we take the artist’s name or the title of the song or the lyrics from the chorus and put it into this Stable Diffusion algorithm. When you put in text, you get an image back. So then we could get these images and play them as a music quiz.

For example, the song ‘Eye of the tiger’ would probably be visualized with various images of close-ups of tigers’ eyes. You then get points based on how many images you need to view before guessing the right song. So you get one image and if you can guess the song with that, you get ten points. If you need to see another image, and you can guess it right that time, you get eight points and so on.

We’re getting the playlists from Spotify, using the Spotify API. And then using a Genious API to look up the lyrics. We’re splitting those up and then using Stable Diffusion to generate the images. And then we’ve also created the frontend for the actual game.

Insights and reflections

One part of the challenge was of course to create an actual game, that is to make the music quiz playable. The main challenge was the amount of work we needed to do, even though we chose to work with pretty simple tech that we knew from before.

We were really interested in this challenge of using Stable Diffusion for something that is nice and enjoyable for anyone. The main thing that surprised us was how good the final results actually were.

If continuing to work on the project, the real challenge would be to make the game a more playable and enjoyable experience for the player without too many delays and waiting time. It would also be interesting to explore how other versions of the tech could be used; like creating quizzes for films, brand makes, recipe pictures or geography.