In exploring the wonderful and ever-growing world of machine learning, we built a music composer that gives us a glimpse of the future, and it looks exciting.
The idea for the ML Lo-Fi Music Composer came about during the Prototyp Weekend (the weekend before the Prototyp Week), when an audio-to-MIDI converter was used on one of the creator's own pieces of music. Although the output was messy, the main theme and melody of the original music were still intact, which led to us investigating the possibilities of generating music with deep learning. The original idea was to take a simple form of music that could easily be converted from audio into MIDI tracks, train ML models on the MIDI information, and then have it play something similar in an endless loop.
The first challenge we encountered in developing the ML Lo-Fi Music Composer was a common one – data collection and normalization. We quickly realized that the concept of audio-to-MIDI conversion was too complex and that lo-fi jazz and hip-hop samples would be easier to collect and normalize. Once this was established, we used Google's Magenta project to normalize MIDI or MusicXML datasets to NoteSequences, which we then used as input for the machine-learning models.
One of the biggest challenges in the development of ML Lo-Fi Music Composer was getting the machine learning models to generate music that was both pleasant and unique. While a decent piece of music would often be generated, it would end up sounding similar to the training data, suggesting overfitting. Time constraints, unfortunately, meant that the models had to be left as they were, and the focus shifted to putting together something that could produce good overall results. We decided to have multiple models generate music patterns, with each model responsible for its own instrument. However, the training data had not been prepared sufficiently for this, so the focus was put on the piano while the bass and drums were played in a predefined loop at a set tempo.
Although the final result did not accurately represent the original idea, we were still happy with what we had achieved. The ML Lo-Fi Music Composer was able to play decent music, albeit slowly. It was a fun way to explore different scenarios in which machine learning can be used to compose music. Although we may use a slightly different approach next time, the project served as a valuable and stimulating learning experience.
With more time and resources, a project like this can be improved. With a larger dataset, more carefully trained models, and more fine-tuning, it could genuinely become something great and useable. If not, it could serve as a foundation for other innovations in the field of machine learning and music composition, at the very least.
We use cookies to give you a better experience when visiting our website. Read more about how we handle cookies