The new app uses AI to enable anyone to create music mashups

Imagine if Cop’s “Every Little Thing She Do Is Magic” opens with Michael Jackson’s “Beat It,” then Cardi B’s “Bodak Yellow” riff meets Jackson. Mixing isn’t an impossible fantasy, but a reality with Mixboard, a tablet app that lets users with no music or editing experience create the songs of their dreams.

This app comes out of the Music Technology Center of the Georgia Institute of Technology, whose director, Professor Gil Weinberg, and his students have been developing mashup tools for years. Now, technology is finally catching up to their original vision. With the help of artificial intelligence (AI), artists source apps that users might not pair naturally and find the best parts of a song for mashups.

“I think everyone can be creative with music,” said Weinberg. “Maybe not everyone knows all the theory needed to create a piece of music at the pitch level, but almost everyone loves music. If we give them a really easy way to put songs in different places, they can create something unique.”

The researchers presented their work in the paper “Mixboard — A Co-Creative Mashup Application for Novices” at the International Conference of New Interfaces for Musical Expression (NIME 2023) in May.

Create Mashups

To create a mashup, users drag and drop up to four songs from either Spotify or the Mixboard’s music library to segment vocals, bass, chords, and drums up to 32 bars. The experience is visual and allows users to follow along as the app combines their selection of songs, rendering them in just seconds. Once done, users can download and share their unique tracks.

“I work on almost every part of the project, from the web interface and backend to creating a new iOS app,” says Raghavasimhan Sankaranarayanan, Ph.D. music technology student. “I’m passionate about making strong, production-ready stuff.”

AI-DJ

AI allows users to focus on song development rather than worrying about technical aspects and music theory. The app uses an open source library of music information capture to determine a song’s tempo, key, and stem, which helps break songs down into individual mixes. Algorithms can stretch the sound to match the tempo or transpose keys to ensure both songs are in the same key. The researchers also provide basic song structure templates to help users create attractive tracks.

“The more we focus on developing AI for creative purposes, the more it can inspire and be seen as unique input that humans cannot provide,” said Weinberg. “Together, with your own creativity and AI creative input, mashups can be something unique and new that you wouldn’t make alone.”

Mixing in User Feedback

To make sure Mixboard worked, the researchers conducted a user survey, led by Human-Computer Interaction master’s student Tommy Ottolin. Their study evaluated web interfaces with 45 subjects between the ages of 18 and 27, not including anyone with more than one year of music mixing or composition experience. Participants were given 30 minutes to test the Mixboard. Next, they completed semi-structured interviews with the researchers followed by a 20-question survey using a five-point Likert scale that asked about things like how much creative control users felt or how easy the app was to use.

“Watching people interact with our Mixboard evokes a variety of emotions in me, including pride, nervousness, and excitement,” says Qinying Lei, a graduate master of music technology. “I love seeing users captivated by Mixboard and being surprised by the unique music they produce.”

The Future of Mixboards

While researchers continue to refine app automation and enhance iOS functionality, they also see much greater potential than just lab experiments. They hoped to commercialize the app but be less constrained by technology and more by licensing restrictions. Weinberg has consulted with Kobi Abayomi, former senior vice president for Data Science at Warner Music Group, now head of Science for Gumbel Demand Acceleration and a member of the advisory board of Ivan Allen College of Liberal Arts.

“Music rights holders should look at apps like Mixboard as a marketing opportunity, not licensing games,” he said. “Getting people interested in music, or specifically your kind of music, is at the heart of forward-thinking music marketing.”

Ultimately, though, Mixboard is about democratizing access to music production and showing that anyone can be creative.

“To hear the resulting mashups get better and better with every modification to our algorithm is just amazing,” said Nitin Hugar, a graduate of the music technology masters. “It’s also fun to try to put together the most unlikely of songs: an Indian song with an African beat and some jazz chords is definitely not what I thought would sound good put together, but I’ve been proven wrong time and time again. This project made me appreciate how music from different regions of the world are interconnected.”

#####

Georgia Institute of Technology, or Georgia Tech, is one of the leading public research universities in the US, developing leaders who advance technology and improve the human condition. The Institute offers degrees in business, computing, design, engineering, liberal arts, and science. More than 45,000 undergraduate and graduate students, representing 50 states and more than 148 countries, study on the main campus in Atlanta, on campuses in France and China, and through distance and online learning. As a leading technology university, Georgia Tech is an engine of economic development for Georgia, the Southeast, and the nation, conducting more than $1 billion in research each year for government, industry, and society.