The Georgia Tech Center for Music Technology, led by Gil Weinberg, has a reputation for doing incredible musical things with robots, with a mix of
creativity and technical expertise in robotics and AI. For example, a cybernetic
second arm for a drummer, a cybernetic third arm for a drummer, and a bunch
of interesting research on ways that robots can dynamically collaborate with
humans in the context of improvisational music. That last thing usually features
Shimon: a four-armed expressive robotic marimba player which can analyze
music in real time and improvise along with human performers.
It’s an impressive thing to watch, but Shimon’s talents were mostly
restricted to riffing on what other human musicians were doing. Now, Shimon
has leveraged deep learning to create structured, coherent, and totally unique
compositions of its very own.
Shimon’s teacher (of sorts) is Georgia Tech Ph.D. student, Mason Bretan.
The melody and harmonic structure that you would hear is the output of a
four-measure-long seed melody running through a neural network that’s been
trained on nearly 5,000 complete songs (including music by Beethoven, The
Beatles, Lady Gaga, Miles Davis, and John Coltrane), along with two million
motifs, riffs, licks, and other foundational musical elements.
It’s important to understand that Shimon isn’t just mushing together
different bits of music that it’s been programmed with or that it’s using some
kind of random-music generator. The special thing about what Shimon is doing is
that its deep neural network has — in effect — listened to those thousands of
songs, and its compositions represent everything it’s learned from analyzing
them. It’s able to generate harmonies and chords, and it focuses (like humans
do) on the overall structure of the composition rather than simply what note
should come next in an existing sequence.
Bretan calls this “higher-level musical semantics.” Shimon’s music isn’t
something that we can necessarily identify with at this point because we’re
hearing the creative output of a deep-learning system. Weinberg calls Shimon’s
music “beautiful, inspiring, and strange.” It’s something with coherence and
structure, but it’s also completely unique.
YOU DESIGN IT
WE MACHINE IT
Photo: Georgia Tech
SERVO 08.2017 19