The first AI-based opera

The ACIDS team worked alongside composer Daniele Ghisi, that brilliantly imagined the music for the opera La machine des monstres.

The goal of this opera (led by French director Jean-François Peyret) was to reinvent a Frankenstein of modern times by presenting on stage a machine that makes music by itself. Influenced by the writings of Alan Turing, a British mathematician considered the father of artificial intelligence, Jean-François Peyret set up with the composer Daniele Ghisi and ACIDS team researcher, Philippe Esling, a machine made up of recurrent neural network trained on audio waveforms and capable of generating infinite sets of similarly sounding audio domains. The final result is quite astounding, with the following audio file (in its raw waveform format) being entirely generated by the algorithm point by point (voice and violin included)

One of the peculiarities in the work of Daniele Ghisi, was also to try to set the musical aspect on the training of the model (akin to the story of Frankenstein), rather than purely on the final output. Hence, you can hear how the machine slowly learns by trials and finally converge, leading to a musical depiction of gradient descent (here each sample corresponds to another day of learning)

You can hear more of the resulting audio from this project here