Introduction
AI startup Mirelo, in collaboration with Kyutai Labs, has officially introduced a new Audio-to-MIDI artificial intelligence model. This technology can take a finished audio recording, automatically identify the instruments playing, and output separate MIDI tracks for each component such as vocals, drums, bass, and keyboards. This is seen as a new step forward in supporting music producers and digital content creators.
Detailed Developments
According to the announcement from Yann LeCun and the development team on July 10, 2026, the collaboration between Mirelo and Kyutai Labs aims to solve the highly complex problem of audio separation and digital instrument conversion. Instead of just separating traditional waveform audio sources (stems), this model goes a step further by digitizing them into musical notes and MIDI control data. Users can now easily edit notes, change virtual instruments, or restructure an entire song from a single source audio file.
Technical & Technology Analysis
The major difference of this Audio-to-MIDI model compared to existing solutions lies in its capacity for simultaneous polyphonic multi-instrument transcription. The system is trained to detect the acoustic characteristics of individual instruments like vocals, drums, bass, and keys within a complex audio mix. The algorithm then maps these frequencies and pitches into precise MIDI events, preserving the expressive nuances of the original recording.
Expert Opinions & Assessments
Industry experts highly appreciate this collaboration as Kyutai Labs is well-known for its advanced open-source AI research in audio. Integrating direct audio-to-MIDI transcription technology will significantly shorten the workflow of audio engineers. However, some producers remain skeptical about the ability to handle overly dense mixes or complex instrumental solos, which can easily lead to pitch detection errors by the AI.
Impact & Future
This technology promises to unlock a new era for music remixing, learning, and preserving heritage music. For the music-making community, this tool will simplify chord analysis and help cover traditional songs using modern instruments. In the future, the combination of Audio-to-MIDI models and generative music models could reshape the entire global music production workflow.