10 AI-Powered Music and Audio Product Case Studies (and 20 feature suggestions)
Artificial Intelligence is supercharging music and audio creation and production. Let's take a look at 10 innovative products that use AI to unlock what would otherwise not be possible for musicians, producers, and content creators. Plus I've proposed two new features for each one!
Think of this list as inspiration for your Product Managers, Designers, and Engineers; competitors to be aware of, or perhaps M&A targets!
1. AIVA (Artificial Intelligence Virtual Artist)
AIVA is an AI composer that creates original music for various purposes, from film scores to video game soundtracks. It analyzes large amounts of classical music to generate new compositions in seconds, a task that would take human composers significantly longer.
Core AI capability: Rapid generation of complex musical compositions
Possible next steps:
Composer Copilot: Real-time collaborative composition with human musicians (thinking of old Casio keyboards)
GENres: Develop genre-specific AI models for more diverse musical styles with better authenticity for each
2. iZotope RX
This audio repair and enhancement software uses AI to clean up and restore audio recordings. Its intelligent algorithms can remove background noise, repair clipping, and separate vocals from instrumentals with unprecedented accuracy. (Disclosure: I served at iZotope Director of Product Marketing from 2020 to 2022)
Core AI capability: Advanced audio restoration and separation
Possible next steps:
Pre-Artifact: Analyze an audio file, predict potential artifacts or issues that might occur during processing (compression, EQ, time-stretching, etc.) and suggest preventive measures or optimal processing parameters
Modernizer: Create an AI model trained on historical recording techniques and equipment. Reconstruct missing frequency content or dynamics based on the era's recording technology to upgrade older recordings.
3. Landr
You didn’t think there could be a list like this without Landr, did you? Landr uses AI to automate mastering. It analyzes the audio and applies processing to optimize sound quality.
Core AI capability: Intelligent audio mastering without human intervention
Possible next steps:
Mix Like: AI-driven mixing suggestions that reference a designated recording or catalog
Bob Ludwig Mode: Develop personalized and signature mastering profiles based on the world’s best mastering engineers and individual user preferences
4. Audioshake
Audioshake is an AI-powered platform that allows music creators and rights holders to create stems from mixed audio tracks. It can separate full songs into individual instrument and vocal tracks with remarkable accuracy, even when the original multitrack files are unavailable.
Core AI capability: High-quality stem separation from mixed audio using advanced machine learning algorithms
Possible next steps:
RemixBot: Automatic stem mixing and mastering, allowing creators to quickly produce alternative versions or remixes
Style Transformer: apply the sonic characteristics of one instrument or vocal to another. For example, make a piano stem sound like it was played on a vintage synthesizer, or transform a modern vocal performance to sound like it was recorded in a specific era or style.
5. Boomy
Boomy allows users to create and distribute original songs using AI. It generates complete tracks based on simple user inputs and helps with music distribution.
Core AI capability: End-to-end music creation and distribution for non-musicians
Possible next steps:
Lyric Assistant: AI-driven lyric generation to complement the music
Gen Vid: Generate music videos to match songs
6. LALAL.AI
This tool uses AI to separate vocals and instrumentals from any audio file. It can isolate specific instruments, making it valuable for remixing and sampling.
Core AI capability: High-quality stem separation from mixed audio
Possible next steps:
ENHANCE: Extend short samples or loops to full-length track suggestions, fill in missing parts of partially isolated stems, generate complementary stems in the style of the original track (give me a new bass line, for example)
Section Selector: Isolate specific instruments or vocals within defined time segments of a track. This could include removing a particular instrument from just the chorus, for example.
7. Soundraw
Soundraw is an AI music generator that creates original, royalty-free music for content creators. Users can customize various aspects of the generated tracks to fit their needs.
Core AI capability: On-demand, customizable music generation
Possible next steps:
Auto Bed: AI-driven background music creation based on the content of video or audio in which it is intended to be used. For example, analyze a podcast and intelligently determine where background music should go, generate it, and return a rough mix.
Adaptive Transitions: identify key moments, mood changes, and pacing. Then generate music that adapts to these transition points, adjusting tempo, instrumentation, and emotional tone as needed.
8. Descript
Descript is an all-in-one audio and video editing platform that leverages AI to streamline the content creation process. Its original core feature is the ability to edit audio and video by editing text, making it incredibly user-friendly for podcasters, video creators, and journalists.
Core AI capabilities: transcription, text-based audio/video editing, voice cloning for overdubbing, filler word removal
Possible next steps:
Interview Curator: Improve and structure interview content, identify and organize topics, suggest optimal structure with chapter markers and segment breaks, and auto trim to improve pacing and coherence while maintaining the essence of the conversation
B-Roll: generate and insert B-roll in video content based on the transcript and context
9. MuseNet
A bit of a throwback, but it’s fun to see the state of the art in 2019. Developed by OpenAI, (the ChatGPT company) MuseNet is an AI model that can generate multi-instrumental music in various styles. It demonstrates an understanding of long-term structure and harmony in music.
Core AI capability: Multi-instrumental music generation with style transfer
Possible next steps:
Jam Live: users can play an instrument or sing with with the AI, which responds and adapts to their playing or singing, learning and incorporating the style of the musician
Multimedia Music: Generate music based on input from other modalities, such as images, video, or text. For example, compose music that matches the style and mood of a particular painting or photograph.
10. Suno AI
Suno AI uses artificial intelligence to generate complete songs, including lyrics and vocals, from text prompts. This tool allows users to create original music without any musical training or equipment.
Core AI capability: Text-to-song generation with vocals and instrumentation
Possible next steps:
Synesthesia-Like: Users can "paint" with colors or shapes, and Suno AI translates these visual inputs into music
Memory Melodizer: Users can input a personal memory or significant life event, and Suno AI creates a song that captures the essence of that memory, incorporating emotional content, key details, and personal significance.