Official Hugging Face Diffusers Implementation of QA-MDT

QAMDT: Quality-Aware Diffusion for Text-to-Music 🎶

QADMT brings a new approach to text-to-music generation by using quality-aware training to tackle issues like low-fidelity audio and weak labeling in datasets.

With a masked diffusion transformer (MDT), QADMT delivers SOTA results on MusicCaps and Song-Describer, enhancing both quality and musicality.

It follows fromthis paper by the University of Science and Technology of China, authored by@changliet al..

Usage:

!git lfs install!gitclone https://huggingface.co/jadechoghari/openmusic qa_mdt

This command will change the folder name fromopenmusic toqa_mdt

pip install -r qa_mdt/requirements.txtpip install xformers==0.0.26.post1pip install torchlibrosa==0.0.9 librosa==0.9.2pip install -q pytorch_lightning==2.1.3 torchlibrosa==0.0.9 librosa==0.9.2 ftfy==6.1.1 braceexpandpip install torch==2.3.0+cu121 torchvision==0.18.0+cu121 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu121

from qa_mdt.pipelineimport MOSDiffusionPipelinepipe = MOSDiffusionPipeline()pipe("A modern synthesizer creating futuristic soundscapes.")

Enjoy the music!! 🎶