Repository files navigation

Demo Video

The basic pipeline:

• Starts a stream_reader that pulls live audio data from any source using PyAudio (soundcard, microphone, ...)
• Reads data from this stream many times per second (eg 1000 updates per second) and stores that data in a fifo buffer
• When triggered by.get_audio_features(), the stream_analyzer, applies a Fast-Fourier-Transform to the most recent audio window in the buffer
• Whenvisualize is enabled, the visualizer displays these FFT features in realtime using a PyGame GUI (I made two display modes: 2D and 3D)

Requirements:

pip install -r requirements.txt

You also might have to:

sudo apt install libasound-dev portaudio19-dev libportaudio2 libportaudiocpp0 (on Ubuntu)or

brew install portaudio (on Mac)

• Other platforms like Mac/Windows should work if PyGame can find your display and Python finds your audio card (these can be tricky withWSL)

Alternatively to pyaudio, you can trysounddevice which sometimes is more compatible with Windows/Mac

• just runpip install sounddevice
• The code to switch between the two sound interfaces is in the__init__ function of the Stream_Analyzer class

Usage:

just runpython run_FFT_analyzer.py and play a sound on your machine!

• I have personally learnedA LOT about sound by watchingthis realtime visualization while listening to music
• You can run the stream_analyzer in headless mode and use the FFT features in any Python Application that requires live musical features

ToDo:

• Implement realtime beat detection / melody extraction on top of FFT features (eg using Harmonic/Percussive decomposition)
• The pygame.transform operations sometimes cause weird visual artifacts (boxes) for some resolution settings --> fix??
• Remove the matplotlib dependency since it's only needed for the colormap of the vis..
• Slow bars decay speed currently depends on how often.get_audio_features() is called --> fix