- Notifications
You must be signed in to change notification settings - Fork25
Closed
Description
lazy_loader loaded modules are not thread-safe: when concurrently accessed from several threads all but the first thread get partially loaded module missing most of functionality.
For examplelibrosa uses lazy_loader to loadresampy and gets the following error:
AttributeError: module 'resampy' has no attribute 'resample'To Reproduce
Run the following snippet:
import librosaimport threadingimport numpy as npdef resample(): librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")for _ in range(5): threading.Thread(target=resample).start()Expected behavior
No errors reported.
Actual output
Traceback (most recent call last):Traceback (most recent call last): File "C:\Build\envs\dec5\lib\threading.py", line 870, in runTraceback (most recent call last): File "C:\Build\envs\dec5\lib\threading.py", line 870, in run File "C:\Build\envs\dec5\lib\threading.py", line 870, in runTraceback (most recent call last): File "C:\Build\envs\dec5\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resampleself._target(*self._args, **self._kwargs)self._target(*self._args, **self._kwargs) File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample self._target(*self._args, **self._kwargs) File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)AttributeError: module 'resampy' has no attribute 'resample' y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis) y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)AttributeError: module 'resampy' has no attribute 'resample'AttributeError: module 'resampy' has no attribute 'resample' y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)AttributeError: module 'resampy' has no attribute 'resample'python-BaseExceptionpython-BaseExceptionpython-BaseExceptionpython-BaseExceptionException in thread Exception in thread Thread-8:Exception in thread Traceback (most recent call last):Thread-5:Traceback (most recent call last): File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_inner File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_innerThread-9:Traceback (most recent call last): File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_inner self.run() File "C:\Build\envs\dec5\lib\threading.py", line 870, in run self.run() File "C:\Build\envs\dec5\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs)self.run() File "C:\Build\envs\dec5\lib\threading.py", line 870, in run File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample self._target(*self._args, **self._kwargs) File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample self._target(*self._args, **self._kwargs) File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis) y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)AttributeError: AttributeError: module 'resampy' has no attribute 'resample'module 'resampy' has no attribute 'resample' librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resamplepython-BaseException y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)AttributeError: module 'resampy' has no attribute 'resample'Exception in thread Thread-6:Traceback (most recent call last): File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_inner self.run() File "C:\Build\envs\dec5\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best") File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)AttributeError: module 'resampy' has no attribute 'resample'Software versions
Windows-10-10.0.22621-SP0Python 3.8.0 (default, Nov 6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)]NumPy 1.23.5SciPy 1.10.1librosa 0.10.1INSTALLED VERSIONS------------------python: 3.8.0 (default, Nov 6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)]librosa: 0.10.1audioread: 3.0.1numpy: 1.23.5scipy: 1.10.1sklearn: 1.3.2joblib: 1.3.2decorator: 5.1.1numba: 0.58.1soundfile: 0.12.1pooch: v1.8.0soxr: 0.3.7typing_extensions: installed, no version number availablelazy_loader: installed, no version number availablemsgpack: 1.0.7numpydoc: Nonesphinx: Nonesphinx_rtd_theme: Nonematplotlib: Nonesphinx_multiversion: Nonesphinx_gallery: Nonemir_eval: Noneipython: Nonesphinxcontrib.rsvgconverter: Nonepytest: Nonepytest_mpl: Nonepytest_cov: Nonesamplerate: Noneresampy: 0.4.2presets: Nonepackaging: 23.2Metadata
Metadata
Assignees
Labels
No labels