Repository files navigation

This repository is the demonstration of the group equivariant Ambisonic signal processing DNNs [1], implemented by the authors.

This repository (except submodules) is released under the specific license.ReadLicense file in this repository before you download and use this software.

The submoduleseld-dcase2019 is under its own license.

The scriptfCGModule.py is fromzlin7/CGNet, which is originally released under the MIT License.

.├── LICENSE├── README.md├── adversarial_attack.py├── article_figure│   └── taslp├── boot_tensorboard.sh├── checkpoints├── dcase19_dataset.py├── docker│   ├── Dockerfile│   └── build.sh├── evaluation.py├── fCGModule.py├── feature_extraction.py├── login_torch_sh.sh├── main.py├── math_util.py├── models.py├── modules.py├── parameter.py├── render_taslp_fig3.py├── render_taslp_fig4.py├── result├── ret_adv├── ret_eval├── run_adversarial_attack.sh├── run_experiment.sh└── seld-dcase2019

We assume the environment thatdocker/Dockerfile appropriately works.

1. Clone this repository.

git clone --recursive https://github.com/nttrd-mdlab/group-equiv-seldcd group-equiv-seld

2. Build the Docker environment.

$cd docker$ ./build.sh> ...> Successfully built 31cc484c9976> Successfully tagged cgdcase:0.2$cd ../

3. Download the dataset files from the link onthis website. You needfoa_dev.z**,metadata_dev.zip,foa_eval.zip,metadata_eval.zip. Then, generate the normalized dataset usingfeature_extraction.py (do not forget to rewrite the path to the downloaded files infeature_extraction.py).

./login_torch_sh.shpython3  feature_extraction.pyexit

4. Start model training.

./run_experiment.sh 0  # Specify the GPU number (0-origin) by argument

Trained model is saved to./checkpoints, and the log is saved to./result.

5. Change experiment conditions by rewritingparameter.py and re-run./run_experiment.sh:

   • Togglemodel=['Conventional', 'Proposed'][1] to[0] to test baseline model.
   • Togglescale_equivariance=True toFalse to disable scale equivariance of proposed method.
   • Switchtrain_rotation_bias=['virtual_rot', 'azi_random', None][0] to[1] to enable rotational data augmentation.
   • Rewritefeature_phase_different_bin=0 toNone to disable time translation invariance of proposed method.

6. Check and compare performance.

Evaluate the trained model

$ ./login_torch_sh.sh 0$ python3 evaluation.py --resume ./checkpoints/(name of checkpoint file).checkpoint$exit

Compare the progress of (being) trained models

$ ./boot_tensorboard.sh

Then, viewhttp://localhost:6006 with your browser.

7. Render the figures on the paper:

$ ./login_torch_sh.sh 0$ python3 render_taslp_fig3.py$ python3 render_taslp_fig4.py$exit

8. Run experiment for adversarial attack.

$ ./run_adversarial_attack.sh 0 ./checkpoints/(name of checkpoint file).checkpoint (output file name)

• [1] R. Sato, K. Niwa, K. Kobayashi, "Ambisonic Signal Processing DNNs Guaranteeing Rotation, Scale and Time Translation Equivariance," IEEE/ACM Trans. ASLP, (to be published), 2021.