![[International Conference on Machine Learning Logo]](/image.pl?url=https%3a%2f%2fproceedings.mlr.press%2fv202%2fassets%2fimages%2flogo-pmlr.svg&f=jpg&w=240)
[edit]
Polarity Is All You Need to Learn and Transfer Faster
Qingyang Wang, Michael Alan Powell, Eric W Bridgeford, Ali Geisa, Joshua T VogelsteinProceedings of the 40th International Conference on Machine Learning, PMLR 202:36264-36284, 2023.
Abstract
Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities areadequately seta priori, then networks learn with less time and data. We also explicitly illustrate situations in whicha priori setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning.
Cite this Paper
BibTeX
@InProceedings{pmlr-v202-wang23ae, title = {Polarity Is All You Need to Learn and Transfer Faster}, author = {Wang, Qingyang and Powell, Michael Alan and Bridgeford, Eric W and Geisa, Ali and Vogelstein, Joshua T}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {36264--36284}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/wang23ae/wang23ae.pdf}, url = {https://proceedings.mlr.press/v202/wang23ae.html}, abstract = {Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities areadequately seta priori, then networks learn with less time and data. We also explicitly illustrate situations in whicha priori setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning.}} Endnote
%0 Conference Paper%T Polarity Is All You Need to Learn and Transfer Faster%A Qingyang Wang%A Michael Alan Powell%A Eric W Bridgeford%A Ali Geisa%A Joshua T Vogelstein%B Proceedings of the 40th International Conference on Machine Learning%C Proceedings of Machine Learning Research%D 2023%E Andreas Krause%E Emma Brunskill%E Kyunghyun Cho%E Barbara Engelhardt%E Sivan Sabato%E Jonathan Scarlett%F pmlr-v202-wang23ae%I PMLR%P 36264--36284%U https://proceedings.mlr.press/v202/wang23ae.html%V 202%X Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities areadequately seta priori, then networks learn with less time and data. We also explicitly illustrate situations in whicha priori setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning. APA
Wang, Q., Powell, M.A., Bridgeford, E.W., Geisa, A. & Vogelstein, J.T.. (2023). Polarity Is All You Need to Learn and Transfer Faster.Proceedings of the 40th International Conference on Machine Learning, inProceedings of Machine Learning Research 202:36264-36284 Available from https://proceedings.mlr.press/v202/wang23ae.html.