Instantly share code, notes, and snippets.
Save akaanirban/621e63237e63bb169126b537d7a1d979 to your computer and use it in GitHub Desktop.
Edit04/11/2021: This gist is quite old now. The current version of PyTorch is 1.8.1, which is miles ahead of version 1.0.1 that I was trying to install when I wrote this gist. Therefore some of the instructions may not apply, or some dependencies may have changed or bugs taken care of. I do not currently have a Raspberry Pi to verify unfortunately. Please proceed with caution. Further, there are may others who have shared their fixes, and direct links to their wheels down in the comments. Cheers !
sudo apt install libopenblas-dev libblas-dev m4 cmake cython python3-yaml libatlas-base-dev- Stop the swap :
sudo dphys-swapfile swapoff - Modify the size of the swap by editing as
rootthe following file :/etc/dphys-swapfile. Modify the valiableCONF_SWAPSIZEand change its value toCONF_SWAPSIZE=2048 - Run following from command prompt:
dphys-swapfile setupto update the changes. - Start the swap back again:
sudo dphys-swapfile swapon - Check if the changes have taken place:
free -m
For raspberry pi 4 there may be an issue with the Gcc and G++ version. Install an older version ans use it
apt-get install gcc-4.9 g++-4.9 export CC=gcc-4.9 export CXX=g++-4.9 export USE_CUDA=0 export USE_MKLDNN=0 export USE_NNPACK=0 export USE_QNNPACK=0 export USE_NUMPY=1 export USE_DISTRIBUTED=0 export NO_CUDA=1 export NO_DISTRIBUTED=1 export NO_MKLDNN=1 export NO_NNPACK=1 export NO_QNNPACK=1 export ONNX_ML=1 ## this is extremely important otherwise you will run into onnx_pb.h error. # other workaround is edit the onnx_pb.h file and add #define ONNX_ML 1 inside it to skipgit clone --recursive https://github.com/pytorch/pytorchcd pytorch- Optionally if you want to install a specific branch e.g. I want to install v1.0.1 because the torchjit is not broken in that branch:
git checkout v1.0.1sudo -E python3 setup.py buildsudo -E python3 setup.py installLife Hack : Always build a wheel and keep it somewhere safe because you have already build from source.
sudo -E python3 setup.py bdist_wheelThe.whl file will be inpytorch/dist folder.
There can be multiple errors. The most irritating one is `fatal error: onnx/onnx.pb.h: No such file or directory
compilation terminated.This is a [bug](https://github.com/onnx/onnx/issues/1947) which is recent. That is why it is imperative to set the flagONNX_ML=, this bypasses theonnx_pb.hfile search. The other hard coded way would be to manually edit the fileonnx_pb.hand set# DEFINE ONNX_ML 1` at the beginning. This is a hack, other features might break because of this.
If you face some procol buffer problem, do thisgit submodule update --remote third_party/protobuf (pytorch/pytorch#22564 (comment))
cd python3>>> import torch>>> import numpy as np>>> a = torch.from_numpy(np.random.randn(1, 100))>>> print(a)torchsmoke commentedMay 3, 2020
Please link my uploaded wheel from version 1.6!wheel
StuartIanNaylor commentedDec 27, 2020
v1.7.1
[ 5%] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wrappers.pb.cc.o[ 5%] Linking CXX static library ../../../lib/libprotobuf.a[ 5%] Built target libprotobufmake: *** [Makefile:141: all] Error 2Traceback (most recent call last): File "setup.py", line 760, in <module> build_deps() File "setup.py", line 315, in build_deps cmake=cmake) File "/home/pi/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2 cmake.build(my_env) File "/home/pi/pytorch/tools/setup_helpers/cmake.py", line 345, in build self.run(build_args, my_env) File "/home/pi/pytorch/tools/setup_helpers/cmake.py", line 141, in run check_call(command, cwd=self.build_dir, env=env) File "/usr/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd)subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 2.guiambros commentedJan 4, 2021
Thank you@akaanirban! I was able to compile v1.7.1 just fine. I didn't need to downgrade gcc; used the default 8.3.0. The protobuf error was fixed with the submodule update, as indicated above.
Just for testing, I also tried nightly 1.8, but it failed at 89%,kernel_function_legacy_test; log below. But 1.7 is fine for my use case, so I'll stick with it for now.
[ 89%] Built target kernel_function_legacy_testmake: *** [Makefile:141: all] Error 2Traceback (most recent call last): File "setup.py", line 717, in <module> build_deps() File "setup.py", line 313, in build_deps cmake=cmake) File "/home/user/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2 cmake.build(my_env) File "/home/user/pytorch/tools/setup_helpers/cmake.py", line 345, in build self.run(build_args, my_env) File "/home/user/pytorch/tools/setup_helpers/cmake.py", line 141, in run check_call(command, cwd=self.build_dir, env=env) File "/usr/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd)subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 2.$Hi@guiambros I am glad that v1.7.1 works fine. When I did this, I was using pytorch 1.0 or something. I would imagine things are a bit different now. Sadly I am not working with raspberry pis at present, so do not know about the current bugs. If you have a different method which you used to compile v1.7.1 and have it somewhere (like a gist or something) , please let me know, I will add a pointer to your gist so that it helps other people!
kevinchan04 commentedFeb 2, 2021
Thank you@akaanirban! I was able to compile v1.7.1 just fine. I didn't need to downgrade gcc; used the default 8.3.0. The protobuf error was fixed with the submodule update, as indicated above.
Just for testing, I also tried nightly 1.8, but it failed at 89%,
kernel_function_legacy_test; log below. But 1.7 is fine for my use case, so I'll stick with it for now.[ 89%] Built target kernel_function_legacy_testmake: *** [Makefile:141: all] Error 2Traceback (most recent call last): File "setup.py", line 717, in <module> build_deps() File "setup.py", line 313, in build_deps cmake=cmake) File "/home/user/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2 cmake.build(my_env) File "/home/user/pytorch/tools/setup_helpers/cmake.py", line 345, in build self.run(build_args, my_env) File "/home/user/pytorch/tools/setup_helpers/cmake.py", line 141, in run check_call(command, cwd=self.build_dir, env=env) File "/usr/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd)subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 2.$
Could you provide your compiled whl file? I met problem when I tried to compile v1.7.1.
Building wheel torch-1.7.0a0
-- Building version 1.7.0a0
cmake --build . --target install --config Release -- -j 4
make: *** No rule to make target 'install'. Stop.
Traceback (most recent call last):
File "setup.py", line 760, in
build_deps()
File "setup.py", line 315, in build_deps
cmake=cmake)
File "/home/ubuntu/pytorch_install/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
cmake.build(my_env)
File "/home/ubuntu/pytorch_install/pytorch/tools/setup_helpers/cmake.py", line 345, in build
self.run(build_args, my_env)
File "/home/ubuntu/pytorch_install/pytorch/tools/setup_helpers/cmake.py", line 141, in run
check_call(command, cwd=self.build_dir, env=env)
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 2.
MR901 commentedFeb 17, 2021
If someone wants the compiled whl file version 1.9 for raspberrypi4.link
pifparfait commentedApr 11, 2021
Hi all, I got this issue from trying to install torch to my raspberry Pi 4, following these steps. Many thanks for any help.
@akaanirban
[ 72%] Building CXX object caffe2/CMakeFiles/conv_to_nnpack_transform_test.dir/transforms/conv_to_nnpack_transform_test.cc.o
[ 73%] Building CXX object caffe2/CMakeFiles/depthwise3x3_conv_op_test.dir/share/contrib/depthwise/depthwise3x3_conv_op_test.cc.o
[ 73%] Building CXX object caffe2/CMakeFiles/pattern_net_transform_test.dir/transforms/pattern_net_transform_test.cc.o
[ 73%] Building CXX object caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state.cc.o
In file included from /tmp/pytorch/caffe2/onnx/helper.h:4:0,
from /tmp/pytorch/caffe2/onnx/backend.h:5,
from /tmp/pytorch/caffe2/python/pybind_state.cc:19:
/tmp/pytorch/third_party/onnx/onnx/onnx_pb.h:52:26: fatal error: onnx/onnx.pb.h: No such file or directory
#include "onnx/onnx.pb.h"
^
compilation terminated.
make[2]: *** [caffe2/CMakeFiles/caffe2_pybind11_state.dir/build.make:63: caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2802: caffe2/CMakeFiles/caffe2_pybind11_state.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 73%] Linking CXX executable ../bin/conv_to_nnpack_transform_test
[ 73%] Built target conv_to_nnpack_transform_test
[ 73%] Linking CXX executable ../bin/depthwise3x3_conv_op_test
[ 73%] Built target depthwise3x3_conv_op_test
[ 73%] Linking CXX executable ../bin/pattern_net_transform_test
[ 73%] Built target pattern_net_transform_test
make: *** [Makefile:141: all] Error 2
Failed to run 'bash ../tools/build_pytorch_libs.sh caffe2'
@pifparfait Sorry, this gist is probably very outdated, since the current pytorch is many many versions ahead. There are many other comments in this page of other developers who have kindly shared the links to their compiled versions which you can probably directly used. Can you check those? Perhaps the one from here:https://gist.github.com/akaanirban/621e63237e63bb169126b537d7a1d979#gistcomment-3634301