Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Commit70c96ca

Browse files
syurkevi9prady9
andauthored
change setup.py to optionally build binaries in source distribution (#245)
* adds initial skbuild configuration load from local lib directory loads local binaries if they are available* remove git from toml dependencies* additional cmake args in setup.py* add description format for pypi* directory corrections for skbuild* load local nvrtc-builtins when possible* make mkl libs static for distribution* Update README to include information about wheels* remove extra import platformCo-authored-by: pradeep <pradeep@arrayfire.com>
1 parent5bead13 commit70c96ca

File tree

6 files changed

+205
-40
lines changed

6 files changed

+205
-40
lines changed

‎CMakeLists.txt

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
cmake_minimum_required(VERSION3.11.0)
2+
project(arrayfire-python)
3+
find_package(PythonExtensionsREQUIRED)
4+
include(FetchContent)
5+
6+
set(CMAKE_MODULE_PATH_OLD${CMAKE_MODULE_PATH})
7+
set(CMAKE_MODULE_PATH"")
8+
set(NO_SONAME)
9+
10+
FetchContent_Declare(
11+
arrayfire
12+
GIT_REPOSITORYhttps://github.com/arrayfire/arrayfire.git
13+
GIT_TAGv3.8
14+
)
15+
FetchContent_MakeAvailable(arrayfire)
16+
set(CMAKE_MODULE_PATH${CMAKE_MODULE_PATH_OLD})
17+
18+
set(ignoreWarning"${SKBUILD}")

‎README.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -25,30 +25,29 @@ def calc_pi_device(samples):
2525

2626
Choosing a particular backend can be done using`af.set_backend(name)` where name is either "_cuda_", "_opencl_", or "_cpu_". The default device is chosen in the same order of preference.
2727

28-
##Requirements
29-
30-
Currently, this project is tested only on Linux and OSX.Youalso need to havethe ArrayFire C/C++ libraryinstalled on your machine. You can get itfrom the following sources.
28+
##Getting started
29+
ArrayFire can be installed from a variety of sources.[Pre-built wheels](https://repo.arrayfire.com/python/wheels/3.8.0/) are available for a number of systems and toolkits. Wheels for some systems are available on PyPI. Unsupported systems will require separate installation of the ArrayFire C/C++ libraries and only the python wrapper will be installed in that case.
30+
Youcan getthe ArrayFire C/C++ library from the following sources:
3131

3232
-[Download and install binaries](https://arrayfire.com/download)
3333
-[Build and install from source](https://github.com/arrayfire/arrayfire)
3434

35-
Please check the following links for dependencies.
36-
37-
-[Linux dependencies](http://www.arrayfire.com/docs/using_on_linux.htm)
38-
-[OSX dependencies](http://www.arrayfire.com/docs/using_on_osx.htm)
39-
40-
##Getting started
41-
42-
**Install the last stable version:**
4335

36+
**Install the last stable version:**
4437
```
4538
pip install arrayfire
4639
```
4740

48-
**Install the development version:**
41+
**Install a pre-built wheel for a specific CUDA toolkit version:**
42+
```
43+
pip install arrayfire==3.8.0+cu112 -f https://repo.arrayfire.com/python/wheels/3.8.0/
44+
# Replace the +cu112 local version with the desired toolkit
45+
```
46+
47+
**Install the development source distribution:**
4948

5049
```
51-
pip install git+git://github.com/arrayfire/arrayfire-python.git@devel
50+
pip install git+git://github.com/arrayfire/arrayfire-python.git@master
5251
```
5352

5453
**Installing offline:**
@@ -57,10 +56,11 @@ pip install git+git://github.com/arrayfire/arrayfire-python.git@devel
5756
cd path/to/arrayfire-python
5857
python setup.py install
5958
```
59+
Rather than installing and building ArrayFire elsewhere in the system, you can also build directly through python by first setting the`AF_BUILD_LOCAL_LIBS=1` environment variable. Additional setup will be required to build ArrayFire, including satisfying dependencies and further CMake configuration. Details on how to pass additional arguments to the build systems can be found in the[scikit-build documentation.](https://scikit-build.readthedocs.io/en/latest/)
6060

6161
**Post Installation:**
6262

63-
Please follow[these instructions](https://github.com/arrayfire/arrayfire-python/wiki) to ensurethe arrayfire-python can find the arrayfire libraries.
63+
If you are not using one of the pre-built wheels, you may need to ensure arrayfire-python can find the installed arrayfire libraries.Please follow[these instructions](https://github.com/arrayfire/arrayfire-python/wiki) to ensurethat arrayfire-python can find the arrayfire libraries.
6464

6565
To run arrayfire tests, you can run the following command from command line.
6666

‎arrayfire/library.py

Lines changed: 74 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
importctypesasct
1616
importtraceback
1717
importos
18+
importsys
1819

1920
c_float_t=ct.c_float
2021
c_double_t=ct.c_double
@@ -492,8 +493,6 @@ class VARIANCE(_Enum):
492493
_VER_MAJOR_PLACEHOLDER="__VER_MAJOR__"
493494

494495
def_setup():
495-
importplatform
496-
497496
platform_name=platform.system()
498497

499498
try:
@@ -527,7 +526,7 @@ def _setup():
527526
ct.windll.kernel32.SetErrorMode(0x0001|0x0002)
528527

529528
ifAF_SEARCH_PATHisNone:
530-
AF_SEARCH_PATH="C:/Program Files/ArrayFire/v"+AF_VER_MAJOR+"/"
529+
AF_SEARCH_PATH="C:/Program Files/ArrayFire/v"+AF_VER_MAJOR+"/"
531530

532531
ifCUDA_PATHisnotNone:
533532
CUDA_FOUND=os.path.isdir(CUDA_PATH+'/bin')andos.path.isdir(CUDA_PATH+'/nvvm/bin/')
@@ -554,7 +553,12 @@ def _setup():
554553
post='.so.'+_VER_MAJOR_PLACEHOLDER
555554

556555
ifAF_SEARCH_PATHisNone:
557-
AF_SEARCH_PATH='/opt/arrayfire-'+AF_VER_MAJOR+'/'
556+
ifos.path.exists('/opt/arrayfire-'+AF_VER_MAJOR+'/'):
557+
AF_SEARCH_PATH='/opt/arrayfire-'+AF_VER_MAJOR+'/'
558+
elifos.path.exists('/opt/arrayfire/'):
559+
AF_SEARCH_PATH='/opt/arrayfire/'
560+
else:
561+
AF_SEARCH_PATH='/usr/local/'
558562

559563
ifCUDA_PATHisNone:
560564
CUDA_PATH='/usr/local/cuda/'
@@ -566,21 +570,46 @@ def _setup():
566570
else:
567571
raiseOSError(platform_name+' not supported')
568572

569-
ifAF_PATHisNone:
570-
os.environ['AF_PATH']=AF_SEARCH_PATH
571-
572-
returnpre,post,AF_SEARCH_PATH,CUDA_FOUND
573+
returnpre,post,AF_PATH,AF_SEARCH_PATH,CUDA_FOUND
573574

574575
class_clibrary(object):
575576

577+
def__find_nvrtc_builtins_libname(self,search_path):
578+
filelist=os.listdir(search_path)
579+
forfinfilelist:
580+
if'nvrtc-builtins'inf:
581+
returnf
582+
returnNone
583+
576584
def__libname(self,name,head='af',ver_major=AF_VER_MAJOR):
577585
post=self.__post.replace(_VER_MAJOR_PLACEHOLDER,ver_major)
578586
libname=self.__pre+head+name+post
579-
ifos.path.isdir(self.AF_PATH+'/lib64'):
580-
libname_full=self.AF_PATH+'/lib64/'+libname
587+
588+
ifself.AF_PATH:
589+
ifos.path.isdir(self.AF_PATH+'/lib64'):
590+
path_search=self.AF_PATH+'/lib64/'
591+
else:
592+
path_search=self.AF_PATH+'/lib/'
593+
else:
594+
ifos.path.isdir(self.AF_SEARCH_PATH+'/lib64'):
595+
path_search=self.AF_SEARCH_PATH+'/lib64/'
596+
else:
597+
path_search=self.AF_SEARCH_PATH+'/lib/'
598+
599+
ifplatform.architecture()[0][:2]=='64':
600+
path_site=sys.prefix+'/lib64/'
601+
else:
602+
path_site=sys.prefix+'/lib/'
603+
604+
path_local=self.AF_PYMODULE_PATH
605+
libpaths= [('',libname),
606+
(path_site,libname),
607+
(path_local,libname)]
608+
ifself.AF_PATH:#prefer specified AF_PATH if exists
609+
libpaths.append((path_search,libname))
581610
else:
582-
libname_full=self.AF_PATH+'/lib/'+libname
583-
return(libname,libname_full)
611+
libpaths.insert(2, (path_search,libname))
612+
returnlibpaths
584613

585614
defset_unsafe(self,name):
586615
lib=self.__clibs[name]
@@ -592,13 +621,19 @@ def __init__(self):
592621

593622
more_info_str="Please look at https://github.com/arrayfire/arrayfire-python/wiki for more information."
594623

595-
pre,post,AF_PATH,CUDA_FOUND=_setup()
624+
pre,post,AF_PATH,AF_SEARCH_PATH,CUDA_FOUND=_setup()
596625

597626
self.__pre=pre
598627
self.__post=post
599628
self.AF_PATH=AF_PATH
629+
self.AF_SEARCH_PATH=AF_SEARCH_PATH
600630
self.CUDA_FOUND=CUDA_FOUND
601631

632+
# prefer locally packaged arrayfire libraries if they exist
633+
af_module=__import__(__name__)
634+
self.AF_PYMODULE_PATH=af_module.__path__[0]+'/'ifaf_module.__path__elseNone
635+
636+
602637
self.__name=None
603638

604639
self.__clibs= {'cuda' :None,
@@ -618,7 +653,7 @@ def __init__(self):
618653
'opencl' :4}
619654

620655
# Try to pre-load forge library if it exists
621-
libnames=self.__libname('forge',head='',ver_major=FORGE_VER_MAJOR)
656+
libnames=reversed(self.__libname('forge',head='',ver_major=FORGE_VER_MAJOR))
622657

623658
try:
624659
VERBOSE_LOADS=os.environ['AF_VERBOSE_LOADS']=='1'
@@ -628,14 +663,15 @@ def __init__(self):
628663

629664
forlibnameinlibnames:
630665
try:
631-
ct.cdll.LoadLibrary(libname)
666+
full_libname=libname[0]+libname[1]
667+
ct.cdll.LoadLibrary(full_libname)
632668
ifVERBOSE_LOADS:
633-
print('Loaded '+libname)
669+
print('Loaded '+full_libname)
634670
break
635671
exceptOSError:
636672
ifVERBOSE_LOADS:
637673
traceback.print_exc()
638-
print('Unable to load '+libname)
674+
print('Unable to load '+full_libname)
639675
pass
640676

641677
c_dim4=c_dim_t*4
@@ -644,24 +680,39 @@ def __init__(self):
644680

645681
# Iterate in reverse order of preference
646682
fornamein ('cpu','opencl','cuda',''):
647-
libnames=self.__libname(name)
683+
libnames=reversed(self.__libname(name))
648684
forlibnameinlibnames:
649685
try:
650-
ct.cdll.LoadLibrary(libname)
686+
full_libname=libname[0]+libname[1]
687+
688+
ct.cdll.LoadLibrary(full_libname)
651689
__name='unified'ifname==''elsename
652-
clib=ct.CDLL(libname)
690+
clib=ct.CDLL(full_libname)
653691
self.__clibs[__name]=clib
654692
err=clib.af_randu(c_pointer(out),4,c_pointer(dims),Dtype.f32.value)
655693
if (err==ERR.NONE.value):
656694
self.__name=__name
657695
clib.af_release_array(out)
658696
ifVERBOSE_LOADS:
659-
print('Loaded '+libname)
697+
print('Loaded '+full_libname)
698+
699+
# load nvrtc-builtins library if using cuda
700+
ifname=='cuda':
701+
nvrtc_name=self.__find_nvrtc_builtins_libname(libname[0])
702+
ifnvrtc_name:
703+
ct.cdll.LoadLibrary(libname[0]+nvrtc_name)
704+
705+
ifVERBOSE_LOADS:
706+
print('Loaded '+libname[0]+nvrtc_name)
707+
else:
708+
ifVERBOSE_LOADS:
709+
print('Could not find local nvrtc-builtins libarary')
710+
660711
break;
661712
exceptOSError:
662713
ifVERBOSE_LOADS:
663714
traceback.print_exc()
664-
print('Unable to load '+libname)
715+
print('Unable to load '+full_libname)
665716
pass
666717

667718
if (self.__nameisNone):
@@ -689,6 +740,7 @@ def parse(self, res):
689740
lst.append(key)
690741
returntuple(lst)
691742

743+
692744
backend=_clibrary()
693745

694746
defset_backend(name,unsafe=False):

‎pyproject.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[build-system]
2+
requires = ["setuptools","wheel","scikit-build","cmake","ninja"]

‎setup.cfg

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
[metadata]
22
name = arrayfire
3-
version = 3.7.20200213
3+
version = 3.8.0
44
description = Python bindings for ArrayFire
55
licence = BSD
66
long_description = file: README.md
7+
long_description_content_type = text/markdown
78
maintainer = ArrayFire
89
maintainer_email = technical@arrayfire.com
910
url = http://arrayfire.com
@@ -15,6 +16,8 @@ classifiers =
1516

1617
[options]
1718
packages = find:
19+
install_requires=
20+
scikit-build
1821

1922
[options.packages.find]
2023
include = arrayfire

‎setup.py

Lines changed: 93 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,98 @@
99
# http://arrayfire.com/licenses/BSD-3-Clause
1010
########################################################
1111

12-
# TODO: Look for af libraries during setup
12+
importos
13+
importre
1314

14-
fromsetuptoolsimportsetup
15+
# package can be distributed with arrayfire binaries or
16+
# just with python wrapper files, the AF_BUILD_LOCAL
17+
# environment var determines whether to build the arrayfire
18+
# binaries locally rather than searching in a system install
19+
20+
AF_BUILD_LOCAL_LIBS=os.environ.get('AF_BUILD_LOCAL_LIBS')
21+
print(f'AF_BUILD_LOCAL_LIBS={AF_BUILD_LOCAL_LIBS}')
22+
ifAF_BUILD_LOCAL_LIBS:
23+
print('Proceeding to build ArrayFire libraries')
24+
else:
25+
print('Skipping binaries installation, only python files will be installed')
26+
27+
AF_BUILD_CPU=os.environ.get('AF_BUILD_CPU')
28+
AF_BUILD_CPU=1ifAF_BUILD_CPUisNoneelseint(AF_BUILD_CPU)
29+
AF_BUILD_CPU_CMAKE_STR='-DAF_BUILD_CPU:BOOL=ON'if (AF_BUILD_CPU==1)else'-DAF_BUILD_CPU:BOOL=OFF'
30+
31+
AF_BUILD_CUDA=os.environ.get('AF_BUILD_CUDA')
32+
AF_BUILD_CUDA=1ifAF_BUILD_CUDAisNoneelseint(AF_BUILD_CUDA)
33+
AF_BUILD_CUDA_CMAKE_STR='-DAF_BUILD_CUDA:BOOL=ON'if (AF_BUILD_CUDA==1)else'-DAF_BUILD_CUDA:BOOL=OFF'
34+
35+
AF_BUILD_OPENCL=os.environ.get('AF_BUILD_OPENCL')
36+
AF_BUILD_OPENCL=1ifAF_BUILD_OPENCLisNoneelseint(AF_BUILD_OPENCL)
37+
AF_BUILD_OPENCL_CMAKE_STR='-DAF_BUILD_OPENCL:BOOL=ON'if (AF_BUILD_OPENCL==1)else'-DAF_BUILD_OPENCL:BOOL=OFF'
38+
39+
AF_BUILD_UNIFIED=os.environ.get('AF_BUILD_UNIFIED')
40+
AF_BUILD_UNIFIED=1ifAF_BUILD_UNIFIEDisNoneelseint(AF_BUILD_UNIFIED)
41+
AF_BUILD_UNIFIED_CMAKE_STR='-DAF_BUILD_UNIFIED:BOOL=ON'if (AF_BUILD_UNIFIED==1)else'-DAF_BUILD_UNIFIED:BOOL=OFF'
42+
43+
ifAF_BUILD_LOCAL_LIBS:
44+
# invoke cmake and build arrayfire libraries to install locally in package
45+
fromskbuildimportsetup
46+
47+
deffilter_af_files(cmake_manifest):
48+
cmake_manifest=list(filter(lambdaname:not (name.endswith('.h')
49+
orname.endswith('.cpp')
50+
orname.endswith('.hpp')
51+
orname.endswith('.cmake')
52+
orname.endswith('jpg')
53+
orname.endswith('png')
54+
orname.endswith('libaf.so')#avoids duplicates due to symlinks
55+
orre.match('.*libaf\.so\.3\..*',name)isnotNone
56+
orname.endswith('libafcpu.so')
57+
orre.match('.*libafcpu\.so\.3\..*',name)isnotNone
58+
orname.endswith('libafcuda.so')
59+
orre.match('.*libafcuda\.so\.3\..*',name)isnotNone
60+
orname.endswith('libafopencl.so')
61+
orre.match('.*libafopencl\.so\.3\..*',name)isnotNone
62+
orname.endswith('libforge.so')
63+
orre.match('.*libforge\.so\.1\..*',name)isnotNone
64+
or'examples'inname),cmake_manifest))
65+
returncmake_manifest
66+
67+
print('Building CMAKE with following configurable variables: ')
68+
print(AF_BUILD_CPU_CMAKE_STR)
69+
print(AF_BUILD_CUDA_CMAKE_STR)
70+
print(AF_BUILD_OPENCL_CMAKE_STR)
71+
print(AF_BUILD_UNIFIED_CMAKE_STR)
72+
73+
74+
setup(
75+
packages=['arrayfire'],
76+
cmake_install_dir='',
77+
cmake_process_manifest_hook=filter_af_files,
78+
include_package_data=False,
79+
cmake_args=[AF_BUILD_CPU_CMAKE_STR,
80+
AF_BUILD_CUDA_CMAKE_STR,
81+
AF_BUILD_OPENCL_CMAKE_STR,
82+
AF_BUILD_UNIFIED_CMAKE_STR,
83+
# todo: pass additional args from environ
84+
'-DCMAKE_BUILD_TYPE:STRING="RelWithDebInfo"',
85+
'-DFG_USE_STATIC_CPPFLAGS:BOOL=OFF',
86+
'-DFG_WITH_FREEIMAGE:BOOL=OFF',
87+
'-DCUDA_architecture_build_targets:STRING=All',
88+
'-DAF_BUILD_DOCS:BOOL=OFF',
89+
'-DAF_BUILD_EXAMPLES:BOOL=OFF',
90+
'-DAF_INSTALL_STANDALONE:BOOL=ON',
91+
'-DAF_WITH_IMAGEIO:BOOL=ON',
92+
'-DAF_WITH_LOGGING:BOOL=ON',
93+
'-DBUILD_TESTING:BOOL=OFF',
94+
'-DAF_BUILD_FORGE:BOOL=ON',
95+
'-DAF_INSTALL_LIB_DIR:STRING=arrayfire',
96+
'-DAF_INSTALL_BIN_DIR:STRING=arrayfire',
97+
'-DFG_INSTALL_LIB_DIR:STRING=arrayfire',
98+
'-DAF_WITH_STATIC_MKL=ON',
99+
]
100+
)
101+
102+
else:
103+
# ignores local arrayfire libraries, will search system instead
104+
fromsetuptoolsimportsetup
105+
setup()
15106

16-
setup()

0 commit comments

Comments
 (0)

[8]ページ先頭

©2009-2025 Movatter.jp