Movatterモバイル変換

[0]ホーム
URL:

Skip to content
Search Gists
Sign in Sign up

Instantly share code, notes, and snippets.

@vsajip
Last activeDecember 15, 2025 14:52

    Select an option

    Save vsajip/4673395 to your computer and use it in GitHub Desktop.
    A script which demonstrates how to extend Python 3.3's EnvBuilder, by installing setuptools and pip in created venvs. This functionality is not provided as an integral part of Python 3.3 because, while setuptools and pip are very popular, they are third-party packages.The script needs Python 3.3 or later; invoke it using"python pyvenvex.py -h"fo…
    #
    # Copyright (C) 2013-2020 Vinay Sajip. New BSD License.
    #
    importos
    importos.path
    fromsubprocessimportPopen,PIPE
    importsys
    fromthreadingimportThread
    fromurllib.parseimporturlparse
    fromurllib.requestimporturlretrieve
    importvenv
    classExtendedEnvBuilder(venv.EnvBuilder):
    """
    This builder installs setuptools and pip so that you can pip or
    easy_install other packages into the created environment.
    :param nodist: If True, setuptools and pip are not installed into the
    created environment.
    :param nopip: If True, pip is not installed into the created
    environment.
    :param progress: If setuptools or pip are installed, the progress of the
    installation can be monitored by passing a progress
    callable. If specified, it is called with two
    arguments: a string indicating some progress, and a
    context indicating where the string is coming from.
    The context argument can have one of three values:
    'main', indicating that it is called from virtualize()
    itself, and 'stdout' and 'stderr', which are obtained
    by reading lines from the output streams of a subprocess
    which is used to install the app.
    If a callable is not specified, default progress
    information is output to sys.stderr.
    """
    def__init__(self,*args,**kwargs):
    self.nodist=kwargs.pop('nodist',False)
    self.nopip=kwargs.pop('nopip',False)
    self.progress=kwargs.pop('progress',None)
    self.verbose=kwargs.pop('verbose',False)
    super().__init__(*args,**kwargs)
    defpost_setup(self,context):
    """
    Set up any packages which need to be pre-installed into the
    environment being created.
    :param context: The information for the environment creation request
    being processed.
    """
    os.environ['VIRTUAL_ENV']=context.env_dir
    ifnotself.nodist:
    self.install_setuptools(context)
    # Can't install pip without setuptools
    ifnotself.nopipandnotself.nodist:
    self.install_pip(context)
    defreader(self,stream,context):
    """
    Read lines from a subprocess' output stream and either pass to a progress
    callable (if specified) or write progress information to sys.stderr.
    """
    progress=self.progress
    whileTrue:
    s=stream.readline()
    ifnots:
    break
    ifprogressisnotNone:
    progress(s,context)
    else:
    ifnotself.verbose:
    sys.stderr.write('.')
    else:
    sys.stderr.write(s.decode('utf-8'))
    sys.stderr.flush()
    stream.close()
    definstall_script(self,context,name,url):
    _,_,path,_,_,_=urlparse(url)
    fn=os.path.split(path)[-1]
    binpath=context.bin_path
    distpath=os.path.join(binpath,fn)
    # Download script into the env's binaries folder
    urlretrieve(url,distpath)
    progress=self.progress
    ifself.verbose:
    term='\n'
    else:
    term=''
    ifprogressisnotNone:
    progress('Installing %s ...%s'% (name,term),'main')
    else:
    sys.stderr.write('Installing %s ...%s'% (name,term))
    sys.stderr.flush()
    # Install in the env
    args= [context.env_exe,fn]
    p=Popen(args,stdout=PIPE,stderr=PIPE,cwd=binpath)
    t1=Thread(target=self.reader,args=(p.stdout,'stdout'))
    t1.start()
    t2=Thread(target=self.reader,args=(p.stderr,'stderr'))
    t2.start()
    p.wait()
    t1.join()
    t2.join()
    ifprogressisnotNone:
    progress('done.','main')
    else:
    sys.stderr.write('done.\n')
    # Clean up - no longer needed
    os.unlink(distpath)
    definstall_setuptools(self,context):
    """
    Install setuptools in the environment.
    :param context: The information for the environment creation request
    being processed.
    """
    url='https://bootstrap.pypa.io/ez_setup.py'
    self.install_script(context,'setuptools',url)
    # clear up the setuptools archive which gets downloaded
    pred=lambdao:o.startswith('setuptools-')ando.endswith('.tar.gz')
    files=filter(pred,os.listdir(context.bin_path))
    forfinfiles:
    f=os.path.join(context.bin_path,f)
    os.unlink(f)
    definstall_pip(self,context):
    """
    Install pip in the environment.
    :param context: The information for the environment creation request
    being processed.
    """
    url='https://bootstrap.pypa.io/get-pip.py'
    self.install_script(context,'pip',url)
    defmain(args=None):
    compatible=True
    ifsys.version_info< (3,3):
    compatible=False
    elifnothasattr(sys,'base_prefix'):
    compatible=False
    ifnotcompatible:
    raiseValueError('This script is only for use with '
    'Python 3.3 or later')
    else:
    importargparse
    parser=argparse.ArgumentParser(prog=__name__,
    description='Creates virtual Python '
    'environments in one or '
    'more target '
    'directories.')
    parser.add_argument('dirs',metavar='ENV_DIR',nargs='+',
    help='A directory to create the environment in.')
    parser.add_argument('--no-setuptools',default=False,
    action='store_true',dest='nodist',
    help="Don't install setuptools or pip in the "
    "virtual environment.")
    parser.add_argument('--no-pip',default=False,
    action='store_true',dest='nopip',
    help="Don't install pip in the virtual "
    "environment.")
    parser.add_argument('--system-site-packages',default=False,
    action='store_true',dest='system_site',
    help='Give the virtual environment access to the '
    'system site-packages dir.')
    ifos.name=='nt':
    use_symlinks=False
    else:
    use_symlinks=True
    parser.add_argument('--symlinks',default=use_symlinks,
    action='store_true',dest='symlinks',
    help='Try to use symlinks rather than copies, '
    'when symlinks are not the default for '
    'the platform.')
    parser.add_argument('--clear',default=False,action='store_true',
    dest='clear',help='Delete the contents of the '
    'environment directory if it '
    'already exists, before '
    'environment creation.')
    parser.add_argument('--upgrade',default=False,action='store_true',
    dest='upgrade',help='Upgrade the environment '
    'directory to use this version '
    'of Python, assuming Python '
    'has been upgraded in-place.')
    parser.add_argument('--verbose',default=False,action='store_true',
    dest='verbose',help='Display the output '
    'from the scripts which '
    'install setuptools and pip.')
    options=parser.parse_args(args)
    ifoptions.upgradeandoptions.clear:
    raiseValueError('you cannot supply --upgrade and --clear together.')
    builder=ExtendedEnvBuilder(system_site_packages=options.system_site,
    clear=options.clear,
    symlinks=options.symlinks,
    upgrade=options.upgrade,
    nodist=options.nodist,
    nopip=options.nopip,
    verbose=options.verbose)
    fordinoptions.dirs:
    builder.create(d)
    if__name__=='__main__':
    rc=1
    try:
    main()
    rc=0
    exceptExceptionase:
    print('Error: %s'%e,file=sys.stderr)
    sys.exit(rc)
    @dotysan
    Copy link

    Hi, and thanks!

    I'm fiddling with a Python-from-scratch script...and found that your venv wrapper doesn't generate an error whenever pip install fails due to a missing bz2 library.

    + set -e+ $HOME/bin/python3.3 pyvenvex.py fooInstalling setuptools ....................................................................................................................................................................................................................................................done.Installing pip .........done.+ echo DONE

    Bit pip is not installed!

    If I add --verbose, it reveals the error. And of course the workaround is to build Python with bzip2 support. But still no exit(!0) status.

    @abbottc
    Copy link

    I found that (on Linux) pip installs to the "local/bin" folder of the venv. But since running "activate" only prepends the "bin" folder to $PATH, a symlink needs to be created so that pip can be run in the venv without specifying a path. I forked your code and added a few lines to do this. If you want to pull the changes, it's here:

    https://gist.github.com/abbottc/7382709

    And thanks for the script!

    @yajo
    Copy link

    Quick shortcut for using:

    $ python3 -c "$(curl https://gist.github.com/vsajip/4673395/raw/3420d9150ce1e9797dc8522fce7386d8643b02a1/pyvenvex.py)" env-dir

    @fabiencastan
    Copy link

    As mentioned, I needed to install bzip2$ pip3 install bz2file.

    I installed without symlinks, but core modules are missing, like os, encodings...

    $ ./pyvenvex-env/bin/python3.4Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linuxType "help", "copyright", "credits" or "license" for more information.>>> import os>>> os<module 'os' from '/usr/lib/python3.4/os.py'>

    @stevepiercy
    Copy link

    @vsajip I forked and made some cosmetic changes to this script that I would like to include in an update to the Python docs:
    https://gist.github.com/stevepiercy/5039c54d65dfad7315411637f335479e/revisions

    I don't know how to submit PRs to gists, so hopefully you can just copy-pasta.

    Python docs:
    https://docs.python.org/3/library/venv.html#an-example-of-extending-envbuilder

    Issue tracker:
    https://bugs.python.org/issue27285

    These changes help to clarify that a "virtual environment" (and not an "env" or "environment") is being built. Thank you for your consideration.

    @whitsonk
    Copy link

    Note... Line 136 tries to download git-pip from 'https://raw.github.com/pypa/pip/master/contrib/get-pip.py'
    That returns a 404
    I recommend the following URL instead...https://github.com/pypa/get-pip

    @Ree26er
    Copy link

    Oh I did that lol

    @daigotanaka
    Copy link

    https://gist.github.com/vsajip/4673395#file-pyvenvex-py-L120
    Looks like Bitbucket repo is not available anymore.
    I made it work again by replacing it withhttps://raw.githubusercontent.com/ActiveState/ez_setup/master/ez_setup.py

    @vsajip
    Copy link
    Author

    vsajip commentedAug 19, 2020•
    edited
    Loading

    Or perhaps the canonical location -https://bootstrap.pypa.io/ez_setup.py - might be better. I've updated the Gist to get those resources fromhttps://bootstrap.pypa.io/ - hopefully that solves the linkrot problem!

    @Mavnus04
    Copy link

    Had to replace the url for pip, output during recent install on Windows 10 machine:

    Installing pip ...ERROR: This script does not work on Python 3.3 The minimum supported Python version is 3.7. Please use https://bootstrap.pypa.io/pip/3.3/get-pip.py instead.done.

    New linkhttps://bootstrap.pypa.io/pip/3.3/get-pip.py worked great. Was able to install packages with
    python -m pip install [packagename] --ignore-requires-python

    Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
    [8]ページ先頭
    ©2009-2025 Movatter.jp