Bug Description

The wsgi_app.py file in the tree allows operators to run Glance API as a proper WSGI app. This has been the default devstack deployment for some time and multiple real clouds in the wild deploy like this. However, an attempt to start an import will be met with an image state of "queued" forever and no tasks will ever start, run, or complete.

(note that this has been a known issue and the Glance team prescribes running standalone eventlet-based glance-api for deployments that need import to work).

Reviewed:https://review.opendev.org/742065
Committed:https://git.openstack.org/cgit/openstack/glance/commit/?id=16a5431c665adadbfcb9909cf3a73334d796c19a
Submitter: Zuul
Branch: master

commit 16a5431c665adadbfcb9909cf3a73334d796c19a
Author: Dan Smith <email address hidden>
Date: Mon Jul 20 20:49:34 2020 -0700

    Make glance-api able to do async tasks in WSGI mode

    This teaches glance-api how to do async threading things when it is
    running in pure-WSGI mode. In order to do that, a refactoring of things
    that currently depend on eventlet is required.

    It adds a [wsgi]/task_pool_threads configuration knob, which is used
    in the case of pure-WSGI and native threads to constrain the number
    of threads in that pool (and thus the task parallelism). This will
    allow tuning by the operator, but also lets us default that to just
    a single thread in the backport of these fixes so that we can avoid
    introducing a new larger footprint in the backport unexpectedly.

    Partial-Bug: #1888713
    Depends-On:https://review.opendev.org/#/c/742047/
    Change-Id: Ie15028b75fb8518ec2b0c0c0386d21782166f759

Reviewed:https://review.opendev.org/742314
Committed:https://git.openstack.org/cgit/openstack/glance/commit/?id=783fa72f481d68962509e708585fea4c163d5bf4
Submitter: Zuul
Branch: master

commit 783fa72f481d68962509e708585fea4c163d5bf4
Author: Dan Smith <email address hidden>
Date: Tue Jul 21 16:25:40 2020 -0700

    Make image conversion use a proper python interpreter for prlimit

    The image conversion plugin does a processutils exec(), which needs
    to spawn python for prlimit support. Under uwsgi, sys.executable
    points to uwsgi itself, which won't work in this case. This introduces
    a [wsgi]/python_interpreter config option (because I don't think
    there is any way to get this from uwsgi itself) which we use for
    the exec. By default, it's sys.executable, which is what is used
    right now so nobody should notice a change unless they need it.

    Note: Making this depend on the devstack change to remove the wsgi
    import restriction so we can get a test on it.

    Partial-Bug: #1888713
    Change-Id: I7cb2e135d6ea2cb21de55060df3f7bf40b3e64b6

Reviewed:https://review.opendev.org/742493
Committed:https://git.openstack.org/cgit/openstack/glance/commit/?id=f4b78606bafd0c118de67f6e1ccdad8e0c5f555d
Submitter: Zuul
Branch: master

commit f4b78606bafd0c118de67f6e1ccdad8e0c5f555d
Author: Dan Smith <email address hidden>
Date: Wed Jul 22 10:03:58 2020 -0700

    Make wsgi_app support graceful shutdown

    This makes us wait for all running threads in the tasks_pool threadpool
    before exiting. Tested with uwsgi via devstack, with a long-running
    image conversion task. Indeed, uwsgi triggers a graceful shutdown, we
    wait for the remaining thread, and then exit when the conversion has
    completed.

    Closes-Bug: #1888713
    Change-Id: I0cec7771ab8ce471607825eb4721fee1c6bdd1e6