Removing eventlet from Swift

General information

https://governance.openstack.org/tc//goals/selected/remove-eventlet.html https://wiki.openstack.org/wiki/Eventlet-removal

Swift eventlet POC

This is a POC at this moment to discuss viable approaches removing eventlet. As of writing this, this makes it possible to run the account-server without eventlet being installed, using gunicorn as the wsgi servier. Gunicorn looks promising at this point and is already part of the global openstack/requirements. Account server unit and functional (non in-process) tests are passing as well. Early benchmarking showed a +/- 15% speed difference to using eventlet.wsgi, though without tunning any settings.

Patches are proposed to the feature/threaded branch: https://review.opendev.org/q/project:openstack/swift+branch:feature/threaded+(status:open+OR+status:merged)

POC state

module imports

The POC replaces a couple of classes (eg. GreenPool, Timeout etc) with dummys to make importing swift modules possible without eventlet being installed. This is enabled by setting the environment variable "DISABLE_EVENTLET=true". It would be nice to do this specifically within config files, however Swift imports quite a bit of eventlet modules (due to dependencies) before we even get to parsing the config files.

Wherever possible eventlet imports have been replaced with their native counterpart, eg. eventlet.green.socket -> socket. Some imports are just disabled for now when not using eventlet to get the minimal POC running the account-server working.

tests

All tests should still pass with eventlet installed. In some cases this required a bit of workarounds to not modify tests heavily at this time, eg adding shutdown_safe() in https://review.opendev.org/c/openstack/swift/+/959193/6/swift/common/wsgi.py to ensure greenio.shutdown_safe() is still being called when testing with eventlet installed.

Getting started

To get started, checkout the last patch (currently #960454):

git fetch ssh://cschwede@review.opendev.org:29418/openstack/swift refs/changes/54/960454/4 && git checkout FETCH_HEAD

It is recommended to use a fresh virtualenv without eventlet being uninstalled. For example:

virtualenv ~/.venv3withouteventlet/
source ~/.venv3withouteventlet/bin/activate
pip install setuptools
python setup.py develop
pip install -r test-requirements.txt 
pip uninstall eventlet

If the environment setting "DISABLE_EVENTLET=true" is set, eventlet imports are (atm partially) replaced and the account-server and some tests can be run without using eventlet. For example:

DISABLE_EVENTLET=true pytest test/unit/account/test_server.py 
DISABLE_EVENTLET=true swift-account-server etc/account-server.conf-sample -v # stays in foreground
DISABLE_EVENTLET=true SWIFT_TEST_IN_PROCESS=false pytest test/functional/test_account.py

Next shortterm steps

Make all tests covering all changes passing without eventlet. This requires a couple of working replacements, eg. in-process wsgi.server, usable Timeout() class etc.
Use a config setting instead of an environment variable to disable eventlet.

Ideas

It might make sense to use asyncio together with eventlet+gunicorn to make migrating things easier. See https://eventlet.readthedocs.io/en/latest/asyncio/migration.html#step-1-switch-to-the-asyncio-hub
Especially for daemons not running a wsgi server (replicators, auditors etc) it might be a viable approach to simply use native threads and ThreadPoolExecutor.

Swift/Eventlet-removal

Contents