https://wiki.openstack.org/w/api.php?action=feedcontributions&user=Frederic+Lepied&feedformat=atom
OpenStack - User contributions [en]
2024-03-28T08:40:16Z
User contributions
MediaWiki 1.28.2
https://wiki.openstack.org/w/index.php?title=Python3&diff=54070
Python3
2014-05-29T07:05:52Z
<p>Frederic Lepied: /* OpenStack clients */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== bytes.decode and unicode.encode ===<br />
<br />
Python has a notion of "default encoding": sys.getdefaultencoding(). On Python 2, the default encoding is ASCII, whereas it is UTF-8 on Python 3.<br />
<br />
Don't write <code>data.decode()</code> or <code>text.encode()</code> without parameter, because you will use a different encoding on Python 2 and Python 3.<br />
<br />
Use an explicit encoding instead. Example: <code>data.decode('utf-8')</code> or <code>text.encode('utf-8')</code>. The right encoding depends on the use case, but UTF-8 is usually a good candidate (it is a superset of ASCII).<br />
<br />
=== safe_decode ===<br />
<br />
Olso Incubator has a function '''safe_decode()''' which can be used to decode a bytes string and pass text strings unchanged.<br />
<br />
The default encoding is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:<br />
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)<br />
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)<br />
<br />
It's safer to explicit the encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".<br />
<br />
Safe usage:<br />
* <code>safe_decode(data, 'utf-8')</code>: decode bytes from UTF-8 or returns data unchanged if it's already a text string<br />
<br />
Unsafe usage:<br />
* <code>safe_decode(data)</code><br />
<br />
By default, the decoder is strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: safe_decode(b'[\xff]', 'ascii', 'ignore') returns '[]'.<br />
<br />
=== safe_encode ===<br />
<br />
Olso Incubator has a function '''safe_encode()''' which can be used to encode a string. Its usage is tricky and you should understand how it works and which encodings are used.<br />
* <code>safe_encode(text)</code> encodes text to the output encoding<br />
* <code>safe_encode(bytes)</code> may decode the string and then reencode to a different encoding if input and output encodings are different<br />
<br />
The default input encoding (<code>incomding</code> parameter) is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:<br />
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)<br />
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)<br />
<br />
The default output encoding (<code>encoding</code> parameter) is UTF-8.<br />
<br />
It's safer to explicit the input encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".<br />
<br />
Safe usage:<br />
* <code>safe_encode(data, incoming='utf-8')</code>: encode text to UTF-8 or returns data unchanged if it's already a bytes string (since the input and output encoding are UTF-8)<br />
<br />
Unsafe usage:<br />
* <code>safe_encode(data)</code><br />
<br />
Example:<br />
* <code>safe_encode(b'\xe9', incoming='latin-1')</code> returns <code>b'\xc3\xa9'</code>.<br />
<br />
By default, the encoder and the decoder are strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: <code>safe_encode(b'[\xff]', incoming='ascii', errors='ignore')</code> returns <code>b'[]'</code>.<br />
<br />
=== logging module and format exceptions ===<br />
<br />
On Python 2, the logging module accepts bytes and text strings. On Python 3, it only accepts text strings. For example, logging.error(b'hello') logs <code>b'hello'</code> instead of <code>'hello'</code>.<br />
<br />
There is no clear rule for format exceptions yet. There are different choices depending on the project:<br />
<br />
* <code>str(exc)</code>: native string, so use bytes on Python 2<br />
* <code>six.text_type(exc)</code>: always use Unicode. It may raise unicode error depending on the exception, be careful. Example of such error in python 2: <code>unicode(Exception("nonascii:\xe9"))</code>.<br />
* <code>six.u(str(exc))</code>: unsafe on Python 2 if str(exc) contains non-ASCII bytes, ex: <code>unicode(str(Exception("\xff")))</code><br />
* <code>LOG.exception(_LE("... %(exc)s ..."), {"exc": exc, ...})</code><br />
<br />
Since logging functions expect text strings on Python 3, logged exceptions should be formatted using <code>str(exc)</code>. Example: <code>LOG.debug(str(exc))</code>.<br />
<br />
=== HTTP ===<br />
<br />
The HTTP protocol is based on '''bytes''':<br />
<br />
* HTTP body contains '''bytes'''. For example, use io.BytesIO for a stream storing an HTTP body.<br />
* HTTPConnection.getresponse().read() returns '''bytes''' (in Python 3, '''str''' which is bytes in Python 2)<br />
* On Python 3, the http.client accepts text for HTTP headers: keys are encoded to ASCII and values to ISO 8859-1 (which is only a small subset of the Unicode charset)<br />
* It looks like Swift encodes internally HTTP headers to UTF-8 (directly using the UTF-8 encoding, not using a MIME encoding like =?UTF-8?Q?...?=. See the HTTP [RFC 2047 http://www.ietf.org/rfc/rfc2047.txt] and [http://stackoverflow.com/questions/4400678/http-header-should-use-what-character-encoding HTTP header should use what character encoding?]<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/fixture/test_logging.py || https://review.openstack.org/#/c/90318/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_openstack_generators.py || https://review.openstack.org/#/c/88124/ || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_imageutils.py || https://review.openstack.org/#/c/90532/ || <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: lightgreen;" | Yes || style="background-color: orange" | Non-voting || style="background-color:red" | No || || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 || Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | [https://pypi.python.org/pypi/python-swiftclient/ On PyPI] || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
<br />
[https://caniusepython3.com/check/4fd5dda2-b1f1-4db4-a636-67cd3276cb6a Porting status] for [https://github.com/openstack/requirements/blob/master/global-requirements.txt global-requirement.txt].<br />
<br />
It's now possible to specify different dependencies for Python 2 and Python 3 using:<br />
* requirements-py2.txt: all dependencies for Python 2 (not only dependencies specific to Python 2)<br />
* requirements-py3.txt: all dependencies for Python 3 (not only dependencies specific to Python 3)<br />
* (same for test-requirements.txt)<br />
<br />
You have to edit tox.ini to specify the right requirements file. Extract of a tox.ini file:<br />
<br />
<pre><br />
...<br />
[testenv:py33]<br />
deps = -r{toxinidir}/requirements-py3.txt<br />
-r{toxinidir}/test-requirements-py3.txt<br />
...<br />
</pre><br />
<br />
See also a patch to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock). Examples:<br />
** neutronclient: https://review.openstack.org/#/c/95786/<br />
** Oslo Incubator: https://review.openstack.org/#/c/93729/<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || Requirements upgraded: https://review.openstack.org/94357<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || <br />
Not released yet.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || eventlet portage to Python 3 in progress. Victor Stinner is working on [http://trollius.readthedocs.org/ Trollius] (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too. Python 3 support as of 0.10.4. Pushed to requirements: https://review.openstack.org/94358 .<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
=== Other OpenStack projects ===<br />
<br />
Python 3 compatible:<br />
<br />
* stackforge/python-jenkins : https://review.openstack.org/#/c/95004/ (py33 gate is voting)<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
* Juno summit notes: https://etherpad.openstack.org/p/juno-cross-project-future-of-python (Oslo) and https://etherpad.openstack.org/p/juno_swift_python3 (Swift)<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=54068
Python3
2014-05-29T07:04:56Z
<p>Frederic Lepied: /* OpenStack clients */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== bytes.decode and unicode.encode ===<br />
<br />
Python has a notion of "default encoding": sys.getdefaultencoding(). On Python 2, the default encoding is ASCII, whereas it is UTF-8 on Python 3.<br />
<br />
Don't write <code>data.decode()</code> or <code>text.encode()</code> without parameter, because you will use a different encoding on Python 2 and Python 3.<br />
<br />
Use an explicit encoding instead. Example: <code>data.decode('utf-8')</code> or <code>text.encode('utf-8')</code>. The right encoding depends on the use case, but UTF-8 is usually a good candidate (it is a superset of ASCII).<br />
<br />
=== safe_decode ===<br />
<br />
Olso Incubator has a function '''safe_decode()''' which can be used to decode a bytes string and pass text strings unchanged.<br />
<br />
The default encoding is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:<br />
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)<br />
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)<br />
<br />
It's safer to explicit the encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".<br />
<br />
Safe usage:<br />
* <code>safe_decode(data, 'utf-8')</code>: decode bytes from UTF-8 or returns data unchanged if it's already a text string<br />
<br />
Unsafe usage:<br />
* <code>safe_decode(data)</code><br />
<br />
By default, the decoder is strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: safe_decode(b'[\xff]', 'ascii', 'ignore') returns '[]'.<br />
<br />
=== safe_encode ===<br />
<br />
Olso Incubator has a function '''safe_encode()''' which can be used to encode a string. Its usage is tricky and you should understand how it works and which encodings are used.<br />
* <code>safe_encode(text)</code> encodes text to the output encoding<br />
* <code>safe_encode(bytes)</code> may decode the string and then reencode to a different encoding if input and output encodings are different<br />
<br />
The default input encoding (<code>incomding</code> parameter) is <code>sys.stdin.encoding or sys.getdefaultencoding()</code>:<br />
* Python 3: the locale encoding, or UTF-8 if sys.stdin is "mocked" (io.StringIO instance)<br />
* Python 2: the locale encoding, or ASCII if stdin is not a TTY or if sys.stdin is "mocked" (StringIO.StringIO instance)<br />
<br />
The default output encoding (<code>encoding</code> parameter) is UTF-8.<br />
<br />
It's safer to explicit the input encoding to not rely on the locale encoding and have the same behaviour even if sys.stdin is "mocked".<br />
<br />
Safe usage:<br />
* <code>safe_encode(data, incoming='utf-8')</code>: encode text to UTF-8 or returns data unchanged if it's already a bytes string (since the input and output encoding are UTF-8)<br />
<br />
Unsafe usage:<br />
* <code>safe_encode(data)</code><br />
<br />
Example:<br />
* <code>safe_encode(b'\xe9', incoming='latin-1')</code> returns <code>b'\xc3\xa9'</code>.<br />
<br />
By default, the encoder and the decoder are strict. You can specify a different error handler using the optional <code>errors</code> parameter. Example: <code>safe_encode(b'[\xff]', incoming='ascii', errors='ignore')</code> returns <code>b'[]'</code>.<br />
<br />
=== logging module and format exceptions ===<br />
<br />
On Python 2, the logging module accepts bytes and text strings. On Python 3, it only accepts text strings. For example, logging.error(b'hello') logs <code>b'hello'</code> instead of <code>'hello'</code>.<br />
<br />
There is no clear rule for format exceptions yet. There are different choices depending on the project:<br />
<br />
* <code>str(exc)</code>: native string, so use bytes on Python 2<br />
* <code>six.text_type(exc)</code>: always use Unicode. It may raise unicode error depending on the exception, be careful. Example of such error in python 2: <code>unicode(Exception("nonascii:\xe9"))</code>.<br />
* <code>six.u(str(exc))</code>: unsafe on Python 2 if str(exc) contains non-ASCII bytes, ex: <code>unicode(str(Exception("\xff")))</code><br />
* <code>LOG.exception(_LE("... %(exc)s ..."), {"exc": exc, ...})</code><br />
<br />
Since logging functions expect text strings on Python 3, logged exceptions should be formatted using <code>str(exc)</code>. Example: <code>LOG.debug(str(exc))</code>.<br />
<br />
=== HTTP ===<br />
<br />
The HTTP protocol is based on '''bytes''':<br />
<br />
* HTTP body contains '''bytes'''. For example, use io.BytesIO for a stream storing an HTTP body.<br />
* HTTPConnection.getresponse().read() returns '''bytes''' (in Python 3, '''str''' which is bytes in Python 2)<br />
* On Python 3, the http.client accepts text for HTTP headers: keys are encoded to ASCII and values to ISO 8859-1 (which is only a small subset of the Unicode charset)<br />
* It looks like Swift encodes internally HTTP headers to UTF-8 (directly using the UTF-8 encoding, not using a MIME encoding like =?UTF-8?Q?...?=. See the HTTP [RFC 2047 http://www.ietf.org/rfc/rfc2047.txt] and [http://stackoverflow.com/questions/4400678/http-header-should-use-what-character-encoding HTTP header should use what character encoding?]<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/fixture/test_logging.py || https://review.openstack.org/#/c/90318/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_openstack_generators.py || https://review.openstack.org/#/c/88124/ || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_imageutils.py || https://review.openstack.org/#/c/90532/ || <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: lightgreen;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 || Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | [https://pypi.python.org/pypi/python-swiftclient/ On PyPI] || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
<br />
[https://caniusepython3.com/check/4fd5dda2-b1f1-4db4-a636-67cd3276cb6a Porting status] for [https://github.com/openstack/requirements/blob/master/global-requirements.txt global-requirement.txt].<br />
<br />
It's now possible to specify different dependencies for Python 2 and Python 3 using:<br />
* requirements-py2.txt: all dependencies for Python 2 (not only dependencies specific to Python 2)<br />
* requirements-py3.txt: all dependencies for Python 3 (not only dependencies specific to Python 3)<br />
* (same for test-requirements.txt)<br />
<br />
You have to edit tox.ini to specify the right requirements file. Extract of a tox.ini file:<br />
<br />
<pre><br />
...<br />
[testenv:py33]<br />
deps = -r{toxinidir}/requirements-py3.txt<br />
-r{toxinidir}/test-requirements-py3.txt<br />
...<br />
</pre><br />
<br />
See also a patch to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock). Examples:<br />
** neutronclient: https://review.openstack.org/#/c/95786/<br />
** Oslo Incubator: https://review.openstack.org/#/c/93729/<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || Requirements upgraded: https://review.openstack.org/94357<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | Yes || || <br />
Not released yet.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || eventlet portage to Python 3 in progress. Victor Stinner is working on [http://trollius.readthedocs.org/ Trollius] (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too. Python 3 support as of 0.10.4. Pushed to requirements: https://review.openstack.org/94358 .<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
=== Other OpenStack projects ===<br />
<br />
Python 3 compatible:<br />
<br />
* stackforge/python-jenkins : https://review.openstack.org/#/c/95004/ (py33 gate is voting)<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
* Juno summit notes: https://etherpad.openstack.org/p/juno-cross-project-future-of-python (Oslo) and https://etherpad.openstack.org/p/juno_swift_python3 (Swift)<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=50439
Python3
2014-04-29T15:35:04Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== OpenStack Summit 2014 at Atlanta (Juno): The Future of Python Support ==<br />
<br />
Doug Hellmann proposed a cross-project workshop: http://summit.openstack.org/cfp/details/316<br />
<br />
Etherpad: https://etherpad.openstack.org/p/juno-cross-project-future-of-python<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || https://review.openstack.org/#/c/90318/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || https://review.openstack.org/#/c/88124/ || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_imageutils.py || https://review.openstack.org/#/c/90532/ || <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=OpenPGP_Web_of_Trust/Juno_Summit&diff=50438
OpenPGP Web of Trust/Juno Summit
2014-04-29T15:24:55Z
<p>Frederic Lepied: /* Juno Summit Key Signing */</p>
<hr />
<div>= Juno Summit Key Signing =<br />
In an effort to build a secure web of trust, some community members will perform an informal exchange of identification and confirm key fingerprints in the Developer Lounge at the OpenStack Juno Design Summit. Exact place and location is to be defined.<br />
<br />
If you plan to join us, please bring one or more forms of photo identification, preferably at least a [http://en.wikipedia.org/wiki/Passport passport], and make sure the real name on your key corresponds to the name on your passport. If you want additional tips, see the [[OpenPGP Web of Trust#Key Signing Process|parent article]].<br />
<br />
If you want to bring cards or pieces of paper with your name and key fingerprint then feel free, but if you '''add your information to the table below before the 14th of April 10:00am UTC''', your key will be added to a text file containing all names and fingerprints, and for which we will check together the validity.<br />
<br />
Please enter your name in alphabetical order.<br />
<br />
{| class="wikitable"<br />
|+ people participating in identification exchange<br />
! Real Name !! IRC Nick !! Key Fingerprint !! Preferred E-mail Address<br />
|-<br />
| Adrien Cunin || Adri2000 || 16F7 F5B3 2D4F 0948 5697 FCBA 37F7 4CBF 350B 810B || adrien@adriencunin.fr<br />
|-<br />
| Andreas Jaeger || Ajaeger || 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 || jaegerandi@gmail.com<br />
|-<br />
| Andreas Jaeger || Ajaeger || EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB || jaegerandi@gmail.com<br />
|-<br />
| Angus Salkeld || asalkeld || 8417 81C4 3BE3 7E25 9E6C ED7B 5AC3 6012 F166 35A8 || angus.salkeld@rackspace.com<br />
|-<br />
| Anne Gentle || annegentle || 563D 74AE 8B7E BDDF 18EB F068 0EE0 3068 5A1E 68E3 || anne@openstack.org<br />
|-<br />
| Alex Baretto || shakamunyi || A634 1E83 45C2 62E6 FA06 3363 E45E 104C 1F32 70F6 || shakamunyi@gmail.com<br />
|-<br />
| Benjamin Nemec || bnemec || 1731 DC5C 9910 BAB2 1EFF 76EA 37A1 19DD 05CB BBAA || nemecb@gmail.com<br />
|-<br />
| Chris Jones || Ng || 6C99 9021 9B3A EC6D 4A28 7EE7 C574 7646 7313 2D75 || cmsj@tenshu.net<br />
|-<br />
| Clint Byrum || SpamapS || F00C 9F2F 4F7E B714 AFAD 1A42 538C 0766 F4BC B38E || clint@fewbar.com<br />
|-<br />
| Devananda van der Veen || devananda || D235 9F32 7586 11A6 D0D5 99DC 845B EE06 7889 83A7 || devananda.vdv@gmail.com<br />
|-<br />
| Dina Belova || DinaBelova || 950D 48B2 3F58 A436 D4D2 A096 57D4 40F6 6E83 8E8B || dbelova@mirantis.com<br />
|-<br />
| Davanum Srinivas || dims || A67E 5FD8 80EB 089F 2317 7967 80D8 3A79 6103 BF59 || davanum@gmail.com<br />
|-<br />
| Dolph Mathews || dolphm || C1B5 629D E7A3 C2E7 3CF3 D665 7BC8 38D0 688A 4993 || dolph.mathews@gmail.com<br />
|-<br />
| Doug Hellmann || dhellmann || E060 113F 190C D011 A8FA B18D 3B6D 06A0 C428 437A || doug@doughellmann.com<br />
|-<br />
| Frederic Lepied || flepied || 6639 924B 8C3E 11BF F460 D573 DBF7 D8BA B931 62BA || frederic.lepied@enovance.com<br />
|-<br />
| Graham Hayes || mugsie || 10A2 36A3 7F24 80AA 7C33 36A9 1EE0 4775 7D28 E972 || graham@hayes.ie<br />
|-<br />
| Jason E. Rist || jrist || 632D 71C5 1C14 AE9B A239 0CD2 CAB1 6138 BAB7 67F8 || jrist@redhat.com<br />
|-<br />
| Jason "Jay" Faulkner || JayF || C6DD CA6C 66C4 C935 C6A4 4CA2 C481 B2FF 1612 2806 || jay@jvf.cc<br />
|-<br />
| Jeremy Stanley || fungi || 97AE 496F C02D EC9F C353 B2E7 48F9 9611 4349 5829 || fungi@yuggoth.org<br />
|-<br />
| Joe Gordon || jogo || 6FE9 D1ED 1D42 472E 39B5 2963 C98F 2334 3CBB DAC4 || joe.gordon0@gmail.com<br />
|-<br />
| John Dickinson || notmyname || 6A86 657F C8CF 99F7 8D8C C86A F0E7 F736 6D92 3037 || me@not.mn<br />
|-<br />
| Mark Atwood || MarkAtwood || 4F48 0261 701C B81F B003 3401 7322 9172 62CC C1E5 || me@mark.atwood.name<br />
|-<br />
| Matthew Oliver || mattoliverau || D1D9 7F7D 2C72 A142 9C71 365D 2EF4 97BF 73A3 79C3 || matt@oliver.net.au<br />
|-<br />
| Matt Wagner || matty_dubs || 9E9D B8DA 7F92 B207 8647 7992 4A9A 24C4 E9ED D380 || matt.wagner@redhat.com<br />
|-<br />
| Matt Wagner || matty_dubs || 5150 9442 00FE 3099 4CA8 D2EA E639 859C 2BE0 2E05 || matt.wagner@redhat.com<br />
|-<br />
| Matthew Treinish || mtreinish || 22D7 C9D2 971F 5185 A7FC CCCC FD12 A0F2 14C9 E177 || mtreinish@kortar.org<br />
|-<br />
| Melanie Witt || melwitt || EC06 A780 62F9 5E1E B89E 992C 44E2 2070 EEA5 8CA1 || melwitt@yahoo-inc.com<br />
|-<br />
| Michael Davies || mrda || 6870 C7B4 FB9C 629C C845 4E89 81C9 6288 F959 ABC5 || michael@the-davies.net<br />
|-<br />
| Nathan Kinder || nkinder || 28E8 CB5B 9567 BF4D 2735 B8BD 96BE E84E D18B E115 || nkinder@redhat.com<br />
|-<br />
| Paul Michali || pmichali || 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83 || pcm@cisco.com<br />
|-<br />
| Robert Collins || lifeless || 8244 0CEA B440 83C7 9431 D2CC 298E 9A19 AAC0 E286 || robertc@robertcollins.net<br />
|-<br />
| Roman Podoliaka || rpodolyaka || FA44 628D 79D8 9149 7C9A 3556 B479 9CF8 7DE7 563D || rpodolyaka@mirantis.com<br />
|-<br />
| Sean Dague || sdague || 4134 CF35 4470 6F86 DFB1 3746 1CAC 45A6 9FE5 AF45 || sean@dague.net<br />
|-<br />
| Sergey Lukjanov || SergeyLukjanov || 71C5 CE23 FE25 A46B 95D2 4BCB 1626 1D61 85DB 0AEC || slukjanov@mirantis.com<br />
|-<br />
| Steve Baker || stevebaker || 8172 9177 B91C 6311 60E5 B2F2 0E06 3036 384E 42CD || sbaker@redhat.com<br />
|-<br />
| Steve Kowalik || StevenK || 6557 4BDD 9F3C 76D0 808C BF36 FDD6 3BAF 588A 553F || stevenk@ubuntu.com<br />
|-<br />
| Thierry Carrez || ttx || 22A7 9430 50DB 1E67 EC2B 641A 507A F890 25B1 0423 || thierry@openstack.org<br />
|-<br />
| Thomas Goirand || zigo || A0B1 A9F3 5089 5613 0E7A 425C D416 AD15 AC6B 43FE || zigo@debian.org<br />
|-<br />
| Troy Toman || troytoman || CC5E B4FA 766F AA62 7EA9 766E 9606 8114 D82A 7C29 || troy@tomanator.com<br />
|-<br />
}</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=50102
Python3
2014-04-26T11:23:02Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== OpenStack Summit 2014 at Atlanta (Juno): The Future of Python Support ==<br />
<br />
Doug Hellmann proposed a cross-project workshop: http://summit.openstack.org/cfp/details/316<br />
<br />
Etherpad: https://etherpad.openstack.org/p/juno-cross-project-future-of-python<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || https://review.openstack.org/#/c/90318/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || https://review.openstack.org/#/c/88124/ || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/test_imageutils.py || https://review.openstack.org/#/c/90532/ || depends on https://review.openstack.org/#/c/90318<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=50016
Python3
2014-04-25T09:19:14Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== OpenStack Summit 2014 at Atlanta (Juno): The Future of Python Support ==<br />
<br />
Doug Hellmann proposed a cross-project workshop: http://summit.openstack.org/cfp/details/316<br />
<br />
Etherpad: https://etherpad.openstack.org/p/juno-cross-project-future-of-python<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || https://review.openstack.org/#/c/90318/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || https://review.openstack.org/#/c/88124/ || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/test_imageutils.py || || tests pass with testtools, but fail with nosetests (because of testscenarios)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3/SprintPycon2014&diff=49075
Python3/SprintPycon2014
2014-04-17T10:38:32Z
<p>Frederic Lepied: /* Synchronize work on OpenStack components */</p>
<hr />
<div>Victor Stinner organizes a sprint to Port OpenStack to Python 3 during 4 days at Montreal (Canada) during Pycon Montreal 2014: between April, 14 (Monday) and April, 17 (Thursday). See [https://us.pycon.org/2014/community/sprints/ Pycon Montreal 2014 sprints].<br />
<br />
== What ==<br />
<br />
The ultimate goal is to port the whole OpenStack project to Python 3. The goal of the sprint is to port OpenStack components and OpenStack dependencies to Python 3, send patches to port as much code as possible.<br />
<br />
If you don't know OpenStack at all, please join us to port modules to Python 3: MySQL-python, boto (see boto3), etc.<br />
<br />
Or come to help us replacing eventlet with Trollius!<br />
<br />
== Who ==<br />
<br />
Anyone knowing Python and OpenStack can participate!<br />
<br />
== When ==<br />
<br />
April, 14 (Monday) and April, 17 (Thursday).<br />
<br />
The next major release of OpenStack is "Icehouse", [https://wiki.openstack.org/wiki/Icehouse_Release_Schedule scheduled for April, 17].<br />
<br />
== Where ==<br />
<br />
Montreal, at [https://us.pycon.org/2014/ Pycon Montreal 2014].<br />
<br />
== Requirements ==<br />
<br />
* Good knowledge of Python<br />
* Bring a laptop<br />
* Create an account on Launchpad: [https://login.launchpad.net/+new_account Launchpad: new account] -- Launchpad is used as OpenID provider, it is at least needed to edit this wiki page, used for the sprint<br />
<br />
To contribute to OpenStack:<br />
<br />
* Basic knowledge of OpenStack<br />
* [http://devstack.org/ Install DevStack] (small setup of OpenStack written to hack OpenStack)<br />
* and OpenStack projects, sign the CLA (can be done during the sprint)<br />
** [https://www.openstack.org/join/register/ Join The OpenStack Foundation: Individual Member (Free)]<br />
** [https://review.openstack.org/ Login on review.openstack.org] and [https://review.openstack.org/#/settings/ssh-keys upload your SSH key]<br />
<br />
== Getting started ==<br />
<br />
We will focus on [https://github.com/openstack/oslo-incubator/ Oslo Incubator], so start by cloning the repository:<br />
<br />
git clone https://github.com/openstack/oslo-incubator.git<br />
<br />
Enter oslo-incubator directory, edit requirements.txt to comment "eventlet" line.<br />
<br />
cd oslo-incubator<br />
vim requirements.txt # comment eventlet line, save, exit<br />
<br />
Install [https://pypi.python.org/pypi/tox tox]:<br />
<br />
pip install tox==1.6.1<br />
<br />
Finally, run unit tests using [https://pypi.python.org/pypi/tox tox] (pip install tox):<br />
<br />
tox -e py33<br />
<br />
If you would like to run a specific unit test:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ python -m testtools.run tests.unit.test_cliutils<br />
<br />
Or you may try:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ nosetests tests/unit/test_cliutils.py<br />
<br />
See also http://haypo-notes.readthedocs.org/openstack.html#tests<br />
<br />
== Port OpenStack components to Python 3 ==<br />
<br />
First, see [[Python3#OpenStack_clients]] for the latest status.<br />
<br />
* oslo incubator: see https://wiki.openstack.org/wiki/Python3#Oslo_Incubator<br />
* oslo.messaging <br />
* swiftclient is almost done<br />
* glanceclient<br />
* neutronclient <br />
* savannaclient<br />
<br />
Other tasks:<br />
<br />
* Replace nose with testr<br />
* Horizon: Replace oslo.sphinx with oslosphinx (drop the dot)<br />
* Replace eventlet with trollius! (very complex right now)<br />
<br />
== Port OpenStack dependencies to Python 3 ==<br />
<br />
First, see [[Python3#Dependencies]] for the latest status.<br />
<br />
* boto: https://github.com/boto/boto/pull/1156 - I sent this PR a long time ago but it hasn't gotten traction. I think the boto team wants to completely rewrite boto (boto3) and do Python 3 support there. i expect that to fracture the community and double the maintenance load (much like Python 3 ;-)) but I think that was what he was leaning towards. You might consider voting for Python 3 support in plain old boto in that PR or on the mailing list...<br />
* jsonrpclib<br />
* mysql-python: 2 pending pull requests: <br />
* https://github.com/farcepest/MySQLdb1/pull/31<br />
* https://github.com/farcepest/MySQLdb1/pull/48<br />
* oslosphinx: https://review.openstack.org/#/c/79311/<br />
* pycadf<br />
* python-ldap<br />
* qpid-python<br />
* rtslib-fb<br />
* sphinxcontrib-docbookrestapi<br />
* sphinxcontrib-httpdomain<br />
* sphinxcontrib-pecanwsme<br />
* sqlalchemy-migrate (in progress)<br />
* suds<br />
* thrift<br />
* websockify<br />
<br />
== Document ==<br />
<br />
* [[Python3|Python 3 wiki page]]<br />
* [http://docs.openstack.org/developer/hacking/ OpenStack Style Guidelines]<br />
* [http://pythonhosted.org/six/ Documentation of the six module]<br />
* [http://haypo-notes.readthedocs.org/python.html#port-python-2-code-to-python-3 Port Python 2 code to Python 3]<br />
* [http://techs.enovance.com/6722/status-of-the-openstack-port-to-python-3-2 Status of the OpenStack port to Python 3]<br />
* [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
* [http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
<br />
== Synchronizations ==<br />
<br />
=== IRC ===<br />
<br />
The IRC channel #openstack-pycon of the IRC server Freenode will be used during the sprint.<br />
<br />
=== Participants ===<br />
<br />
* Chmouel Boudjnah<br />
* Cyril Roelandt (remote)<br />
* Doug Hellman (will review patches, remote)<br />
* Julien Danjou (interested to review patches, remote)<br />
* Frederic Lepied<br />
* Victor Stinner (organizer)<br />
* Éric Araujo (Wednesday)<br />
<br />
=== Synchronize work on OpenStack components ===<br />
<br />
Any change in OpenStack must be carefully reviewed. Don't expect to see your changes commited immediatly, you will probably have to discuss with the "core developers" of each project to enhance your patch, to make it fit the coding style of the project.<br />
<br />
If you want to work on a component (ex: neutron, nova, etc.), please write the component name with your name below:<br />
<br />
* <component>: <developer name><br />
<br />
If you send a patch, please add it to the list below.<br />
<br />
* oslo-incubator: https://review.openstack.org/#/c/87336/<br />
* oslo-incubator: https://review.openstack.org/#/c/87376/<br />
* oslo-incubator: https://review.openstack.org/#/c/87404/<br />
* oslo-incubator: https://review.openstack.org/#/c/87413/<br />
* oslo-incubator: https://review.openstack.org/#/c/88087/<br />
* oslo-incubator: https://review.openstack.org/#/c/87973/<br />
* oslo-incubator: https://review.openstack.org/#/c/88124/<br />
* oslo-incubator: https://review.openstack.org/#/c/74433/<br />
* oslo-incubator: https://review.openstack.org/#/c/74728/<br />
* swiftclient: https://review.openstack.org/#/c/87287/<br />
* swiftclient: https://review.openstack.org/#/c/84104/<br />
* swiftclient: https://review.openstack.org/#/c/84102/<br />
<br />
=== Synchronize work on OpenStack dependencies ===<br />
<br />
If you want to work on an OpenStack dependency, please modify the list below:<br />
<br />
* <python project>: <developer name><br />
<br />
If you send a patch, pull request, whatever, please add it to the list below:<br />
<br />
* <project>: <url to patch></div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=49048
Python3
2014-04-17T02:32:44Z
<p>Frederic Lepied: /* Python 3 Status of OpenStack projects */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || https://review.openstack.org/#/c/88124/ || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/test_imageutils.py || || tests pass with testtools, but fail with nosetests (because of testscenarios)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: orange" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=49046
Python3
2014-04-17T02:01:03Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || || depends on https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/test_imageutils.py || || tests pass with testtools, but fail with nosetests (because of testscenarios)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: orange" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=49003
Python3
2014-04-16T21:31:50Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || https://review.openstack.org/#/c/88087/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || || depends on https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/test_imageutils.py || || tests pass with testtools, but fail with nosetests (because of testscenarios)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: orange" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3/SprintPycon2014&diff=48918
Python3/SprintPycon2014
2014-04-16T14:46:33Z
<p>Frederic Lepied: /* Synchronize work on OpenStack components */</p>
<hr />
<div>Victor Stinner organizes a sprint to Port OpenStack to Python 3 during 4 days at Montreal (Canada) during Pycon Montreal 2014: between April, 14 (Monday) and April, 17 (Thursday). See [https://us.pycon.org/2014/community/sprints/ Pycon Montreal 2014 sprints].<br />
<br />
== What ==<br />
<br />
The ultimate goal is to port the whole OpenStack project to Python 3. The goal of the sprint is to port OpenStack components and OpenStack dependencies to Python 3, send patches to port as much code as possible.<br />
<br />
If you don't know OpenStack at all, please join us to port modules to Python 3: MySQL-python, boto (see boto3), etc.<br />
<br />
Or come to help us replacing eventlet with Trollius!<br />
<br />
== Who ==<br />
<br />
Anyone knowing Python and OpenStack can participate!<br />
<br />
== When ==<br />
<br />
April, 14 (Monday) and April, 17 (Thursday).<br />
<br />
The next major release of OpenStack is "Icehouse", [https://wiki.openstack.org/wiki/Icehouse_Release_Schedule scheduled for April, 17].<br />
<br />
== Where ==<br />
<br />
Montreal, at [https://us.pycon.org/2014/ Pycon Montreal 2014].<br />
<br />
== Requirements ==<br />
<br />
* Good knowledge of Python<br />
* Bring a laptop<br />
* Create an account on Launchpad: [https://login.launchpad.net/+new_account Launchpad: new account] -- Launchpad is used as OpenID provider, it is at least needed to edit this wiki page, used for the sprint<br />
<br />
To contribute to OpenStack:<br />
<br />
* Basic knowledge of OpenStack<br />
* [http://devstack.org/ Install DevStack] (small setup of OpenStack written to hack OpenStack)<br />
* and OpenStack projects, sign the CLA (can be done during the sprint)<br />
** [https://www.openstack.org/join/register/ Join The OpenStack Foundation: Individual Member (Free)]<br />
** [https://review.openstack.org/ Login on review.openstack.org] and [https://review.openstack.org/#/settings/ssh-keys upload your SSH key]<br />
<br />
== Getting started ==<br />
<br />
We will focus on [https://github.com/openstack/oslo-incubator/ Oslo Incubator], so start by cloning the repository:<br />
<br />
git clone https://github.com/openstack/oslo-incubator.git<br />
<br />
Enter oslo-incubator directory, edit requirements.txt to comment "eventlet" line.<br />
<br />
cd oslo-incubator<br />
vim requirements.txt # comment eventlet line, save, exit<br />
<br />
Install [https://pypi.python.org/pypi/tox tox]:<br />
<br />
pip install tox==1.6.1<br />
<br />
Finally, run unit tests using [https://pypi.python.org/pypi/tox tox] (pip install tox):<br />
<br />
tox -e py33<br />
<br />
If you would like to run a specific unit test:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ python -m testtools.run tests.unit.test_cliutils<br />
<br />
Or you may try:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ nosetests tests/unit/test_cliutils.py<br />
<br />
See also http://haypo-notes.readthedocs.org/openstack.html#tests<br />
<br />
== Port OpenStack components to Python 3 ==<br />
<br />
First, see [[Python3#OpenStack_clients]] for the latest status.<br />
<br />
* oslo incubator: see https://wiki.openstack.org/wiki/Python3#Oslo_Incubator<br />
* oslo.messaging <br />
* swiftclient is almost done<br />
* glanceclient<br />
* neutronclient <br />
* savannaclient<br />
<br />
Other tasks:<br />
<br />
* Replace nose with testr<br />
* Horizon: Replace oslo.sphinx with oslosphinx (drop the dot)<br />
* Replace eventlet with trollius! (very complex right now)<br />
<br />
== Port OpenStack dependencies to Python 3 ==<br />
<br />
First, see [[Python3#Dependencies]] for the latest status.<br />
<br />
* boto: https://github.com/boto/boto/pull/1156 - I sent this PR a long time ago but it hasn't gotten traction. I think the boto team wants to completely rewrite boto (boto3) and do Python 3 support there. i expect that to fracture the community and double the maintenance load (much like Python 3 ;-)) but I think that was what he was leaning towards. You might consider voting for Python 3 support in plain old boto in that PR or on the mailing list...<br />
* jsonrpclib<br />
* mysql-python: 2 pending pull requests: <br />
* https://github.com/farcepest/MySQLdb1/pull/31<br />
* https://github.com/farcepest/MySQLdb1/pull/48<br />
* oslosphinx: https://review.openstack.org/#/c/79311/<br />
* pycadf<br />
* python-ldap<br />
* qpid-python<br />
* rtslib-fb<br />
* sphinxcontrib-docbookrestapi<br />
* sphinxcontrib-httpdomain<br />
* sphinxcontrib-pecanwsme<br />
* sqlalchemy-migrate (in progress)<br />
* suds<br />
* thrift<br />
* websockify<br />
<br />
== Document ==<br />
<br />
* [[Python3|Python 3 wiki page]]<br />
* [http://docs.openstack.org/developer/hacking/ OpenStack Style Guidelines]<br />
* [http://pythonhosted.org/six/ Documentation of the six module]<br />
* [http://haypo-notes.readthedocs.org/python.html#port-python-2-code-to-python-3 Port Python 2 code to Python 3]<br />
* [http://techs.enovance.com/6722/status-of-the-openstack-port-to-python-3-2 Status of the OpenStack port to Python 3]<br />
* [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
* [http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
<br />
== Synchronizations ==<br />
<br />
=== IRC ===<br />
<br />
The IRC channel #openstack-pycon of the IRC server Freenode will be used during the sprint.<br />
<br />
=== Participants ===<br />
<br />
* Chmouel Boudjnah<br />
* Cyril Roelandt (remote)<br />
* Doug Hellman (will review patches, remote)<br />
* Julien Danjou (interested to review patches, remote)<br />
* Frederic Lepied<br />
* Victor Stinner (organizer)<br />
* Éric Araujo (Wednesday)<br />
<br />
=== Synchronize work on OpenStack components ===<br />
<br />
Any change in OpenStack must be carefully reviewed. Don't expect to see your changes commited immediatly, you will probably have to discuss with the "core developers" of each project to enhance your patch, to make it fit the coding style of the project.<br />
<br />
If you want to work on a component (ex: neutron, nova, etc.), please write the component name with your name below:<br />
<br />
* <component>: <developer name><br />
<br />
If you send a patch, please add it to the list below:<br />
<br />
* oslo-incubator: https://review.openstack.org/#/c/87336/2<br />
* oslo-incubator: https://review.openstack.org/#/c/87376/<br />
* oslo-incubator: https://review.openstack.org/#/c/87404/<br />
* oslo-incubator: https://review.openstack.org/#/c/87413/<br />
<br />
=== Synchronize work on OpenStack dependencies ===<br />
<br />
If you want to work on an OpenStack dependency, please modify the list below:<br />
<br />
* <python project>: <developer name><br />
<br />
If you send a patch, pull request, whatever, please add it to the list below:<br />
<br />
* <project>: <url to patch></div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3/SprintPycon2014&diff=48917
Python3/SprintPycon2014
2014-04-16T14:46:10Z
<p>Frederic Lepied: /* Synchronize work on OpenStack dependencies */</p>
<hr />
<div>Victor Stinner organizes a sprint to Port OpenStack to Python 3 during 4 days at Montreal (Canada) during Pycon Montreal 2014: between April, 14 (Monday) and April, 17 (Thursday). See [https://us.pycon.org/2014/community/sprints/ Pycon Montreal 2014 sprints].<br />
<br />
== What ==<br />
<br />
The ultimate goal is to port the whole OpenStack project to Python 3. The goal of the sprint is to port OpenStack components and OpenStack dependencies to Python 3, send patches to port as much code as possible.<br />
<br />
If you don't know OpenStack at all, please join us to port modules to Python 3: MySQL-python, boto (see boto3), etc.<br />
<br />
Or come to help us replacing eventlet with Trollius!<br />
<br />
== Who ==<br />
<br />
Anyone knowing Python and OpenStack can participate!<br />
<br />
== When ==<br />
<br />
April, 14 (Monday) and April, 17 (Thursday).<br />
<br />
The next major release of OpenStack is "Icehouse", [https://wiki.openstack.org/wiki/Icehouse_Release_Schedule scheduled for April, 17].<br />
<br />
== Where ==<br />
<br />
Montreal, at [https://us.pycon.org/2014/ Pycon Montreal 2014].<br />
<br />
== Requirements ==<br />
<br />
* Good knowledge of Python<br />
* Bring a laptop<br />
* Create an account on Launchpad: [https://login.launchpad.net/+new_account Launchpad: new account] -- Launchpad is used as OpenID provider, it is at least needed to edit this wiki page, used for the sprint<br />
<br />
To contribute to OpenStack:<br />
<br />
* Basic knowledge of OpenStack<br />
* [http://devstack.org/ Install DevStack] (small setup of OpenStack written to hack OpenStack)<br />
* and OpenStack projects, sign the CLA (can be done during the sprint)<br />
** [https://www.openstack.org/join/register/ Join The OpenStack Foundation: Individual Member (Free)]<br />
** [https://review.openstack.org/ Login on review.openstack.org] and [https://review.openstack.org/#/settings/ssh-keys upload your SSH key]<br />
<br />
== Getting started ==<br />
<br />
We will focus on [https://github.com/openstack/oslo-incubator/ Oslo Incubator], so start by cloning the repository:<br />
<br />
git clone https://github.com/openstack/oslo-incubator.git<br />
<br />
Enter oslo-incubator directory, edit requirements.txt to comment "eventlet" line.<br />
<br />
cd oslo-incubator<br />
vim requirements.txt # comment eventlet line, save, exit<br />
<br />
Install [https://pypi.python.org/pypi/tox tox]:<br />
<br />
pip install tox==1.6.1<br />
<br />
Finally, run unit tests using [https://pypi.python.org/pypi/tox tox] (pip install tox):<br />
<br />
tox -e py33<br />
<br />
If you would like to run a specific unit test:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ python -m testtools.run tests.unit.test_cliutils<br />
<br />
Or you may try:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ nosetests tests/unit/test_cliutils.py<br />
<br />
See also http://haypo-notes.readthedocs.org/openstack.html#tests<br />
<br />
== Port OpenStack components to Python 3 ==<br />
<br />
First, see [[Python3#OpenStack_clients]] for the latest status.<br />
<br />
* oslo incubator: see https://wiki.openstack.org/wiki/Python3#Oslo_Incubator<br />
* oslo.messaging <br />
* swiftclient is almost done<br />
* glanceclient<br />
* neutronclient <br />
* savannaclient<br />
<br />
Other tasks:<br />
<br />
* Replace nose with testr<br />
* Horizon: Replace oslo.sphinx with oslosphinx (drop the dot)<br />
* Replace eventlet with trollius! (very complex right now)<br />
<br />
== Port OpenStack dependencies to Python 3 ==<br />
<br />
First, see [[Python3#Dependencies]] for the latest status.<br />
<br />
* boto: https://github.com/boto/boto/pull/1156 - I sent this PR a long time ago but it hasn't gotten traction. I think the boto team wants to completely rewrite boto (boto3) and do Python 3 support there. i expect that to fracture the community and double the maintenance load (much like Python 3 ;-)) but I think that was what he was leaning towards. You might consider voting for Python 3 support in plain old boto in that PR or on the mailing list...<br />
* jsonrpclib<br />
* mysql-python: 2 pending pull requests: <br />
* https://github.com/farcepest/MySQLdb1/pull/31<br />
* https://github.com/farcepest/MySQLdb1/pull/48<br />
* oslosphinx: https://review.openstack.org/#/c/79311/<br />
* pycadf<br />
* python-ldap<br />
* qpid-python<br />
* rtslib-fb<br />
* sphinxcontrib-docbookrestapi<br />
* sphinxcontrib-httpdomain<br />
* sphinxcontrib-pecanwsme<br />
* sqlalchemy-migrate (in progress)<br />
* suds<br />
* thrift<br />
* websockify<br />
<br />
== Document ==<br />
<br />
* [[Python3|Python 3 wiki page]]<br />
* [http://docs.openstack.org/developer/hacking/ OpenStack Style Guidelines]<br />
* [http://pythonhosted.org/six/ Documentation of the six module]<br />
* [http://haypo-notes.readthedocs.org/python.html#port-python-2-code-to-python-3 Port Python 2 code to Python 3]<br />
* [http://techs.enovance.com/6722/status-of-the-openstack-port-to-python-3-2 Status of the OpenStack port to Python 3]<br />
* [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
* [http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
<br />
== Synchronizations ==<br />
<br />
=== IRC ===<br />
<br />
The IRC channel #openstack-pycon of the IRC server Freenode will be used during the sprint.<br />
<br />
=== Participants ===<br />
<br />
* Chmouel Boudjnah<br />
* Cyril Roelandt (remote)<br />
* Doug Hellman (will review patches, remote)<br />
* Julien Danjou (interested to review patches, remote)<br />
* Frederic Lepied<br />
* Victor Stinner (organizer)<br />
* Éric Araujo (Wednesday)<br />
<br />
=== Synchronize work on OpenStack components ===<br />
<br />
Any change in OpenStack must be carefully reviewed. Don't expect to see your changes commited immediatly, you will probably have to discuss with the "core developers" of each project to enhance your patch, to make it fit the coding style of the project.<br />
<br />
If you want to work on a component (ex: neutron, nova, etc.), please write the component name with your name below:<br />
<br />
* <component>: <developer name><br />
<br />
If you send a patch, please add it to the list below:<br />
<br />
* https://review.openstack.org/#/c/87336/2<br />
* https://review.openstack.org/#/c/87376/<br />
* https://review.openstack.org/#/c/87404/<br />
* https://review.openstack.org/#/c/87413/<br />
<br />
=== Synchronize work on OpenStack dependencies ===<br />
<br />
If you want to work on an OpenStack dependency, please modify the list below:<br />
<br />
* <python project>: <developer name><br />
<br />
If you send a patch, pull request, whatever, please add it to the list below:<br />
<br />
* <project>: <url to patch></div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48913
Python3
2014-04-16T14:31:45Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || depends on pycadf ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || || depends on https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48912
Python3
2014-04-16T14:30:34Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_base_report.py || https://review.openstack.org/#/c/87973/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || || depends on https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48909
Python3
2014-04-16T13:42:42Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || || depends on https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48906
Python3
2014-04-16T13:37:07Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_base_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48722
Python3
2014-04-14T23:47:58Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || https://review.openstack.org/87413 ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_base_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48720
Python3
2014-04-14T22:56:20Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || https://review.openstack.org/#/c/87404/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_base_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:lightgreen;" | Yes || No tests :) || style="background-color: lightgreen;" | In the git repo, not on PyPI || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48689
Python3
2014-04-14T20:26:16Z
<p>Frederic Lepied: /* Python 3 Status of OpenStack projects */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_base_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_views.py || https://review.openstack.org/87376 ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange" | In the git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3/SprintPycon2014&diff=48652
Python3/SprintPycon2014
2014-04-14T17:34:00Z
<p>Frederic Lepied: /* Synchronize work on OpenStack dependencies */</p>
<hr />
<div>Victor Stinner organizes a sprint to Port OpenStack to Python 3 during 4 days at Montreal (Canada) during Pycon Montreal 2014: between April, 14 (Monday) and April, 17 (Thursday). See [https://us.pycon.org/2014/community/sprints/ Pycon Montreal 2014 sprints].<br />
<br />
== What ==<br />
<br />
The ultimate goal is to port the whole OpenStack project to Python 3. The goal of the sprint is to port OpenStack components and OpenStack dependencies to Python 3, send patches to port as much code as possible.<br />
<br />
If you don't know OpenStack at all, please join us to port modules to Python 3: MySQL-python, boto (see boto3), etc.<br />
<br />
Or come to help us replacing eventlet with Trollius!<br />
<br />
== Who ==<br />
<br />
Anyone knowing Python and OpenStack can participate!<br />
<br />
== When ==<br />
<br />
April, 14 (Monday) and April, 17 (Thursday).<br />
<br />
The next major release of OpenStack is "Icehouse", [https://wiki.openstack.org/wiki/Icehouse_Release_Schedule scheduled for April, 17].<br />
<br />
== Where ==<br />
<br />
Montreal, at [https://us.pycon.org/2014/ Pycon Montreal 2014].<br />
<br />
== Requirements ==<br />
<br />
* Good knownledge of Python<br />
* Bring a laptop<br />
* Create an account on Launchpad: [https://login.launchpad.net/+new_account Launchpad: new account] -- Launchpad is used as OpenID provider, it is at least needed to edit this wiki page, used for the sprint<br />
<br />
To contribute to OpenStack:<br />
<br />
* Basic knownledge of OpenStack<br />
* [http://devstack.org/ Install DevStack] (small setup of OpenStack written to hack OpenStack)<br />
* and OpenStack projects, sign the CLA (can be done during the sprint)<br />
** [https://www.openstack.org/join/register/ Join The OpenStack Foundation: Individual Member (Free)]<br />
** [https://review.openstack.org/ Login on review.openstack.org] and [https://review.openstack.org/#/settings/ssh-keys upload your SSH key]<br />
<br />
== Getting started ==<br />
<br />
We will focus on [https://github.com/openstack/oslo-incubator/ Olso Incubator], so start by cloning the repository:<br />
<br />
git clone https://github.com/openstack/oslo-incubator.git<br />
<br />
Enter olso-incubator directory, edit requirements.txt to comment "eventlet" line.<br />
<br />
cd olso-incubator<br />
vim requirements.txt # comment eventlet line, save, exit<br />
<br />
Install tox:<br />
<br />
pip install tox==1.6.1<br />
<br />
Finally, run unit tests using [https://pypi.python.org/pypi/tox tox] (pip install tox):<br />
<br />
tox -e py33<br />
<br />
If you would like to run a specific unit test:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ python -m testtools.run tests.unit.test_cliutils<br />
<br />
Or you may try:<br />
<br />
$ . .tox/py33/bin/activate<br />
$ nosetests tests/unit/test_cliutils.py<br />
<br />
See also http://haypo-notes.readthedocs.org/openstack.html#tests<br />
<br />
== Port OpenStack components to Python 3 ==<br />
<br />
First, see [[Python3#OpenStack_clients]] for the latest status.<br />
<br />
* oslo incubator: see https://wiki.openstack.org/wiki/Python3#Oslo_Incubator<br />
* oslo.messaging <br />
* swiftclient is almost done<br />
* glanceclient<br />
* neutronclient <br />
* savannaclient<br />
<br />
Other tasks:<br />
<br />
* Replace nose with testr<br />
* Horizon: Replace oslo.sphinx with oslosphinx (drop the dot)<br />
* Replace eventlet with trollius! (very complex right now)<br />
<br />
== Port OpenStack dependencies to Python 3 ==<br />
<br />
First, see [[Python3#Dependencies]] for the latest status.<br />
<br />
* boto<br />
* jsonrpclib<br />
* mysql-python: 2 pending pull requests<br />
* oslosphinx: https://review.openstack.org/#/c/79311/<br />
* pycadf<br />
* python-ldap<br />
* qpid-python<br />
* rtslib-fb<br />
* sphinxcontrib-docbookrestapi<br />
* sphinxcontrib-httpdomain<br />
* sphinxcontrib-pecanwsme<br />
* sqlalchemy-migrate (in progress)<br />
* suds<br />
* thrift<br />
* websockify<br />
<br />
== Document ==<br />
<br />
* [[Python3|Python 3 wiki page]]<br />
* [http://docs.openstack.org/developer/hacking/ OpenStack Style Guidelines]<br />
* [http://pythonhosted.org/six/ Documentation of the six module]<br />
* [http://haypo-notes.readthedocs.org/python.html#port-python-2-code-to-python-3 Port Python 2 code to Python 3]<br />
* [http://techs.enovance.com/6722/status-of-the-openstack-port-to-python-3-2 Status of the OpenStack port to Python 3]<br />
* [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
* [http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
<br />
== Synchronizations ==<br />
<br />
=== IRC ===<br />
<br />
The IRC channel #openstack-pycon of the IRC server Freenode will be used during the sprint.<br />
<br />
=== Participants ===<br />
<br />
* Chmouel Boudjnah<br />
* Cyril Roelandt (remote)<br />
* Doug Hellman (will review patches, remote)<br />
* Julien Danjou (interested to review patches, remote)<br />
* Frederic Lepied<br />
* Victor Stinner (organizer)<br />
* Éric Araujo (Tuesday)<br />
<br />
=== Synchronize work on OpenStack components ===<br />
<br />
Any change in OpenStack must be carefully reviewed. Don't expect to see your changes commited immediatly, you will probably have to discuss with the "core developers" of each project to enhance your patch, to make it fit the coding style of the project.<br />
<br />
If you want to work on a component (ex: neutron, nova, etc.), please write the component name with your name below:<br />
<br />
* <component>: <developer name><br />
<br />
If you send a patch, please add it to the list below:<br />
<br />
* <nowiki>https://review.openstack.org/#/...</nowiki><br />
<br />
=== Synchronize work on OpenStack dependencies ===<br />
<br />
If you want to work on an OpenStack dependency, please modify the list below:<br />
<br />
* <python project>: <developer name><br />
<br />
If you send a patch, pull request, whatever, please add it to the list below:<br />
<br />
* <project>: <url to patch><br />
<br />
* oslo-incubator: https://review.openstack.org/#/c/87336/</div>
Frederic Lepied
https://wiki.openstack.org/w/index.php?title=Python3&diff=48651
Python3
2014-04-14T17:32:27Z
<p>Frederic Lepied: /* Oslo Incubator */</p>
<hr />
<div>This page tracks the progress of Python 3 effort porting for OpenStack.<br />
<br />
== Python 3 ==<br />
<br />
[http://techs.enovance.com/6521/openstack_python3 Why should OpenStack move to Python 3 right now?]<br />
:''Python 3 is usually seen as the new Python version which breaks compatibility and raises new Unicode issues. Python 3 is much more than that. It’s a new clean language which has a more consistent syntax. It has many new features, not less than 15 new modules. Python 3 is already well supported by major Linux distributions, whereas Python 2.7 reached its end-of-life. Slowly, some bugs cannot be fixed in Python 2.7 anymore and are only fixed in the latest Python 3 release. Python 3 is now 5 years old and considered as a mature programming language.''<br />
<br />
== Pycon Montreal 2014: Sprint Port OpenStack to Python 3 ==<br />
<br />
Enovance organizes a sprint to Port OpenStack to Python 3 during 4 days: between April, 14 (Monday) and April, 17 (Thursday). See the page [[Python3/SprintPycon2014]].<br />
<br />
== Port Python 2 code to Python 3 ==<br />
<br />
OpenStack project chose to use the same code base for Python 2 and Python 3. The [http://pythonhosted.org/six/ Six: Python 2 and 3 Compatibility Library] helps to write code working on both versions. OpenStack must still support Python 2.6 for RHEL, but not Python 2.5 and older. Debian Stable provides Python 3 but only Python 3.2, so u'unicode' syntax should be avoided (use six.u('unicode') instead).<br />
<br />
=== Common patterns ===<br />
<br />
* Replace dict.iteritems() with six.iteritems(dict)<br />
* Replace iterator.next() with next(iterator)<br />
* Replace basestring with six.string_types<br />
* Replace unicode with six.text_type<br />
<br />
=== References to port Python 2 code to Python 3 ===<br />
* [http://python3porting.com/ Porting to Python 3 Book] by Lennart Regebro, especially the [http://python3porting.com/differences.html Language differences and workarounds].<br />
* [http://docs.python.org/dev/howto/pyporting.html HOWTO: Porting Python 2 Code to Python 3] by Brett Cannon<br />
* [https://wiki.python.org/moin/PortingPythonToPy3k Porting Python Code to 3.x]<br />
* [http://code.google.com/p/python-incompatibility/ python-incompatibility]: Demonstrates incompatibilities between Python versions.<br />
<br />
=== Common pitfalls ===<br />
<br />
==== What is a string ? ====<br />
You should definitely not talk about "strings" in your commit logs/reviews. In Python 2, a 'string' is bytes; in Python 3, it's a Unicode text string. The following code snippet may help in understanding the difference:<br />
<br />
Python 2:<br />
>>> type('foo')<br />
<type 'str'><br />
>>> type(u'foo')<br />
<type 'unicode'><br />
>>> type(b'foo')<br />
<type 'str'><br />
>>> isinstance('foo', six.text_type)<br />
False<br />
>>> isinstance(u'foo', six.text_type)<br />
True<br />
>>> bytes is str<br />
True<br />
>>> b'foo'[0]<br />
'f'<br />
<br />
Python 3:<br />
>>> type('foo')<br />
<class 'str'><br />
>>> type(u'foo')<br />
<class 'str'><br />
>>> type(b'foo')<br />
<class 'bytes'><br />
>>> isinstance('foo', six.text_type)<br />
True<br />
>>> isinstance(b'foo', six.text_type)<br />
False<br />
>>> bytes is str<br />
False<br />
>>> b'foo'[0]<br />
102<br />
<br />
== Python 3 of OpenStack Dependencies ==<br />
<br />
'''Blocker Pointer:''' it's not yet possible to specify different list of dependencies for Python 2 and Python 3. For example, mox only works on Python 2, mox3 can be used on Python 3.<br />
<br />
* Julien Danjou proposed to add requirements-py3.txt: [https://review.openstack.org/#/c/58770/ openstack/requirements patch] and [https://review.openstack.org/#/c/63236/ openstack-dev/pbr patch]<br />
* An alternative is to support markers in requirements (in pip): [https://github.com/pypa/pip/issues/1433 pip issue: Support markers in setup(install_requires)?]; [https://github.com/pypa/pip/pull/1472 Victor Stinner's pull request: "parse requirements in markers"]<br />
<br />
OpenStack Dependencies:<br />
<br />
* mox: use mox3 or port tests on mock which works on Python 3 (mock has been integrated in Python 3.3 as unittest.mock)<br />
* eventlet: not available on Python 3 yet, alternatives: [http://docs.python.org/dev/library/asyncio.html asyncio] ([http://code.google.com/p/tulip/ Tulip] for Python 3.3+/[https://bitbucket.org/haypo/trollius Trollius] for Python 2), [http://www.tornadoweb.org/ Tornado]<br />
<br />
== Portage in progress ==<br />
<br />
* Oslo Messaging: Portage in Progress by Victor Stinner ([https://review.openstack.org/#/dashboard/9107 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* heatclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* neutronclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
* glanceclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Portage done ==<br />
* keystoneclient: Portage in Progress by Cyril Roelandt ([https://review.openstack.org/#/dashboard/8122 dashboard])<br />
<br />
== Python 3 Status of OpenStack projects ==<br />
<br />
See also [[Python3Deps]].<br />
<br />
=== Oslo Incubator ===<br />
<br />
'''BLOCKER BUG:''' Tests using testscenarios fail on Python 3 with nosetests because of this bug:<br />
https://bugs.launchpad.net/testscenarios/+bug/872887<br />
<br />
Recently merged reviews:<br />
<br />
* https://review.openstack.org/#/c/79781/<br />
<br />
<br />
{|class="wikitable sortable"<br />
|-<br />
! Test (full path) !! Patches !! Comment<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/config/test_generator.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/crypto/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migration_common.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_models.py || https://review.openstack.org/#/c/80307/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/db/sqlalchemy/test_options.py || https://review.openstack.org/#/c/80627/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_migrate_cli.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_utils.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/db/sqlalchemy/test_sqlalchemy.py || ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/fixture/test_logging.py || ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/middleware/test_request_id.py || https://review.openstack.org/#/c/80336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_sizelimit.py || https://review.openstack.org/#/c/80450/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/middleware/test_audit.py || || <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_guru_meditation_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_base_report.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_openstack_generators.py || ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/reports/test_views.py || ||<br />
|-<br />
<br />
| style="background-color: lightblue" | tests/unit/rpc/test_common.py || https://review.openstack.org/#/c/80533/ || The RPC code in the incubator is deprecated in favor of oslo.messaging. --[[User:Doug-hellmann|doug-hellmann]] ([[User talk:Doug-hellmann|talk]]) 15:40, 14 April 2014 (UTC)<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/scheduler/test_base_filter.py || https://review.openstack.org/#/c/80321/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/scheduler/test_weights.py || https://review.openstack.org/#/c/87336/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_cliutils || https://review.openstack.org/#/c/74433/ ||<br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_fileutils || https://review.openstack.org/#/c/74728/ ||<br />
|- <br />
<br />
| style="background-color: lightgreen" | tests/unit/test_gettext.py || https://review.openstack.org/#/c/80534/ || (issue with testscenarios) <br />
|-<br />
<br />
| style="background-color: red" | tests/unit/test_imageutils.py || || nosetests + testscenarios failure<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_jsonutils.py || https://review.openstack.org/#/c/80370/ ||<br />
|- <br />
<br />
| style="background-color: red" | tests/unit/test_log.py || || depends on https://review.openstack.org/#/c/80534/ <br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_quota.py || https://review.openstack.org/#/c/80564/ ||<br />
|-<br />
<br />
| style="background-color: lightgreen" | tests/unit/test_strutils.py || <br />
* https://review.openstack.org/#/c/80571/ (safe_encode)<br />
* https://review.openstack.org/#/c/80573/<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== OpenStack clients ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ceilometerclient python-ceilometerclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color:lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-cinderclient python-cinderclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting ||https://review.openstack.org/#/c/73844/ || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ganttclient python-ganttclient] || ? || ? || ? || ? ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-glanceclient python-glanceclient] || style="background-color: orange;" | In Progress || style="background-color: orange" | Non-voting || style="background-color:red" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-heatclient python-heatclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange" | In the git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ironicclient python-ironicclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-keystoneclient python-keystoneclient] || style="background-color: lightgreen;" | Yes || style="background-color:lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-marconiclient python-marconiclient]|| style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-melangeclient python-melangeclient] || ? || ? || ? || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-novaclient python-novaclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-neutronclient python-neutronclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || || Differences between Python 2 and 3 ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-openstackclient python-openstackclient] || style="background-color: orange" | In Progress || style="background-color: orange" | Non-Voting || style="background-color: red" | No || || Works with glanceclient HEAD<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-savannaclient python-savannaclient] || style="background-color: orange;" | In progress || style="background-color: orange" | Non-voting || || || https://review.openstack.org/#/c/73128/<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-swiftclient python-swiftclient] || style="background-color: orange;" | In progress || style="background-color: orange;" | Non-voting || style="background-color:red" |`No || Differences between Python 2 and 3 || <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-tuskarclient python-tuskarclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | On PyPI || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-troveclient python-troveclient] || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen" | On PyPI || ||<br />
|}<br />
<br />
=== Core OpenStack projects ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Trove classifiers !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ceilometer ceilometer] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-glanceclient<br />
* python-swiftclient<br />
* sphinxcontrib-docbookrestapi (tests only)<br />
* sphinxcontrib-httpdomain (tests only)<br />
* sphinxcontrib-pecanwsme (tests only)<br />
* sqlalchemy-migrate<br />
* swift (tests only)<br />
* thrift (which is blocking happybase)<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/cinder cinder] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* python-glanceclient<br />
* python-swiftclient<br />
* rtslib-fb<br />
* sqlalchemy-migrate<br />
* suds<br />
* taskflow<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/glance glance] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* nose-exclude (tests only)<br />
* nose-htmloutput (tests only)<br />
* openstack.nose_plugin (test only)<br />
* oslo.messaging<br />
* oslosphinx<br />
* paste<br />
* python-cinderclient<br />
* python-swiftclient<br />
* qpid-python (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/heat heat] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslosphinx (tests only)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient (tests only)<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
* qpid-python<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/horizon horizon] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* django-compressor<br />
* django-openstack-auth<br />
* eventlet<br />
* hacking (tests)<br />
* nose-exclude (tests)<br />
* nosehtmloutput (tests)<br />
* openstack.nose-plugin (tests)<br />
* oslo.sphinx (tests)<br />
* python-ceilometerclient<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-heatclient<br />
* python-neutronclient<br />
* python-swiftclient<br />
* python-troveclient<br />
||<br />
|-<br />
<br />
<br />
<br />
| [https://pypi.python.org/pypi/keystone keystone] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (tests only)<br />
* oslo.messaging<br />
* oslosphinx (tests only)<br />
* pam<br />
* paste<br />
* pycadf<br />
* python-ldap (tests only)<br />
* sqlalchemy-migrate<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/neutron neutron] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* eventlet<br />
* hacking (hacking)<br />
* jsonrpclib<br />
* oslo.rootwrap<br />
* paste<br />
* python-neutronclient<br />
||<br />
|-<br />
<br />
<br />
| [https://pypi.python.org/pypi/nova nova] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* boto<br />
* eventlet<br />
* hacking (tests only)<br />
* mysql-python (tests only)<br />
* oslo.messaging<br />
* oslo.rootwrap<br />
* oslosphinx (tests only)<br />
* paste<br />
* pycadf<br />
* python-cinderclient<br />
* python-glanceclient<br />
* python-neutronclient<br />
* sqlalchemy-migrate<br />
* suds<br />
* websockify<br />
||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/swift swift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* dnspython<br />
* eventlet<br />
* hacking (tests only)<br />
* netifaces<br />
* nosehtmloutput (tests only)<br />
* openstack.nose-plugin (tests only)<br />
* python-swiftclient (tests only)<br />
||<br />
|-<br />
<br />
|}<br />
<br />
=== Dependencies ===<br />
{|class="wikitable sortable"<br />
|-<br />
! Project !! Python 3 compatibility !! CI tests running? !! Python 3 classifiers ? !! Blocked by !! Comment<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/boto boto] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || See https://github.com/boto/boto3 (experimental)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-compressor django-compressor] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://github.com/django-compressor/django-compressor/issues/484<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/django-openstack-auth django-openstack-auth] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Ported: https://review.openstack.org/#/dashboard/8122<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/dnspython dnspython] || style="background-color:lightgreen;" | Yes || N/A|| style="background-color: lightgreen;" | Yes || || Must use the [https://pypi.python.org/pypi/dnspython3/ Python 3 version], see https://github.com/rthalley/dnspython/issues/60<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/ecdsa ecdsa] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: orange;" | In the Git repo || ||Py3 support merge before the 0.10 release (see https://github.com/warner/python-ecdsa/commits/master)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/eventlet eventlet] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Victor Stinner is working on Trollius (asyncio for Python 2) which may replace eventlet: [http://techs.enovance.com/6562/asyncio-openstack-python3 Use the new asyncio module and Trollius in OpenStack]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/hacking hacking] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Cyril Roelandt patch: [https://review.openstack.org/#/c/77585/ Make hacking Python 3 compatible]<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/jsonrpclib jsonrpclib] || style="background-color:red;" | No || N/A || style="background-color: red;" | No || || The project seems dead :(<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/mysql-python mysql-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || 2 pull requests for Python 3 (https://github.com/farcepest/MySQLdb1/pulls). The projects is being renamed to moist (https://github.com/farcepest/moist), Python 3 support might happen there. <br />
|-<br />
<br />
| [https://pypi.python.org/pypi/netifaces netifaces] || style="background-color:red;" | No || N/A|| style="background-color: red;" | No || || Patch sent by Victor Stinner (in private): [https://bitbucket.org/haypo/misc/src/tip/openstack/netifaces_python3.patch netifaces_python3.patch], Debian has patches too<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nose-exclude nose-exclude] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/kgrandis/nose-exclude/issue/10/test-failures-with-python-3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/nosehtmloutput nosehtmloutput] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No ||<br />
* nose-exclude (tests only)<br />
* openstack.nose-plugin<br />
||<br />
* https://bugs.launchpad.net/ubuntu/+source/python-nosehtmloutput/+bug/1287247<br />
* https://review.openstack.org/#/c/80956/<br />
|-<br />
| [https://pypi.python.org/pypi/nosexcover nosexcover] || style="background-color:lightgreen;" | No || N/A || style="background-color: lightgreen;" | On PyPI || || Python 3 support since 1.0.9<br />
|-<br />
| [https://pypi.python.org/pypi/openstack.nose-plugin openstack.nose-plugin] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.vmware oslo.vmware] || style="background-color:red;" | No || style="background-color: lightgreen;" | Voting || style="background-color: lightgreen;" | Yes || suds || <br />
|-<br />
| [https://pypi.python.org/pypi/oslo.config oslo.config] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Voting || style="background-color: orange;" | In the git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.messaging oslo.messaging] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.rootwrap oslo.rootwrap] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI (1.1.0) || ||<br />
|-<br />
| [https://pypi.python.org/pypi/oslosphinx oslosphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://review.openstack.org/#/c/79311/<br />
|-<br />
| [https://pypi.python.org/pypi/oslo.sphinx oslo.sphinx] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Must be replaced by oslosphinx (without the dot)<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pam pam] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The fork [https://pypi.python.org/pypi/simplepam simplepam] works on Python 2 and 3<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paramiko paramiko] || style="background-color:lightgreen;" | Yes || N/A || style="background-color: lightgreen;" | On PyPI || || Get https://review.openstack.org/#/c/81132/ merged<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/paste paste] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || https://bitbucket.org/ianb/paste/pull-request/9/python-3-support/diff<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/pycadf pycadf] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/python-ldap python-ldap] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || The project seems dead.<br />
|-<br />
<br />
| [https://pypi.python.org/pypi/qpid-python qpid-python] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/rtslib-fb rtslib-fb] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-docbookrestapi sphinxcontrib-docbookrestapi] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-httpdomain sphinxcontrib-httpdomain] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sphinxcontrib-pecanwsme sphinxcontrib-pecanwsme] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/sqlalchemy-migrate sqlalchemy-migrate] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No|| <br />
* hacking<br />
* ibm-db-sa<br />
* scripttest<br />
||<br />
|-<br />
| [https://pypi.python.org/pypi/suds suds] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || || Dead project<br />
|-<br />
| [https://pypi.python.org/pypi/taskflow taskflow] || style="background-color:lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | In the Git repo, not on PyPI || ||<br />
|-<br />
| [https://pypi.python.org/pypi/thrift thrift] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
| [https://pypi.python.org/pypi/websockify websockify] || style="background-color:red;" | No || style="background-color: red;" | No || style="background-color: red;" | No || ||<br />
|-<br />
<br />
|}<br />
<br />
== Reports at OpenStack Summits ==<br />
<br />
* Havana summit notes: https://etherpad.openstack.org/p/havana-python3<br />
* Icehouse summit notes: https://etherpad.openstack.org/p/IcehousePypyPy3<br />
<br />
== Dependencies ==<br />
<br />
This is an attempt to document which OpenStack dependencies work under Python 3. We're starting with oslo, since it's the base of everything else, then gradually including other projects.<br />
<br />
The [https://pypi.python.org/pypi/caniusepython3 caniusepython3] tool can be used to do quick yes/no Python 3 support checks.<br />
<br />
=== oslo.config ===<br />
<br />
==== pip-requires ====<br />
* <span style="color: green;">argparse</span> - included with Python 2.7+. The separate package is only needed for 2.6.<br />
<br />
==== test-requires ====<br />
* <span style="color: orange;">mox</span> - mox3 supports Python, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: green;">nose</span> - supports Python 3<br />
* <span style="color: green;">nose-exclude</span> - 2.6-2.7, 3.1-3.3<br />
* <span style="color: green;">testtools</span> - 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">coverage</span> - supports 2.3-3.3<br />
* <span style="color: green;">sphinx</span> - supports Python 3<br />
<br />
=== oslo-incubator ===<br />
<br />
==== pip-requires ====<br />
<br />
* <span style="color: green;">PasteDeploy</span>: supports 2.5-3.3<br />
* <span style="color: green;">WebOb</span>: support Python 3<br />
* <span style="color: red;">eventlet</span>: NO (MAJOR PAIN POINT) (gevent doesn't either, though there are some old forks that tried)<br />
* <span style="color: orange;">greenlet</span>: supports 2.4-3.2 (lack of 3.3 may be false negative)<br />
* <span style="color: green;">lxml</span>: supports 2.4-3.3<br />
* <span style="color: green;">routes</span>: supports 2.6-3.3<br />
* <span style="color: green;">iso8601</span>: NO (perhaps try [http://labix.org/python-dateutil python-dateutil], specifically the parser module?)<br />
* <span style="color: orange;">anyjson</span>: 2.4-3.1 (lack of 3.2-3.3 may be false negatives)<br />
* <span style="color: green;">kombu</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">argparse</span>: included in Python 2.7+<br />
* <span style="color: green;">stevedore</span>: supports 2.7, 3.2, and 3.3<br />
* <span style="color: green;">SQLAlchemy</span>: supports Python 3 (exact version not given)<br />
* <span style="color: red;">qpid-python</span>: NO<br />
<br />
==== test-requires ====<br />
<br />
* <span style="color: green;">distribute</span>: 2.4-3.3<br />
* <span style="color: green;">coverage</span>: 2.3-3.3<br />
* <span style="color: green;">fixtures</span>: supports Python 3 (exact version not given)<br />
* <span style="color: green;">mock</span>: 2.5-3.3<br />
* <span style="color: orange;">mox</span>: use mox3, or replace mox with mock (which is now included in python 3.3)<br />
* <span style="color: red;">mysql-python</span>: NO (maybe try pymysql instead?)<br />
* <span style="color: green;">nose</span>: Supports Python 3<br />
* <span style="color: green;">nose-exclude</span>: 2.6-2.7, 3.1-3.3<br />
* <span style="color: red;">nosehtmloutput</span>: ?<br />
* <span style="color: green;">pep8</span>: Supports Python 3<br />
* <span style="color: green;">pyflakes</span>: Supports Python 3<br />
* <span style="color: green;">pylint</span>: Supports Python 3 (tested with Python 3.2)<br />
* <span style="color: green;">pyzmq</span>: Supports 2.6-2.7, 3.2+<br />
* <span style="color: green;">redis</span>: Supports 2.5-2.7, 3.2+<br />
* <span style="color: green;">setuptools-git</span>: 2.4-2.7, 3.1-3.3<br />
* <span style="color: green;">sphinx</span>: Supports Python 3<br />
* <span style="color: green;">testtools</span>: 2.6-2.7, 3.2-3.3<br />
* <span style="color: green;">webtest</span>: 2.6-2.7, 3.2-3.3</div>
Frederic Lepied