Difference between revisions of "PyMySQL evaluation"
(→Drivers Under Consideration) |
Mike Bayer (talk | contribs) |
||
Line 43: | Line 43: | ||
{|class="wikitable sortable" | {|class="wikitable sortable" | ||
|- | |- | ||
− | ! Project !! PyPi hosted !! | + | ! Project !! PyPi hosted !! Eventlet friendly !! Python 3 compatibility !! Maturity and/or stability !! Comment |
|- | |- | ||
− | | MySQL-Python || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: red;" | No || style="background-color: lightgreen;" | Yes || Can be monkeypatched by eventlet, but | + | | MySQL-Python || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: red;" | No || style="background-color: lightgreen;" | Yes || Can be monkeypatched by eventlet, but only to enable thread pooling |
|- | |- | ||
| mysqlclient || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || MySQL-python fork, so should be similar to it | | mysqlclient || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || MySQL-python fork, so should be similar to it | ||
|- | |- | ||
− | | OurSQL || style="background-color: lightgreen;" | Yes || style="background-color: red;" | No || style="background-color: | + | | OurSQL || style="background-color: lightgreen;" | Yes || style="background-color: red;" | No || style="background-color: orange;" | Yes, but not Pypi hosted || style="background-color: orange;" | ??? || Has not seen commits/releases in two years |
|- | |- | ||
| MySQL-Connector-Python || style="background-color: red;" | No || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || The official Oracle-supported driver for MySQL | | MySQL-Connector-Python || style="background-color: red;" | No || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes || The official Oracle-supported driver for MySQL | ||
Line 58: | Line 58: | ||
== PyMySQL Overview == | == PyMySQL Overview == | ||
− | + | TODO |
Revision as of 17:13, 14 January 2015
Contents
PyMySQL Evaluation
This page will capture issues related to Openstack moving to the PyMySQL driver for MySQL/MariaDB dbapi access.
Rationale
While the MySQL-Python driver is a very mature and stable driver, it does not provide compatibility with either Python 3 or with eventlet monkeypatching (MySQL-Python can be monkeypatched with eventlet, but this feature disabled by default and missed in documentation). So OpenStack's usage of MySQL-Python, combined with the fact that concurrency is provided by eventlet, means that we currently have fully serialized database access within a single process, that is, only one database command occurs at a time within an Openstack Python process.
Drivers Under Consideration
The two drivers that are known to provide eventlet-monkeypatch compatibility are MySQL-Connector and PyMySQL, as they are written in pure Python. The two other well-known drivers for MySQL are MySQL-Python and OurSQL, both of which are written in C and offer no explicit support for eventlet or async. They can reportedly be built to support gevent using a system called Greenify, but the maturity and/or stabiltiy of this system is unknown.
A comparison at http://www.diamondtin.com/2014/sqlalchemy-gevent-mysql-python-drivers-comparison/ illustrates performance metrics observed with all four of these drivers, with MySQL-python built both without and with the "greenify" system.
A summary of the status of all four drivers is as follows.
MySQL-Python
MySQL-Python is the most widely used Python driver. However, it does not support Python 3, and it does not support async systems unless used within a thread pool. While a system known as Greenify can potentially resolve the latter situation, the Python 3 limitation is still a deal-breaker - pull request with Python 3 support was proposed to MySQL-python in April 2014 and still not merged.
mysqlclient
mysqlclient is a fork of MySQL-python. It add Python 3.3 support and merges some pull requests.
OurSQL
OurSQL is an alternative MySQL driver that is also written in C. It features fast performance and uses a different execution model than MySQL-python, using prepared statements. OurSQL does have a Python 3 port hosted on launchpad but is not integrated with OurSQL itself nor is it published on Pypi. Because it is written in C, it does not offer compatibility with async systems including eventlet-style monkeypatching.
Besides the shaky Python 3 support and lack of eventlet support, OurSQL is also not maintained, seeing its last master commit on 2012-06-05.
MySQL-Connector-Python
MySQL-connector-Python is a pure Python MySQL driver, and is now released under the auspices of the MySQL project itself, as owned by Oracle. MySQL-connector-Python supports both Python 3 as well as eventlet monkeypatching, and is well maintained. It is endorsed by Oracle as the official Oracle-supported driver for MySQL, so to that extent, it is in most ways Openstack's first choice in driver. However, Oracle refuses to publish MySQL-connector-Python on Pypi, which is critical to the Openstack infrastructure. Repeated attempts to communicate with Oracle in order to resolve this issue have not made any progress. Therefore, for this one unfortunate reason, MySQL-connector-Python will not have a place in the Openstack ecosystem unless this issue is resolved.
PyMySQL
As this document is named "PyMySQL Evaluation", one would expect that it has achieved our highest ratings for quality and appropriateness. But as should now be apparent, at the moment PyMySQL is winning pretty much because all the alternatives are worse :).
PyMySQL meets all of Openstack's criterion for a driver: it is fully open source, hosted on Github, released on Pypi, is actively maintained, is written in pure Python so is eventlet-monkeypatch compatible, and is fully Python 3 compatible.
MySQL DB Drivers Comparison
Project | PyPi hosted | Eventlet friendly | Python 3 compatibility | Maturity and/or stability | Comment |
---|---|---|---|---|---|
MySQL-Python | Yes | Partial | No | Yes | Can be monkeypatched by eventlet, but only to enable thread pooling |
mysqlclient | Yes | Partial | Yes | Yes | MySQL-python fork, so should be similar to it |
OurSQL | Yes | No | Yes, but not Pypi hosted | ??? | Has not seen commits/releases in two years |
MySQL-Connector-Python | No | Yes | Yes | Yes | The official Oracle-supported driver for MySQL |
PyMySQL | Yes | Yes | Yes | Yes |
PyMySQL Overview
TODO