Jump to: navigation, search

Difference between revisions of "PyMySQL evaluation"

(Drivers Under Consideration)
Line 43: Line 43:
 
{|class="wikitable sortable"
 
{|class="wikitable sortable"
 
|-
 
|-
! Project !! PyPi hosted !! Eventlen friendly !! Python 3 compatibility !! Maturity and/or stabiltiy !! Comment
+
! Project !! PyPi hosted !! Eventlet friendly !! Python 3 compatibility !! Maturity and/or stability !! Comment
 
|-
 
|-
| MySQL-Python || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: red;" | No  || style="background-color: lightgreen;" | Yes || Can be monkeypatched by eventlet, but nobody does it
+
| MySQL-Python || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: red;" | No  || style="background-color: lightgreen;" | Yes || Can be monkeypatched by eventlet, but only to enable thread pooling
 
|-
 
|-
 
| mysqlclient || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: lightgreen;" | Yes  || style="background-color: lightgreen;" | Yes || MySQL-python fork, so should be similar to it
 
| mysqlclient || style="background-color: lightgreen;" | Yes || style="background-color: orange;" | Partial || style="background-color: lightgreen;" | Yes  || style="background-color: lightgreen;" | Yes || MySQL-python fork, so should be similar to it
 
|-
 
|-
| OurSQL || style="background-color: lightgreen;" | Yes || style="background-color: red;" | No || style="background-color: red;" | No || style="background-color: orange;" | ??? || Doesent seems to be maintained,
+
| OurSQL || style="background-color: lightgreen;" | Yes || style="background-color: red;" | No || style="background-color: orange;" | Yes, but not Pypi hosted || style="background-color: orange;" | ??? || Has not seen commits/releases in two years
 
|-
 
|-
 
| MySQL-Connector-Python || style="background-color: red;" | No || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes  || style="background-color: lightgreen;" | Yes || The official Oracle-supported driver for MySQL
 
| MySQL-Connector-Python || style="background-color: red;" | No || style="background-color: lightgreen;" | Yes || style="background-color: lightgreen;" | Yes  || style="background-color: lightgreen;" | Yes || The official Oracle-supported driver for MySQL
Line 58: Line 58:
 
== PyMySQL Overview ==
 
== PyMySQL Overview ==
  
TODO
+
TODO

Revision as of 17:13, 14 January 2015

PyMySQL Evaluation

This page will capture issues related to Openstack moving to the PyMySQL driver for MySQL/MariaDB dbapi access.

Rationale

While the MySQL-Python driver is a very mature and stable driver, it does not provide compatibility with either Python 3 or with eventlet monkeypatching (MySQL-Python can be monkeypatched with eventlet, but this feature disabled by default and missed in documentation). So OpenStack's usage of MySQL-Python, combined with the fact that concurrency is provided by eventlet, means that we currently have fully serialized database access within a single process, that is, only one database command occurs at a time within an Openstack Python process.

Drivers Under Consideration

The two drivers that are known to provide eventlet-monkeypatch compatibility are MySQL-Connector and PyMySQL, as they are written in pure Python. The two other well-known drivers for MySQL are MySQL-Python and OurSQL, both of which are written in C and offer no explicit support for eventlet or async. They can reportedly be built to support gevent using a system called Greenify, but the maturity and/or stabiltiy of this system is unknown.

A comparison at http://www.diamondtin.com/2014/sqlalchemy-gevent-mysql-python-drivers-comparison/ illustrates performance metrics observed with all four of these drivers, with MySQL-python built both without and with the "greenify" system.

A summary of the status of all four drivers is as follows.

MySQL-Python

MySQL-Python is the most widely used Python driver. However, it does not support Python 3, and it does not support async systems unless used within a thread pool. While a system known as Greenify can potentially resolve the latter situation, the Python 3 limitation is still a deal-breaker - pull request with Python 3 support was proposed to MySQL-python in April 2014 and still not merged.

mysqlclient

mysqlclient is a fork of MySQL-python. It add Python 3.3 support and merges some pull requests.

OurSQL

OurSQL is an alternative MySQL driver that is also written in C. It features fast performance and uses a different execution model than MySQL-python, using prepared statements. OurSQL does have a Python 3 port hosted on launchpad but is not integrated with OurSQL itself nor is it published on Pypi. Because it is written in C, it does not offer compatibility with async systems including eventlet-style monkeypatching.

Besides the shaky Python 3 support and lack of eventlet support, OurSQL is also not maintained, seeing its last master commit on 2012-06-05.

MySQL-Connector-Python

MySQL-connector-Python is a pure Python MySQL driver, and is now released under the auspices of the MySQL project itself, as owned by Oracle. MySQL-connector-Python supports both Python 3 as well as eventlet monkeypatching, and is well maintained. It is endorsed by Oracle as the official Oracle-supported driver for MySQL, so to that extent, it is in most ways Openstack's first choice in driver. However, Oracle refuses to publish MySQL-connector-Python on Pypi, which is critical to the Openstack infrastructure. Repeated attempts to communicate with Oracle in order to resolve this issue have not made any progress. Therefore, for this one unfortunate reason, MySQL-connector-Python will not have a place in the Openstack ecosystem unless this issue is resolved.

PyMySQL

As this document is named "PyMySQL Evaluation", one would expect that it has achieved our highest ratings for quality and appropriateness. But as should now be apparent, at the moment PyMySQL is winning pretty much because all the alternatives are worse :).

PyMySQL meets all of Openstack's criterion for a driver: it is fully open source, hosted on Github, released on Pypi, is actively maintained, is written in pure Python so is eventlet-monkeypatch compatible, and is fully Python 3 compatible.

MySQL DB Drivers Comparison

Project PyPi hosted Eventlet friendly Python 3 compatibility Maturity and/or stability Comment
MySQL-Python Yes Partial No Yes Can be monkeypatched by eventlet, but only to enable thread pooling
mysqlclient Yes Partial Yes Yes MySQL-python fork, so should be similar to it
OurSQL Yes No Yes, but not Pypi hosted  ??? Has not seen commits/releases in two years
MySQL-Connector-Python No Yes Yes Yes The official Oracle-supported driver for MySQL
PyMySQL Yes Yes Yes Yes

PyMySQL Overview

TODO