Revision as of 15:21, 14 January 2015

PyMySQL Evaluation

This page will capture issues related to Openstack moving to the PyMySQL driver for MySQL/MariaDB dbapi access.

Rationale

While the MySQL-Python driver is a very mature and stable driver, it does not provide compatibility with either Python 3 or with eventlet monkeypatching (MySQL-Python can be monkeypatched with eventlet, but this feature disabled by default and missed in documentation). So OpenStack's usage of MySQL-Python, combined with the fact that concurrency is provided by eventlet, means that we currently have fully serialized database access within a single process, that is, only one database command occurs at a time within an Openstack Python process.

Drivers Under Consideration

The two drivers that are known to provide eventlet-monkeypatch compatibility are MySQL-Connector and PyMySQL, as they are written in pure Python. The two other well-known drivers for MySQL are MySQL-Python and OurSQL, both of which are written in C and offer no explicit support for eventlet or async. They can reportedly be built to support gevent using a system called Greenify, but the maturity and/or stabiltiy of this system is unknown.

A comparison at http://www.diamondtin.com/2014/sqlalchemy-gevent-mysql-python-drivers-comparison/ illustrates performance metrics observed with all four of these drivers, with MySQL-python built both without and with the "greenify" system.

A summary of the status of all four drivers is as follows.

MySQL-Python

MySQL-Python is the most widely used Python driver. However, it does not support Python 3, and it does not support async systems unless used within a thread pool. While a system known as Greenify can potentially resolve the latter situation, the Python 3 limitation is still a deal-breaker - pull request with Python 3 support was proposed to MySQL-python in April 2014 and still not merged.

mysqlclient

mysqlclient is a fork of MySQL-python. It add Python 3.3 support and merges some pull requests.

OurSQL

OurSQL is an alternative MySQL driver that is also written in C. It features fast performance and uses a different execution model than MySQL-python, using prepared statements. OurSQL does have a Python 3 port hosted on launchpad but is not integrated with OurSQL itself nor is it published on Pypi. Because it is written in C, it does not offer compatibility with async systems including eventlet-style monkeypatching.

Besides the shaky Python 3 support and lack of eventlet support, OurSQL is also not maintained, seeing its last master commit on 2012-06-05.

MySQL-Connector-Python

MySQL-connector-Python is a pure Python MySQL driver, and is now released under the auspices of the MySQL project itself, as owned by Oracle. MySQL-connector-Python supports both Python 3 as well as eventlet monkeypatching, and is well maintained. It is endorsed by Oracle as the official Oracle-supported driver for MySQL, so to that extent, it is in most ways Openstack's first choice in driver. However, Oracle refuses to publish MySQL-connector-Python on Pypi, which is critical to the Openstack infrastructure. Repeated attempts to communicate with Oracle in order to resolve this issue have not made any progress. Therefore, for this one unfortunate reason, MySQL-connector-Python will not have a place in the Openstack ecosystem unless this issue is resolved.

PyMySQL

As this document is named "PyMySQL Evaluation", one would expect that it has achieved our highest ratings for quality and appropriateness. But as should now be apparent, at the moment PyMySQL is winning pretty much because all the alternatives are worse :).

PyMySQL meets all of Openstack's criterion for a driver: it is fully open source, hosted on Github, released on Pypi, is actively maintained, is written in pure Python so is eventlet-monkeypatch compatible, and is fully Python 3 compatible.

PyMySQL Overview

TODO