Revision as of 16:30, 31 August 2010

This page serves as a proposal to unify the command-line and configuration file options processing for OpenStack projects.

<<TableOfContents()>>

Current status

Command-line options processing

Currently, Nova uses `python-gflags` for processing command-line options. This style includes the following way of specifying program options:

In a module, you do this:

#!highlight python
import flags
FLAGS = flags.FLAGS

DEFINE_string('connection_type', 'libvirt', 'libvirt, xenapi, or fake')

The above would define an option "--connection_type" that can appear on the command line (or in a special "flag file", see below)

While a coder defines their module-level flags in the module they are working on (as opposed to a single global file), there is no support for either:

• enforcement of module-level option names
• auto-prefixing of module name to option name
  • for example, having an option in `/nova/datastore.py` called "driver" automatically be labeled "--datastore-driver" or "--nova-datastore-driver"

In Swift, there is a mixture of usage of either the older `getopt` module or the newer `optparse` module (which has now been deprecated in favour of the `argparse` module in 2.7).

Configuration file processing

In Nova, there is no support for configuration file processing other than the use of gflag's `--flag-file=FILE` argument, which indicates a file that can contain a non-standard newline-delimited list of flag options, like so:

--connection_type=fake
--s3_port=3334

As mentioned, this is non-standard, and does not conform to any RFC regarding configuration files.

In Swift, some things use the `ConfigParser` module for allowing configuration options in files. There doesn't seem to be any support, however, for configuration option groups in common configuration files for specific modules (see proposal below)

Proposal for a unified options processing module

I propose consolidating all of the CLI and configuration file options processing into a single common module: openstack.common.config

This module would expose an API for adding system and module-level program options in a simple, straightforward manner, with automatic support for program options entered into standard configuration files.

Adding program options

Program options may be added using openstack.common.config.add_module_option.

Example: `/nova/datastore.py`

Current options stuff (abbreviated):

#!highlight python
from nova import flags


FLAGS = flags.FLAGS
flags.DEFINE_string('redis_host', '127.0.0.1',
                    'Host that redis is running on.')
flags.DEFINE_integer('redis_port', 6379,
                    'Port that redis is running on.')

Would become:

#!highlight python
from openstack.common import config

config.add_module_option(__name__, '--redis-host', '127.0.0.1',
                         'Host that redis is running on')
config.add_module_option(__name__, '--redis-port', 6379,
                         'Port that redis is running on', type="int")

Since the underlying processing engine would be optparse, you can use any of the keyword arguments to openstack.common.config.add_module_option that you would use with optparse.[[OptionParser]].add_option, as illustrated below:

#!highlight python
from openstack.common import config

_engines = ['redis', 'sqlite', 'memory']

config.add_module_option(__name__, '--engine', default='redis',
                         choices=_engines, metavar="DRIVER",
                         help="Driver to use for data storage "
                         " (default: %default)")

Retrieving program options

Configuration files

References

• Python optparse module