- 1 Before you Jump into the Code
- 2 Being a Good Neutron Developer (and becoming a Core Developer)
- 3 Making a First Code Commit
- 4 Proposing and Implementing a Feature
- 5 Developing a Neutron Plugin
- 6 API Extensions
- 7 How to debug Neutron (and other OpenStack projects probably )
- 8 Developing in Linux
Before you Jump into the Code
Before developing a plugin yourself, make sure you know how OpenStack Networking (Neutron [formerly Quantum]) works by reading the OpenStack Networking Admin Guide and asking questions.
Subscribe to the openstack-dev email list. . Email sent to this list about OpenStack Network should have a subject line that starts with "[Neutron]".
You'll also want to take care of a few OpenStack-wide requirements, described in HowToContribute . In particular:
- signing a Contributors License Agreement (http://wiki.openstack.org/CLA)
- getting a launchpad account are very important (http://launchpad.net)
If you're feeling adventurous, you can sign up to see the set of bugs + changes flowing into Neutron:
- subscribe to neutron bug mail: https://bugs.launchpad.net/neutron/+subscribe
- sign up to see changes proposed to the openstack/neutron and openstack/python-neutron projects: https://review.openstack.org/#/settings/projects
Being a Good Neutron Developer (and becoming a Core Developer)
Good developers write high quality code, and help maintain the quality of the neutron codebase by offering helpful and consistent reviews of other code proposed for Neutron.
Good OpenStack Network team members regularly attend the team meeting on IRC (2100 UTC on Mondays on the #openstack-meeting channel on freenode). These team meetings are for us to check-in about progress on key community deliverables, identify key reviews, and hash out any design differences that may be blocking forward progress.
Good OpenStack Network team members also participate in neutron discussions on the mailing list, monitor and fix open bugs, and help monitor the email lists and Launchpad to help answer questions about neutron. You can sign up as an "Answers contact" here: https://answers.launchpad.net/neutron/
People who earn the respect of the OpenStack Networking community for their effort and skill reviewing code can become "core developers". Core devs are the heart of any open source project. If you're wondering how to increase your chances of becoming a core developer, I would suggest reviewing existing code commits at https://review.openstack.org/#/q/status:open+project:openstack/neutron,n,z . Anyone can sign up for a launchpad account to review, and reviewing helps the other core members to get to know you, and judge your skills as a code reviewer. Each review you work on will have at least two other core developers working on it, since two core developers are needed to approve any patch. I'd also suggest identifying one or two community projects that you can work on to demonstrate your commitment to the community and help convince core devs that you can design good code (for ideas, see: http://wiki.openstack.org/NeutronStarterBugs).
A general rule of the OpenStack Networking community is that you're free to disagree with people during email discussions, IRC meetings, and code reviews, as long as you do so respectfully.
Making a First Code Commit
The best way to get started with OpenStack Networking is probably to review and test some proposed changes to Neutron (bookmark these links):
A review will always include the command to pull this particular branch of neutron for testing. So it's useful to take your devstack setup (see above), pull down these changes, confirm that they work as expected, then review the code (devstack will do a git checkout of each project into /opt/stack, so you can easily find the code in /opt/stack/neutron).
A good next step would then be to try and fix a minor bug or add a test case. For ideas, see: NeutronStarterBugs.
After making any changes to the code, be sure that unit tests still run cleanly.
When running from source, be sure you know how to run unit tests, as you'll need to do this before submitting any code:
Executing tox will run both the unit tests and check style compliance (pep8 + HACKING.rst), and both must pass for code to be accepted upstream.
To run just the unit tests:
tox -e py27
To run just the style compliance checks:
tox -e pep8
Finally, see these instructions on OpenStack Gerritt Workflow to understand how to submit code for review and inclusion within Neutron.
Proposing and Implementing a Feature
When a significant change is required to the existing implementation, or a new feature is being considered, it is first proposed in the form of a blueprint (specification). A new blueprint can be added here: https://launchpad.net/neutron (please make sure you target it to the relevant milestone). Here is a template as a reference for creating a Neutron Blueprint: http://wiki.openstack.org/Neutron/BlueprintTemplate
Developing a Neutron Plugin
If you are trying to enable the use of OpenStack with a new network switching technology, then you probably want to develop a Neutron plugin. You've come to the right place.
What is a Neutron Plugin?
Neutron exposes a logical API to define network connectivity between devices from other OpenStack services (e.g., vNICs from Nova VMs). The logical connectivity described using the API must be translated into actually configuration on virtual and/or physical switches. This is where a Neutron plugin comes in. The Neutron plugin is able to talk to one or more types of switches and dynamically reconfigures the switches based on API calls.
What Code Do I Need to Write
The Neutron code-base has a python API that corresponds to the set of API calls you must implement to be a Neutron plugin. Within the source code directory, look at neutron/neutron_plugin_base.py .
The code-base helps out by providing some useful sqlalchemy bindings to store logical API entities (networks, ports) if you choose to build your plugin around a SQL database.
A plugin usually consists of code to:
- Store information about the current logical network configuration (e.g., a SQL database)
- Determine and store information about the "mapping" of the logical model to the physical network (e.g., picking a VLAN to represent a logical network).
- Communicate with one or more types of switches to configure them according to implement that mapping. This may be in the form of an agent running on the hypervisor, or code that remotely logs into a switch and reconfigures it.
Also, if your switching technology requires nova-compute to create vNICs in a special way, you may need to create a special "vif-plugging" module to be used with your switch. However, given the existing set of plugins, it is very likely that there is already the type of vif-plugging you need. Its important that vif-plugging is kept as simple as possible to avoid complicating nova with networking details. For example, you should not pass a VLAN ID to nova, you should just have nova create the basic device and have a neutron plugin agent take care of setting up connectivity to that vlan.
There should not be a need to modify any other nova code, as this code uses only the logical Neutron API that is the same across all plugins.
Useful Next Steps
- Read the Neutron Overview in the Admin guide to get an understanding of the basic components: http://docs.openstack.org/trunk/openstack-network/admin/content/ch_overview.html
- Browse the Neutron API guide to understand the API that each plugin must implement: http://docs.openstack.org/api/openstack-network/2.0/content/index.html
- Get the Neutron code (http://launchpad.net/neutron) and run it according to the admin guide.
- Check out the existing plugins in the neutron/plugins directory to get an idea of what a plugin looks like in practice
- See an example of nova-compute vif-plugging. In the nova codebase, see nova/virt/libvirt/vif.py.
Adding a Plugin to the Public Neutron Repository
Neutron plugins may be open source or proprietary. Some of the open source plugins are maintained in the main Neutron repository (http://www.github.com/openstack/neutron), while others are maintained in their own separate repository. As a project, we encourage plugins to be kept in the main repository for several reasons (a more cohesive developer community, better code sharing across plugins, and more uniform testing of plugins). However, we must also make sure that any plugins in the main repository remain well maintained. As a result, any plugin that is in the main repo must have at least one active member of the neutron core developer team willing to maintain it. This may be achieved by having an existing core dev agree to maintain a plugin, or by having one of the developers of the plugin become a member of the core developer team.
Note: being a core developer includes community duties beyond just the maintenance of a particular plugin. This includes participating in team meetings, a minimum of 2-3 hours a week of code reviews, and an expectation that the developer will also help with other development tasks that are generally helpful to the community.
Note: We strongly suggest that your initial plugin commit focuses only the the core Neutron plugin capabilities and is as simple as possible. This increases the chances that you can convince someone else to be the maintainer, and also simplifies the initial review (initial reviews are often the hardest, as the committer is still learning Neutron coding/style requirements).
- Q: Can I run multiple plugins at the same time?
- A: No, only a single plugin can be run at a time, for a given Neutron API. That is because a "plugin" is really chunk of code that called to "implement" a particular API call. Just because there is one plugin, however, does not mean that you can only talk to one type of switch. One "plugin" can have multiple "drivers" that talk to different types of switches. For example, the Cisco plugin talks to multiple types of switches. There is no formal driver interface, but we encourage people to write the code that talks to a switch in a general way such that other plugins will be able to leverage it. A "driver" will usually be code that is able to talk to a particular switch model or family of switches. "drivers" usually will have methods for standard provisioning operations, like putting a port on a particular VLAN as the result of an attachment.
A. Yes, use the "meta-plugin" that calls code from two different existing plugins (https://github.com/openstack/neutron/tree/master/neutron/plugins/metaplugin)
API Extensions allow a plugin to extend the Neutron API in order to expose more information. This information could be required to implement advanced functionality that is specific to a particular plugin, or to expose a capability before it has been incorporated into an official Neutron API.
We strongly believe that the right way to "propose" new functionality for an official API is to first expose it as an extension. Code is a very concrete definition, and writing the code and an implementation of that code often exposes important details that might not have come up if the discussion of an API was merely abstract.
- Creating Extensions:
- Extension files should be placed in the extensions folder located at: neutron/extensions .
- The extension file should have a class with the same name as the filename. This class should implement the contract required by the extension framework. See ExtensionDescriptor class in neutron/api/extensions.py for details.
- To stop a file in the extensions folder from being loaded as an extension, the filename should start with an "_". For an example of an extension file look at Foxinsocks class in neutron/tests/unit/extensions/foxinsocks.py. The unit tests in neutron/tests/unit/test_extensions.py document all the ways in which you can use extensions
- Associating plugins with extensions:
- A Plugin can advertise all the extensions it supports through the 'supported_extension_aliases' attribute. Eg:
class SomePlugin: ... supported_extension_aliases = ['extension1_alias', 'extension2_alias', 'extension3_alias'] Any extension not in this list will not be loaded for the plugin
- Extension Interfaces for plugins (optional). The extension can mandate an interface that plugins have to support with the 'get_plugin_interface' method in the extension. For an example see the FoxInSocksPluginInterface in foxinsocks.py.
There are three types of extensions in Neutron:
- Resources introduce a new "entity" to the API. In Neutron, current entities are "networks" and "ports".
- Action extensions tend to be "verbs" that act on a resource. For example, in Nova, there is a "server" resource, and a "rebuild" action on that server resource: http://docs.openstack.org/cactus/openstack-compute/developer/openstack-compute-api-1.1/content/Rebuild_Server-d1e3538.html#d2278e1430 . Core Neutron API doesn't really use "actions" as part of the core API, but extensions certainly could.
- Request extensions allow one to add new values to existing requests objects. For example, if you were doing a POST to create a server, but wanted to add a new 'widgetType' attribute for the server object.
Looking at Foxinsox, along with the tests that use it can be helpful here: * neutron/tests/unit/extensions/foxinsocks.py * neutron/tests/unit/test_extensions.py
You can also look at other extensions in neutron/extensions, or even nova extensions in nova/api/openstack/compute/contrib/ (these are documented here: http://nova.openstack.org/api_ext/index.html)
Usually the best thing to do is find another extension that does something similar to what you are doing, then use it as a template.
How to debug Neutron (and other OpenStack projects probably )
Since most of OpenStack projects share the same infrastructure, the findings below may be valid for debugging other OpenStack projects.
Here is a list of tools that can (or cant) be used in order to debug Neutron. Please add your findings to the list.
- Eclipse pydev - Free. It works! (Thanks to gong yong sheng). You need to modify quantum-server and __init__.py as following: From: eventlet.monkey_patch() To: eventlet.monkey_patch(os=False, thread=False)
- PyCharm - Commercial. It works.
PyCharm (3.4.1) In the Settings menu, under Python Debugger and enable "Gevent compatible debugging
- Python pdb - Free. It works with no code changes.However it is a CLI based tool. Prefer another tool if you can.
- Wing IDE - It is a commercial python IDE that can handle multi threaded python application. It works without code change.
Developing in Linux
1. Make a virtual machine (however you do that on your OS) 1. Login to it. 1. If you have an HTTP proxy, export it as http_proxy - it makes virtual env recreation much easier. 1. Install development dependencies:
apt-get install python-dev python-virtualenv git
yum install python-devel python-virtualenv git
1. Run some tests: