[ out of date, updates for Juno in progress ]
One document, one mission. - author's note"
Parity related efforts in Icehouse were primarily analysis, documentation, and tracking of related blueprints, bugs and reviews. Endeavors for Juno need to be very specific in order to get further. There are five areas that need serious focus:
- quality and performance
- gate performance
- test coverage
- migration paths
Pragmatically, it is pointless to discuss this without considering resourcing. 5 areas require serious focus implies 5 principle resources at minimum. Additionally, a core should be either be one of these resources or, perhaps optimally, "in the loop". Besides facilitating reviews a core may be better positioned to intervene when a patch or effort risks breaking parity related functionality. Ideally, persons with a parity focus (or at least keeps it in mind) should be actively involved with all related efforts (e.g. DVR, HA, etc.). In short, parity is a sort of cross cutting concern and it needs sufficient representation across the relevant efforts to be achieved. Under-resourcing risks any practical progress requiring mulitple cycles.
While an overall objective of deprecating nova-networking may be an OpenStack goal, this is a distraction. The focus is to provide a superior option for OpenStack users to migrate to that need more flexibility, features performance and scalabilty.
What Needs To Be Done?
Quality and Performance
Specifically, the quality and performance of the OpenSource components of Neutron have to be beyond reasonable reproach.
- Functionality must be reliable with well understood code paths and state transitions.
- Configuration and use should be, if not precisely intuitive, be sensible within the eyes of the user. A simple deployment should not be unduly complex to configure. Complexity should mount in direct correlation to configuration.
- Operations should have reasonable completion times and not be unduly related to number of concurrent operations (e.g. while "batching" is acceptable for some things, delay by design in provisioning is unaccepable)
- Test coverage- the review queue for tests needs constant attention!
- Real world perspective and vigilence - reasonable subjective understanding of real world use cases! The persons working on parity should be, besides extending and amplifying the tempest test suite need to really know how neutron is behaving in a realistic deployments. Even simple ones. Some things are not reasonable to put in tempest, but are still reasonable tests. How long does it take to assign floating IPs? What's the impact of multiple concurrent spawns? Can you break things by just doing a bunch of stuff using the client concurrently? This is just a matter of really knowing what the product is doing.
Each parity team member should almost appear as though they are infra team members. Where the gate is wrong, the gate needs to be made right. This is not just a matter of making the tests right, but making sure that the infrastructure has what it needs to do what is needed! Poor performance of neutron as a whole in the gate is a parity blocker.
Nova is both the client and the server with respect to neutron. The workings of these interactions cannot be ignored and have to be extremely well understood. This implies some straddling of the teams and getting to know the related goals and objectives of the active nova efforts. Active representation is essential.
(There are people who are currently straddling both, but we can use more!)
mlavalle and rossella_s have been spearheading this effort to date. These efforts need to be considered as directly relevant.
There is an overlap with interoperability and quality/performance and migration paths. This is effectively a cross-cutting concern that partly defines what aspects of quality and interoperability are specific to parity. That is, if it will never affect a migrating user, it is not likely a parity concern. Realistically, there are infinite possibilties of deployments so this it is impractical to empirically define all migration paths. Parity principals should be familiar with the nova network manager types, multi-host, L3 functionality, etc. so that they can conceptualize how a user may migrate. This is not a "nice to have". It is essential and also part of why overall neutron project awareness is important. If a design decision is made that explicitly and deliberately hinders a basic migration path, then it needs to be mitigated.
Roles of Cores and Principals
Vigilence on the review queues is essential. Support for patches that enhance migration potential, performance, etc. must be given in a timely fashion as is intervention in deviations. Core involvement facilitates proper emphasis where required. REMEMBER however that anyone can -1.. In short, the role of the the principal is to be actively involved in at least one related activity, safeguarding migration paths, stability, etc. and otherwise contributing to closing perceived feature and functionality gaps that are migration obstacles. Principals attend the biweekly parity meetings and remain in close contact with the other principals in the interim. Situations will arise where principals must work in closely directed efforts to rapidly overcome particular obstacles (geurilla tactics? hero sprints?). A principal need not be core, but the demands on overall knowledge and the complexity of some of the problem areas does imply a certain amount of experience.
For the purposes of continuity and progress, a principal should be in a positiion to commit to the parity effort for a minimum of a complete cycle.
Approach to Organization
It is more important to be active in the principal areas than to have frequent formal meetings. Most principal areas are addressed under other guises weekly at the IRC meeting. A meeting every two weeks for the principals is worthwhile to start, accelerating or spreading out as the overall situation demands. Interoperability should continue as a weekly item in the neutron team meetings. Individuals wishing to contribute but are unable to commit to a protracted effort for a cycle are still valuable and all contributions are welcome. If a contributor steps in, the principals must effectively sponsor the effort, providing timely feedback, review support, etc. to make the most of these contributions. This implies a relationship between the number of non-principal and principal contributors, but it is unlikely that this would become a limitation beyond that of the core/non-core contributor ratio in the neutron project as a whole.
Moving Forward and Reporting Status
Reporting parity is a more of state of what is broken than what it is not. A tag denoting an issue as network-parity related should be incorporated into the blueprints and launchpad bug system.
Two new blueprints need to be written (or resurrected as the case may be) with a realistic and proper design and plan, including testing:
- simplified all-in-one with "external access" (wording?)
Both of these issues are largely concerned with resolving remaining migration path issues that are completely unaddressed within nova.
Additional blueprints are required:
- Provide Flat Network Migration Path
- Provide FlatDHCP Network Migration Path
- Provide VLAN Migration Path
These blueprints are the muster points for new code and bugs that directly impact the respective migration path. In addition to these specific migration paths:
- Real World Migrations from Nova Network
This last blueprint may be defined as a parent or the dependencies may be defined such that the last cannot reasonably be completed without the others also being completed.
The blueprints for the specific areas are important for several reasons:
- it brings the specific efforts of the principals *in process" and allows the activities to be planned for and reported as logical groupings in a given cycle
- it provides an opportunity for the neutron project as a whole to agree on priorities and relevance of the particular efforts
These blueprints cannot be placeholders, but must be properly defined bodies of work. If it is not possible to define them so that they are first class blueprints that may be approved (or not) in a regular fashion than more work is required in their definition. Dependencies on existing blueprints and bugs should be indicated where relevant. These blueprints should ideally be in a completed state for the spring summit in 2014. Plans and efforts for mitigating risks based on dependencies should be addressed as soon as possible.
The experience with Icehouse with respect to parity is that it seemed like everyone was working on something that was parity related, but was not actually for the sake of parity. This is partly the motivation behind organizing parity related efforts as a sort of "team member with ulterior motives" across the relevant teams. It also results renders the co-ordination with the related teams moot, or at the very least innate. The challenge in this approach is maintaining focus on parity and direction
What is beyond this point is historical and needs revision.
- Icehouse Summit QA Neutron session etherpad
- Icehouse Summit on testing with mulitple nodes
- Icehouse Summit on negative testing
bzs... don't you wish you could suck content from gerrit.
Default implementation (openvswitch)
e.g. How long does it take for a floating IP to take effect
- node counts
- network counts
- tenant counts
- dhcp agents
- l3 agents
- multiple processes
- answer to the multi-host (fault isolation) question
- real/better HA
The FlatNetworkManager (thanks rkukura for spelling this out!): (There are interop bugs related to security groups that may break this!)
(would this work for FlatDHCPNetworkManager as well?) (the aforementioned security group related issue would not affect the flat dhcp scenario the same way)
- Icehouse summit Distributed Router (possible multi-host approach)