Difference between revisions of "Meetings/Neutron-DVR"
Brian-haley (talk | contribs) m |
Brian-haley (talk | contribs) m |
||
Line 189: | Line 189: | ||
==== Jobs ==== | ==== Jobs ==== | ||
* DVR+HA multinode job is now run for all patches, currently non-voting | * DVR+HA multinode job is now run for all patches, currently non-voting | ||
− | ** https://review.openstack.org/#/c/ | + | ** https://review.openstack.org/#/c/455406/ |
** Will monitor and make sure it's stable | ** Will monitor and make sure it's stable | ||
Revision as of 15:10, 20 April 2017
The OpenStack Networking L3 DVR Sub-team holds public meetings as advertised on OpenStack IRC Meetings Calendar. If you are unable to attend, please check the most recent logs.
Contents
- 1 Meetings
- 2 Agenda
- 2.1 Meeting April 19th, 2017
- 2.2 Topics for Discussion
- 2.2.1 Bugs (Swami)
- 2.2.1.1 Bugs That Need to be Closed
- 2.2.1.2 New Bugs this week
- 2.2.1.3 High Priority Bugs in progress
- 2.2.1.4 Categorized Bugs
- 2.2.1.5 RFE
- 2.2.1.6 Existing Functionality Broken Bugs
- 2.2.1.7 Scale and Performance Impact Bugs
- 2.2.1.8 New Features Bugs
- 2.2.1.9 Refactor or Cleanup Bugs
- 2.2.1.10 WishList Bugs
- 2.2.1.11 WatchList Bugs
- 2.2.1.12 Bugs Closed Recently
- 2.2.2 Performance/Scalability
- 2.2.3 Gate (haleyb)
- 2.2.4 Stable backports (haleyb)
- 2.2.5 Open Discussion
- 2.2.1 Bugs (Swami)
- 2.3 Meeting commands
Meetings
- Weekly on Wednesday at 1500 UTC
- IRC channel:
#openstack-meeting-alt
on freenode - Chair: Brian Haley (haleyb), Swaminathan Vasudevan (Swami)
- Meetings, with their notes and logs, will be found under http://eavesdrop.openstack.org/meetings/neutron_dvr
Agenda
Meeting April 19th, 2017
Announcements (haleyb)
- When adding items below I'd like to try to get feedback on whether they are a MUST FIX, SHOULD FIX, or GOOD TO HAVE, NEEDS TRIAGE and TRIAGED
Topics for Discussion
Bugs (Swami)
All DVR bugs should be tagged and listed here: https://bugs.launchpad.net/neutron/+bugs?field.tag=l3-dvr-backlog
Bugs That Need to be Closed
- None
New Bugs this week
- https://bugs.launchpad.net/neutron/+bug/1682228 - Cross Address scopes traffic with floatingip limitation in DVR.
- https://bugs.launchpad.net/neutron/+bug/1682345 - Neutron Metering agent RPC need to send notification to specific hosts that are configured.
High Priority Bugs in progress
- https://bugs.launchpad.net/neutron/+bug/1672345 (NEEDS TRIAGE) - Symptoms look like it is a duplicate of 'allowed-address-pairs'
- https://bugs.launchpad.net/neutron/+bug/1657981 (Needs Triage) - FloatingIP not reachable after the compute node restarts while the agent is down.
- https://bugs.launchpad.net/neutron/+bug/1644231 (TRIAGED) - fip router config is not created if the vm ports attached to FIPs have no device_owner
- Needs help message update saying "compute:" prefix needed in ports manually-created for instances
- https://review.openstack.org/#/c/425919/ (NEEDS REVIEW)
- https://bugs.launchpad.net/neutron/+bug/1632540 (NEEDS TRIAGE) - L3 agent logs fills up quick (stable/mitaka tag 8.1.2 - need more info on reproducer)
- https://review.openstack.org/434863 (NEEDS REVIEW) - this will fix some of the issue
- https://bugs.launchpad.net/neutron/+bug/1606741 (SHOULD FIX) - Metadata error with dvr_snat on compute hosts.
- Lowered to Medium
- https://review.openstack.org/352686 (Expired)
Categorized Bugs
RFE
- https://bugs.launchpad.net/neutron/+bug/1563879 (RFE) - DVR should route packets to Instances behind the L2 Gateway
- https://bugs.launchpad.net/neutron/+bug/1557290 (RFE) - DVR FIP agent gateway does not pass traffic directed at fixed IP
- https://bugs.launchpad.net/neutron/+bug/1577488 (RFE) - "Fast exit" for compute node egress flows when using DVR
- https://review.openstack.org/#/c/283757/ (Merged)
- https://review.openstack.org/#/c/355062/ (Needs Review)
- https://bugs.launchpad.net/neutron/+bug/1583694 (RFE) - DVR support for Allowed_address_pair port that are bound to multiple ACTIVE VM ports
- https://review.openstack.org/#/c/320669/ (WIP)(Server side patch)
- https://review.openstack.org/#/c/323618/ (WIP)(Agent side patch)
Existing Functionality Broken Bugs
- https://bugs.launchpad.net/neutron/+bug/1526855 - (SHOULD FIX) - VMs fail to get metadata in large scale environments
- https://bugs.launchpad.net/neutron/+bug/1447227 (GOOD TO HAVE) - Connecting two or more distributed routers to a subnet. (No one is working on it)
- https://bugs.launchpad.net/neutron/+bug/1552070 (GOOD TO HAVE) - Functional test changes to address dual stack for DVR
Scale and Performance Impact Bugs
New Features Bugs
- https://bugs.launchpad.net/neutron/+bug/1504039 - LinuxBridge DVR (Spec & code)
Refactor or Cleanup Bugs
- https://bugs.launchpad.net/neutron/+bug/1538369 - Refactor request (Not a real bug)
WishList Bugs
- https://bugs.launchpad.net/neutron/+bug/1518819 - Ability to specify a gateway when adding a subnet without cidr (GOOD TO HAVE)
- https://bugs.launchpad.net/neutron/+bug/1450067 (Low) - ML2 and L3 plugin exposes dvr extension even if ovs is unused. (No owners yet) (GOOD TO HAVE)
- https://bugs.launchpad.net/neutron/+bug/1476469 (TRIAGE ME) - with DVR, a VM can't use floating IP and VPN at the same time
- This is not a bug, since floating IP can't be used with the VPN
- https://bugs.launchpad.net/neutron/+bug/1620824 (TRIAGED) - Neutron DVR(SNAT) steals FIP traffic
- https://review.openstack.org/#/c/366297/ (needs review) (Currently not using reference L2pop and using tcp_loose=0).
WatchList Bugs
Bugs Closed Recently
- https://bugs.launchpad.net/neutron/+bug/1612804 (MUST FIX) - test_shelve_instance fails with sshtimeout (One instance of failure seen in gate)
- Closed since not seen recently
- https://bugs.launchpad.net/neutron/+bug/1597461 (FIXED) - Two masters after reboot of controller when HA enabled. Also seen with DVR
- https://review.openstack.org/357458 (merged)
- https://bugs.launchpad.net/neutron/+bug/1675187 (MUST FIX) - Floating IPs not removed on rfp interface in qrouter
- https://bugs.launchpad.net/neutron/+bug/1647432 (SHOULD FIX) - Multiple SIGHUPs to keepalived might trigger re-election
- https://bugs.launchpad.net/neutron/+bug/1653633 (EXPIRED) - fwaas v1 with DVR: l3 agent can't restore the NAT rules for floatingIP (Can't be reproduced in the master branch)
- not sure it's FWaaS-specific
- https://bugs.launchpad.net/neutron/+bug/1668277 (SHOULD FIX) - When enable l2pop, after deleting the DVR port, the associated flow entries still exists
- https://bugs.launchpad.net/neutron/+bug/1629539 (NEEDS TRIAGE) - Broken distributed virtual router w/ lbaas v1
- Against Mitaka, expired
- https://bugs.launchpad.net/neutron/+bug/1658060 (Needs triage) - FirewallNotFound exceptions when deleting the firewall in FWaaS-DVR
- https://bugs.launchpad.net/neutron/+bug/1541406 - IPv6 Prefix Delegation does not work with DVR (SHOULD FIX)
- https://bugs.launchpad.net/neutron/+bug/1571676 (Re-opened) - After binding a floating IP to VM, the static route can't work in DVR
- https://review.openstack.org/#/c/308068/ (Patch Merged)
- https://bugs.launchpad.net/neutron/+bug/1557290 (SHOULD FIX) - DVR not able to forward traffic to private IP from FIP namespace
- Duplicate of Fast-exit RFE
- https://bugs.launchpad.net/neutron/+bug/1654991 (SHOULD FIX) Allow all migration of routers
- https://review.openstack.org/#/c/376550/ (Patch merged)
- https://bugs.launchpad.net/neutron/+bug/1506567 (SHOULD FIX) - No information from metering agent
- https://review.openstack.org/377108 (Patch merged)
- https://bugs.launchpad.net/neutron/+bug/1644415 (SHOULD FIX) - DVR-SNAT mode with HA does not clean up the fip route rule in namespace
- https://review.openstack.org/#/c/404571/ (Patch merged)
- https://bugs.launchpad.net/neutron/+bug/1414559 (OVS drops RARP packets)
- in neutron https://review.openstack.org/#/c/246898/ (MERGED) Notify vif plugged event
- and nova https://review.openstack.org/#/c/246910/ (not making progress) Notify vif plugged event
- https://bugs.launchpad.net/neutron/+bug/1403455 (MUST FIX) - neutron netns cleanup does not cleanup all subprocesses
- Live migration
- https://review.openstack.org/#/c/286855/ (Tempest change) - Needs work - must merge before Nova patch
- https://review.openstack.org/275420 (Merged) - Neutron Server side DVR change
- https://review.openstack.org/#/c/260738/ (Merged) - L3 Agent side patch
- https://bugs.launchpad.net/neutron/+bug/1631513 (MUST FIX) - Race condition in update_gateway_port when two simultaneous router update occurs for the same router
- https://bugs.launchpad.net/neutron/+bug/1641535 (MUST FIX) - FIP failed to remove in router's standby node
- https://bugs.launchpad.net/neutron/+bug/1612192 (MUST FIX) - L3 DVR: Unable to complete operation on subnet (Related to HA)
- Not sure this is critical any more, as it's mostly seen in the SFC and OVN jobs, ~0 in the dvr-multinode-full job
- https://bugs.launchpad.net/neutron/+bug/1612192 (MUST FIX) - L3 DVR: Unable to complete operation on subnet (Related to HA)
- Not sure this is critical any more, as it's mostly seen in the SFC and OVN jobs, ~0 in the dvr-multinode-full job
- https://bugs.launchpad.net/neutron/+bug/1593354 (MUST FIX) - SNAT HA failed due to missing NAT rule and sg- interface. (May be a duplicate)
- Not reproducible with Newton, might be only Mitaka (Not Seen in Mitaka so closed)
- https://bugs.launchpad.net/neutron/+bug/1531065 - (Expired) duplicate fetch subnet_id in get_subnet_for_dvr (GOOD TO HAVE)
- https://bugs.launchpad.net/neutron/+bug/1585165 (SHOULD FIX) - Floating ip not reachable after VM live migration (Seen in Liberty with DVR) - Seems to me like a duplicate but should triage it more on the symptoms.
- https://bugs.launchpad.net/neutron/+bug/1625333 (TRIAGED) - FloatingIP GARP fails. Log seen in the l3 agent and VM not able to ping
- Need a reproducer, only on RHEL
- https://bugs.launchpad.net/neutron/+bug/1580648 (FIXED) - Two HA routers in master state during functional test
- Patch already pushed in but the issue is still seen. (Not able to reproduce)
- https://bugs.launchpad.net/neutron/+bug/1602614 (MUST FIX) - DVR+L3 HA Loss during failover is higher
- https://bugs.launchpad.net/neutron/+bug/1456073 (High) - Block Migration with FIP breaks with DVR (MUST FIX)
- https://review.openstack.org/275073 (Nova side change) - Needs review
- https://bugs.launchpad.net/neutron/+bug/1462154 (With DVR Pings to floating IPs replied with fixed-ips if VMs are on the same network)
- https://review.openstack.org/246855 (Merged) (Reverted)
Performance/Scalability
Gate (haleyb)
Failures
Neutron Failure Rate dashboard - http://grafana.openstack.org/dashboard/db/neutron-failure-rate
- The DVR/LinuxBridge/Multi-node failure rate was high yesterday for the non-voting jobs (April 4th)
- Need to investigate why to make sure nothing is lurking (might cover in L3 meeting as well)
- Bugs in Nova:
- https://bugs.launchpad.net/nova/+bug/1524898 (Volume based live migration aborted unexpectedly)
- https://bugs.launchpad.net/nova/+bug/1535232 (live-migration ci failure on nfs shared storage)
Jobs
- DVR+HA multinode job is now run for all patches, currently non-voting
- https://review.openstack.org/#/c/455406/
- Will monitor and make sure it's stable
Stable backports (haleyb)
General information
Ihar created a page tracking all the potential backports from Mitaka to the stable releases. I have been going through it with the help of Swami to get stable/liberty, well, more stable. Bugs are removed from list as they merge.
We need to continue to be aggressive at proactively backporting fixes to the stable branches
This is a list of bugs with a fix that has been committed to the master branch, and are tagged with 'neutron-proactive-backport-potential+l3-dvr-backlog'
Open Discussion
- ???
Meeting commands
/join #openstack-meeting-alt
#startmeeting neutron_dvr
#chair Swami
#topic Announcements
#undo topic
#link https://wiki.openstack.org/wiki/Meetings/Neutron-DVR
#action haleyb will get something specific done this week
...
#endmeeting