Revision as of 07:01, 10 February 2016

Meetings

Weekly on Wednesday at 1500 UTC
IRC channel: #openstack-meeting-alt on freenode
Chair: Brian Haley (haleyb), Swaminathan Vasudevan (Swami)
Meetings, with their notes and logs, will be found under http://eavesdrop.openstack.org/meetings/neutron_dvr

Agenda

Meeting February 10th, 2016

Announcements (haleyb)

Haleyb out on Vaccation this week.
This week lot more patches have merged.
- Scheduling refactor patches have merged.
- SNAT-HA for DVR Server side patch have merged.

We still have a couple of high bugs.
- Live Migration Bug/ FloatingIP ping comes through the SNAT namespace.
- When going through them I'd like to try to get feedback on whether they are a MUST FIX, SHOULD FIX, or GOOD TO HAVE
- If it needs a rebase, do it soon after Jenkins notifies you
- If you just need reviews, ping people on irc or post a comment in the review

Topics for Discussion

Bugs (Swami)

All DVR bugs should be tagged and listed here: https://bugs.launchpad.net/neutron/+bugs?field.tag=l3-dvr-backlog

New Bugs this week

Categorized Bugs

Gate Test Failures

https://bugs.launchpad.net/neutron/+bug/1450604 (Medium) - Fix or stabilize Multi-node Job

Existing Functionality Broken Bugs

https://bugs.launchpad.net/neutron/+bug/1541406 - IPv6 Prefix Delegation does not work with DVR ( SHOULD FIX)

** https://review.openstack.org/#/c/277657/ ( Related Patch) - Needs Review.

https://bugs.launchpad.net/neutron/+bug/1533216 ( Low) - Item allocator should return same value for key (GOOD TO HAVE)
- https://review.openstack.org/266379 ( Related Patch) - Requires Review.

https://bugs.launchpad.net/neutron/+bug/1456073 (High) - Block Migration with FIP breaks with DVR (MUST FIX)
- https://review.openstack.org/275073 (Nova side change) - Needs review
- https://review.openstack.org/275420 (Neutron Server side DVR change) - Needs review
- https://review.openstack.org/#/c/260738/ (Needs review) ( L3 Agent side patch) - Needs review

- related bug https://bugs.launchpad.net/neutron/+bug/1414559 with fixes: (OVS drops RARP packets)
  - in neutron https://review.openstack.org/#/c/246898/ (Needs review) Notify vif plugged event ( related)
  - and nova https://review.openstack.org/#/c/246910/ (Needs review) Notify vif plugged event ( related)

https://bugs.launchpad.net/neutron/+bug/1462154 (High) - DVR pings to floatingip replied with fixed ips (SHOULD FIX)
- https://review.openstack.org/246855
- Doesn't seem to be making progress?

https://bugs.launchpad.net/neutron/+bug/1445255 (Low) - DVR Floatingip to unbound port does not work (Allowed address pair) (SHOULD FIX)
- https://review.openstack.org/#/c/254439/ (Needs review)

https://bugs.launchpad.net/neutron/+bug/1447227 (Low) - Connecting two or more distributed routers to a subnet. (No one is working on it) (GOOD TO HAVE)

Scale and Performance Impact Bugs

New Features Bugs

https://bugs.launchpad.net/neutron/+bug/1504039 - LinuxBridge DVR (Spec & code)

Refactor or Cleanup Bugs

https://bugs.launchpad.net/neutron/+bug/1531065 - (Low) duplicate fetch subnet_id in get_subnet_for_dvr (GOOD TO HAVE)
- https://review.openstack.org/#/c/263563/ - (Needs review)
https://bugs.launchpad.net/neutron/+bug/1538369 - Refactor request ( Not a real bug)

WishList Bugs

https://bugs.launchpad.net/neutron/+bug/1518819 - Ability to specify a gateway when adding a subnet without cidr (GOOD TO HAVE)
https://bugs.launchpad.net/neutron/+bug/1529439 - unify validate_agent_router_combination exceptions for dvr agent_mode (GOOD TO HAVE)
https://bugs.launchpad.net/neutron/+bug/1450067 (Low) - ML2 and L3 plugin exposes dvr extension even if ovs is unused. (No owners yet) (GOOD TO HAVE)

WatchList Bugs

https://bugs.launchpad.net/neutron/+bug/1444014 (Incomplete) (StaleDataError) - on Watch
https://bugs.launchpad.net/neutron/+bug/1521815 (Low)- (DVR functional Job failure) - on Watch state

Bugs Closed

https://bugs.launchpad.net/neutron/+bug/1365473 (High) - (DVR SNAT HA - Server Side) (Should change this to HIGH) Right now it is WishList (Merged)
https://bugs.launchpad.net/neutron/+bug/1540259 (Low)- Use list should be True DVR portbinding - Merged ( CAn be closed)
https://bugs.launchpad.net/neutron/+bug/1526175 (Medium) - L3 HA routers end up in DVR Node (medium) (Abandoned)
https://bugs.launchpad.net/neutron/+bug/1522824 (High) - Test_shelve_instance test failure. (MUST FIX) ( Merged)
- https://review.openstack.org/#/c/253569/ - Set PENDING_BUILD status for ports after update in DB - Abandoned in favor of https://review.openstack.org/#/c/215467 ( Merged)
https://bugs.launchpad.net/neutron/+bug/1510796 (Low) - Function sync_routers always calls get_dvr_sync_data in ha scenario Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1538163 (UNDECIDED) - Race condition in removing a router on dvr service port delete - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1524908 (Undecided - May be a Low) - Router may be removed from SNAT agent by accident - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1536110 (Undecided) - OVS agent should fail if can't get DVR mac address - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1535928 (Undecided) - Duplicate IPtables rule detected - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1508869 (Low) - handle port host change (live migration) - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1524020 (Medium) - DVR ARP update call count can be reduced. - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1524291 (Low) - Code duplication - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1512199 - (Medium) This bug has been confirmed and re-opened this week
- Seen in Master branch as well - probably introduced due to regression. - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1499785 (Low) - Static routes are not added to the router namespace - Merged (Can be closed)
https://bugs.launchpad.net/neutron/+bug/1499787 (Low) - Static routes are added to snat namespace without checking for gateway. - Merged (Can be closed)
- https://review.openstack.org/#/c/228026/ (reviewed - final comments addressed)
https://bugs.launchpad.net/neutron/+bug/1424096 (Undecided) - Delete port with admin context. - Merged (Can be closed)
- https://review.openstack.org/#/c/257938/ - (Patch in merge conflict - reviewed)
https://bugs.launchpad.net/neutron/+bug/1530179 -(Medium) get_subnet_for_dvr() returns wrong gateway mac - Merged (Can be closed)
- https://review.openstack.org/#/c/262574/ - (Needs review)
https://bugs.launchpad.net/neutron/+bug/1505575 Fatal memory consumption by neutron-server with DVR at scale - Merged (Can be closed)
- https://review.openstack.org/#/c/234067/ (sync in chunks)

Discuss a plan to address the gate failures (haleyb)

Current failure rate: https://goo.gl/L1WODG
- Single node check job failure rate has recovered from last week issues, same as non-dvr job as of today.
- Multi-node non-DVR job failures are still over 50% (migrate volume error)
- Multi-node DVR job over 70%
  - Filed https://bugs.launchpad.net/bugs/1538387 yesterday (OVS l2pop exception)
    - https://review.openstack.org/272986 out for review
  - Next noticeable issue: "DVR: Unable to retrieve subnet information for subnet_id"
    - https://launchpad.net/bugs/1454921
    - https://review.openstack.org/#/c/272025/ (needs update)
Debug the gate in the current failure state (Identify owners who can own this)
https://bugs.launchpad.net/neutron/+bug/1515360 - Tempest "SSHTimeout" failures (GOOD TO HAVE) - This patch seem to have some timing issues and so we are seeing more failures for the pings.
- https://review.openstack.org/#/c/247748/ (add debugging code to l3-agent) - Still pings are not consistent with the tempest.
https://drive.google.com/file/d/0B4kh-7VVPWlPMkdrQWFFdjNsSnM/view?usp=sharing - Logstash failures captured. Any logstash errors can be captured here

Performance/Scalability (obondarev)

https://blueprints.launchpad.net/neutron/+spec/improve-dvr-l3-agent-binding (SHOULD HAVE)
- https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/improve-dvr-l3-agent-binding
- All changes have been merged!

Open Discussion

Meeting commands

/join #openstack-meeting-alt
#startmeeting neutron_dvr
#topic Announcements
#undo topic
#link https://wiki.openstack.org/wiki/Meetings/Neutron-DVR
#action haleyb will get something specific done this week
#chair Swami
...
#endmeeting

@@ Line 8: / Line 8: @@
 = Agenda =
-== Meeting February 3rd, 2016 ==
+== Meeting February 10th, 2016 ==
 === Announcements (haleyb) ===
-* We still have a lot of bugs (below)
+* Haleyb out on Vaccation this week.
+* This week lot more patches have merged.
+** Scheduling refactor patches have merged.
+** SNAT-HA for DVR Server side patch have merged.
+* We still have a couple of high bugs.
+** Live Migration Bug/ FloatingIP ping comes through the SNAT namespace.
 ** When going through them I'd like to try to get feedback on whether they are a MUST FIX, SHOULD FIX, or GOOD TO HAVE
 ** If it needs a rebase, do it soon after Jenkins notifies you
@@ Line 18: / Line 24: @@
 == Topics for Discussion ==
-=== Live Migration issues with DVR (Swami) ===
+=== Bugs (Swami) ===
-* We have started a discussion on the live migration issue with DVR and how to address it from the neutron side. As a team we need to get consensus on this ideas before we work with the Nova team on this issue.
-**https://bugs.launchpad.net/neutron/+bug/1456073 (discussion thread)
-** Any updates - Are we all in sync with nova-neutron notifier requirement and also with the changes proposed for the neutron side to address the live migration.
-** Live migration was discussed with the Nova team during the mid-cycle meet-up. Based on the discussion we have upload a couple of patches for early review ( Work in progress patches)
-** https://review.openstack.org/275073 (Nova change)
-** https://review.openstack.org/275420 (Neutron Server side DVR change)
-** https://review.openstack.org/#/c/260738/ ( Neutron Agent side DVR change)
-** NOTE: Based on the current internal tests, these patches seem to fix the Live migration and floatingip issues.
-=== Bugs (Swami) ===
 All DVR bugs should be tagged and listed here: https://bugs.launchpad.net/neutron/+bugs?field.tag=l3-dvr-backlog
 ==== New Bugs this week ====
-* https://bugs.launchpad.net/neutron/+bug/1541406 - IPv6 Prefix Delegation does not work with DVR
-* https://bugs.launchpad.net/neutron/+bug/1540259 - Use list should be True DVR portbinding
-* https://bugs.launchpad.net/neutron/+bug/1538369 - Refactor request ( Not a real bug)
 ==== Categorized Bugs ====
 ==== Gate Test Failures ====
-* https://bugs.launchpad.net/neutron/+bug/1522824 (High) - Test_shelve_instance test failure. (MUST FIX)
-** https://review.openstack.org/#/c/253569/ - Set PENDING_BUILD status for ports after update in DB - Abandoned in favor of https://review.openstack.org/#/c/215467
 * https://bugs.launchpad.net/neutron/+bug/1450604 (Medium) - Fix or stabilize Multi-node Job
 ==== Existing Functionality Broken Bugs ====
+* https://bugs.launchpad.net/neutron/+bug/1541406 - IPv6 Prefix Delegation does not work with DVR ( SHOULD FIX)
+ ** https://review.openstack.org/#/c/277657/ ( Related Patch) - Needs Review.
+* https://bugs.launchpad.net/neutron/+bug/1533216 ( Low) - Item allocator should return same value for key (GOOD TO HAVE)
+** https://review.openstack.org/266379 ( Related Patch) - Requires Review.
 * https://bugs.launchpad.net/neutron/+bug/1456073 (High) - Block Migration with FIP breaks with DVR (MUST FIX)
-** related bug https://bugs.launchpad.net/neutron/+bug/1414559 with fixes:
+** https://review.openstack.org/275073 (Nova side change) - Needs review
-*** in neutron https://review.openstack.org/#/c/246898/ (Needs review)
+** https://review.openstack.org/275420 (Neutron Server side DVR change) - Needs review
-*** and nova https://review.openstack.org/#/c/246910/ (Needs review)
+** https://review.openstack.org/#/c/260738/ (Needs review) ( L3 Agent side patch) - Needs review
-*** https://review.openstack.org/#/c/259299/ (WIP)
-*** https://review.openstack.org/#/c/259171/ (WIP)
+** related bug https://bugs.launchpad.net/neutron/+bug/1414559 with fixes: (OVS drops RARP packets)
-*** https://review.openstack.org/#/c/260738/ (WIP)
+*** in neutron https://review.openstack.org/#/c/246898/ (Needs review) Notify vif plugged event ( related)
+*** and nova https://review.openstack.org/#/c/246910/ (Needs review) Notify vif plugged event ( related)
 * https://bugs.launchpad.net/neutron/+bug/1462154 (High) - DVR pings to floatingip replied with fixed ips (SHOULD FIX)
 ** https://review.openstack.org/246855
 ** Doesn't seem to be making progress?
 * https://bugs.launchpad.net/neutron/+bug/1445255 (Low) - DVR Floatingip to unbound port does not work (Allowed address pair) (SHOULD FIX)
 **https://review.openstack.org/#/c/254439/ (Needs review)
 * https://bugs.launchpad.net/neutron/+bug/1447227 (Low) - Connecting two or more distributed routers to a subnet. (No one is working on it) (GOOD TO HAVE)
@@ Line 64: / Line 64: @@
 ==== New Features Bugs ====
-* https://bugs.launchpad.net/neutron/+bug/1365473 (High) - (DVR SNAT HA - Server Side) (Should change this to HIGH) Right now it is WishList
-** https://review.openstack.org/#/c/143169/ (Needs core blessings) (MUST FIX)
 * https://bugs.launchpad.net/neutron/+bug/1504039 - LinuxBridge DVR (Spec & code)
@@ Line 71: / Line 70: @@
 * https://bugs.launchpad.net/neutron/+bug/1531065 - (Low)  duplicate fetch subnet_id in get_subnet_for_dvr (GOOD TO HAVE)
 ** https://review.openstack.org/#/c/263563/ - (Needs review)
+* https://bugs.launchpad.net/neutron/+bug/1538369 - Refactor request ( Not a real bug)
 ==== WishList Bugs ====
@@ Line 82: / Line 82: @@
 ==== Bugs Closed ====
+* https://bugs.launchpad.net/neutron/+bug/1365473 (High) - (DVR SNAT HA - Server Side) (Should change this to HIGH) Right now it is WishList (Merged)
+* https://bugs.launchpad.net/neutron/+bug/1540259 (Low)- Use list should be True DVR portbinding - Merged ( CAn be closed)
 * https://bugs.launchpad.net/neutron/+bug/1526175 (Medium) - L3 HA routers end up in DVR Node (medium) (Abandoned)
+* https://bugs.launchpad.net/neutron/+bug/1522824 (High) - Test_shelve_instance test failure. (MUST FIX) ( Merged)
+** https://review.openstack.org/#/c/253569/ - Set PENDING_BUILD status for ports after update in DB - Abandoned in favor of https://review.openstack.org/#/c/215467  ( Merged)
 * https://bugs.launchpad.net/neutron/+bug/1510796 (Low) - Function sync_routers always calls get_dvr_sync_data in ha scenario Merged (Can be closed)
 * https://bugs.launchpad.net/neutron/+bug/1538163 (UNDECIDED) - Race condition in removing a router on dvr service port delete - Merged (Can be closed)

Difference between revisions of "Meetings/Neutron-DVR"