Jump to: navigation, search

Difference between revisions of "Meetings/QATeamMeeting"

(Proposed Agenda for June 26 2014 (2200 UTC))
m (Agenda for next Office hours)
 
(605 intermediate revisions by 51 users not shown)
Line 1: Line 1:
 +
__TOC__
  
 
= Weekly QA Team meeting =
 
= Weekly QA Team meeting =
The OpenStack QA Team holds public weekly meetings in <code><nowiki>#openstack-meeting</nowiki></code>, '''Thursday alternating every week between 17:00 UTC and 22:00 UTC''.<br />
+
The [[QA|OpenStack Quality Assurance (QA)]] team Develop, maintain, and initiate tools and plans to ensure the upstream stability and quality of OpenStack, and its release readiness at any point during the release cycle.
Everyone interested in testing, quality assurance, performance engineering, etc, should attend!
 
  
The meeting minutes of can be found here: [http://eavesdrop.openstack.org/meetings/qa meeting minutes on everdrop]
+
The [[QA|OpenStack Quality Assurance (QA)]] team holds public weekly Office hours:
 +
* Channel:  <code><nowiki>#openstack-qa</nowiki></code>
 +
* Time: Tuesday every week at 15:00 UTC.
 +
* ical - http://eavesdrop.openstack.org/#QA_Team_Office_hours
  
''Rules for adding to the agenda'':
+
Everyone interested in testing, quality assurance, performance engineering, etc, should attend! The meeting minutes of previous meetings can be found here: [http://eavesdrop.openstack.org/meetings/qa meeting minutes on eavesdrop]
* anyone is free to add to proposed agenda, please do so by 1 hr before the meeting
+
 
 +
'''''Rules for adding to the agenda''''':
 +
* anyone is free to add to proposed agenda, please do so by 1 hr before the Office hours
 
* if you add an item to the agenda, please include your ircnick (i.e. sdague) to the end of your item, so we know who did, and who should lead the discussion
 
* if you add an item to the agenda, please include your ircnick (i.e. sdague) to the end of your item, so we know who did, and who should lead the discussion
  
=== Proposed Agenda for June 26 2014 (2200 UTC) ===
 
* Specs Review
 
* Blueprints
 
** Future of https://blueprints.launchpad.net/tempest/+spec/nova-api-test-inheritance ?(mtreinish)
 
** Status updates
 
* Grenade
 
* Neutron testing
 
* Bugs
 
* Critical Reviews
 
* Outstanding code reviews
 
** https://review.openstack.org/#/q/status:open+project:openstack/tempest,n,z
 
 
== Previous meetings ==
 
All previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/qa/.
 
 
=== Agenda for June 19 2014 (1700 UTC) ===
 
* Specs Review
 
** https://review.openstack.org/94473 (tempest config script)
 
*** What is going on with this spec? (submitted by boris42)
 
*** Does any one have code they are working on ahead of this spec being approved?
 
**  https://review.openstack.org/#/c/101232/ (restructure scenario - yfried)
 
* Blueprints
 
** Status updates
 
* Grenade
 
* Neutron full voting on icehouse and juno (afazekas)
 
* Neutron testing
 
** scenario specific gate (yfried)
 
*** sdague objects to some LBaaS scenario patches due to duration. could we move the scenarios to a separate gate?
 
* Bugs
 
* Critical Reviews
 
** Convert scenario test tearDown to addCleanup https://review.openstack.org/#/c/62101/ (mtreinish & yfried)
 
** Force connection auth as part of RemoteClient in scenario https://review.openstack.org/#/c/92573/ (yfried)
 
** Adds Cross Host Scenario https://review.openstack.org/77816 (yfried)
 
  
 +
'''''Tips to Chair the Office hours''''':
 +
* Start this meeting using: #startmeeting qa
 +
* If nothing much to discuss then we can always finish Office hours any time before 1 hour.
 +
* Announcement and Priority Items progress (first 2 items of Agenda) can be skipped and need not to be tracked every week.
 +
* End this meeting using: #endmeeting
  
=== Agenda for June 12 2014 (2200 UTC) ===
+
== Agenda for next Office hours ==
* Specs Review
+
PING LIST-
** Client response checking https://review.openstack.org/93037 -- What about xml?
+
* Announcement and Action Item (Optional)
*** a) Put response code checks using the response part of the schema for json in all xml clients (quite a bit more work)
+
* Zed Priority Items progress
*** b) Don't touch xml clients and accept that some (perhaps most) response checking for xml is lost when the current explicit checks in tests are removed
+
** go through the zed priority items list and try to plan the work / highlight milestones
* Blueprints
+
*** https://etherpad.opendev.org/p/qa-zed-priority
** Status updates
+
* OpenStack Events Updates and Planning
* Disabling nova v3 API tests by default - http://lists.openstack.org/pipermail/openstack-dev/2014-June/037370.html (mriedem)
+
** <This includes the OpenStack Summit and PTG updates and planning>
* Experimental tag (cyeoh)
+
** Summit & PTG
** tag for APIs which are thought to be stable, but for one cycle in extreme circumstances may have a breaking backwards incompatible change
+
*** latest PTG was held on April 4th-8th virtually - https://www.openstack.org/ptg/
*** Nova tasks API is probably an example
+
*** PTG etherpad tracking the topics for the Zed cycle
*** Experimental tag would indicate that the test is only run against the latest project devel tree
+
**** https://etherpad.opendev.org/p/qa-zed-ptg
* Neutron testing
+
* Gate Status / Fix Checks
* Bugs
+
** <not just QA projects gate but any other project gate also where QA team can help>
* Critical Reviews
+
** Gate Blocker Fix / Urgent Change
* Tempest tests for Barbican
+
*** https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade)
** blueprint - https://blueprints.launchpad.net/tempest/+spec/add-basic-tests-for-barbican
+
* Periodic jobs Status Checks
 +
** Periodic stable: https://zuul.openstack.org/builds?job_name=tempest-full-yoga&job_name=tempest-full-xena&job_name=tempest-full-wallaby-py3&job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&pipeline=periodic-stable
 +
** Periodic master: https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic
 +
* Distros check:
 +
** Centos 9: https://zuul.openstack.org/builds?job_name=tempest-full-centos-9-stream&job_name=devstack-platform-centos-9-stream&skip=0
 +
** Fedora: https://zuul.openstack.org/builds?job_name=devstack-platform-fedora-latest&skip=0
 +
** openEuler: https://zuul.openstack.org/builds?job_name=devstack-platform-openEuler-20.03-SP2+&skip=0
 +
** Debian: https://zuul.openstack.org/builds?job_name=devstack-platform-debian-bullseye&skip=0
 +
* Sub Teams highlights (Sub Teams means individual projects under QA program)
 +
** Tempest
 +
*** https://review.openstack.org/#/q/project:openstack/tempest+status:open
 +
** Patrole
 +
*** https://review.openstack.org/#/q/project:openstack/patrole+status:open
 +
** Devstack
 +
*** https://review.openstack.org/#/q/project:openstack/devstack+status:open
 +
** Grenade
 +
*** https://review.openstack.org/#/q/project:openstack/grenade+status:open
 +
** Hacking
 +
*** https://review.opendev.org/#/q/project:openstack/hacking+status:open
 +
** Changes with Review-Priority == +1
 +
*** https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade)
 
* Open Discussion
 
* Open Discussion
** Multi-node testing for migrations with Nova API?  What needs to happen? (mriedem)
+
** [everyone feel free to add their topics under Open Discussion prior to the Office hours with your irc nickname]
 +
** (gmann) This came up during pain point discussion in TC. With fewer contributors issues in QA (especially devstack where we have more incoming requests/changes), how about expanding the devstack team with potential candidates?  
 +
*** https://etherpad.opendev.org/p/pain-point-elimination
 +
* Bug Triage (last 30 min)
 +
** https://etherpad.openstack.org/p/qa-bug-triage-zed
  
=== Agenda for June 5 2014 (1700 UTC) ===
+
== Previous meetings ==
* Mid-Cycle Meetup (mtreinish)
+
* All previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/qa/.
** https://wiki.openstack.org/wiki/Qa_Infra_Meetup_2014
+
* Old agendas are stored on [[Meetings/QATeamMeeting/OldAgendas]]
* Unstable API testing in Tempest (mtreinish)
 
** https://review.openstack.org/95789
 
* Specs Review
 
** https://review.openstack.org/#/c/94473/
 
** https://review.openstack.org/#/c/97589/ (asselin)
 
* Blueprints
 
** Status updates
 
* Neutron testing
 
* Bugs
 
* Critical Reviews
 
 
 
=== Agenda for May 29 2014 (2200 UTC) ===
 
* Specs Review
 
* Blueprints
 
** Status updates
 
* Neutron testing
 
* Bugs
 
* Critical Reviews
 
 
 
=== Agenda for May 22 2014 (1700 UTC) ===
 
* Blueprint Purge (mtreinish)
 
* Specs Review
 
* Blueprints
 
** Status updates
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
* Critical Reviews
 
* Summit follow-up (andreaf)
 
 
 
 
 
=== Agenda for May 8 2014 (2200 UTC) ===
 
* Reminder about summit etherpads
 
** https://wiki.openstack.org/wiki/Summit/Juno/Etherpads#QA
 
* Proposal to move success response checking to clients (dkranz)
 
* Can we turn on voting of ironic jobs (recent creds change broke it)? (adam_g)
 
* Specs Review
 
* Blueprints
 
** Status updates
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
* Critical Reviews
 
** https://review.openstack.org/#/c/92573/ - connection verification as part of RemoteClient init
 
 
 
=== Agenda for April 24 2014 (1700 UTC) ===
 
* Oslo Liaison (mtreinish)
 
** https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons
 
* Summit sessions (mtreinish)
 
** https://etherpad.openstack.org/p/Juno-QA-design-summit-topics
 
* Specs Review
 
* Blueprints
 
** Current state
 
*** SSH stress test - https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floating-ip (julien-llp)
 
** Status updates from people
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
* Critical Reviews
 
 
 
=== Agenda for April 17 2014 (2200 UTC) ===
 
* Summit sessions (mtreinish)
 
** https://etherpad.openstack.org/p/Juno-QA-design-summit-topics
 
* Oslo Liaison (mtreinish)
 
** https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons
 
* Blueprints
 
** Current state
 
*** https://blueprints.launchpad.net/tempest/+spec/nova-api-attribute-test There is a question about whether
 
the returned dict schemas should set additionalProperties to false. Perhaps they should because adding one is supposed
 
to require an extension, but they do not in these reviews.
 
** Specs Review
 
** Status updates from people
 
* Neutron testing
 
** Work has continued in reviewing api tests. 19 patchsets merged out of 28 we are tracking for Icehouse
 
** Following patchsets require only one more +2 to merge
 
*** https://review.openstack.org/#/c/67547
 
*** https://review.openstack.org/#/c/63723
 
*** https://review.openstack.org/#/c/66541
 
** During past Monday Neutron weekly IRC meeting, a session during the design summit was proposed. Neutron core developers supported the idea. Session added here by mlavalle: http://summit.openstack.org/cfp/details/366
 
* Heat testing
 
* Bugs
 
* Critical Reviews
 
 
 
 
 
===  Agenda for April 10 2014 (1700 UTC) ===
 
* Summit sessions (mtreinish)
 
** https://etherpad.openstack.org/p/Juno-QA-design-summit-topics
 
* Blueprints
 
** Current state
 
** Specs Review
 
** Status updates from people
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
* Critical Reviews
 
 
 
=== Agenda for April 3 2014 (2200 UTC) ===
 
* Summit sessions (mtreinish)
 
** https://etherpad.openstack.org/p/Juno-QA-design-summit-topics
 
* Blueprints
 
** Current state
 
** Specs Review
 
** Status updates from people
 
*** https://blueprints.launchpad.net/tempest/+spec/cinder-v2-api-tests(zhikunliu)
 
****we have already enabled basic cinder v2 tests. Should we port all v1 tests to v2 like nova v2 to v3? Is it the right direction?
 
* Network Scenarios - call to arms (mlavalle&yfried)
 
* Tempest branching strategy (dkranz)
 
** Many distros, and now refstack, lag behind OpenStack releases. How can we enable more work to happen on master after a release?
 
*** Make sure tempest can run against previous releases as we do with client libraries?
 
*** Initial result of master against stable/havana nova-network (not including heat/ceilo/swift): http://paste.openstack.org/show/74966/
 
*** Whole transcript with error stacks: http://paste.openstack.org/show/74990/  148 failures out of 2009
 
*** Etherpad to track these issues here: https://etherpad.openstack.org/p/master-against-havana-errors
 
* Critical Reviews
 
 
 
=== Agenda for March 27 2014 (1700 UTC) ===
 
* QA Specs Repo Proposals (sdague)
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
* Critical Reviews
 
 
 
=== Agenda for March 20 2014 (2200 UTC) ===
 
* Blueprints
 
** QA Specs repository
 
** Current state
 
** Proposal
 
** Status updates from people
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
** Bug day review (maurosr)
 
* Critical Reviews
 
 
 
=== Agenda for March 13 2014 (1700 UTC) ===
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
* Running tempest as non-admin (dkranz)
 
* qa-specs gerrit repository
 
 
 
=== Agenda for March 6 2014 (2200 UTC) ===
 
* Finding a volunteer to organize a bug day
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
* Neutron testing
 
* Heat testing
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
* Strategy for creating schemas for negative auto-gen (dkranz)
 
 
 
===  Agenda for February 27 2014 (1700 UTC) ===
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
* Neutron testing
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
* What is our strategy for getting fail on log errors turned back on? (dkranz)
 
 
 
=== Agenda for February 20 2014 (2200 UTC) ===
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
*** https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/neutron-advanced-scenarios,n,z (yfried)
 
**** https://review.openstack.org/#/c/73973/ (Minor chages)
 
**** https://review.openstack.org/#/c/69567/ (Refactor cross_tenant)
 
**** https://review.openstack.org/#/c/66879/ (Refactor network_basic_ops)
 
**** https://review.openstack.org/#/c/52994/ (Prevent overlapping CIDRs)
 
**** https://review.openstack.org/#/c/69361/ (hotplug nic)
 
*** https://review.openstack.org/#/q/project:openstack/tempest+branch:master+topic:bp/multi-keystone-api-version-tests,n,z (andreaf)
 
**** https://review.openstack.org/#/c/74699/ (Fixes to auth.py - by maurosr)
 
**** https://review.openstack.org/#/c/68140/ (Auth unit tests - by maurosr)
 
**** https://review.openstack.org/#/c/73704/ (Refactor manager base class)
 
**** https://review.openstack.org/#/c/74387/ (Introduce Credentials class - pending unit tests)
 
* Neutron testing
 
** mlavalle won't be able to attend this meeting. This is the update:
 
*** api test has continued over the past few days, with 14 active contributors
 
*** Message was sent to the ML with a list of all the api tests patchsets to facilitate review by the neutron and Tempest core developers
 
**** Already seeing Tempest core reviewers approving and merging code. Thanks! Let's keep it up
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
* Other
 
** Service dependency decorators in tests (andreaf)
 
 
 
=== Agenda for February 13 2014 (1700 UTC) ===
 
* Removing nose support (mtreinish)
 
* Blueprints
 
** Current state
 
** Proposal
 
*** Fuzzy testing framework (mkoderer)
 
** Nova v3 testing in scenario tests (andreaf)
 
** Status updates from people
 
*** https://blueprints.launchpad.net/tempest/+spec/make-tempest-pluggable (vponomaryov)
 
**** There are 2 commits ready for review, that are related to bp:
 
**** https://review.openstack.org/#/c/62429/ (exceptions)
 
**** https://review.openstack.org/#/c/69985/ (config)
 
**** Need to decide, is it acceptable? Its pros and cons?
 
* Neutron testing
 
* API tests in the Neutron tree (marun)
 
** https://review.openstack.org/#/c/72585/ (api test in the neutron tree)
 
** https://review.openstack.org/#/c/72588/ (api test run with tempest rest clients)
 
* Graduation requirements for incubated project w.r.t Tempest (malini)
 
**From TC ' Project must have a basic devstack-gate job set up' - what constitutes a basic devstack-gate ?
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
 
 
=== Agenda for February 6 2014 (2200 UTC) ===
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
* Neutron testing
 
* When can we enable failing jobs with bogus log ERRORs (dkranz)
 
* Criteria for accepting tests that cannot run normally in the gate (dkranz)
 
** multi-node, require configuration not present in a current gate job
 
** What precisely is the policy for accepting in tempest? Is running nightly downstream acceptable?
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
 
 
=== Agenda for January 30 2014 (1700 UTC) ===
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
* Neutron testing
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
 
 
=== Agenda for January 23 2014 (2200 UTC) ===
 
* Gate Updates (sdague)
 
** Current State of Things
 
** New Gating model - http://lists.openstack.org/pipermail/openstack-dev/2014-January/025140.html
 
* Blueprints
 
** Current state
 
** Proposal
 
** Status updates from people
 
*** https://blueprints.launchpad.net/tempest/+spec/negative-tests
 
**** Any major comments/issues?
 
**** Do we really need to support xml in the future?
 
**** What is needed to get this in?
 
**** Should schemas go in classes as in the patch, or elsewhere?
 
* Neutron testing
 
** mlavalle to report latest status of api testing
 
* Bugs
 
** Critical / High bug review
 
* Critical Reviews
 
 
 
 
 
=== Agenda for January 16 2014 (1700 UTC) ===
 
* Blueprints
 
** Current state
 
*** Negative test status/discussion (dkranz)
 
*** Config verification status/discussion (rahmu)
 
** Proposal
 
*** Start using Six to prepare for an eventual Python 3 migration (rahmu)
 
** Status updates from people
 
* Bugs
 
** Critical / High bug review
 
* Neutron testing (mlavalle)
 
* Marconi testing (malini)
 
**https://review.openstack.org/#/q/topic:bp/add-basic-marconi-tests,n,z
 
* Ironic testing (agordeev)
 
** https://etherpad.openstack.org/p/IronicDevstackTesting
 
** related [http://lists.openstack.org/pipermail/openstack-dev/2014-January/024332.html discussion in openstack-dev], [http://lists.openstack.org/pipermail/openstack-dev/2014-January/024273.html initial message]
 
* Critical Reviews
 
 
 
=== Agenda for January 9 2014 (2200 UTC) ===
 
* Blueprints
 
** Current state
 
*** Negative test status/discussion (dkranz)
 
** Status updates from people
 
** Reconsideration of https://blueprints.launchpad.net/tempest/+spec/refactor-rest-client, according to changes (vponomaryov):
 
*** https://review.openstack.org/#/c/62923/
 
*** https://review.openstack.org/#/c/64948/
 
* Bugs
 
** Critical / High bug review
 
** Closing invalid Tempest bugs
 
* Neutron testing (mlavalle)
 
** Gap analysis for api tests is complete
 
*** Approach taken was very detailed and assumed that, to the extent possible, we want to test every attribute, as many options as possible and every operation
 
*** Negative tests were removed from the gap analysis, in light of "generative" tool under development in Tempest
 
*** email was sent on 1/6 to the ML to remind the community about this effort and provide an update
 
*** Seeing people assigning themselves tests from the gap analysis, but not as many as desirable. mlavalle has started assigning himself tests from the list. We can also reduce the gap next week during the sprint in Montreal
 
** Progress in creating new stable Tempest tests
 
*** Recently merged
 
**** https://review.openstack.org/#/c/59029/ rossella_s (api xml client)
 
**** https://review.openstack.org/#/c/61529/ rossella_s (api floating ip's)
 
**** https://review.openstack.org/#/c/62614/ enikanorov (api refactor network client)
 
**** https://review.openstack.org/#/c/55133/ mouad benchchaoui (api)
 
*** In review cycle:
 
**** https://review.openstack.org/#/c/63723/ Ann Kamyshnikova (api floating ip)
 
**** https://review.openstack.org/#/c/61118/ Ann Kamyshnikova (api lbaas)
 
**** https://review.openstack.org/#/c/63627/ yfried (scenario floating ip's)
 
**** https://review.openstack.org/#/c/58697/ Elena Ezhova (scenario lbaas)
 
**** https://review.openstack.org/#/c/65120/ Miguel Lavalle (lbaas agent scheduler)
 
**** https://review.openstack.org/#/c/65120/ yfried (api fwaas)
 
**** https://review.openstack.org/#/c/55146/ yfried (scenario network connectivity)
 
**** https://review.openstack.org/#/c/62962/ salv-orlando (make router creation for tenant isolation optional)
 
**** https://review.openstack.org/#/c/60008/ Evgeny Fedoruk (Extending quota support for neutron LBaaS entities)
 
**** https://review.openstack.org/#/c/50542/ ChenZheng (Test for the neutron api with provider extension)
 
**** https://review.openstack.org/#/c/63625/ QianLin
 
**** https://review.openstack.org/#/c/63999/2 Ann Kamyshnikova (external network extension)
 
**** https://review.openstack.org/#/c/64271 Elena Ezhova  (api binding extended attributes for ports)
 
* Critical Reviews
 
 
 
=== Agenda for December 19 2013 (1700 UTC) ===
 
* Cancelling meetings for the rest of the year (mtreinish)
 
* Blueprints
 
** Current state
 
** Status updates from people
 
** Needing some attention (gfidente)
 
*** https://blueprints.launchpad.net/tempest/+spec/fail-gate-on-log-errors (close or not?)
 
*** https://blueprints.launchpad.net/tempest/+spec/quantum-basic-api (seems to me pretty hard to track/implement, yet useful)
 
*** https://blueprints.launchpad.net/tempest/+spec/refactor-rest-client (there is actual work here, approve?)
 
*** https://blueprints.launchpad.net/tempest/+spec/keystoneclient-api (approve/low?)
 
*** https://blueprints.launchpad.net/tempest/+spec/multi-keystone-api-version-tests (maybe we should just use latest?)
 
*** https://blueprints.launchpad.net/tempest/+spec/swift-test-without-keystone (is it a good idea at all?)
 
* Bugs (sdague)
 
** Critical / High bug review
 
** Closing invalid Tempest bugs
 
* Neutron testing
 
* Bug status
 
* On new tests tracking (plus and cons of wiki/etherpad/blueprint) (gfidente)
 
* Critical Reviews
 
 
 
=== Agenda for December 12 2013 (2200 UTC) ===
 
 
 
* Blueprints
 
** Current state
 
** Status updates from people
 
* Neutron testing
 
** Progress on API tests gap analysis in https://etherpad.openstack.org/p/icehouse-summit-qa-neutron (mlavalle)
 
** Report on https://bugs.launchpad.net/neutron/+bug/1251448 (sixth top bug in the gate this week) and its relationship to https://bugs.launchpad.net/tempest/+bug/1259282 (mlavalle)
 
* Bug status
 
** How should we handle bug traffic after we finish the current purge to make sure the noise doesn't come back?
 
** Rotating dispatcher for incoming bugs?
 
** Ask people to put a logstash query in bug tickets that are about test failures?
 
** https://etherpad.openstack.org/p/tempest-bug-triage
 
* Critical Reviews
 
* We should consider putting a fast-track on heat and ceilometer tests since they are integrated but lacking. (dkranz)
 
* Should we have more specialization on the Core review team? e.g. I am comfortable with the nova v3 patches, but no where on neutron. (dkranz)
 
 
 
=== Agenda for December 5 2013 ===
 
* Meeting time change proposal
 
* Getting additional folks signed up as point people
 
** Meeting organizer - mtreinish voluteered
 
** Blueprint Triage point person - need volunteer
 
** Bug Triage point person - need volunteer
 
* Blueprints
 
** Current state
 
** Status updates from people
 
* Neutron testing
 
** Progress on API tests gap analysis in https://etherpad.openstack.org/p/icehouse-summit-qa-neutron (mlavalle)
 
** Wiki page for new comers on how to contribute Neutron API tests (mlavalle)
 
* Critical Reviews
 
* How can we make sure reviews don't slide >1 week (dkranz)
 
 
 
=== Agenda for November 21 2013 ===
 
*  Blueprints
 
* Negative testing (mkoderer/dkranz)
 
* Neutron testing: Discussion of what needs to happen for the sprint in January to take place / succeed? What do we need to accomplish by mid-December to make sure the sprint happens? (mlavalle)
 
* Critical Reviews
 
 
 
=== Agenda for November 14 2013 ===
 
* Possible Meeting Time change to include .jp (sdague)
 
* Blueprints
 
** Blueprints breakdown (sdague)
 
** Negative testing (mkoderer/dkranz)
 
* Neutron testing
 
** Negative tests coverage: neutron status (EmilienM)
 
** Progress on https://bugs.launchpad.net/swift/+bug/1224001 (mlavalle)
 
* How should we sync the teams (neutron/scenario test)? One BP with work items, bugs or ML? (mkoderer)
 
* Onboarding documentation - where should we put it? (mkoderer)
 
* Critical Reviews (sdague)
 
 
 
=== Agenda for October 24 2013 ===
 
* Design Summit Schedule (sdague)
 
** http://icehousedesignsummit.sched.org/ - figure out if there are last minute changes we need (note: many time blocks are fixed due to conflicts, so wiggle room is limited)
 
* Neutron job status (mtreinish)
 
* Scope and place for performance testing such as Rally (dkranz)
 
* Status and roll-out plan for failing the gate on log errors (dkranz)
 
* State of 'smoke' tagging: can we make it useful? (dkranz)
 
* Tempest config reorg and cleanup (mtreinish)
 
 
 
=== Agenda for October 17 2013 ===
 
* Unblocking of Ceilometer QA testing (jd__, sileht)
 
** https://review.openstack.org/#/c/49394/
 
 
 
=== Agenda for October 10 2013 ===
 
* Blueprints (sdague)
 
** started cleaning up blueprints, need to start a purge of all the Unknown items
 
* Neutron job status (sdague)
 
* Elastic Recheck Top issues (mtreinish)
 
* Stable branch timing (sdague)
 
* Design Summit Initial Planning (sdague)
 
** QA sessions needed at DS - https://etherpad.openstack.org/icehouse-qa-session-planning
 
* Lot's of new negative tests. Do we want that? (dkranz)
 
** some are "validating strings", like passing long strings or checking for invalid vs. non existent uuids (gfidente)
 
** some are "traversing" the components, like ensuring we can't attach resources to non existent servers (gfidente)
 
** I've the feeling they should not be tagged as 'smoke' ... and maybe a lot not even 'gate'?
 
* Rollout plan for failing builds that pass tempest but with spurious log ERRORs (dkranz)
 
** https://review.openstack.org/#/c/50795/
 
 
 
=== Agenda for October 3 2013 ===
 
* Blueprints (sdague)
 
** started cleaning up blueprints, need to start a purge of all the Unknown items
 
* Neutron job status (sdague)
 
* Elastic Recheck Top issues (mtreinish)
 
* Stable branch timing (sdague)
 
* Design Summit Initial Planning (sdague)
 
** QA sessions needed at DS - https://etherpad.openstack.org/icehouse-qa-session-planning
 
 
 
=== Agenda for September 26 2013 ===
 
* Blueprints (sdague)
 
** neutron testing status (mlavalle)
 
** Bogus errors in logs (dkranz)
 
* rework "skip test" functionality, reprises mkoderer's proposal (gfidente)
 
* Critical Reviews (sdague)
 
=== Agenda for September 19 2013 ===
 
* Blueprints (mtreinish)
 
** neutron testing status (mlavalle)
 
* Critical Reviews (mtreinish)
 
* How to handle bug fixes in launchpad (mtreinish)
 
* Bogus errors in logs one more time (dkranz)
 
 
 
=== Agenda for September 12 2013 ===
 
* Leader for next week's meeting - sdague on a plane back from LinuxCon during this time
 
* New core members (sdague)
 
* Summit planning (sdague)
 
** my intent is to wait until October to evaluate the agenda we want, however as ideas come forward now feel free to register
 
* Blueprints (sdague)
 
* neutron testing status (mlavalle)
 
* whitebox test removal (sdague)
 
** need a volunteer to ensure these tests are covered in Nova unit tests
 
* Critical Reviews (sdague)
 
* OpenStack qa repo for test plans that can be pointed at from blueprints (mkollaro)
 
 
 
=== Agenda for August 29 2013 ===
 
 
 
* Blueprints (mtreinish)
 
** testr status (mtreinish)
 
*** Do we need a way to "incubate" new tests to avoid flakiness with parallel testr? (dkranz)
 
*** Now that parallel testr is in place (wow) maybe we should organize another bug triage day? (gfidente)
 
** stress tests (mkoderer)
 
** neutron testing status (mlavalle)
 
* Update on slow heat gating job (dkranz)
 
* Critical Reviews (mtreinish)
 
* Testing client library compatibility (dkranz)
 
* Devstack independent tempest usage (afazekas)
 
* BP proposal: rework "skip test" functionaly (mkoderer)
 
 
 
 
 
=== Agenda for August 22 2013 ===
 
* Blueprints (mtreinish)
 
** testr parallel status (mtreinish)
 
** stress tests (mkoderer)
 
* Critical Reviews (mtreinish)
 
* Heat test merge (afazekas)
 
* Devstack independent tempest usage (afazekas)
 
* Adding test plans for new features into the tempest repo (mkollaro)
 
 
 
=== Agenda for August 15 2013 ===
 
* Blueprints (mtreinish)
 
** testr parallel status (mtreinish)
 
** stress tests (mkoderer)
 
* Critical Reviews (mtreinish)
 
 
 
=== Agenda for August 8 2013 ===
 
* Volunteers to run meeting while sdague out (sdague)
 
* Blueprints (sdague)
 
* Critical Reviews (sdague)
 
* Stress tests scope and behaviour, also see reviews on https://review.openstack.org/#/c/39752 (gfidente)
 
* Status of slow heat tests (dkranz)
 
 
 
=== Agenda of August 1 2013 ===
 
''See the [http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-08-01-16.59.html meeting minutes for July 1 on eavesdrop].''
 
 
 
* Blueprints (sdague)
 
* Critical Reviews (sdague)
 
* Call for more Core reviewers (dkranz)
 
 
 
=== Agenda of July 25 2013 ===
 
''See the [http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-07-25-17.00.html meeting minutes for July 25 on eavesdrop].''
 
 
 
* Blueprints (sdague)
 
** Current state of implementation
 
** Stress Tests https://blueprints.launchpad.net/tempest/+spec/stress-tests (mkoderer)
 
* WebDav status codes in nova? Consistently using the 404 or 422 on actions.
 
* Adding test cases with skip attribute vote? Exact rule (afazekas)
 
* py26 compatibility (afazekas)
 
* Consistent reviewing (afazekas)
 
* Critical Reviews (sdague)
 
* Separate heat job and slow tests in general (dkranz)
 
 
 
 
 
===Agenda of July 18 2013 ===
 
''See the [http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-07-18-17.00.html meeting minutes for July 18 on eavesdrop].''
 
 
 
* Blueprints (sdague)
 
** Havana-2 status check in - https://launchpad.net/tempest/+milestone/havana-2 (sdague .. et al)
 
** Testr Progress - https://blueprints.launchpad.net/tempest/+spec/speed-up-tempest (mtreinish)
 
** Stress Tests (new Blueprint and findings) - https://blueprints.launchpad.net/tempest/+spec/stress-tests (mkoderer)
 
* Mailing list move (voting on moving to -dev) (sdague)
 
* White Box Tests (owner for discussion?)
 
** Should they really be unit tests?
 
** Current state of implementation
 
** Integration in CI
 
** Adding test cases with skip attribute (afazekas)
 
* Critical Reviews (sdague)
 
 
 
=== Agenda of July 11 2013 ===
 
''See the [http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-07-11-17.03.html meeting minutes for July 11 on eavesdrop].''
 
 
 
* QA Program Business
 
** PTL Nominations close - see if we need to run election
 
** Program Description
 
* Blueprints
 
** Havana-2 status check in - https://launchpad.net/tempest/+milestone/havana-2
 
** Testr Progress - https://blueprints.launchpad.net/tempest/+spec/speed-up-tempest
 
* Critical Reviews
 
* Stress Tests in CI
 
* Full Open Stack networking gate job
 
* New stress tests in tempest --[[User:M-koderer|m-koderer]] ([[User talk:M-koderer|talk]]) 16:36, 11 July 2013 (UTC)
 
 
 
 
 
=== Agenda of June 27 2013 ===
 
''See the [http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-06-27-17.01.html meeting minutes for June 27 on eavesdrop].''
 
 
 
* ''nothing in Agenda ... only QA as OpenStack Program, QA Leader and testr topics were discussed little bit''
 

Latest revision as of 07:54, 9 May 2022

Weekly QA Team meeting

The OpenStack Quality Assurance (QA) team Develop, maintain, and initiate tools and plans to ensure the upstream stability and quality of OpenStack, and its release readiness at any point during the release cycle.

The OpenStack Quality Assurance (QA) team holds public weekly Office hours:

Everyone interested in testing, quality assurance, performance engineering, etc, should attend! The meeting minutes of previous meetings can be found here: meeting minutes on eavesdrop

Rules for adding to the agenda:

  • anyone is free to add to proposed agenda, please do so by 1 hr before the Office hours
  • if you add an item to the agenda, please include your ircnick (i.e. sdague) to the end of your item, so we know who did, and who should lead the discussion


Tips to Chair the Office hours:

  • Start this meeting using: #startmeeting qa
  • If nothing much to discuss then we can always finish Office hours any time before 1 hour.
  • Announcement and Priority Items progress (first 2 items of Agenda) can be skipped and need not to be tracked every week.
  • End this meeting using: #endmeeting

Agenda for next Office hours

PING LIST-

Previous meetings