Jump to: navigation, search

Meetings/QATeamMeeting

Contents

Weekly QA Team meeting

The OpenStack QA Team holds public weekly meetings in #openstack-meeting, 'Thursday alternating every week between 17:00 UTC and 09:00 UTC.
Everyone interested in testing, quality assurance, performance engineering, etc, should attend!

The meeting minutes of can be found here: meeting minutes on eavesdrop

Rules for adding to the agenda:

  • anyone is free to add to proposed agenda, please do so by 1 hr before the meeting
  • if you add an item to the agenda, please include your ircnick (i.e. sdague) to the end of your item, so we know who did, and who should lead the discussion

Proposed Agenda for August 27th 2015 (1700 UTC)

Previous meetings

All previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/qa/.

Agenda for August 20th 2015 (0900 UTC)

Agenda for August 13th 2015 (1700 UTC)


Agenda for August 6th 2015 (0900 UTC)

Agenda for July 30th 2015 (1700 UTC)

Agenda for July 16th 2015 (1700 UTC)

Agenda for July 9th 2015 (2200 UTC)

Agenda for July 2nd 2015 (1700 UTC)

Agenda for June 25th 2015 (2200 UTC)

Agenda for June 18th 2015 (17000 UTC)

  • Specs Reviews
  • Blueprints
    • Status Updates
  • Deprecation of user account options in tempest.conf
    • Really only need test-accounts for real clouds with cleanup
    • Tenant isolation is good for isolating failures
    • How/when can we deprecate?
      • Add gate job with test accounts
      • Stop trying to support non-isolated user/demo accounts in tools (cleanup, tempest config, etc.)
  • DevStack
  • Grenade
  • Bugs
  • Critical Reviews

Agenda for June 11th 2015 (2200 UTC)

Agenda for June 4th 2015 (1700 UTC)

Agenda for May 28th 2015 (2200 UTC)

Agenda for May 14th 2015 (2200 UTC)

Agenda for May 7th 2015 (1700 UTC)

Agenda for April 30th 2015 (2200 UTC)

Agenda for April 23rd 2015 (1700 UTC)

Agenda for April 16th 2015 (2200 UTC)

Agenda for April 9th 2015 (1700 UTC)

Agenda for April 2nd 2015 (2200 UTC)

Agenda for March 19th 2015 (2200 UTC)

Agenda for March 12th 2015 (1700 UTC)

Agenda for March 5th 2015 (2200 UTC)

Regular agenda if there is sufficient time after removal reviews:

Agenda for February 26th 2015 (1700 UTC)

Agenda for February 19th 2015 (2200 UTC)

Agenda for February 12th 2015 (1700 UTC)

Agenda for February 5th 2015 (2200 UTC)

Agenda for January 22nd 2015 (2200 UTC)

Agenda for January 15th 2015 (1700 UTC)

Agenda for January 8th 2015 (2200 UTC)

Agenda for December 18th 2014 (1700 UTC)

Agenda for December 11th 2014 (2200 UTC)

  • Spec Reviews
  • Blueprints
    • Status Updates
  • Proposed plan for moving neutron rest client to tempest lib (dkranz)
    • Refactor rest client so it can be separated from tempest
    • Same for neutron client, including applying the clients-return-one-value spec
      • This requires a "big bang" so we need to stop merging neutron patches while this is in review
    • Copy rest and neutron client to neutron and get it working there, fixing things as needed
    • Copy any changes back to tempest
    • Move rest and neutron client to tempest-lib
    • neutron replaces copied client code with reference to tempest-lib
  • DevStack
  • Grenade
  • Bugs
  • Critical Reviews

Agenda for December 4th 2014 (1700 UTC)

  • Tagging removal of Tempest XML support? (mtreinish)
  • Spec Reviews
  • Blueprints
    • Status Updates
  • DevStack
  • Grenade
  • Future of "large-ops" testing
    • does current test need 6 gates?
    • alternate test in stress/scenario
  • Bugs
  • Critical Reviews

Agenda for November 20th 2014 (1700 UTC)

Agenda for October 30th 2014 (2200 UTC)

Agenda for October 23rd 2014 (1700 UTC)

Agenda for October 16th 2014 (2200 UTC)


Agenda for October 9th 2014 (1700 UTC)

Agenda for October 2nd 2014 (2200 UTC)

  • tempest-2 tag (mtreinish)
  • Blueprints
    • Status Updates
      • bp resource-cleanup: 4 migrate_* patches to go, plus hacking rule and drop safe_setup, then first phase is complete. Cleanup of resource_cleanup methods going on in parallel. Phase 2 to start next week.
      • bp test-accounts: non-isolated jobs verified ok. Using pre-provisioned credentials still fails because of credentials leaks. Wait on resource-cleanup bp phase one to be done, retest and if ok we can merge
      • bp tempest-client-scenarios: masayuki's bp - I worked last week on the cleanup patches, and thanks to all the reviews, they are all merged, except for the cleanup of requirements.txt proposed by masayuki.
  • DevStack
  • Grenade
  • Bugs
  • Critical Reviews
  • Summit topic brainstorming

Agenda for September 25th 2014 (1700 UTC)

Agenda for September 18th 2014 (2200 UTC)

Agenda for September 11th 2014 (1700 UTC)

  • Summit topic brainstorming (mtreinish)
  • Results of Bug Day
    • Went from 124 New to 71. Not so much participation.
    • We need more real-time triage. Rotating weekly bug triage?
  • Specs Review
  • Blueprints
    • Status Updates
    • First tempest-lib release (mtreinish)
  • Devstack
  • Grenade
  • Bugs
  • Critical Reviews
    • Move success response checking of Swift to tempest clients https://review.openstack.org/#/c/117193/ (dmorita)
      • Swift's core members say Swift's policy of response code is responding 2xx series on success. They believe they should not define one or two specific code(s) for successful response.
      • On the other hand, Kenichi makes an assertion that OpenStack's policy says a response code should not be changed, so we should make tests more strictly.
      • I can understand both assertions but we have to decide which idea to be chosen.

Agenda for September 4th 2014 (2200 UTC)

Agenda for August 28 2014 (1700 UTC)

Agenda for August 21 2014 (2200 UTC)

  • Devstack now part of the QA program
  • Starting a QA liaison system (like oslo) (mtreinish)
  • Can we start using smoke tag for its intended purpose? (dkranz)
    • Except in neutron which still uses smoke in icehouse runs
  • Specs Review
  • Blueprints
    • Status Updates
  • Grenade
  • Neutron testing
  • Bugs
  • Critical Reviews

Agenda for August 14 2014 (1700 UTC)

  • Name for tempest library (mtreinish)
  • Specs Review
  • Blueprints
    • Status Updates
  • Grenade
  • Neutron testing
  • Bugs
  • Critical Reviews

Agenda for July 31 2014 (1700 UTC)

  • Specs Review
  • Prioritizing Neutron over Nova-Network
  • Blueprints
    • Status Updates
  • Grenade
  • Neutron testing
  • Bugs
  • Critical Reviews
  • No meeting next week

Agenda for July 24 2014 (2200 UTC)

  • Volunteer to run next 2 meetings on July 31st and August 7th (mtreinish)
  • Mid-cycle Meetup Summary (mtreinish)
  • Blueprints
  • Specs Review
  • Now that there is agreement to move api/func testing to projects, should we start that (virtually) now? (dkranz)
    • Reclaim smoke tag (except for neutron until neutron-full is enabled everywhere)
    • Use smoke to tag a small set of sanity api tests or those with some cross-project implications
    • Start gating each project with all scenario plus full api tests of its test directory, and smoke for the others
  • Grenade
  • Neutron testing
  • Bugs
  • Critical Reviews
  • Proposal of temp review policies (andreaf - I'll try to make it for the meeting, else details below)
    • New scenario tests shall use tempest client. Rebase on in-flight bp changes?
    • New API tests shall not include return code validation. I think introducing return code validation should be a separate patch as it can be reviewed and tested independently
    • New scenario / API neutron tests on hold (no +A) until full job voting on neutron and tempest


Agenda for July 10 2014 (2200 UTC)

Agenda for July 3 2014 (1700 UTC)

Agenda for June 26 2014 (2200 UTC)

Agenda for June 19 2014 (1700 UTC)


Agenda for June 12 2014 (2200 UTC)

  • Specs Review
    • Client response checking https://review.openstack.org/93037 -- What about xml?
      • a) Put response code checks using the response part of the schema for json in all xml clients (quite a bit more work)
      • b) Don't touch xml clients and accept that some (perhaps most) response checking for xml is lost when the current explicit checks in tests are removed
  • Blueprints
    • Status updates
  • Disabling nova v3 API tests by default - http://lists.openstack.org/pipermail/openstack-dev/2014-June/037370.html (mriedem)
  • Experimental tag (cyeoh)
    • tag for APIs which are thought to be stable, but for one cycle in extreme circumstances may have a breaking backwards incompatible change
      • Nova tasks API is probably an example
      • Experimental tag would indicate that the test is only run against the latest project devel tree
  • Neutron testing
  • Bugs
  • Critical Reviews
  • Tempest tests for Barbican
  • Open Discussion
    • Multi-node testing for migrations with Nova API? What needs to happen? (mriedem)

Agenda for June 5 2014 (1700 UTC)

Agenda for May 29 2014 (2200 UTC)

  • Specs Review
  • Blueprints
    • Status updates
  • Neutron testing
  • Bugs
  • Critical Reviews

Agenda for May 22 2014 (1700 UTC)

  • Blueprint Purge (mtreinish)
  • Specs Review
  • Blueprints
    • Status updates
  • Neutron testing
  • Heat testing
  • Bugs
  • Critical Reviews
  • Summit follow-up (andreaf)


Agenda for May 8 2014 (2200 UTC)

Agenda for April 24 2014 (1700 UTC)

Agenda for April 17 2014 (2200 UTC)

the returned dict schemas should set additionalProperties to false. Perhaps they should because adding one is supposed to require an extension, but they do not in these reviews.


Agenda for April 10 2014 (1700 UTC)

Agenda for April 3 2014 (2200 UTC)

Agenda for March 27 2014 (1700 UTC)

  • QA Specs Repo Proposals (sdague)
  • Blueprints
    • Current state
    • Proposal
    • Status updates from people
  • Neutron testing
  • Heat testing
  • Bugs
  • Critical Reviews

Agenda for March 20 2014 (2200 UTC)

  • Blueprints
    • QA Specs repository
    • Current state
    • Proposal
    • Status updates from people
  • Neutron testing
  • Heat testing
  • Bugs
    • Bug day review (maurosr)
  • Critical Reviews

Agenda for March 13 2014 (1700 UTC)

  • Blueprints
    • Current state
    • Proposal
    • Status updates from people
  • Neutron testing
  • Heat testing
  • Bugs
    • Critical / High bug review
  • Critical Reviews
  • Running tempest as non-admin (dkranz)
  • qa-specs gerrit repository

Agenda for March 6 2014 (2200 UTC)

  • Finding a volunteer to organize a bug day
  • Blueprints
    • Current state
    • Proposal
    • Status updates from people
  • Neutron testing
  • Heat testing
  • Bugs
    • Critical / High bug review
  • Critical Reviews
  • Strategy for creating schemas for negative auto-gen (dkranz)

Agenda for February 27 2014 (1700 UTC)

  • Blueprints
    • Current state
    • Proposal
    • Status updates from people
  • Neutron testing
  • Bugs
    • Critical / High bug review
  • Critical Reviews
  • What is our strategy for getting fail on log errors turned back on? (dkranz)

Agenda for February 20 2014 (2200 UTC)

Agenda for February 13 2014 (1700 UTC)

Agenda for February 6 2014 (2200 UTC)

  • Blueprints
    • Current state
    • Proposal
    • Status updates from people
  • Neutron testing
  • When can we enable failing jobs with bogus log ERRORs (dkranz)
  • Criteria for accepting tests that cannot run normally in the gate (dkranz)
    • multi-node, require configuration not present in a current gate job
    • What precisely is the policy for accepting in tempest? Is running nightly downstream acceptable?
  • Bugs
    • Critical / High bug review
  • Critical Reviews

Agenda for January 30 2014 (1700 UTC)

  • Blueprints
    • Current state
    • Proposal
    • Status updates from people
  • Neutron testing
  • Bugs
    • Critical / High bug review
  • Critical Reviews

Agenda for January 23 2014 (2200 UTC)


Agenda for January 16 2014 (1700 UTC)

Agenda for January 9 2014 (2200 UTC)

Agenda for December 19 2013 (1700 UTC)

Agenda for December 12 2013 (2200 UTC)

Agenda for December 5 2013

  • Meeting time change proposal
  • Getting additional folks signed up as point people
    • Meeting organizer - mtreinish voluteered
    • Blueprint Triage point person - need volunteer
    • Bug Triage point person - need volunteer
  • Blueprints
    • Current state
    • Status updates from people
  • Neutron testing
  • Critical Reviews
  • How can we make sure reviews don't slide >1 week (dkranz)

Agenda for November 21 2013

  • Blueprints
  • Negative testing (mkoderer/dkranz)
  • Neutron testing: Discussion of what needs to happen for the sprint in January to take place / succeed? What do we need to accomplish by mid-December to make sure the sprint happens? (mlavalle)
  • Critical Reviews

Agenda for November 14 2013

  • Possible Meeting Time change to include .jp (sdague)
  • Blueprints
    • Blueprints breakdown (sdague)
    • Negative testing (mkoderer/dkranz)
  • Neutron testing
  • How should we sync the teams (neutron/scenario test)? One BP with work items, bugs or ML? (mkoderer)
  • Onboarding documentation - where should we put it? (mkoderer)
  • Critical Reviews (sdague)

Agenda for October 24 2013

  • Design Summit Schedule (sdague)
  • Neutron job status (mtreinish)
  • Scope and place for performance testing such as Rally (dkranz)
  • Status and roll-out plan for failing the gate on log errors (dkranz)
  • State of 'smoke' tagging: can we make it useful? (dkranz)
  • Tempest config reorg and cleanup (mtreinish)

Agenda for October 17 2013

Agenda for October 10 2013

  • Blueprints (sdague)
    • started cleaning up blueprints, need to start a purge of all the Unknown items
  • Neutron job status (sdague)
  • Elastic Recheck Top issues (mtreinish)
  • Stable branch timing (sdague)
  • Design Summit Initial Planning (sdague)
  • Lot's of new negative tests. Do we want that? (dkranz)
    • some are "validating strings", like passing long strings or checking for invalid vs. non existent uuids (gfidente)
    • some are "traversing" the components, like ensuring we can't attach resources to non existent servers (gfidente)
    • I've the feeling they should not be tagged as 'smoke' ... and maybe a lot not even 'gate'?
  • Rollout plan for failing builds that pass tempest but with spurious log ERRORs (dkranz)

Agenda for October 3 2013

  • Blueprints (sdague)
    • started cleaning up blueprints, need to start a purge of all the Unknown items
  • Neutron job status (sdague)
  • Elastic Recheck Top issues (mtreinish)
  • Stable branch timing (sdague)
  • Design Summit Initial Planning (sdague)

Agenda for September 26 2013

  • Blueprints (sdague)
    • neutron testing status (mlavalle)
    • Bogus errors in logs (dkranz)
  • rework "skip test" functionality, reprises mkoderer's proposal (gfidente)
  • Critical Reviews (sdague)

Agenda for September 19 2013

  • Blueprints (mtreinish)
    • neutron testing status (mlavalle)
  • Critical Reviews (mtreinish)
  • How to handle bug fixes in launchpad (mtreinish)
  • Bogus errors in logs one more time (dkranz)

Agenda for September 12 2013

  • Leader for next week's meeting - sdague on a plane back from LinuxCon during this time
  • New core members (sdague)
  • Summit planning (sdague)
    • my intent is to wait until October to evaluate the agenda we want, however as ideas come forward now feel free to register
  • Blueprints (sdague)
  • neutron testing status (mlavalle)
  • whitebox test removal (sdague)
    • need a volunteer to ensure these tests are covered in Nova unit tests
  • Critical Reviews (sdague)
  • OpenStack qa repo for test plans that can be pointed at from blueprints (mkollaro)

Agenda for August 29 2013

  • Blueprints (mtreinish)
    • testr status (mtreinish)
      • Do we need a way to "incubate" new tests to avoid flakiness with parallel testr? (dkranz)
      • Now that parallel testr is in place (wow) maybe we should organize another bug triage day? (gfidente)
    • stress tests (mkoderer)
    • neutron testing status (mlavalle)
  • Update on slow heat gating job (dkranz)
  • Critical Reviews (mtreinish)
  • Testing client library compatibility (dkranz)
  • Devstack independent tempest usage (afazekas)
  • BP proposal: rework "skip test" functionaly (mkoderer)


Agenda for August 22 2013

  • Blueprints (mtreinish)
    • testr parallel status (mtreinish)
    • stress tests (mkoderer)
  • Critical Reviews (mtreinish)
  • Heat test merge (afazekas)
  • Devstack independent tempest usage (afazekas)
  • Adding test plans for new features into the tempest repo (mkollaro)

Agenda for August 15 2013

  • Blueprints (mtreinish)
    • testr parallel status (mtreinish)
    • stress tests (mkoderer)
  • Critical Reviews (mtreinish)

Agenda for August 8 2013

  • Volunteers to run meeting while sdague out (sdague)
  • Blueprints (sdague)
  • Critical Reviews (sdague)
  • Stress tests scope and behaviour, also see reviews on https://review.openstack.org/#/c/39752 (gfidente)
  • Status of slow heat tests (dkranz)

Agenda of August 1 2013

See the meeting minutes for July 1 on eavesdrop.

  • Blueprints (sdague)
  • Critical Reviews (sdague)
  • Call for more Core reviewers (dkranz)

Agenda of July 25 2013

See the meeting minutes for July 25 on eavesdrop.

  • Blueprints (sdague)
  • WebDav status codes in nova? Consistently using the 404 or 422 on actions.
  • Adding test cases with skip attribute vote? Exact rule (afazekas)
  • py26 compatibility (afazekas)
  • Consistent reviewing (afazekas)
  • Critical Reviews (sdague)
  • Separate heat job and slow tests in general (dkranz)


Agenda of July 18 2013

See the meeting minutes for July 18 on eavesdrop.

Agenda of July 11 2013

See the meeting minutes for July 11 on eavesdrop.


Agenda of June 27 2013

See the meeting minutes for June 27 on eavesdrop.

  • nothing in Agenda ... only QA as OpenStack Program, QA Leader and testr topics were discussed little bit