Jump to: navigation, search

Difference between revisions of "Meetings/Rally"

(Replaced content with " The page has been moved to https://rally.readthedocs.io")
(59 intermediate revisions by 6 users not shown)
Line 1: Line 1:
=  Weekly Rally meeting =
The [https://launchpad.net/Rally Rally] project team holds a weekly (starting from Oct 29th 2013) team meeting in <code><nowiki>#openstack-meeting</nowiki></code>:
* Weekly on Mondays at [http://www.worldclock.com/world_clock.html 1400 UTC]
The page has been moved to https://rally.readthedocs.io
= Weekly Rally release meeting =
At this meeting, we do not discuss common topics but release-specific stuff only, including the list of critical patches that have to be merged in the next release; Held in IRC, in the  <code><nowiki>#openstack-rally</nowiki></code> channel (irc.freenode.net).
* Held on Mondays at  [http://www.worldclock.com/world_clock.html 1300 UTC]
= Next meeting =
NOTE: Include your IRC nickname with your agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. Please have any supporting links ready. You might want to put this on your calendar if you are adding items.
== Agenda template==
  * [irc_nickname] Topic name. In additional some general question to the community
    - Link to the patch, additional links or information.
Stuck Reviews :
  * [irc_nickname] Topic name. In additional some general question to the community
    - Link to the patch, additional links or information.
== Agenda ==
'''April 3rd, 2016 14:00 UTC'''
  * [stpierre] Unsticking https://review.openstack.org/#/c/277225/15
'''March 28th, 2016 14:00 UTC'''
* [amaretskiy] Balancing users: maybe do not support random choice at all, just switch to balancing silently?
    - https://review.openstack.org/#/c/229896/
= Previous meetings =
'''February 29th, 2016 14:00 UTC'''
  * [stpierre] How to test complex functional cases (e.g., https://review.openstack.org/#/c/282918/)
  * [amaretskiy] Let's discuss Web UI for trends report
      - http://logs.openstack.org/21/276721/4/check/gate-rally-dsvm-rally/0425a53/rally-plot/trends.html.gz
      - https://review.openstack.org/#/c/276721/
'''February 22th, 2016 14:00 UTC'''
  * [stpierre] 0.3.0 release was buggy; need more transparency, possibly branching/change freeze to protect soon-to-be-released code
  * [andreykurilin] Using *.openstack.org URLs - https://review.openstack.org/#/c/282646
    - for docs ( http://docs.openstack.org/developer/rally/ )
    - for git ( https://git.openstack.org/cgit/openstack/rally )
  * [andreykurilin] plugins based on entry-points
  * [stpierre] keystone v3 support
    - https://review.openstack.org/#/c/282918/1
    - https://review.openstack.org/#/c/277225/12
'''February 15th, 2016 14:00 UTC'''
* [anreykurillin] keystone can kill Rally (Agreed: almost agreed))
'''February 8th, 2016 14:00 UTC'''
* [ikhudoshyn] Current state of Rally gate jobs -- testing against Keystone API v2.0 and v3 (Agreed: Left mos jobs in rally-ci against v2.0 and move devstack-based jobs in rally-ci to v3)
* [rvasilets] About changing coverage script behaviour (Agreed:file the bug)
* [Open Discussion] Green tests of heat siege workload
'''January 18th, 2016 14:00 UTC'''
* [amaretskiy] command 'rally version' always returns version that was determined once during the installation,
  so this command lies to those users (mostly developers) who uses rally directly from git repository
  without re-installation. Any ideas how to deal with it?
* [rvasilets] Copy-paste code in cinder wrapper. Unification of usage of cinderclient and cinder wrapper in cinder utils.
* [ikhudoshyn] Issue with DB migration -- alembic fails to create foreign key for a table referring itself
'''January 4th, 2015 14:00 UTC'''
* [amaretskiy] Chain of patches for improvements of scenario output (spec ''improve_scenario_output_format.rst'').
  Here is how the result looks in HTML report: http://goo.gl/CKfuzV. All patches are ready for review:
    - https://review.openstack.org/#/c/254851/
    - https://review.openstack.org/#/c/254261/
    - https://review.openstack.org/#/c/255336/
    - https://review.openstack.org/#/c/261542/
    - https://review.openstack.org/#/c/257886/
    - https://review.openstack.org/#/c/252344/
    - https://review.openstack.org/#/c/257990/
* [amaretskiy] Do we plan to make a resources cleanup check affecting job status? (https://goo.gl/5m80qC)
* [amaretskiy] Idea: how about filling task tag with input file name by default?
'''December 28th, 2015 14:00 UTC'''
  * [ikhudoshyn] Progress with Distributed Runner
    - https://review.openstack.org/#/c/254780/
    - https://review.openstack.org/#/c/261559/
'''December 21st, 2015 14:00 UTC'''
  * [stpierre] Deprecation of wait_for() creates ugliness.
    * It seems like we need a generic wait routine. If wait_for() isn't it, let's create one so we don't have to do stuff like https://review.openstack.org/#/c/253406/
  * [andreykurilin] NoSQL db(Mongo?!) vs SQL for storing results
  * [amaretskiy] There was a question asked why StreamingAlgorithm.result() raises RallyException - maybe return None instead
'''Stuck Reviews :'''
  * [amaretskiy] Significant improvements for scenario output according to the spec improve_scenario_output_format.rst.
    There is a chain of patches ready for review. Example how does this look in report: http://goo.gl/pH4FYm
    - https://review.openstack.org/#/c/254316/
    - https://review.openstack.org/#/c/254851/
    - https://review.openstack.org/#/c/254261/
    - https://review.openstack.org/#/c/255293/
    - https://review.openstack.org/#/c/255336/
    - https://review.openstack.org/#/c/257886/
    - https://review.openstack.org/#/c/252344/
'''December 14th, 2015 14:00 UTC'''
'''December 7th, 2015 14:00 UTC'''
* [ikhudoshyn] Preparation for a Distributed Runner (Info: Before implementing distributed runner itself expected two preparation steps
  1. Runner should return chunks (or batches) of TaskResult's rather then individual ones
  2. Runner should aggregate SLA data and return SLAChecker instances)
'''November 30th, 2015 14:00 UTC'''
  * [chenli]  Question about the rally-as-a-service refactor.
                  Is the structure will be like :  db <-> objects <->...<->rally-api <---> rally-client  ?
                  What we will use outside db & objects module ? dict ? object returned by objects modules ?
                  Do we consider using versioned-objects nova/cinder using in the future ?
                  On the client side, do we need a extra-layer to get data from object (when under not raas mode, do rest when raas mode)
                  data_serialize -> api -> cli
                  Using dict in all client code ?
'''November 23th, 2015 14:00 UTC'''
* [amaretskiy] Do we need a hacking check with `topy' tool? (Agreed: make grammar nazi job)
* [andreykurilin] Rally 0.1.2 will be the last release with py26 support? (Agreed: removing support of py26)
* [rvasilets] Add monasca plugin to Rally. How we will add monasca into the gate. Should it be non-voting, separate job or experimental? (Info: It will be non voting job)
'''November 16th, 2015 14:00 UTC'''
* [redixin] Unsafe self.context usage by contexts. Any context may damage data of any other context.
* [amaretskiy] Proposal of improvements for scenario output format
  - https://review.openstack.org/#/c/237632/
'''November 9th, 2015 14:00 UTC'''
* [cjmarti] Blueprint discussion. Add the hability to re-run the tempest tests that failed in the last test execution.
  - https://blueprints.launchpad.net/rally/+spec/re-run-failed-tempest-tests
* [ikhudoshyn] Distributed load generation spec
  - https://review.openstack.org/#/c/236979/
* [amaretskiy] Propose to increase minimal Jinja2 version in requirements
  - https://review.openstack.org/#/c/240556/
* [rvasilets] Do we have volunteers  to fix https://bugs.launchpad.net/rally/+bug/1420755 bug?
'''November 2st, 2015 14:00 UTC'''
  * [andreykurilin] Sudo access for rally user in docker container.
    - https://answers.launchpad.net/rally/+question/273388
  * [rvasilets] Fix timeout for scenario runners:
    - https://review.openstack.org/#/c/235910/
'''October 19st, 2015 14:00 UTC'''
* [ikhudoshyn] Distributed load generation (Info: By the links you could find a draft for the spec. Also a couple of patches which contain some independent parts)
      - https://review.openstack.org/#/c/236979/
      - https://review.openstack.org/#/c/238547/
      - https://review.openstack.org/#/c/239406/
'''October 19st, 2015 14:00 UTC'''
* [idegtiarov] Mandatory demand for unique resources names in Ceilometer context. Ceilometer team is interested to have optional possibility to set resources names manually.
      - https://review.openstack.org/#/c/192130/
* [rvasilets] New Rally dashboard
      - https://goo.gl/BcCAkU https://review.openstack.org/#/c/235957/
* [stpierre] Bug (?) in @types processing of images created by context
      - https://bugs.launchpad.net/rally/+bug/1503744
* [amaretskiy] plans and status for upcoming changes in output charts in HTML report
* [ikhudoshyn] Distributed load generation - spec
      - https://review.openstack.org/#/c/236979/
'''October 12st, 2015 14:00 UTC'''
* [mbonell] Blueprint discussion. New command to install Tempest's plugins in rally verify (Agreed: make some code and do the git review).
  - https://blueprints.launchpad.net/rally/+spec/install-tempest-plugins
* [lcazares] Blueprint discussion. Add optional parameters at rally container creation to auto-register an OpenStack deployment.
  - https://blueprints.launchpad.net/rally/+spec/rally-container-autodeployment
* [amaretskiy] What are the most critical bugs and features (required) in HTML reports?
* [redixin] Docker pull downloads all tags by default
    'docker pull rallyforge/rally' on fresh system needs more then one gigabyte to download.
'''October 5st, 2015 14:00 UTC'''
  * [boris-42]  Bug reports housekeeping (Agreed:  We will reduce size of patch queue and amount of bugs in project. Bug manager will be ilyakhudoshin.  Try keep like less then 30 open bugs)
'''September 28st, 2015 14:00 UTC'''
  * [redixin] rally-ci updates. (Info: Now it supports cloud images. Also it support btrfs (previously was only zfs). Added lots of small changes like timeouts and triggering jobs by ref-updated. Current rally-ci configuration may be found here: https://github.com/redixin/rci-config/blob/master/config.yaml Status of rally-ci may be seen here https://blueprints.launchpad.net/rally-ci)
  * [boris-42] Rally 0.1.0 updates (Info: we did it! Here are release notes https://rally.readthedocs.org/en/latest/release_notes/latest.html)
  * [boris-42] About Rally 0.1.1 (Info: Should cut new Release 0.1.1 mostly because of serious bug in Rally reports)
  * [boris-42] Critical bugs in reports (Info: patch https://review.openstack.org/#/c/228403/)
  * [ylobankov] Creating Tempest resources (Info: started work on this https://review.openstack.org/#/c/226652/)
  * [rvasilets] A small refactor of Rally install script:
    - https://review.openstack.org/#/c/190931/4
'''September 21st, 2015 14:00 UTC'''
  * [amaretskiy] New Rally release model (Info: now we have flexible ability to publish major and minor releases)
  * [amaretskiy] Discuss Rally Tags (See [openstack-dev] [openstack-operators][tc][tags])
  * [andreykurilin] Status for implementation of 'rally task abort' command
      - https://review.openstack.org/#/c/161636/
  * [stpierre] The users context acts differently for the v3 and v2 APIs; in the former case, it auto-adds the user to a "member" role, while it doesn't in v2. (Bug: https://bugs.launchpad.net/rally/+bug/1492177)
'''September 14th, 2015 14:00 UTC'''
  * [kun] argument for using atomic actions or not
    In the patch https://review.openstack.org/#/c/220200/, the output http://paste.openstack.org/show/451164/ of task seems bad, but:
      - if not using atomic actions, the output keeps meaningless from real business
      - if using atomic actions, the record time represent a combine of multiple calls, how do operators know what bad happened inside it?
  * [rvasilets] How to use Rally API in our functional tests?
      - https://review.openstack.org/#/c/198772/ As I understand from this log http://logs.openstack.org/72/198772/8/check/gate-rally-dsvm-cli/f2114f2/console.html.gz#_2015-08-07_15_13_59_652 rally db is empty because "no such table: deployments". Is any idea how to use it right in func tests?
  * [boris-42] Rally Tags for Big Tent
'''September 7th, 2015 14:00 UTC'''
  * [yfried] "Compose scenario" vs "new input task format:
    We have 2 specs that want to enhance the execution of existing tasks. Can we please define if/how they relate
    to each other?
      - https://github.com/openstack/rally/blob/master/doc/specs/in-progress/new_rally_input_task_format.rst
      - https://review.openstack.org/#/c/215762/
  * [boris-42] Rally Info & Plugin reference
  * [boris-42] What is important for Rally 0.1.0 release
'''August 31th, 2015 14:00 UTC'''
  * [amaretskiy] Status for patches related to HTML reports and new charts classes
    - https://review.openstack.org/#/c/214695/
    - https://review.openstack.org/#/c/146814/
    - https://review.openstack.org/#/c/169828/
    - https://review.openstack.org/#/c/215598/
  * [ikhudoshyn] Object-oriented design of task configuration
    - https://review.openstack.org/#/c/211160/
  * [boris-42] Rally Plugin reference
'''August 24th, 2015 14:00 UTC'''
  * [andreykurilin] The critical number of outdated patches on review.
    - There are a lot of patches on review without any updates at last 2-4 weeks. What we should do with them? Maintain or Abandon(best-practice from OpenStack community)?
  * [boris-42] Updates about switching to Plugin base.
'''August 17th, 2015'''
  * [kun_huang] [short news] chinese openstack hackathon (There is a meetup in this week, openstack hackathon (china))
  * [rvasilets] 'rally task abort' now is green! Welcome to review. https://review.openstack.org/#/c/161636/
  * [boris-42] Switching Scenario to plugin base https://review.openstack.org/#/c/212831/ After this patch we will be able to do few next things:
    - common deprecation mechanism for all types of plugins
    - common (scenario like) validation mechanism for all types of plugins
    - common types mechanism and replace a bunch of bad code from rally engine/framework
  * [boris-42] Where is 0.1.0 release? (Agreed: need to switch to plugin base scenarios and implement task aborts. Probably in two weeks)
'''August 10th, 2015'''
  * [amaretskiy ]New classes for HTML report charts. https://review.openstack.org/#/c/159458 (Agreed: They are already done, need review for it)
  * [kun] ]Vm workload https://docs.google.com/document/d/1FQicnj6aIdbfYts9ofYdz0dUEf9zyssWqDjQR40h268/edit (Agreed: the next step is to create servers_ext context)
  * [kun] What's the state of Solaris support?
  * [ikhudoshyn] Refactor BenchmarkEngine https://review.openstack.org/#/c/209046/ (Agreed: take a look at this)
'''August 3th, 2015'''
  * Swift context for Sahara in Rally https://review.openstack.org/#/c/177732/ (Agreed: boris-42 will take a look at them)
  * Ability to specify directory with murano package files https://review.openstack.org/#/c/195677/ (Agreed: boris-42 believe that rvasilets should just ping other cores to review it)
  * Add 'description' element to scenario config https://review.openstack.org/181307 (Agreed: this information should be somewhere used not just added. yingjun will add this to the task reports)
'''July 27th, 2015'''
  * Make Rally framework independent from OpenStack https://review.openstack.org/#/c/203911/
  * Create_and_list_<singular> (e.g., create_and_list_image) or create_and_list_<plural> (e.g., create_and_list_users)? (Agreed: we should use plural everywhere and make hacking  rule for this)
  * Verify performance with Rally: compare performance of scenarios after 2 runs (Agreed: We will finish this task someday)
  * Replace depraceted oslo.db code https://review.openstack.org/#/c/205530/ (Agreed: Looks reasonable)
'''July 20th, 2015'''
  * [boris-42] Meeting agenda and how to make people work on it not at the very last moment (Agreed: meeting manager sends out reminder on Thursday)
  * [boris-42] Splitting Rally framework and OpenStack (updates) (Boris: Need "patch that will make our context responsible for mapping resources for each scenario iteration. e.g. I will move this code https://github.com/openstack/rally/blob/master/rally/task/runner.py#L47-L58 to ContextManager. I am going to publish this patch today later")
  * [boris-42] Support of benchmarking multi API versions of projects with same plugins (Boris: "We will have special context for openstack_api_versions where you can specify info what python clients to create and there will be validator for all scenarios (which versions they support) and a bit magic in osclients and openstack.scenario to make this work.")
  * [amaretskiy] Status for patches related to new streaming Reports generation: (Need code review)
    - [https://review.openstack.org/#/c/201611/ more streaming algorithms]
    - [https://review.openstack.org/#/c/159458/ new classes for Reports generation]
    - [https://review.openstack.org/#/c/146814/ use new classes and chunks for reports]
    - [https://review.openstack.org/#/c/169828/ add LoadProfile chart to reports]
  * Ironic benchmarks '''led by rvasilets''': (Need code review)
    - https://review.openstack.org/#/c/186064/ need code review
'''July 13th, 2015:'''
  * Supporting Solaris (Agreed: We will support Solaris and we will start from CI for it)
  * Should we support py26? (Need: Talk without dhellmann about cuping release versions in requriments)
  * Stucked specs. https://review.openstack.org/#/c/182245/ (Agreed Split "Refactoring Rally cleanup": 1) consistent resource naming; 2) the actual cleanup.)
  * Stucked patches (pain) https://review.openstack.org/#/q/project:openstack/rally+branch:master+topic:bp/vm-workloads-framework,n,z (Agree: Cores take a look at it)
  * Who is in charge for moderating 2 next meetings? (Agree: stpierre is going to be in charge for next two meethings)
  * Murano benchmarks https://review.openstack.org/#/c/195677/ (Need: code review)
  * Cinder benchmarks (Cinder is going to make rally job voting after https://review.openstack.org/#/c/194180/ )
  * Ironic benchmarks https://review.openstack.org/#/c/186064/ (Agree: Ask somebody from Ironic core team for review)
'''July 6th 2015:'''
  * Rally Meeting Agenda (Agreed: rotate each 2 weeks meeting manager; write summaries by hand using meetbot stuff https://wiki.debian.org/MeetBot)
  * Ironic benchmarks https://review.openstack.org/#/c/186064/. Issues with recursion wile Ironic task execute http://logs.openstack.org/64/186064/14/experimental/gate-rally-dsvm-ironic-rally/9b0a8d7/console.html.gz#_2015-06-30_14_39_09_363 (Agreed: try to understand why Ironic benchmarks doesn't work and why CLI of ironic works; boris-42 will review Ironic patch https://review.openstack.org/#/c/186064/)
  * Use DDT framework in tests. Sample of DDT way to test https://github.com/openstack/rally/blob/master/tests/unit/plugins/openstack/scenarios/manila/test_utils.py#L117-L133 (Agreed: using ddt for newly added tests, low priority to rewite)
  * Optimize Rally imports https://review.openstack.org/#/c/196493/ (Agreed: That patch should be merged =))
  * Some review work update https://etherpad.openstack.org/p/rally_review_work.  (Action: do the changes with rally dashboard to fit kun_huang requirments. Agreed: review the oldest patches firstly)
'''June 29th 2015:'''
  * Fixtures upgrades that is block Rally gates
  * Optimize Rally imports to reduce start time
  * Open discussion
Logs form the meeting above here http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-29-14.01.log.html
'''June 22th 2015:'''
  * Broken cinder attach volume update (https://review.openstack.org/193828)
  * Rally task abort (finished)
  * Rally CI updates(format of config file, web ui, etc.)
  * Rally monasca job
  * Open discussion
Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-22-14.11.log.html
'''June 15th 2015:'''
  * Rally gates are/were blocked (ways to became more isolated from such mistakes)
  * Delays in next release
  * Open discussion
Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-15-14.11.log.html
'''June 8th 2015:'''
  * Plugin base (finished)
  * Rally core structure changes
  * Rally as a Lib & Rally as a Service (spec https://review.openstack.org/#/c/182245/)
  * Rally Infra updates (https://review.openstack.org/#/c/185031/)
  * Open discussion
Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-08-14.09.log.html
'''May 25th 2015:'''
  * New HTML reports classes (https://review.openstack.org/#/c/159458/, https://review.openstack.org/#/c/146814/, https://review.openstack.org/#/c/169828/)
  * New script for gate jobs (https://review.openstack.org/#/c/175549/)
  * Fuel benchmarks (https://review.openstack.org/#/c/178731/)
  * Specs in the review
  * Open discussion
Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-05-25-14.01.log.html
'''May 18th 2015:'''
  * [Spec] Enhance Rally API (https://review.openstack.org/182245)
  * Fuel benchmarks (https://review.openstack.org/#/c/178731/, https://review.openstack.org/#/c/181270/, https://review.openstack.org/#/c/182873/)
  * Splitting plugins in Rally (https://review.openstack.org/#/q/project:openstack/rally+branch:master+topic:bp/split-plugins,n,z)
  * Where to find answers (http://rally.readthedocs.org/en/latest/project_info.html)
  * Open discussion
Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-05-18-14.04.log.html
'''April 28th 2015:'''
  * Rally QA week
  * Upcoming Rally 0.0.4 release: progress on critical patches
  * Spec on refactoring scenario utils: review and discussion (https://review.openstack.org/#/c/172831/)
  * Spec on in-tree functional tests: review and discussion (https://review.openstack.org/#/c/166487/)
  * Open discussion
Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-04-28-17.00.log.html
= Meeting logs =
* [http://eavesdrop.openstack.org/meetings/rally/2016/ Meeting logs for 2016]
* [http://eavesdrop.openstack.org/meetings/rally/2015/ Meeting logs for 2015]
* [http://eavesdrop.openstack.org/meetings/rally/2014/ Meeting logs for 2014]
* [http://eavesdrop.openstack.org/meetings/rally/2013/ Meeting logs for 2013]
= Have any questions? =
Ping boris-42 at <code><nowiki>#openstack-rally</nowiki></code>

Latest revision as of 13:01, 30 October 2017

The page has been moved to https://rally.readthedocs.io