Jump to: navigation, search

Difference between revisions of "Meetings/Rally"

m (Previous meetings)
Line 24: Line 24:
== Agenda ==
== Agenda ==
'''October 31th, 2016 14:00 UTC'''
'''November 7th, 2016 14:00 UTC'''
   * [astudenov] Let's discuss how db api should save task results data in chunks
   * [andreykurilin] updates from summit
   - https://review.openstack.org/#/c/391421/
   * [andreykurilin] rally plugins versioning
= Previous meetings =
= Previous meetings =

Revision as of 13:43, 2 November 2016

Weekly Rally meeting

The Rally project team holds a weekly (starting from Oct 29th 2013) team meeting in #openstack-meeting:

Weekly Rally release meeting

At this meeting, we do not discuss common topics but release-specific stuff only, including the list of critical patches that have to be merged in the next release; Held in IRC, in the #openstack-rally channel (irc.freenode.net).

Next meeting

NOTE: Include your IRC nickname with your agenda items so that you can be called upon in the meeting and arrive at the meeting promptly if placing items in agenda. Please have any supporting links ready. You might want to put this on your calendar if you are adding items.

Agenda template

 * [irc_nickname] Topic name. In additional some general question to the community
   - Link to the patch, additional links or information.
Stuck Reviews :
 * [irc_nickname] Topic name. In additional some general question to the community
   - Link to the patch, additional links or information.


November 7th, 2016 14:00 UTC

 * [andreykurilin] updates from summit
 * [andreykurilin] rally plugins versioning

Previous meetings

October 31th, 2016 14:00 UTC

 * [astudenov] Let's discuss how db api should save task results data in chunks
 - https://review.openstack.org/#/c/391421/

October 10th, 2016 14:00 UTC

 * [amaretskiy] Let's discuss term "scenario" - should this term be replaced completely by "workload"
    or maybe we can give it specific meaning?
     - https://rally.readthedocs.io/en/latest/glossary.html#scenario
 * [ylobankov] New Rally verify HTML report https://review.openstack.org/#/c/343504/
 * [rvasiltes] Add sla section to the samples

September 26th, 2016 14:00 UTC

  * [rvasilets] A lot of people interested in https://review.openstack.org/#/c/234195/
    - the idea is  to do some working runner with changing `rps` value and then in other patches optimize it

July 25th, 2016 14:00 UTC

 * [rvasilets] How to optimize sync requirements workflow

July 11th, 2016 14:00 UTC

 * [amaretskiy] We have Rally-as-a-Service demonstration - let's discuss it and decide (Agreed: will review)
 * [boris-42] Rally mascot (Agreed: most votes for white ermine in rally jacket)

July 4th, 2016 14:00 UTC

 * [rvasilets] I want to implement context resource cleanup
  - https://github.com/openstack/rally/blob/master/rally/plugins/openstack/context/designate/zones.py#L61
 * [amaretskiy] How about to add a new chapter about HTML report to project documentation? Sometimes we have questions from community about report, for example "What is 90%ile and 95%ile in table?", "Why iterations are started from 1?", "What does 'Aggregated' and 'Per iteration' mean?" and so on.

June 13th, 2016 14:00 UTC

 * [amaretskiy] Class RallyException does not provide i18n good enough without subclassing - we still have to use gettext in this case to split i18n part of message from non-translated args.
   Here is a patch which demonstrates how we can solve this problem:
      - https://review.openstack.org/#/c/328978/

May 30th, 2016 14:00 UTC

 * [rvasilets] Priorities for this week for Rally team

April 25th, 2016 14:00 UTC

 * [amaretskiy] Let's define what messages we always wrap with rally.commin.i18n._ and what we don't translate.  It would be great to save this rule somewhere.
 * [amaretskiy] How to prevent unexpected changing of mutable data (for example context configuration)?
    -  https://review.openstack.org/#/c/307340/ FrozenDict looks good for first sign, however it (at least for now) does not prevent changing of nested data (http://paste.openstack.org/show/495134/)
    - copy.deepcopy() works fine but does not raise exceptions, which is also bad

April 18th, 2016 14:00 UTC

 * [andreykurilin] Release 0.4.0
 * [andreykurilin] Planning for next release
 * [amaretskiy] Let's revive DB refactoring spec and give it high priority, since we can not effectively switch to new task format without this spec
    - https://review.openstack.org/#/c/167060/
 * [yanyanhu] Basic support for Senlin service in Rally:
    - https://review.openstack.org/298109
    - https://review.openstack.org/307170
    - https://review.openstack.org/307050
 * [andreykurilin] Make Glance V2 the default version.

April 3rd, 2016 14:00 UTC

 * [stpierre] Unsticking https://review.openstack.org/#/c/277225/15

March 28th, 2016 14:00 UTC

* [amaretskiy] Balancing users: maybe do not support random choice at all, just switch to balancing silently?
    - https://review.openstack.org/#/c/229896/

February 29th, 2016 14:00 UTC

 * [stpierre] How to test complex functional cases (e.g., https://review.openstack.org/#/c/282918/)
 * [amaretskiy] Let's discuss Web UI for trends report
     - http://logs.openstack.org/21/276721/4/check/gate-rally-dsvm-rally/0425a53/rally-plot/trends.html.gz
     - https://review.openstack.org/#/c/276721/

February 22th, 2016 14:00 UTC

 * [stpierre] 0.3.0 release was buggy; need more transparency, possibly branching/change freeze to protect soon-to-be-released code
 * [andreykurilin] Using *.openstack.org URLs - https://review.openstack.org/#/c/282646
   - for docs ( http://docs.openstack.org/developer/rally/ )
   - for git ( https://git.openstack.org/cgit/openstack/rally )
 * [andreykurilin] plugins based on entry-points
 * [stpierre] keystone v3 support
   - https://review.openstack.org/#/c/282918/1
   - https://review.openstack.org/#/c/277225/12

February 15th, 2016 14:00 UTC

* [anreykurillin] keystone can kill Rally (Agreed: almost agreed))

February 8th, 2016 14:00 UTC

* [ikhudoshyn] Current state of Rally gate jobs -- testing against Keystone API v2.0 and v3 (Agreed: Left mos jobs in rally-ci against v2.0 and move devstack-based jobs in rally-ci to v3)
* [rvasilets] About changing coverage script behaviour (Agreed:file the bug)
* [Open Discussion] Green tests of heat siege workload

January 18th, 2016 14:00 UTC

* [amaretskiy] command 'rally version' always returns version that was determined once during the installation,
  so this command lies to those users (mostly developers) who uses rally directly from git repository
  without re-installation. Any ideas how to deal with it?
* [rvasilets] Copy-paste code in cinder wrapper. Unification of usage of cinderclient and cinder wrapper in cinder utils.
* [ikhudoshyn] Issue with DB migration -- alembic fails to create foreign key for a table referring itself

January 4th, 2015 14:00 UTC

* [amaretskiy] Chain of patches for improvements of scenario output (spec improve_scenario_output_format.rst).
  Here is how the result looks in HTML report: http://goo.gl/CKfuzV. All patches are ready for review:
    - https://review.openstack.org/#/c/254851/
    - https://review.openstack.org/#/c/254261/
    - https://review.openstack.org/#/c/255336/
    - https://review.openstack.org/#/c/261542/
    - https://review.openstack.org/#/c/257886/
    - https://review.openstack.org/#/c/252344/
    - https://review.openstack.org/#/c/257990/
* [amaretskiy] Do we plan to make a resources cleanup check affecting job status? (https://goo.gl/5m80qC)
* [amaretskiy] Idea: how about filling task tag with input file name by default?

December 28th, 2015 14:00 UTC

 * [ikhudoshyn] Progress with Distributed Runner
   - https://review.openstack.org/#/c/254780/
   - https://review.openstack.org/#/c/261559/

December 21st, 2015 14:00 UTC

 * [stpierre] Deprecation of wait_for() creates ugliness.
   * It seems like we need a generic wait routine. If wait_for() isn't it, let's create one so we don't have to do stuff like https://review.openstack.org/#/c/253406/
 * [andreykurilin] NoSQL db(Mongo?!) vs SQL for storing results
 * [amaretskiy] There was a question asked why StreamingAlgorithm.result() raises RallyException - maybe return None instead
Stuck Reviews :
 * [amaretskiy] Significant improvements for scenario output according to the spec improve_scenario_output_format.rst.
   There is a chain of patches ready for review. Example how does this look in report: http://goo.gl/pH4FYm
   - https://review.openstack.org/#/c/254316/
   - https://review.openstack.org/#/c/254851/
   - https://review.openstack.org/#/c/254261/
   - https://review.openstack.org/#/c/255293/
   - https://review.openstack.org/#/c/255336/
   - https://review.openstack.org/#/c/257886/
   - https://review.openstack.org/#/c/252344/

December 14th, 2015 14:00 UTC

December 7th, 2015 14:00 UTC

* [ikhudoshyn] Preparation for a Distributed Runner (Info: Before implementing distributed runner itself expected two preparation steps
  1. Runner should return chunks (or batches) of TaskResult's rather then individual ones
  2. Runner should aggregate SLA data and return SLAChecker instances)

November 30th, 2015 14:00 UTC

 * [chenli]  Question about the rally-as-a-service refactor.
                  Is the structure will be like :   db <-> objects <->...<->rally-api <---> rally-client   ?
                  What we will use outside db & objects module ? dict ? object returned by objects modules ? 
                  Do we consider using versioned-objects nova/cinder using in the future ?
                  On the client side, do we need a extra-layer to get data from object (when under not raas mode, do rest when raas mode)
                  data_serialize -> api -> cli
                  Using dict in all client code ?

November 23th, 2015 14:00 UTC

* [amaretskiy] Do we need a hacking check with `topy' tool? (Agreed: make grammar nazi job)
* [andreykurilin] Rally 0.1.2 will be the last release with py26 support? (Agreed: removing support of py26)
* [rvasilets] Add monasca plugin to Rally. How we will add monasca into the gate. Should it be non-voting, separate job or experimental? (Info: It will be non voting job)

November 16th, 2015 14:00 UTC

* [redixin] Unsafe self.context usage by contexts. Any context may damage data of any other context.
* [amaretskiy] Proposal of improvements for scenario output format
  - https://review.openstack.org/#/c/237632/

November 9th, 2015 14:00 UTC

* [cjmarti] Blueprint discussion. Add the hability to re-run the tempest tests that failed in the last test execution. 
  - https://blueprints.launchpad.net/rally/+spec/re-run-failed-tempest-tests
* [ikhudoshyn] Distributed load generation spec
  - https://review.openstack.org/#/c/236979/
* [amaretskiy] Propose to increase minimal Jinja2 version in requirements
  - https://review.openstack.org/#/c/240556/
* [rvasilets] Do we have volunteers  to fix https://bugs.launchpad.net/rally/+bug/1420755 bug?

November 2st, 2015 14:00 UTC

 * [andreykurilin] Sudo access for rally user in docker container.
   - https://answers.launchpad.net/rally/+question/273388
 * [rvasilets] Fix timeout for scenario runners:
   - https://review.openstack.org/#/c/235910/

October 19st, 2015 14:00 UTC

* [ikhudoshyn] Distributed load generation (Info: By the links you could find a draft for the spec. Also a couple of patches which contain some independent parts)
     - https://review.openstack.org/#/c/236979/
     - https://review.openstack.org/#/c/238547/
     - https://review.openstack.org/#/c/239406/

October 19st, 2015 14:00 UTC

* [idegtiarov] Mandatory demand for unique resources names in Ceilometer context. Ceilometer team is interested to have optional possibility to set resources names manually.
     - https://review.openstack.org/#/c/192130/
* [rvasilets] New Rally dashboard
     - https://goo.gl/BcCAkU https://review.openstack.org/#/c/235957/
* [stpierre] Bug (?) in @types processing of images created by context
     - https://bugs.launchpad.net/rally/+bug/1503744
* [amaretskiy] plans and status for upcoming changes in output charts in HTML report
* [ikhudoshyn] Distributed load generation - spec
     - https://review.openstack.org/#/c/236979/

October 12st, 2015 14:00 UTC

* [mbonell] Blueprint discussion. New command to install Tempest's plugins in rally verify (Agreed: make some code and do the git review).
  - https://blueprints.launchpad.net/rally/+spec/install-tempest-plugins 
* [lcazares] Blueprint discussion. Add optional parameters at rally container creation to auto-register an OpenStack deployment.
  - https://blueprints.launchpad.net/rally/+spec/rally-container-autodeployment
* [amaretskiy] What are the most critical bugs and features (required) in HTML reports?
* [redixin] Docker pull downloads all tags by default
   'docker pull rallyforge/rally' on fresh system needs more then one gigabyte to download.

October 5st, 2015 14:00 UTC

 * [boris-42]  Bug reports housekeeping (Agreed:  We will reduce size of patch queue and amount of bugs in project. Bug manager will be ilyakhudoshin.  Try keep like less then 30 open bugs)

September 28st, 2015 14:00 UTC

 * [redixin] rally-ci updates. (Info: Now it supports cloud images. Also it support btrfs (previously was only zfs). Added lots of small changes like timeouts and triggering jobs by ref-updated. Current rally-ci configuration may be found here: https://github.com/redixin/rci-config/blob/master/config.yaml Status of rally-ci may be seen here https://blueprints.launchpad.net/rally-ci)
 * [boris-42] Rally 0.1.0 updates (Info: we did it! Here are release notes https://rally.readthedocs.org/en/latest/release_notes/latest.html)
 * [boris-42] About Rally 0.1.1 (Info: Should cut new Release 0.1.1 mostly because of serious bug in Rally reports)
 * [boris-42] Critical bugs in reports (Info: patch https://review.openstack.org/#/c/228403/)
 * [ylobankov] Creating Tempest resources (Info: started work on this https://review.openstack.org/#/c/226652/)
 * [rvasilets] A small refactor of Rally install script:
   - https://review.openstack.org/#/c/190931/4

September 21st, 2015 14:00 UTC

 * [amaretskiy] New Rally release model (Info: now we have flexible ability to publish major and minor releases)
 * [amaretskiy] Discuss Rally Tags (See [openstack-dev] [openstack-operators][tc][tags])
 * [andreykurilin] Status for implementation of 'rally task abort' command
     - https://review.openstack.org/#/c/161636/
 * [stpierre] The users context acts differently for the v3 and v2 APIs; in the former case, it auto-adds the user to a "member" role, while it doesn't in v2. (Bug: https://bugs.launchpad.net/rally/+bug/1492177)

September 14th, 2015 14:00 UTC

 * [kun] argument for using atomic actions or not
   In the patch https://review.openstack.org/#/c/220200/, the output http://paste.openstack.org/show/451164/ of task seems bad, but:
     - if not using atomic actions, the output keeps meaningless from real business
     - if using atomic actions, the record time represent a combine of multiple calls, how do operators know what bad happened inside it?
 * [rvasilets] How to use Rally API in our functional tests?
     - https://review.openstack.org/#/c/198772/ As I understand from this log http://logs.openstack.org/72/198772/8/check/gate-rally-dsvm-cli/f2114f2/console.html.gz#_2015-08-07_15_13_59_652 rally db is empty because "no such table: deployments". Is any idea how to use it right in func tests?
 * [boris-42] Rally Tags for Big Tent

September 7th, 2015 14:00 UTC

 * [yfried] "Compose scenario" vs "new input task format:
   We have 2 specs that want to enhance the execution of existing tasks. Can we please define if/how they relate
   to each other?
     - https://github.com/openstack/rally/blob/master/doc/specs/in-progress/new_rally_input_task_format.rst
     - https://review.openstack.org/#/c/215762/
 * [boris-42] Rally Info & Plugin reference 
 * [boris-42] What is important for Rally 0.1.0 release

August 31th, 2015 14:00 UTC

 * [amaretskiy] Status for patches related to HTML reports and new charts classes
   - https://review.openstack.org/#/c/214695/
   - https://review.openstack.org/#/c/146814/
   - https://review.openstack.org/#/c/169828/
   - https://review.openstack.org/#/c/215598/
 * [ikhudoshyn] Object-oriented design of task configuration
   - https://review.openstack.org/#/c/211160/
 * [boris-42] Rally Plugin reference

August 24th, 2015 14:00 UTC

 * [andreykurilin] The critical number of outdated patches on review.
   - There are a lot of patches on review without any updates at last 2-4 weeks. What we should do with them? Maintain or Abandon(best-practice from OpenStack community)?
 * [boris-42] Updates about switching to Plugin base.

August 17th, 2015

 * [kun_huang] [short news] chinese openstack hackathon (There is a meetup in this week, openstack hackathon (china))
 * [rvasilets] 'rally task abort' now is green! Welcome to review. https://review.openstack.org/#/c/161636/
 * [boris-42] Switching Scenario to plugin base https://review.openstack.org/#/c/212831/ After this patch we will be able to do few next things:
   - common deprecation mechanism for all types of plugins
   - common (scenario like) validation mechanism for all types of plugins
   - common types mechanism and replace a bunch of bad code from rally engine/framework
 * [boris-42] Where is 0.1.0 release? (Agreed: need to switch to plugin base scenarios and implement task aborts. Probably in two weeks)

August 10th, 2015

 * [amaretskiy ]New classes for HTML report charts. https://review.openstack.org/#/c/159458 (Agreed: They are already done, need review for it)
 * [kun] ]Vm workload https://docs.google.com/document/d/1FQicnj6aIdbfYts9ofYdz0dUEf9zyssWqDjQR40h268/edit (Agreed: the next step is to create servers_ext context)
 * [kun] What's the state of Solaris support?
 * [ikhudoshyn] Refactor BenchmarkEngine https://review.openstack.org/#/c/209046/ (Agreed: take a look at this)

August 3th, 2015

 * Swift context for Sahara in Rally https://review.openstack.org/#/c/177732/ (Agreed: boris-42 will take a look at them)
 * Ability to specify directory with murano package files https://review.openstack.org/#/c/195677/ (Agreed: boris-42 believe that rvasilets should just ping other cores to review it)
 * Add 'description' element to scenario config https://review.openstack.org/181307 (Agreed: this information should be somewhere used not just added. yingjun will add this to the task reports)

July 27th, 2015

 * Make Rally framework independent from OpenStack https://review.openstack.org/#/c/203911/
 * Create_and_list_<singular> (e.g., create_and_list_image) or create_and_list_<plural> (e.g., create_and_list_users)? (Agreed: we should use plural everywhere and make hacking  rule for this)
 * Verify performance with Rally: compare performance of scenarios after 2 runs (Agreed: We will finish this task someday)
 * Replace depraceted oslo.db code https://review.openstack.org/#/c/205530/ (Agreed: Looks reasonable)

July 20th, 2015

 * [boris-42] Meeting agenda and how to make people work on it not at the very last moment (Agreed: meeting manager sends out reminder on Thursday)
 * [boris-42] Splitting Rally framework and OpenStack (updates) (Boris: Need "patch that will make our context responsible for mapping resources for each scenario iteration. e.g. I will move this code https://github.com/openstack/rally/blob/master/rally/task/runner.py#L47-L58 to ContextManager. I am going to publish this patch today later")
 * [boris-42] Support of benchmarking multi API versions of projects with same plugins (Boris: "We will have special context for openstack_api_versions where you can specify info what python clients to create and there will be validator for all scenarios (which versions they support) and a bit magic in osclients and openstack.scenario to make this work.")
 * [amaretskiy] Status for patches related to new streaming Reports generation: (Need code review)
    - more streaming algorithms
    - new classes for Reports generation
    - use new classes and chunks for reports
    - add LoadProfile chart to reports
 * Ironic benchmarks led by rvasilets: (Need code review)
    - https://review.openstack.org/#/c/186064/ need code review

July 13th, 2015:

 * Supporting Solaris (Agreed: We will support Solaris and we will start from CI for it)
 * Should we support py26? (Need: Talk without dhellmann about cuping release versions in requriments)
 * Stucked specs. https://review.openstack.org/#/c/182245/ (Agreed Split "Refactoring Rally cleanup": 1) consistent resource naming; 2) the actual cleanup.)
 * Stucked patches (pain) https://review.openstack.org/#/q/project:openstack/rally+branch:master+topic:bp/vm-workloads-framework,n,z (Agree: Cores take a look at it)
 * Who is in charge for moderating 2 next meetings? (Agree: stpierre is going to be in charge for next two meethings)
 * Murano benchmarks https://review.openstack.org/#/c/195677/ (Need: code review)
 * Cinder benchmarks (Cinder is going to make rally job voting after https://review.openstack.org/#/c/194180/ )
 * Ironic benchmarks https://review.openstack.org/#/c/186064/ (Agree: Ask somebody from Ironic core team for review)

July 6th 2015:

 * Rally Meeting Agenda (Agreed: rotate each 2 weeks meeting manager; write summaries by hand using meetbot stuff https://wiki.debian.org/MeetBot)
 * Ironic benchmarks https://review.openstack.org/#/c/186064/. Issues with recursion wile Ironic task execute http://logs.openstack.org/64/186064/14/experimental/gate-rally-dsvm-ironic-rally/9b0a8d7/console.html.gz#_2015-06-30_14_39_09_363 (Agreed: try to understand why Ironic benchmarks doesn't work and why CLI of ironic works; boris-42 will review Ironic patch https://review.openstack.org/#/c/186064/)
 * Use DDT framework in tests. Sample of DDT way to test https://github.com/openstack/rally/blob/master/tests/unit/plugins/openstack/scenarios/manila/test_utils.py#L117-L133 (Agreed: using ddt for newly added tests, low priority to rewite)
 * Optimize Rally imports https://review.openstack.org/#/c/196493/ (Agreed: That patch should be merged =))
 * Some review work update https://etherpad.openstack.org/p/rally_review_work.  (Action: do the changes with rally dashboard to fit kun_huang requirments. Agreed: review the oldest patches firstly)

June 29th 2015:

 * Fixtures upgrades that is block Rally gates
 * Optimize Rally imports to reduce start time
 * Open discussion

Logs form the meeting above here http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-29-14.01.log.html

June 22th 2015:

 * Broken cinder attach volume update (https://review.openstack.org/193828)
 * Rally task abort (finished)
 * Rally CI updates(format of config file, web ui, etc.)
 * Rally monasca job
 * Open discussion

Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-22-14.11.log.html

June 15th 2015:

 * Rally gates are/were blocked (ways to became more isolated from such mistakes)
 * Delays in next release
 * Open discussion

Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-15-14.11.log.html

June 8th 2015:

 * Plugin base (finished)
 * Rally core structure changes
 * Rally as a Lib & Rally as a Service (spec https://review.openstack.org/#/c/182245/)
 * Rally Infra updates (https://review.openstack.org/#/c/185031/)
 * Open discussion

Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-06-08-14.09.log.html

May 25th 2015:

 * New HTML reports classes (https://review.openstack.org/#/c/159458/, https://review.openstack.org/#/c/146814/, https://review.openstack.org/#/c/169828/)
 * New script for gate jobs (https://review.openstack.org/#/c/175549/)
 * Fuel benchmarks (https://review.openstack.org/#/c/178731/)
 * Specs in the review
 * Open discussion

Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-05-25-14.01.log.html

May 18th 2015:

 * [Spec] Enhance Rally API (https://review.openstack.org/182245)
 * Fuel benchmarks (https://review.openstack.org/#/c/178731/, https://review.openstack.org/#/c/181270/, https://review.openstack.org/#/c/182873/)
 * Splitting plugins in Rally (https://review.openstack.org/#/q/project:openstack/rally+branch:master+topic:bp/split-plugins,n,z)
 * Where to find answers (http://rally.readthedocs.org/en/latest/project_info.html)
 * Open discussion

Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-05-18-14.04.log.html

April 28th 2015:

 * Rally QA week
 * Upcoming Rally 0.0.4 release: progress on critical patches
 * Spec on refactoring scenario utils: review and discussion (https://review.openstack.org/#/c/172831/)
 * Spec on in-tree functional tests: review and discussion (https://review.openstack.org/#/c/166487/)
 * Open discussion

Logs form the meeting above http://eavesdrop.openstack.org/meetings/rally/2015/rally.2015-04-28-17.00.log.html

Meeting logs

Have any questions?

Ping boris-42 at #openstack-rally