Jump to: navigation, search

Difference between revisions of "Meetings/InfraTeamMeeting"

m (→‎Agenda for next meeting)
(→‎Agenda for next meeting)
(48 intermediate revisions by 6 users not shown)
Line 3: Line 3:
 
= Weekly Project Infrastructure team meeting =
 
= Weekly Project Infrastructure team meeting =
  
−
The OpenDev Team holds public weekly meetings in <code><nowiki>#opendev-meeting</nowiki></code>, Tuesdays at 1900 UTC. Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend.
+
The OpenDev Team holds public weekly meetings in <code><nowiki>#opendev-meeting</nowiki></code> on OFTC, Tuesdays at 1900 UTC. Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend.
  
 
Please feel free to add agenda items (and your IRC nick in parenthesis).
 
Please feel free to add agenda items (and your IRC nick in parenthesis).
Line 13: Line 13:
 
* Actions from last meeting
 
* Actions from last meeting
  
−
* Specs approval
+
* Specs Review
 +
** Mailman 3 spec https://review.opendev.org/810990
  
−
* Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.)
+
* Topics
−
** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management]
+
** Improving OpenDev's CD throughput (clarkb 20211026)
−
*** topic:update-cfg-mgmt
+
*** We can run many of our jobs in parallel in all of our CD pipelines. But this requires we properly document/address dependencies
−
*** Zuul as CD engine
+
**** Need to understand our job dependencies and properly note them in Zuul config or address them by combining jobs.
−
** OpenDev
+
***** Example 1: Combine service-gitea-lb and service-gitea jobs.
−
*** Gerrit account inconsistencies
+
***** Example 2: Combine letsencrypt and nameserver jobs
−
**** All preferred emails lack external ids issues have been corrected. All group loops have been corrected.
+
***** Example 3: Have all jobs with webserver config express a dependency on the letsencrypt job
−
**** Workaround is we can stop Gerrit, push to external ids directly, reindex accounts (and groups?), start gerrit, then clear accounts caches (and groups caches?)
+
**** Suggest we document the known job dependencies in a human readable format, then encode this into zuul, then we can switch to parallel runs.
−
**** Next steps
+
**** https://review.opendev.org/c/opendev/system-config/+/807672
−
***** More "dangerous" list has been generated. Should still be safe-ish particularly if we disable the accounts first.
+
***** should list dependencies for all jobs
−
*** Configuration tuning
+
***** zuul doesn't trigger on this?  not sure on best approach to make it mergable
−
**** Reduce the number of ssh threads. Possibly create bot/batch user groups and thread counts as part of this.
+
**** https://review.opendev.org/c/opendev/base-jobs/+/807807
−
**** https://groups.google.com/g/repo-discuss/c/BQKxAfXBXuo Upstream conversation with people struggling with similar problems.
+
***** currently every executor adds keys for bridge, then logs in and clones system-config before running playbooks
−
*** Update our base job's nodeset
+
***** this change makes split jobs to do this.  however, production remains the same as both are called.
−
**** https://review.opendev.org/789098
+
**** https://review.opendev.org/c/opendev/system-config/+/807808
−
 
+
***** this is a follow-on that adds a base job to clone system-config, and stops the other production jobs re-cloning.
−
* General topics
+
***** this job must run first, but then all other jobs can run in parallel, as they are all in the same buildset and using the same "view" of system-config for that particular run
−
** Picking up steam on Puppet -> Ansible rewrites (clarkb 20210518)
+
** Gerrit Account cleanups (clarkb 20211026)
−
*** Enable Xenial -> Bionic/Focal system upgrades
+
*** 33 conflicts remain. Clarkb has written notes on proposed plans for each user in the comments of review02:~clarkb/gerrit_user_cleanups/audit-results-annotated.yaml
−
*** https://etherpad.opendev.org/p/infra-puppet-conversions-and-xenial-upgrades Start capturing TODO list here
+
** Fedora 34 test node booting problems (clarkb 20211026)
−
*** Zuul is done. Mailman next
+
*** Changes to Fedora's kernel packaging broke Xen
−
**** https://review.opendev.org/c/opendev/system-config/+/789622
+
*** Not yet sure if that may have also somehow broken OVH and iweb.
−
** Refreshing non LE certs that expire in just under a month (clarkb 20210518)
+
** Begin planning for Gerrit 3.4 upgrade (clarkb 20211026)
−
*** Shutdown/remove: ask.o.o
+
*** Read through release notes and identify areas of concern.
−
*** Refresh as yearly cert or try to LE: ethercalc, wiki, translate, storyboard
+
**** https://www.gerritcodereview.com/3.4.html
−
*** Meeting with foundation to discuss: openstackid and openstackid-dev
+
*** Do we need to modify system-config-run-review-3.4 or system-config-upgrade-review jobs to test any specific behaviors?
−
** Cleanup of too small swap setups on newer servers (clarkb 20210518)
+
*** Probably worth holding a 3.4 test node and doing a skim for any unexpected behaviors.
−
*** make_swap.sh was updated to accidentally limit swap size to 8MB when we wanted an 8GB limit
 
−
** Removing registration requirement from our IRC channels (clarkb 20210518)
 
−
*** TheJulia asks if we think this is still necessary.
 
−
*** Looking at the last month or so of eavesdrop logs it seems we had ~1.5 spam attempts in the unregistered channel
 
−
*** https://review.opendev.org/c/openstack/project-config/+/791818
 
−
** Block Storage API reference (rosmaita 20210525)
 
−
*** https://docs.openstack.org/api-ref/block-storage/index.html
 
−
*** The Cinder team is removing v2 during Xena.  Ordinarily we'd also delete the v2 api-ref as no longer relevant in master (which is true). However, it might be useful for people who have procrastinated moving their scripts, etc., to v3.
 
−
*** The api-ref isn't branched, like the regular docs.  Is there a way to build and publish the v2 part from stable/wallaby?
 
−
** Scheduling project renames (clarkb 20210518)
 
−
*** Our playbook(s) that do renames likely need updating since the last gerrit upgrade.
 
−
**** We can test this with our functional testing of gerrit too.
 
  
 
* Open discussion
 
* Open discussion
Line 60: Line 49:
 
(any additions should mention original->new full names and link to the corresponding project-config rename change in Gerrit)
 
(any additions should mention original->new full names and link to the corresponding project-config rename change in Gerrit)
  
−
* (lajoskatona) x/tap-as-a-service -> openstack/tap-as-a-service: https://review.opendev.org/c/openstack/project-config/+/790093
+
* Rename foo/example -> bar/example: https://review.opendev.org/123456
−
* foo/example -> bar/example: https://review.opendev.org/#/c/123456
 
  
 
== Previous meetings ==
 
== Previous meetings ==
 
Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/
 
Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/

Revision as of 23:40, 25 October 2021

Weekly Project Infrastructure team meeting

The OpenDev Team holds public weekly meetings in #opendev-meeting on OFTC, Tuesdays at 1900 UTC. Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend.

Please feel free to add agenda items (and your IRC nick in parenthesis).

Agenda for next meeting

  • Announcements
  • Actions from last meeting
  • Topics
    • Improving OpenDev's CD throughput (clarkb 20211026)
      • We can run many of our jobs in parallel in all of our CD pipelines. But this requires we properly document/address dependencies
        • Need to understand our job dependencies and properly note them in Zuul config or address them by combining jobs.
          • Example 1: Combine service-gitea-lb and service-gitea jobs.
          • Example 2: Combine letsencrypt and nameserver jobs
          • Example 3: Have all jobs with webserver config express a dependency on the letsencrypt job
        • Suggest we document the known job dependencies in a human readable format, then encode this into zuul, then we can switch to parallel runs.
        • https://review.opendev.org/c/opendev/system-config/+/807672
          • should list dependencies for all jobs
          • zuul doesn't trigger on this? not sure on best approach to make it mergable
        • https://review.opendev.org/c/opendev/base-jobs/+/807807
          • currently every executor adds keys for bridge, then logs in and clones system-config before running playbooks
          • this change makes split jobs to do this. however, production remains the same as both are called.
        • https://review.opendev.org/c/opendev/system-config/+/807808
          • this is a follow-on that adds a base job to clone system-config, and stops the other production jobs re-cloning.
          • this job must run first, but then all other jobs can run in parallel, as they are all in the same buildset and using the same "view" of system-config for that particular run
    • Gerrit Account cleanups (clarkb 20211026)
      • 33 conflicts remain. Clarkb has written notes on proposed plans for each user in the comments of review02:~clarkb/gerrit_user_cleanups/audit-results-annotated.yaml
    • Fedora 34 test node booting problems (clarkb 20211026)
      • Changes to Fedora's kernel packaging broke Xen
      • Not yet sure if that may have also somehow broken OVH and iweb.
    • Begin planning for Gerrit 3.4 upgrade (clarkb 20211026)
      • Read through release notes and identify areas of concern.
      • Do we need to modify system-config-run-review-3.4 or system-config-upgrade-review jobs to test any specific behaviors?
      • Probably worth holding a 3.4 test node and doing a skim for any unexpected behaviors.
  • Open discussion

Upcoming Project Renames

(any additions should mention original->new full names and link to the corresponding project-config rename change in Gerrit)

Previous meetings

Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/

Retrieved from "https://wiki.openstack.org/w/index.php?title=Meetings/InfraTeamMeeting&oldid=179758"