Jump to: navigation, search

Difference between revisions of "Meetings/InfraTeamMeeting"

(add item to open discussion for 18 May meeting)
(Agenda for next meeting)
 
(318 intermediate revisions by 14 users not shown)
Line 3: Line 3:
 
= Weekly Project Infrastructure team meeting =
 
= Weekly Project Infrastructure team meeting =
  
The OpenDev Team holds public weekly meetings in <code><nowiki>#opendev-meeting</nowiki></code>, Tuesdays at 1900 UTC. Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend.
+
The OpenDev Team holds public weekly meetings in <code><nowiki>#opendev-meeting</nowiki></code> on OFTC, Tuesdays at 1900 UTC. Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend.
  
 
Please feel free to add agenda items (and your IRC nick in parenthesis).
 
Please feel free to add agenda items (and your IRC nick in parenthesis).
Line 10: Line 10:
  
 
* Announcements
 
* Announcements
** Clark out during May 11 meeting time. Need a volunteer chair or can cancel that meeting.
 
  
 
* Actions from last meeting
 
* Actions from last meeting
  
* Specs approval
+
* Specs Review
  
* Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.)
+
* Topics
** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management]
+
** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627)
*** topic:update-cfg-mgmt
+
*** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades
*** Zuul as CD engine
+
*** https://review.opendev.org/q/topic:jitsi_meet-jammy-update
** OpenDev
+
*** Started looking at the wiki there are rough notes at: https://etherpad.opendev.org/p/opendev-bionic-server-upgrades#L58
*** Gerrit account inconsistencies
+
** MariaDB Upgrades (clarkb 20240220)
**** All preferred emails lack external ids issues have been corrected. All group loops have been corrected.
+
*** Relying on the container image MARIADB_AUTO_UPGRADE flag
**** Workaround is we can stop Gerrit, push to external ids directly, reindex accounts (and groups?), start gerrit, then clear accounts caches (and groups caches?)
+
*** Etherpad and mailman are done. Gerrit and Gitea are all that remain.
**** Next steps
+
** AFS Mirror cleanups (clarkb 20240220)
***** More "dangerous" list has been generated. Should still be safe-ish particularly if we disable the accounts first.
+
*** Ubuntu Xenial is next but currently busy with PTG, Release, and other tasks.
*** Configuration tuning
+
*** Can followup with webserver log processing to determine which other mirrors may be dead.
**** Reduce the number of ssh threads. Possibly create bot/batch user groups and thread counts as part of this.
+
** Building Ubuntu Noble Nodes (clarkb 20240416)
**** https://groups.google.com/g/repo-discuss/c/BQKxAfXBXuo Upstream conversation with people struggling with similar problems.
+
*** https://review.opendev.org/c/opendev/glean/+/915907 Glean updates for python3.12 support
*** Update our base job's nodeset
+
*** Appears we will need a debootstrap from debian unstable to build noble
**** https://review.opendev.org/789098
+
** Etherpad 2.0.x Upgrade planning (clarkb 20240416)
*** Gerrit global config simplification
+
*** https://review.opendev.org/c/opendev/system-config/+/914119
**** https://review.opendev.org/789383
+
*** Etherpad 2.0.2 removes the old APIKEY API auth method and now uses Oauth2.0 client_credentials instead. This new system is not yet released, but we are working to determine how to use it pre release.
 
+
** Add reference to the project-team-guide in the fail ci msg (fungi 20240423)
* General topics
+
*** https://review.opendev.org/914189
** Picking up steam on Puppet -> Ansible rewrites (clarkb 20210504)
+
** Gerrit 3.9 Upgrade Planning (clarkb 20240423)
*** Enable Xenial -> Bionic/Focal system upgrades
+
*** https://www.gerritcodereview.com/3.9.html Release Notes
*** https://etherpad.opendev.org/p/infra-puppet-conversions-and-xenial-upgrades Start capturing TODO list here
+
*** Downgrade is possible with an offline reindex using the old war.
*** Zuul service host updates in progress now. Zuul scheduler is last remaining server that needs an upgrade in the zuul cluster.
+
*** Need to compile an upgrade checklist (go over breaking changes and also test the downgrade).
** openEuler patches (ianw 20210504)
+
*** Server could stand to be upgraded as well but I think it is probably best to decouple these concerns.
*** TC level, or being too officious
+
** Wiki SSL Cert Renewal (clarkb 20240423)
** InMotion Cloud reorganization (clarkb 20210504)
+
*** We have just under 4 weeks to renew the cert. This is one of the last (maybe the only) manually managed ssl cert.
*** Currently limited by number of IP addresses available
+
*** Will probably delay until we have less than a week left before processing it.
*** We could deploy an executor in the cloud then set up test nodes without direct external access
 
*** Any held nodes would need floating IPs attached to them to be accessible
 
** Removing registration requirement from our IRC channels (clarkb 20210504)
 
*** TheJulia asks if we think this is still necessary.
 
*** Looking at the last month or so of eavesdrop logs it seems we had ~1.5 spam attempts in the unregistered channel
 
** Switching artifact signing keys from RSA to ECC
 
*** https://review.opendev.org/789062
 
  
 
* Open discussion
 
* Open discussion
** "Toggle CI" in Gerrit web UI (rosmaita 20210518)
 
*** I thought this was on the [https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes 3.2 post-upgrade etherpad] but I can't find it.  We used to have this button in the old web interface
 
*** The Cinder team is looking for a solution to Change Log pollution from our 3rd party CI systems
 
**** example: https://review.opendev.org/c/openstack/cinder/+/790796/
 
*** actual reviewer comments get lost in the Change Log
 
*** the "Only comments" toggle hides all the "Added to cc:" lines, which is nice.  It also hides the Zuul "build succeeded/failed" comments ... what do we need to do so this will work for other CI responses?
 
  
 
== Upcoming Project Renames ==
 
== Upcoming Project Renames ==
 
(any additions should mention original->new full names and link to the corresponding project-config rename change in Gerrit)
 
(any additions should mention original->new full names and link to the corresponding project-config rename change in Gerrit)
 +
Changes should have their topic set to project-rename.
  
* (lajoskatona) x/tap-as-a-service -> openstack/tap-as-a-service: https://review.opendev.org/c/openstack/project-config/+/790093
+
* Rename example/foo -> example/bar: https://review.opendev.org/c/openstack/project-config/+/123456
* foo/example -> bar/example: https://review.opendev.org/#/c/123456
 
  
 
== Previous meetings ==
 
== Previous meetings ==
 
Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/
 
Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/

Latest revision as of 22:13, 22 April 2024

Weekly Project Infrastructure team meeting

The OpenDev Team holds public weekly meetings in #opendev-meeting on OFTC, Tuesdays at 1900 UTC. Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend.

Please feel free to add agenda items (and your IRC nick in parenthesis).

Agenda for next meeting

  • Announcements
  • Actions from last meeting
  • Specs Review
  • Topics
    • Upgrading Bionic servers to Focal/Jammy (clarkb 20230627)
    • MariaDB Upgrades (clarkb 20240220)
      • Relying on the container image MARIADB_AUTO_UPGRADE flag
      • Etherpad and mailman are done. Gerrit and Gitea are all that remain.
    • AFS Mirror cleanups (clarkb 20240220)
      • Ubuntu Xenial is next but currently busy with PTG, Release, and other tasks.
      • Can followup with webserver log processing to determine which other mirrors may be dead.
    • Building Ubuntu Noble Nodes (clarkb 20240416)
    • Etherpad 2.0.x Upgrade planning (clarkb 20240416)
    • Add reference to the project-team-guide in the fail ci msg (fungi 20240423)
    • Gerrit 3.9 Upgrade Planning (clarkb 20240423)
      • https://www.gerritcodereview.com/3.9.html Release Notes
      • Downgrade is possible with an offline reindex using the old war.
      • Need to compile an upgrade checklist (go over breaking changes and also test the downgrade).
      • Server could stand to be upgraded as well but I think it is probably best to decouple these concerns.
    • Wiki SSL Cert Renewal (clarkb 20240423)
      • We have just under 4 weeks to renew the cert. This is one of the last (maybe the only) manually managed ssl cert.
      • Will probably delay until we have less than a week left before processing it.
  • Open discussion

Upcoming Project Renames

(any additions should mention original->new full names and link to the corresponding project-config rename change in Gerrit) Changes should have their topic set to project-rename.

Previous meetings

Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/