Difference between revisions of "Meetings/InfraTeamMeeting"
< Meetings
Clark Boylan (talk | contribs) (→Agenda for next meeting) |
Clark Boylan (talk | contribs) (→Agenda for next meeting) |
||
| (57 intermediate revisions by 5 users not shown) | |||
| Line 10: | Line 10: | ||
* Announcements | * Announcements | ||
| − | |||
* Actions from last meeting | * Actions from last meeting | ||
| Line 17: | Line 16: | ||
* Topics | * Topics | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
** Upgrading Old Servers (clarkb 20230627) | ** Upgrading Old Servers (clarkb 20230627) | ||
| − | *** https://etherpad.opendev.org/p/opendev | + | *** https://etherpad.opendev.org/p/opendev-server-upgrade-planning Central tracking document which may link to more host specific documents |
| − | |||
| − | |||
| − | |||
| − | |||
*** Next on the list are graphite and backup servers | *** Next on the list are graphite and backup servers | ||
| − | *** | + | *** backup03.ca-ymq-1.vexxhost.opendev.org has been launched and is being backed up too |
| + | **** https://review.opendev.org/c/opendev/system-config/+/995420 Starting backup02 removal here | ||
*** Remember to use launch-node's --config-drive flag when booting new Noble nodes in Rax Classic | *** Remember to use launch-node's --config-drive flag when booting new Noble nodes in Rax Classic | ||
| − | ** | + | ** Deploying a Prometheus for Server Metrics (clarkb 20260331) |
| − | *** https://review.opendev.org/c/opendev/system-config/+/ | + | *** https://review.opendev.org/c/opendev/system-config/+/980840 |
| − | *** | + | *** This change and its child deploy prometheus with node exporter to collect server metrics |
| − | ** | + | *** These two changes simplify the setup and testing of prometheus and node exporter |
| − | *** | + | **** https://review.opendev.org/c/zuul/zuul-jobs/+/994564 manage /etc/hosts with public IPs |
| − | *** https://review.opendev.org/c/ | + | **** https://review.opendev.org/c/opendev/system-config/+/994565 Use public IPs in system-config-run jobs |
| − | ** | + | ** Larger VM sizes for tests (corvus 20260618) |
| − | *** | + | *** corvus has been testing python 3.14 with zuul; zuul unit tests now use slightly more than 8GB under 3.14 |
| − | *** clarkb | + | *** We have 16gb nodes, but in two clouds, rax-classic and vexxhost, they have fewer vcpus than their 8gb counterparts, so we need to use 32gb nodes to compensate |
| − | ** | + | *** Are we okay with this? Alternatives? |
| − | *** | + | ** Dealing with alien zuul config errors in the openstack tenant (frickler 20260617) |
| − | *** https:// | + | *** Currently there are still 185 zuul config errors in the openstack tenant, despite my year-long struggle to get rid of them. |
| − | + | *** Most of these are from "alien" repos (74 airship, 29 starlingx) that I have no motivation to fix myself with my OpenStack hats on | |
| − | ** Gitea 1. | + | *** Efforts to motivate these projects to clean up their errors themselves have mostly failed |
| − | *** https://review.opendev.org/c/opendev/system-config/+/ | + | *** I still believe that cleaning these up and being able to easily identify fresh errors is important for the healthyness of the CI setup as a whole |
| − | ** Gitea | + | *** One pretty strong action would be to move these repos into their own tenant(s) or a different shared one like opendev |
| − | *** | + | *** I acknowledge that without further work this would break their CI setup, but I'm questioning now whether that impact would be worse than the impact the current situation has on my ability to maintain the OpenStack CI |
| − | *** | + | *** Other ideas or opinions are welcome |
| − | *** | + | *** clarkb reached out to starlingx and airship about this |
| − | *** | + | **** Airship indicated they would like to avoid the extra work involved in setting up a separate tenant |
| − | *** https://review.opendev.org/c/opendev/system-config/+/ | + | **** clarkb pointed out to them that they would need to fix their zuul config errors and be reachable via email or matrix at a bare minumum if we want to make that work. |
| − | ** | + | **** https://lists.starlingx.io/archives/list/starlingx-discuss@lists.starlingx.io/thread/YQVACUR4OCX74ZULHAJ4AD44MHGY37YI/ |
| − | *** We | + | ** Gitea 1.26.4 Upgrade (clarkb 20260622) |
| − | *** | + | *** https://review.opendev.org/c/opendev/system-config/+/994326 Upgrade Gitea to 1.26.4 |
| − | *** | + | *** Its time to upgrade to the next Gitea bugfix release |
| + | ** Bump Anubis difficult to 5 (clarkb 20260630) | ||
| + | *** There is some evidence that bots are regularly solving the Anubis challenge | ||
| + | *** The challenges are slowing them down enough that services continue to be mostly responsive | ||
| + | *** Should we increase the difficulty one level to slow them down even futher? | ||
| + | *** This will impact regular users too which is likely the primary consideration we should make. | ||
| + | *** https://review.opendev.org/c/opendev/system-config/+/995096 | ||
| + | ** Planning Gerrit Project Renames (clarkb 20260622) | ||
| + | *** We have a request to rename x/cursive to openstack/cursive | ||
| + | *** Any concern with project ownership doing that? The current group membership includes people from Johns Hopkins University and OpenStack Barbican | ||
| + | *** Aiming for July 9 at ~2100 UTC | ||
* Open discussion | * Open discussion | ||
| Line 73: | Line 64: | ||
Changes should have their topic set to project-rename. | Changes should have their topic set to project-rename. | ||
| − | * Rename | + | * Rename x/cursive -> openstack/cursive: https://review.opendev.org/c/openstack/project-config/+/990122 (stephenfin, fungi) |
== Previous meetings == | == Previous meetings == | ||
Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/ | Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/ | ||
Latest revision as of 14:53, 30 June 2026
Contents
Weekly Project Infrastructure team meeting
The OpenDev Team holds public weekly meetings in #opendev-meeting on OFTC, Tuesdays at 1900 UTC. Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend.
Please feel free to add agenda items (and your IRC nick in parenthesis).
Agenda for next meeting
- Announcements
- Actions from last meeting
- Specs Review
- Topics
- Upgrading Old Servers (clarkb 20230627)
- https://etherpad.opendev.org/p/opendev-server-upgrade-planning Central tracking document which may link to more host specific documents
- Next on the list are graphite and backup servers
- backup03.ca-ymq-1.vexxhost.opendev.org has been launched and is being backed up too
- https://review.opendev.org/c/opendev/system-config/+/995420 Starting backup02 removal here
- Remember to use launch-node's --config-drive flag when booting new Noble nodes in Rax Classic
- Deploying a Prometheus for Server Metrics (clarkb 20260331)
- https://review.opendev.org/c/opendev/system-config/+/980840
- This change and its child deploy prometheus with node exporter to collect server metrics
- These two changes simplify the setup and testing of prometheus and node exporter
- https://review.opendev.org/c/zuul/zuul-jobs/+/994564 manage /etc/hosts with public IPs
- https://review.opendev.org/c/opendev/system-config/+/994565 Use public IPs in system-config-run jobs
- Larger VM sizes for tests (corvus 20260618)
- corvus has been testing python 3.14 with zuul; zuul unit tests now use slightly more than 8GB under 3.14
- We have 16gb nodes, but in two clouds, rax-classic and vexxhost, they have fewer vcpus than their 8gb counterparts, so we need to use 32gb nodes to compensate
- Are we okay with this? Alternatives?
- Dealing with alien zuul config errors in the openstack tenant (frickler 20260617)
- Currently there are still 185 zuul config errors in the openstack tenant, despite my year-long struggle to get rid of them.
- Most of these are from "alien" repos (74 airship, 29 starlingx) that I have no motivation to fix myself with my OpenStack hats on
- Efforts to motivate these projects to clean up their errors themselves have mostly failed
- I still believe that cleaning these up and being able to easily identify fresh errors is important for the healthyness of the CI setup as a whole
- One pretty strong action would be to move these repos into their own tenant(s) or a different shared one like opendev
- I acknowledge that without further work this would break their CI setup, but I'm questioning now whether that impact would be worse than the impact the current situation has on my ability to maintain the OpenStack CI
- Other ideas or opinions are welcome
- clarkb reached out to starlingx and airship about this
- Airship indicated they would like to avoid the extra work involved in setting up a separate tenant
- clarkb pointed out to them that they would need to fix their zuul config errors and be reachable via email or matrix at a bare minumum if we want to make that work.
- https://lists.starlingx.io/archives/list/starlingx-discuss@lists.starlingx.io/thread/YQVACUR4OCX74ZULHAJ4AD44MHGY37YI/
- Gitea 1.26.4 Upgrade (clarkb 20260622)
- https://review.opendev.org/c/opendev/system-config/+/994326 Upgrade Gitea to 1.26.4
- Its time to upgrade to the next Gitea bugfix release
- Bump Anubis difficult to 5 (clarkb 20260630)
- There is some evidence that bots are regularly solving the Anubis challenge
- The challenges are slowing them down enough that services continue to be mostly responsive
- Should we increase the difficulty one level to slow them down even futher?
- This will impact regular users too which is likely the primary consideration we should make.
- https://review.opendev.org/c/opendev/system-config/+/995096
- Planning Gerrit Project Renames (clarkb 20260622)
- We have a request to rename x/cursive to openstack/cursive
- Any concern with project ownership doing that? The current group membership includes people from Johns Hopkins University and OpenStack Barbican
- Aiming for July 9 at ~2100 UTC
- Upgrading Old Servers (clarkb 20230627)
- Open discussion
Upcoming Project Renames
(any additions should mention original->new full names and link to the corresponding project-config rename change in Gerrit) Changes should have their topic set to project-rename.
- Rename x/cursive -> openstack/cursive: https://review.opendev.org/c/openstack/project-config/+/990122 (stephenfin, fungi)
Previous meetings
Previous meetings, with their notes and logs, can be found at http://eavesdrop.openstack.org/meetings/infra/ and earlier at http://eavesdrop.openstack.org/meetings/ci/
