| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-25 | |||
| 10:15:03 | stephenfin | whatever risk exists with these changes also exists with the main change, so I don't see any reason to not include this if we're doing the RPC bump (which I do think we should, fwiw) | |
| 10:15:06 | stephenfin | gibi: pretty much | |
| 10:15:49 | gibi | I will take the risk of breaking things and I will use you both to fix any breakage if happens | |
| 10:15:58 | gibi | is that fair? | |
| 10:16:03 | stephenfin | wfm | |
| 10:16:31 | gibi | bauzas: ? | |
| 10:16:49 | bauzas | gibi: use your stick. | |
| 10:17:08 | bauzas | :p | |
| 10:17:17 | gibi | I have broom ;) | |
| 10:17:40 | bauzas | well, I don't ask you fifty nuances of -1s | |
| 10:18:10 | bauzas | ok, lemme squash the series then | |
| 10:18:17 | gibi | bauzas: OK | |
| 10:18:17 | bauzas | and hold your breathes | |
| 10:18:19 | gibi | I will +2it | |
| 10:18:35 | gibi | and we will hold up RC1 if the gate will need some recheck | |
| 10:23:36 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Bump the Compute RPC API to version 6.0 https://review.opendev.org/c/openstack/nova/+/761452 | |
| 10:23:55 | bauzas | gibi: dansmith: stephenfin: ^ squashed rebase | |
| 10:23:59 | bauzas | on your pencils, folks. | |
| 10:24:34 | stephenfin | Done | |
| 10:26:11 | gibi | on it | |
| 10:40:54 | gibi | bauzas: do we need to mention the RPC major bump in the releasenotes? | |
| 10:41:45 | stephenfin | I think bauzas is on dad duty | |
| 10:41:49 | stephenfin | probably wise | |
| 10:41:51 | gibi | or it is totally implemantation detail | |
| 10:42:03 | gibi | anyhow that can be done separately even on the stable/wallaby branch after the cut | |
| 10:42:48 | stephenfin | actually, the only release notes I can find are for when we drop RPC versions | |
| 10:42:55 | stephenfin | releasenotes/notes/conductor_rpcapi_v2_drop-9893c27bb32d9786.yaml | |
| 10:43:05 | stephenfin | so we can wait I guess | |
| 10:46:02 | gibi | yeah it expected to work transparently for the users and admins | |
| 10:46:06 | gibi | I geuss | |
| 10:47:03 | bauzas | gibi: actually, that's a good question | |
| 10:47:06 | gibi | stephenfin: could you check back to the prelude that is looks good to you? https://review.opendev.org/c/openstack/nova/+/782172 | |
| 10:47:15 | stephenfin | sure | |
| 10:47:18 | gibi | thanks | |
| 10:48:11 | stephenfin | yup, looks like my comments were addressed so lgtm | |
| 10:48:16 | gibi | thanks | |
| 11:22:27 | stephenfin | gibi: FYI https://storyboard.openstack.org/#!/story/2008750 | |
| 11:22:47 | stephenfin | I don't think that's a regression so it should block placement RC1 but we'll probably want to fix that soonish | |
| 11:22:51 | stephenfin | *shouldn't | |
| 11:29:02 | gibi | stephenfin: ack. ping me if I can help reviewing the fix | |
| 11:31:06 | gibi | lyarwood: when you are around there is a list of doc fixed piled up behind https://review.opendev.org/c/openstack/nova/+/781434 could you check back to that? | |
| 11:46:41 | openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Update the architecture diagram https://review.opendev.org/c/openstack/nova/+/782979 | |
| 11:46:41 | lyarwood | gibi: ack, will do now | |
| 11:58:06 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/stein: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782962 | |
| 12:00:21 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782982 | |
| 12:00:50 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782962 | |
| 12:01:47 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/rocky: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782983 | |
| 12:02:17 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782984 | |
| 12:07:19 | openstackgerrit | Stephen Finucane proposed openstack/placement master: Correctly handle integrity errors on MySQL 8.x https://review.opendev.org/c/openstack/placement/+/782988 | |
| 12:18:00 | gibi | lyarwood: thanks | |
| 12:20:56 | noonedeadpunk | hey everyone! we see super weird issue with one of our patches where we update nova version to SHA 3de7fb7c327db348d04d15d4cd3c4f811a336126 and trying to perform V->X upgrade and run tempest. | |
| 12:21:18 | noonedeadpunk | So tempest fails with `Details: (TestServerBasicOps:test_server_basic_ops) Server 40ce5b33-d2d8-470e-b8c8-776c53efce8c failed to reach ACTIVE status and task state "None" within the required time (600 s). Current status: BUILD. Current task state: scheduling.` | |
| 12:21:51 | noonedeadpunk | No issues neither in compute nor in scheduler. And instance got rescheduled to compute, but never starts there | |
| 12:22:05 | noonedeadpunk | https://zuul.opendev.org/t/openstack/build/85c40976164c4b1785aefe0c1a4ddab9/log/logs/host/nova-compute.service.journal-11-24-25.log.txt#17356 | |
| 12:23:21 | noonedeadpunk | And SHA `5b65812ef2c6ac46bea30e2e603284eac232339e` was working properly (but yeah, that's an SHA before merging libvirt 6 requirement) | |
| 12:24:19 | noonedeadpunk | are we missing some config change here? | |
| 12:27:39 | lyarwood | V to W right? | |
| 12:27:48 | lyarwood | where can I find the tempest logs? | |
| 12:28:08 | lyarwood | If it cast from the conductor didn't make it to the compute then this would smell like a rabbitmq issue tbh | |
| 12:30:09 | jrosser | the tempest log is here https://zuul.opendev.org/t/openstack/build/85c40976164c4b1785aefe0c1a4ddab9/log/logs/openstack/aio1-utility/tempest_run.log.txt | |
| 12:36:50 | noonedeadpunk | lyarwood: it makes to the compute, since compute has `Instance 40ce5b33-d2d8-470e-b8c8-776c53efce8c has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 256, 'VCPU': 1}}` | |
| 12:38:01 | noonedeadpunk | Except debug messages there's a single warning in the compute log, which is `This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.` But I guess it should not be critical? | |
| 12:48:59 | noonedeadpunk | lyarwood: sorry, you was right, it's rabbit issue :( | |
| 12:49:16 | noonedeadpunk | found the reason here, thanks for the clue! | |
| 12:50:49 | lyarwood | noonedeadpunk: yeah that message above isn't part of the actual request it's from a periodic job on the compute that notices that an instance was scheduled but hasn't started yet | |
| 12:50:56 | lyarwood | noonedeadpunk: np | |
| 13:31:14 | sean-k-mooney | noonedeadpunk: im guessing the compute node was not reciving rpc becuase the topic was lost on the queue? | |
| 13:33:20 | noonedeadpunk | sean-k-mooney: no, because we were using rabbitmq with self-signed SSL but without CA | |
| 13:33:29 | noonedeadpunk | which is not allowed in X | |
| 13:33:35 | noonedeadpunk | but was in V | |
| 13:34:13 | noonedeadpunk | and in that patch we upgraded not only nova, but also u-c, so yeah :( | |
| 13:35:58 | noonedeadpunk | (it's eventually because of the some amqp dependancy) | |
| 13:39:02 | sean-k-mooney | ah ok | |
| 13:44:26 | openstackgerrit | Merged openstack/nova master: Wallaby 23.0.0 prelude section https://review.opendev.org/c/openstack/nova/+/782172 | |
| 13:45:49 | openstackgerrit | Merged openstack/nova master: docs: Add a real-time guide https://review.opendev.org/c/openstack/nova/+/781434 | |
| 13:46:58 | openstackgerrit | Merged openstack/nova master: docs: Remove duplicate TPM extra spec info https://review.opendev.org/c/openstack/nova/+/781435 | |
| 13:48:18 | openstackgerrit | Merged openstack/nova master: docs: Remove duplicated PCI passthrough extra spec info https://review.opendev.org/c/openstack/nova/+/781436 | |
| 13:49:15 | openstackgerrit | Merged openstack/nova master: docs: Add SEV guide https://review.opendev.org/c/openstack/nova/+/781437 | |
| 13:51:25 | gibi | \o/ | |
| 14:12:34 | openstackgerrit | Radomir Dopieralski proposed openstack/python-novaclient master: When creating a client, pass the default logger https://review.opendev.org/c/openstack/python-novaclient/+/783010 | |
| 14:13:18 | Luzi | lyarwood, could you take a look at https://review.opendev.org/c/openstack/nova/+/782430 ? i would like to propose the backport for stein too and start writing the ossn soon | |
| 14:15:37 | openstackgerrit | Merged openstack/nova master: Bump the Compute RPC API to version 6.0 https://review.opendev.org/c/openstack/nova/+/761452 | |
| 14:16:04 | lyarwood | Luzi: done | |
| 14:16:59 | Luzi | lyarwood, thank you :) | |
| 14:21:42 | sean-k-mooney | gibi: actully if there is a followup patch to sev there are a few thigns we shoudl add | |
| 14:23:09 | sean-k-mooney | gibi: multi numa guest may fail to boot due to a qemu issue https://bugzilla.redhat.com/show_bug.cgi?id=1814502 and we shoudl be recommending that hw:mem_page_size be set since sev is locking memory | |
| 14:23:10 | openstack | sean-k-mooney: Error: Error getting bugzilla.redhat.com bug #1814502: NotPermitted | |
| 14:23:36 | sean-k-mooney | oh thats se to private i guess | |
| 14:24:15 | openstackgerrit | Mark Goddard proposed openstack/nova master: Check volume encryption keys prior to resize and migration https://review.opendev.org/c/openstack/nova/+/781079 | |
| 14:28:22 | gibi | sean-k-mooney: you can still propose a reno extension for the RC1. | |
| 14:28:57 | gibi | sean-k-mooney: after today you need a bug that fix on master then backport to stable/wallaby and propose an RC2 | |
| 14:34:41 | sean-k-mooney | well its a docs fix not reno the new admin/sev.rst doc | |
| 14:34:58 | sean-k-mooney | so ya we can fix it later | |
| 14:35:17 | sean-k-mooney | its not urgent | |
| 14:37:32 | gibi | sean-k-mooney: OK | |
| 14:38:26 | bauzas | just looking up fast, have we a regression candidate ? | |
| 14:39:04 | bauzas | and wow for the rcp bump being merged \o/ | |
| 14:44:02 | stephenfin | Hmm, my ZNC bouncer went down :-( | |
| 14:44:16 | stephenfin | Guess that's what I get for 20€ a year or whatever the VPS cost | |
| 14:46:15 | gibi | bauzas: what regression candidate? :) | |