Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-03-25
10:09:42 stephenfin *rosy
10:09:46 stephenfin :)
10:10:09 stephenfin and the successful CI jobs prove as much
10:12:38 stephenfin bauzas: the grenade job runs resize tests https://zuul.opendev.org/t/openstack/build/07e5714177574a1fb513f4ae6b83bbc5/log/job-output.txt#4617
10:12:53 stephenfin e.g. tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm
10:12:58 bauzas yup, was looking into it
10:13:39 bauzas I knew we had tempest tests but I was wondering whether we were calling them with the grenade multinode job
10:13:41 gibi I OK, so we have to move forward somehow. and I guess as a PTL I need to make the decision
10:13:54 stephenfin so we've got test coverage, the tests are passing and the code is sane (IMO at least)
10:15:03 stephenfin whatever risk exists with these changes also exists with the main change, so I don't see any reason to not include this if we're doing the RPC bump (which I do think we should, fwiw)
10:15:06 stephenfin gibi: pretty much
10:15:49 gibi I will take the risk of breaking things and I will use you both to fix any breakage if happens
10:15:58 gibi is that fair?
10:16:03 stephenfin wfm
10:16:31 gibi bauzas: ?
10:16:49 bauzas gibi: use your stick.
10:17:08 bauzas :p
10:17:17 gibi I have broom ;)
10:17:40 bauzas well, I don't ask you fifty nuances of -1s
10:18:10 bauzas ok, lemme squash the series then
10:18:17 gibi bauzas: OK
10:18:17 bauzas and hold your breathes
10:18:19 gibi I will +2it
10:18:35 gibi and we will hold up RC1 if the gate will need some recheck
10:23:36 openstackgerrit Sylvain Bauza proposed openstack/nova master: Bump the Compute RPC API to version 6.0 https://review.opendev.org/c/openstack/nova/+/761452
10:23:55 bauzas gibi: dansmith: stephenfin: ^ squashed rebase
10:23:59 bauzas on your pencils, folks.
10:24:34 stephenfin Done
10:26:11 gibi on it
10:40:54 gibi bauzas: do we need to mention the RPC major bump in the releasenotes?
10:41:45 stephenfin I think bauzas is on dad duty
10:41:49 stephenfin probably wise
10:41:51 gibi or it is totally implemantation detail
10:42:03 gibi anyhow that can be done separately even on the stable/wallaby branch after the cut
10:42:48 stephenfin actually, the only release notes I can find are for when we drop RPC versions
10:42:55 stephenfin releasenotes/notes/conductor_rpcapi_v2_drop-9893c27bb32d9786.yaml
10:43:05 stephenfin so we can wait I guess
10:46:02 gibi yeah it expected to work transparently for the users and admins
10:46:06 gibi I geuss
10:47:03 bauzas gibi: actually, that's a good question
10:47:06 gibi stephenfin: could you check back to the prelude that is looks good to you? https://review.opendev.org/c/openstack/nova/+/782172
10:47:15 stephenfin sure
10:47:18 gibi thanks
10:48:11 stephenfin yup, looks like my comments were addressed so lgtm
10:48:16 gibi thanks
11:22:27 stephenfin gibi: FYI https://storyboard.openstack.org/#!/story/2008750
11:22:47 stephenfin I don't think that's a regression so it should block placement RC1 but we'll probably want to fix that soonish
11:22:51 stephenfin *shouldn't
11:29:02 gibi stephenfin: ack. ping me if I can help reviewing the fix
11:31:06 gibi lyarwood: when you are around there is a list of doc fixed piled up behind https://review.opendev.org/c/openstack/nova/+/781434 could you check back to that?
11:46:41 openstackgerrit Lee Yarwood proposed openstack/nova master: docs: Update the architecture diagram https://review.opendev.org/c/openstack/nova/+/782979
11:46:41 lyarwood gibi: ack, will do now
11:58:06 openstackgerrit Vlad Gusev proposed openstack/nova stable/stein: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782962
12:00:21 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782982
12:00:50 openstackgerrit Lee Yarwood proposed openstack/nova stable/stein: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782962
12:01:47 openstackgerrit Lee Yarwood proposed openstack/nova stable/rocky: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782983
12:02:17 openstackgerrit Lee Yarwood proposed openstack/nova stable/queens: Improve error log when snapshot fails https://review.opendev.org/c/openstack/nova/+/782984
12:07:19 openstackgerrit Stephen Finucane proposed openstack/placement master: Correctly handle integrity errors on MySQL 8.x https://review.opendev.org/c/openstack/placement/+/782988
12:18:00 gibi lyarwood: thanks
12:20:56 noonedeadpunk hey everyone! we see super weird issue with one of our patches where we update nova version to SHA 3de7fb7c327db348d04d15d4cd3c4f811a336126 and trying to perform V->X upgrade and run tempest.
12:21:18 noonedeadpunk So tempest fails with `Details: (TestServerBasicOps:test_server_basic_ops) Server 40ce5b33-d2d8-470e-b8c8-776c53efce8c failed to reach ACTIVE status and task state "None" within the required time (600 s). Current status: BUILD. Current task state: scheduling.`
12:21:51 noonedeadpunk No issues neither in compute nor in scheduler. And instance got rescheduled to compute, but never starts there
12:22:05 noonedeadpunk https://zuul.opendev.org/t/openstack/build/85c40976164c4b1785aefe0c1a4ddab9/log/logs/host/nova-compute.service.journal-11-24-25.log.txt#17356
12:23:21 noonedeadpunk And SHA `5b65812ef2c6ac46bea30e2e603284eac232339e` was working properly (but yeah, that's an SHA before merging libvirt 6 requirement)
12:24:19 noonedeadpunk are we missing some config change here?
12:27:39 lyarwood V to W right?
12:27:48 lyarwood where can I find the tempest logs?
12:28:08 lyarwood If it cast from the conductor didn't make it to the compute then this would smell like a rabbitmq issue tbh
12:30:09 jrosser the tempest log is here https://zuul.opendev.org/t/openstack/build/85c40976164c4b1785aefe0c1a4ddab9/log/logs/openstack/aio1-utility/tempest_run.log.txt
12:36:50 noonedeadpunk lyarwood: it makes to the compute, since compute has `Instance 40ce5b33-d2d8-470e-b8c8-776c53efce8c has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 256, 'VCPU': 1}}`
12:38:01 noonedeadpunk Except debug messages there's a single warning in the compute log, which is `This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.` But I guess it should not be critical?
12:48:59 noonedeadpunk lyarwood: sorry, you was right, it's rabbit issue :(
12:49:16 noonedeadpunk found the reason here, thanks for the clue!
12:50:49 lyarwood noonedeadpunk: yeah that message above isn't part of the actual request it's from a periodic job on the compute that notices that an instance was scheduled but hasn't started yet
12:50:56 lyarwood noonedeadpunk: np
13:31:14 sean-k-mooney noonedeadpunk: im guessing the compute node was not reciving rpc becuase the topic was lost on the queue?
13:33:20 noonedeadpunk sean-k-mooney: no, because we were using rabbitmq with self-signed SSL but without CA
13:33:29 noonedeadpunk which is not allowed in X
13:33:35 noonedeadpunk but was in V
13:34:13 noonedeadpunk and in that patch we upgraded not only nova, but also u-c, so yeah :(
13:35:58 noonedeadpunk (it's eventually because of the some amqp dependancy)
13:39:02 sean-k-mooney ah ok
13:44:26 openstackgerrit Merged openstack/nova master: Wallaby 23.0.0 prelude section https://review.opendev.org/c/openstack/nova/+/782172
13:45:49 openstackgerrit Merged openstack/nova master: docs: Add a real-time guide https://review.opendev.org/c/openstack/nova/+/781434
13:46:58 openstackgerrit Merged openstack/nova master: docs: Remove duplicate TPM extra spec info https://review.opendev.org/c/openstack/nova/+/781435
13:48:18 openstackgerrit Merged openstack/nova master: docs: Remove duplicated PCI passthrough extra spec info https://review.opendev.org/c/openstack/nova/+/781436
13:49:15 openstackgerrit Merged openstack/nova master: docs: Add SEV guide https://review.opendev.org/c/openstack/nova/+/781437
13:51:25 gibi \o/
14:12:34 openstackgerrit Radomir Dopieralski proposed openstack/python-novaclient master: When creating a client, pass the default logger https://review.opendev.org/c/openstack/python-novaclient/+/783010
14:13:18 Luzi lyarwood, could you take a look at https://review.opendev.org/c/openstack/nova/+/782430 ? i would like to propose the backport for stein too and start writing the ossn soon
14:15:37 openstackgerrit Merged openstack/nova master: Bump the Compute RPC API to version 6.0 https://review.opendev.org/c/openstack/nova/+/761452
14:16:04 lyarwood Luzi: done
14:16:59 Luzi lyarwood, thank you :)
14:21:42 sean-k-mooney gibi: actully if there is a followup patch to sev there are a few thigns we shoudl add
14:23:09 sean-k-mooney gibi: multi numa guest may fail to boot due to a qemu issue https://bugzilla.redhat.com/show_bug.cgi?id=1814502 and we shoudl be recommending that hw:mem_page_size be set since sev is locking memory
14:23:10 openstack sean-k-mooney: Error: Error getting bugzilla.redhat.com bug #1814502: NotPermitted
14:23:36 sean-k-mooney oh thats se to private i guess
14:24:15 openstackgerrit Mark Goddard proposed openstack/nova master: Check volume encryption keys prior to resize and migration https://review.opendev.org/c/openstack/nova/+/781079
14:28:22 gibi sean-k-mooney: you can still propose a reno extension for the RC1.
14:28:57 gibi sean-k-mooney: after today you need a bug that fix on master then backport to stable/wallaby and propose an RC2

Earlier   Later