| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-17 | |||
| 19:35:25 | gmann | lyarwood: ^^ | |
| 19:35:32 | gmann | lbragstad: sorry, wrong mention. | |
| 19:36:44 | lbragstad | gmann no worries - http://paste.openstack.org/show/803682/ is what i'm seeing | |
| 19:36:56 | lbragstad | line 49 shows the project-scoped token response | |
| 19:38:22 | lbragstad | mm - nevermind... i used test-config to setup nova.conf, and n-api must not have been bounced after the config update | |
| 19:38:48 | gmann | lbragstad: it should not allowed | |
| 19:39:12 | lbragstad | yep - you're right, i restarted the nova-api and it looks like it picked up the oslo_policy configs | |
| 19:39:23 | gmann | ohk, for policy change n-api does not need to restart but yes for config change it need | |
| 19:39:46 | gmann | lbragstad: nice. | |
| 22:29:34 | melwitt | lyarwood: no apology needed, will hit those. unrelated, curious if you've seen this before for gate failure in swap volume "Failure rebasing volume /dev/sdb on vdb.: libvirt.libvirtError: internal error: unable to execute QEMU command 'drive-mirror': Could not open '/dev/sdb': Permission denied" https://zuul.opendev.org/t/openstack/build/c9515f1edc774e74a84e91a7dee50074/log/compute1/logs/screen-n-cpu.txt#2138 seeing it only on stein | |
| 22:29:35 | melwitt | and train | |
| 23:26:23 | eandersson | ops melwitt missed that you had already re-checked the rocky patch :D | |
| 23:26:36 | eandersson | *stein | |
| 23:26:56 | melwitt | eandersson: np, maybe it will give us better luck :) | |
| 23:27:20 | eandersson | fingerscrossed! | |
| #openstack-nova - 2021-03-18 | |||
| 00:37:37 | melwitt | same root failure during create of cinder resources on stable/stein "libvirt.libvirtError: operation failed: 'serial' is deprecated, please use the corresponding option of '-device' instead Could not open '/dev/sdb': Permission denied" https://zuul.opendev.org/t/openstack/build/e1050a661fec47b49d6d7babc89d71d3/log/logs/screen-n-cpu.txt#3267 | |
| 00:37:57 | melwitt | why can't it open /dev/sdb -_- | |
| 00:39:07 | melwitt | rather, why is it Permission Denied. argh. | |
| 00:42:38 | melwitt | opened https://bugs.launchpad.net/openstack-gate/+bug/1919920 fwiw | |
| 00:42:40 | openstack | Launchpad bug 1919920 in OpenStack-Gate "volume related tests sometimes fail with " Could not open '/dev/sdb': Permission denied" on stable/train and stable/stein" [Undecided,New] | |
| 05:40:35 | openstackgerrit | Merged openstack/nova stable/ussuri: libvirt: Increase incremental and max sleep time during device detach https://review.opendev.org/c/openstack/nova/+/757306 | |
| 06:48:41 | openstackgerrit | Merged openstack/nova stable/pike: [stable-only] Move grenade jobs to experimental https://review.opendev.org/c/openstack/nova/+/781134 | |
| 07:02:36 | eandersson | The whole settings in the database is one of the hardest things for new operators to troubleshoot | |
| 07:03:04 | eandersson | When changing the names of rmq server there seems to be no errors in the logs for some reasons | |
| 07:03:20 | eandersson | And if operators are not aware of that "feature" it is probably the last place they will oook | |
| 07:03:23 | eandersson | *look | |
| 07:04:30 | eandersson | I guess maybe because the old rabbitmq servers are still online, so it's just a really bad "split brain". | |
| 07:05:31 | eandersson | At least for us a common scenario has been to have RabbitMQ and OpenStack services on the same node, but when scaling out having dedicated rmq servers is a thing | |
| 07:06:05 | eandersson | but it's not clear to anyone that the rmq config is actually in the database | |
| 07:06:17 | eandersson | (and again super difficult to troubleshoot) | |
| 09:35:23 | stephenfin | lyarwood, gibi: So in opting for HTTP 409, I was actually following two existing examples: we do this already for both SEV and vTPM issues | |
| 09:37:48 | stephenfin | Everything I've read about HTTP 403 suggests it's to do with authorization issues. That's why we use it for policy. There's are no issues with authorization here. The issue is that the server understood the request but can't support it. HTTP 409 seemed the best of a bad bunch | |
| 09:38:44 | stephenfin | HTTP 400 doesn't seem right either. That would suggest that the request was incorrectly formed. Again, it's correct but we simply can't support it | |
| 09:39:23 | stephenfin | lyarwood, gibi: https://github.com/openstack/nova/blob/master/nova/exception.py#L528-L537 | |
| 09:44:07 | lyarwood | stephenfin: right, I saw the examples and I was initially okay with this | |
| 09:44:46 | lyarwood | stephenfin: I honestly don't care either way tbh, both are a compromise AFAICT | |
| 09:45:18 | lyarwood | stephenfin: lets chat with gmann later and agree which way to land on this before the deadline tomorrow that gibi raised yesterday | |
| 09:45:26 | stephenfin | I think so too. I thought there was an explicit "the server understands this request but can't service it" code, but I can't find one | |
| 09:45:47 | stephenfin | Sounds good. I'll propose a spec update on the assumption that we're going with HTTP 409. I can update if gmann talks us around | |
| 09:47:53 | lyarwood | sounds good | |
| 09:47:57 | lyarwood | ops snap | |
| 09:48:09 | lyarwood | stephenfin: how's the hangover btw? | |
| 09:48:39 | stephenfin | from all the pints in the pubs I wasn't allowed into? | |
| 09:48:40 | stephenfin | :) | |
| 09:49:26 | lyarwood | stephenfin: fair point, I did stumble across this yesterday and suddenly wanted a pint https://soundsofthepub.com/ | |
| 09:50:39 | lyarwood | 24 days until beer gardens open | |
| 09:50:43 | lyarwood | and I bet it rains, all day. | |
| 09:50:45 | stephenfin | hahaha | |
| 09:50:49 | stephenfin | nice one | |
| 09:51:20 | gibi | stephenfin, lyarwood: yeah, as I said before I have no hard opininon. And also nova is not consistent about such error code so nothing to force ourselves into. | |
| 09:51:40 | kashyap | lyarwood: What do you mean 24 days? Is it some UK deadline? | |
| 09:51:48 | kashyap | Morning, BTW | |
| 09:51:49 | lyarwood | kashyap: yes | |
| 09:52:09 | kashyap | lyarwood: Sweet. On rain ... paradise is a well-watered garden, ya know :D | |
| 09:52:12 | lyarwood | kashyap: beer gardens (pubs) and outside dining allowed in ~24 days. | |
| 09:52:18 | gibi | stephenfin: I think the understood, but cannot support is a 500 code like 501 | |
| 09:52:27 | kashyap | lyarwood: Nice. I thought I'll slowly go senile and die before seeing any of these | |
| 09:52:53 | kashyap | (I mean, they're making similar murmurs here too of opening by 01May) | |
| 09:54:49 | gibi | stephenfin: but the https://specs.openstack.org/openstack/api-wg/guidelines/http/response-codes.html#use-of-501-not-implemented disagrees with me about 501 | |
| 09:56:18 | gibi | if we follow that guide ^^ then we should use 400 | |
| 09:56:42 | gibi | but I agree to wait for gmann to get up and make a decision | |
| 10:00:46 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/victoria: DNM - Testing glance backports to fix bug #1919993 https://review.opendev.org/c/openstack/nova/+/781237 | |
| 10:00:48 | openstack | bug 1919993 in Glance "test_image_web_download_import_with_bad_url failing on stable/victoria" [Undecided,New] https://launchpad.net/bugs/1919993 | |
| 10:01:17 | lyarwood | dansmith: ^ re https://bugs.launchpad.net/glance/+bug/1919993 and https://review.opendev.org/c/openstack/glance/+/781236 - when you're online could you take a swing at these glance backports to unblock stable/victoria for Nova? | |
| 10:01:36 | lyarwood | branchless tempest ftw once again | |
| 10:07:11 | lyarwood | elod: ^ btw, that should unblock stable/victoria, not sure if you're in stable-glance-core via stable-maint? | |
| 10:15:18 | elod | lyarwood: yes I have +2 right there as well, I'll review it now | |
| 10:18:40 | lyarwood | thanks | |
| 10:20:14 | gibi | stephenfin: I need your sphinx experties in https://review.opendev.org/c/openstack/nova/+/780401/1/nova/api/openstack/compute/rest_api_version_history.rst#1145 | |
| 10:20:35 | gibi | stephenfin: the current way is fine by me but maybe there is other possibilities | |
| 10:23:03 | stephenfin | gibi: That works, but we could/should probably just update the release note in novaclient | |
| 10:23:28 | stephenfin | i.e. add https://review.opendev.org/c/openstack/nova/+/780401/1/nova/api/openstack/compute/rest_api_version_history.rst#392 | |
| 10:24:44 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: trivial: Fix spelling, formatting of vDPA spec https://review.opendev.org/c/openstack/nova-specs/+/781260 | |
| 10:24:44 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Change error code when vDPA is unsupported https://review.opendev.org/c/openstack/nova-specs/+/781259 | |
| 10:26:02 | gibi | stephenfin: so have an explicit well named anchor in the nova doc and update the novaclient reno to refer to the well named anchor? | |
| 10:26:14 | stephenfin | I think that would be a better approach, yes | |
| 10:26:20 | stephenfin | Personally | |
| 10:26:47 | gibi | OK lets do that | |
| 10:26:57 | gibi | I can propose the fix | |
| 10:27:05 | stephenfin | Sounds good. Happy to review | |
| 10:27:09 | gibi | cool | |
| 11:10:30 | openstackgerrit | Lee Yarwood proposed openstack/nova master: compute: Reject requests to commit intermediary snapshot of an inactive instance https://review.opendev.org/c/openstack/nova/+/781138 | |
| 11:12:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Delegate OVS plug to os-vif https://review.opendev.org/c/openstack/nova/+/602432 | |
| 11:20:23 | lyarwood | gibi: mind if I add a cinder cross project session in the PTG pad, have a topic. | |
| 11:20:38 | gibi | lyarwood: please do | |
| 11:28:10 | lyarwood | done thanks | |
| 11:50:07 | lyarwood | elod: https://review.opendev.org/c/openstack/nova/+/758733/ is ready for review btw | |
| 12:31:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add a resource limits guide https://review.opendev.org/c/openstack/nova/+/781433 | |
| 12:31:09 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Remove duplicate TPM extra spec info https://review.opendev.org/c/openstack/nova/+/781435 | |
| 12:31:09 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add a real-time guide https://review.opendev.org/c/openstack/nova/+/781434 | |
| 12:31:10 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add SEV guide https://review.opendev.org/c/openstack/nova/+/781437 | |
| 12:31:10 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Remove duplicated PCI passthrough extra spec info https://review.opendev.org/c/openstack/nova/+/781436 | |
| 12:31:11 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Change formatting of hypervisor config guides https://review.opendev.org/c/openstack/nova/+/781439 | |
| 12:31:11 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add CPU models guide https://review.opendev.org/c/openstack/nova/+/781438 | |
| 12:31:12 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add libvirt misc doc https://review.opendev.org/c/openstack/nova/+/781440 | |
| 12:31:29 | stephenfin | Some good docs there. I was holding off until after feature freeze ^ | |
| 12:36:45 | kashyap | stephenfin: Cool; a separate CPU models guide is good; instead of that info buried elsewhere | |
| 12:37:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add image metadata property reference guide https://review.opendev.org/c/openstack/nova/+/756867 | |
| 12:37:10 | kashyap | stephenfin: We could also link docs like these into it: something like "See also: https://docs.openstack.org/nova/latest/admin/mitigation-for-Intel-MDS-security-flaws.html" | |
| 12:37:31 | kashyap | If only to make it more discoverable; /me adds a note in the review | |