| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-09-13 | |||
| 17:39:21 | bauzas | okay, I gonna call it a wrap too, but I'll poke around after dinner, since we have a couple of changes upfront in the gate | |
| 17:42:12 | opendevreview | Sylvain Bauza proposed openstack/nova master: Add a Bobcat prelude section https://review.opendev.org/c/openstack/nova/+/894940 | |
| 17:42:29 | dansmith | bauzas: I've seen this fail twice this morning already, not sure if something changed or not: https://b5ee1ed3653a458879e1-60fa9bbec8248937c3af4b3a8047f40b.ssl.cf2.rackcdn.com/893540/6/check/nova-live-migration/6e12356/testr_results.html | |
| 17:42:57 | bauzas | dansmith: yeah that's a known bug | |
| 17:43:13 | bauzas | https://bugs.launchpad.net/neutron/+bug/1940425 | |
| 17:43:32 | bauzas | it's old but its occurrences raised recently | |
| 17:43:45 | dansmith | bauzas: yeah, but I haven't seen it very much in the past few months, but several times today now | |
| 17:43:50 | dansmith | okay | |
| 17:44:27 | dansmith | critical but unfixed in neutron since 2021..eesh | |
| 17:44:29 | bauzas | I haven't pinged lajoskatona ralonsoh nd the other neutron cores | |
| 17:44:37 | bauzas | (yet) | |
| 17:45:20 | bauzas | dansmith: well, I guess they just don't check the number of critical bugs they have on a weekly basis, that's it :D | |
| 18:18:06 | gmann | dansmith: seems we need to put this whole things under cinder check https://review.opendev.org/c/openstack/nova/+/893540/6/roles/run-evacuate-hook/files/test_evacuate.sh#58 | |
| 18:18:22 | gmann | it is failing there https://zuul.opendev.org/t/openstack/build/5efc0aaf88874c45b2eb68d8e9cf4a0d/console | |
| 18:18:24 | dansmith | gmann: dammit | |
| 18:18:45 | dansmith | I was just going to fix one thing, see how it goes, fix another, etc | |
| 18:18:54 | dansmith | you keep pointing out all the problems and are making me look bad :) | |
| 18:29:53 | opendevreview | Dan Smith proposed openstack/nova master: Make our nova-ovs-hybrid-plug job omit cinder https://review.opendev.org/c/openstack/nova/+/893540 | |
| 18:43:42 | lajoskatona | dansmithm bauzas: I think we have now this bug for that issue: https://bugs.launchpad.net/neutron/+bug/2033887 and this is the patch series for that: https://review.opendev.org/q/Ifc2d37e2042fad43dd838821953defd99a5f8665 | |
| 18:45:41 | dansmith | lajoskatona: col | |
| 18:45:44 | dansmith | er cool even | |
| 18:54:02 | gmann | dansmith: you doing so much work and on gate stability makes you always better and not bad :) | |
| 18:54:24 | dansmith | heh | |
| 19:03:44 | bauzas | dansmith: fwiw, also seeing more kernel crashes like https://bb94d2825af897cfcd12-f6f1806a4829a343b7540be166a34ea9.ssl.cf5.rackcdn.com/860829/4/check/nova-next/bf91602/testr_results.html | |
| 19:04:05 | bauzas | again an already known issue but more failures | |
| 19:04:53 | dansmith | bauzas: always with no space left on device? | |
| 19:05:08 | bauzas | don't think so | |
| 19:05:18 | bauzas | I have to doublecheck tho | |
| 19:05:24 | dansmith | that's during boot, so it looks more like maybe a broken image creation or if it's on ceph, maybe a backend problem | |
| 19:06:05 | dansmith | oh volume boot, so yeah something wrong with the actual volume | |
| 19:06:21 | bauzas | yeah probably a ceph RC | |
| 19:06:26 | dansmith | it never mounted and thus couldn't pivot over and the ENOSPC comes from having not mounted something writable on the target | |
| 19:20:52 | bauzas | hmpfff | |
| 19:21:35 | bauzas | Sep 13 13:49:37.042893 np0035238841 nova-compute[40581]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' | |
| 19:21:54 | dansmith | always logged by brick AFAIK :( | |
| 19:23:17 | bauzas | I wish I would be Neo 'operator, get me all the knowledge about chopters, err. volume bindings in OpenStack" | |
| 19:24:40 | bauzas | "Tank, I need a pilot program for B-212 helicopter. Hurry.� | |
| 19:24:40 | bauzas | "Tank, I need a pilot program for B-212 helicopter. Hurry.” | |
| 19:24:54 | bauzas | (found the right catchphrase) | |
| 19:25:11 | opendevreview | Dan Smith proposed openstack/nova master: Make our nova-ovs-hybrid-plug job omit cinder https://review.opendev.org/c/openstack/nova/+/893540 | |
| 19:25:16 | dansmith | gmann: looks like it worked this time, but one fix on the cleanup part | |
| 19:25:55 | dansmith | "choppers" or "copters" not "chopters" :P | |
| 19:25:57 | bauzas | so the backref thing is probably about to be punted | |
| 19:26:13 | bauzas | yeah, damn me, it's late | |
| 19:26:27 | bauzas | and when I was young, this was dubbed in French | |
| 19:27:08 | bauzas | so, the multipathd error is just a "normal error" ? | |
| 19:27:14 | bauzas | that's fun | |
| 19:29:32 | JayF | multipathd is only needed for some wacky hardware; we have a similar setup in IPA where we opportunistically load it | |
| 19:31:04 | dansmith | by wacky you mean almost anyone using FC :) | |
| 19:34:18 | JayF | heh, I wasn't exactly sure what storage tech so you know, wacky hardware ;) | |
| 19:34:29 | JayF | nothing we have in CI/gate was more the point :D | |
| 20:08:46 | opendevreview | Merged openstack/nova master: Update compute rpc alias for bobcat https://review.opendev.org/c/openstack/nova/+/893744 | |
| 21:32:21 | opendevreview | Merged openstack/nova master: Revert "Make compute node rebalance safter" https://review.opendev.org/c/openstack/nova/+/894944 | |
| #openstack-nova - 2023-09-14 | |||
| 05:53:41 | opendevreview | Nobuhiro MIKI proposed openstack/nova-specs master: Re-propose "Add maxphysaddr support for Libvirt" for 2024.1 Caracal https://review.opendev.org/c/openstack/nova-specs/+/895135 | |
| 07:41:27 | sahid | sean-k-mooney: o/ we have discussed about using tenant isolation, but that is not really responding our use case an now all vms scheduled for that given tenant are targeted to that az | |
| 07:42:51 | sahid | we would like that, the scheduler consider the new az for some specified tenants | |
| 07:43:03 | sahid | any idea about how we can achieve that? | |
| 07:44:11 | sahid | basically for a given tenant allowed for az3 the return used by the scheduler will be [az1, az2, az3], and for the other tenants only [az1, az2] | |
| 07:46:05 | sahid | bauzas: o/ you may have some idea ^ | |
| 07:46:57 | sahid | i'm thinking about to fix a bit multitenantisolation to acheive that | |
| 08:24:45 | opendevreview | Merged openstack/nova master: Add service version for Bobcat https://review.opendev.org/c/openstack/nova/+/893749 | |
| 08:27:20 | bauzas | sahid: looking (I was afk) | |
| 08:28:22 | bauzas | so you would like to return different AZs based on the tenant ? | |
| 08:28:55 | bauzas | if so, yeah, it's https://github.com/openstack/nova/blob/master/nova/scheduler/request_filter.py#L95 | |
| 08:37:08 | sahid | hum... | |
| 09:09:19 | sahid | that does not really work, we will have to duplicate our aggregates and set filter_tenant_id to all the duplicate ones | |
| 09:11:01 | sahid | I guess in that way, for a given tenant it will be possible to schedule on as1, az2, az3 using filter_tenant_id and the other will continue on az1 and az2 | |
| 09:49:40 | zigo | stephenfin: FYI, I also found that freezer-api probably needs some sqlalchemy 2.x love too... | |
| 09:49:49 | zigo | TypeError: LegacyEngineFacade.from_config() got an unexpected keyword argument 'autocommit' | |
| 09:49:50 | zigo | ... | |
| 09:50:17 | zigo | failures=511 | |
| 09:50:38 | zigo | Is it possible that just removing the autocommit argument is enough? | |
| 09:52:23 | sean-k-mooney | is freezer still supported | |
| 09:52:29 | zigo | Wooops... s/freezer/watcher/ | |
| 09:52:44 | zigo | My bad. | |
| 09:53:05 | sean-k-mooney | i have almost the same qustion for watcher :P although i had seen that mention at least this side of the pandemic | |
| 09:53:28 | zigo | To the contrary of freezer, there was some commits for this cycle. | |
| 09:54:30 | sean-k-mooney | there was like 8 commit in watcher most procedural | |
| 09:54:43 | sean-k-mooney | but i guess its stil offically supported | |
| 09:56:06 | sean-k-mooney | tobias-urdin: you were making some changes to watcher eiarler in the year. is that a project you have use of personaly | |
| 09:56:50 | sean-k-mooney | just wondering if patches were created how likely is it that they would be merged/reviewd | |
| 11:31:14 | opendevreview | Amit Uniyal proposed openstack/nova master: WIP:Adds device tagging functional tests https://review.opendev.org/c/openstack/nova/+/895162 | |
| 11:51:09 | tobias-urdin | sean-k-mooney: we do not use it, nothing i actively work on, i just did some drive-by fixes for oslo.messaging or fixing ci probably | |
| 11:57:50 | sean-k-mooney | tobias-urdin: ack ya that is what i was assumign i was just wondering if there were any active contibutors left | |
| 12:24:30 | bauzas | ralonsoh: while https://review.opendev.org/c/openstack/neutron/+/893447 is now merged (the neutron revert), we still have CI failures today https://zuul.opendev.org/t/openstack/build/01dd3befff5e42cfb0c0f667b8e7e30b | |
| 12:28:54 | bauzas | elodilles: I guess you know the current situation in Nova | |
| 12:29:06 | bauzas | we still need to await a few merges before RC1 | |
| 12:30:53 | bauzas | hmpfff https://zuul.opendev.org/t/openstack/builds?job_name=nova-live-migration&skip=0 | |
| 12:31:05 | bauzas | 30% of failures I'd say | |
| 12:44:09 | mgariepy | is it possible to force nova a device_type for pci devices ? | |
| 12:44:55 | sean-k-mooney | no | |
| 12:45:51 | mgariepy | i wiant to only do some pci passthrough on a couple of A100 | |
| 12:46:14 | sean-k-mooney | you can filter by the pci adress | |
| 12:46:39 | sean-k-mooney | with support for regxes or bash globs if you dont want to just use the adress | |
| 12:46:51 | mgariepy | https://github.com/openstack/nova/blob/stable/2023.1/nova/pci/stats.py#L682C1-L683C1 | |
| 12:47:52 | sean-k-mooney | yes if the alais is not device_type pf then we filter them out when considering candiates | |
| 12:48:10 | sean-k-mooney | that does not mean you cant reqeust PF for passthrough | |
| 12:48:35 | sean-k-mooney | you jsut have to set the device_type to type-PF in the pci ailais | |
| 12:49:22 | sean-k-mooney | so inestead of type-pci or type-vf https://github.com/openstack/nova/blob/stable/2023.1/nova/pci/request.py#L34 you woudl set type-PF | |
| 12:50:50 | sean-k-mooney | the alias and device list work toghtere to selec tthe device. the alias is how you describe the request and the device_list (pci whitelist) is how you declar what device may be allocated | |