| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-07-15 | |||
| 19:05:25 | opendevreview | Balazs Gibizer proposed openstack/nova master: [func test] ports with both bw and pps resources https://review.opendev.org/c/openstack/nova/+/792394 | |
| 19:05:26 | opendevreview | Balazs Gibizer proposed openstack/nova master: Parse extended resource request from the port data https://review.opendev.org/c/openstack/nova/+/800085 | |
| 19:05:28 | opendevreview | Balazs Gibizer proposed openstack/nova master: Transfer RequestLevelParams from ports to scheduling https://review.opendev.org/c/openstack/nova/+/791506 | |
| 19:07:06 | amodi | spatel: yes so it looks like u need to make it 'available' first, the instance creation error won't persist | |
| 19:07:36 | amodi | im not an expert, so that was my best guess | |
| 19:09:19 | spatel | mostly they are available when you create but this is very odd | |
| 19:16:19 | opendevreview | Balazs Gibizer proposed openstack/nova master: Support boot with extended resource request https://review.opendev.org/c/openstack/nova/+/800086 | |
| 19:16:20 | opendevreview | Balazs Gibizer proposed openstack/nova master: Support move ops with extended resource request https://review.opendev.org/c/openstack/nova/+/800087 | |
| 19:18:50 | opendevreview | Balazs Gibizer proposed openstack/nova master: [func test]Refactor interface attach with qos https://review.opendev.org/c/openstack/nova/+/800088 | |
| 19:18:51 | opendevreview | Balazs Gibizer proposed openstack/nova master: Support interaface attach / detach with new resource request format https://review.opendev.org/c/openstack/nova/+/800089 | |
| 19:18:54 | opendevreview | Balazs Gibizer proposed openstack/nova master: [func test] move unshelve test to the proper place https://review.opendev.org/c/openstack/nova/+/793621 | |
| 20:18:23 | spatel | found my problem, on that AZ all the compute nodes where disabled in openstack that is why it was showing not available :) | |
| 20:18:27 | spatel | sorry for confusion | |
| 21:28:07 | JayF | Is there a known issue with stable/wallaby CI? I've been trying to get https://review.opendev.org/c/openstack/nova/+/799772 through the gate and there's a barrage of random-seeming failures. The change itself is in the ironic driver; the failures are clearly unrelated. | |
| 21:28:27 | JayF | I feel a little bad rechecking something 5+ times to get it through, so I thought I'd check to ensure there's not any known issues with stable/wallaby gate. | |
| 21:30:23 | JayF | A majority, but not all of the failures have been a 400 error around Invalid Volume (details in comments on the patch) | |
| 21:31:11 | sean-k-mooney | at least on of the failures is the device detach bug for volume detach | |
| 21:31:24 | sean-k-mooney | Device detach failed for vdb: Run out of retry while detaching device vdb with device alias virtio-disk1 | |
| 21:31:31 | JayF | That sounds like it matches the error I was referring to | |
| 21:31:32 | sean-k-mooney | JayF: that is a known isue | |
| 21:31:42 | JayF | Is there a backportable patch for it? Or just keep slamming rechecks | |
| 21:31:59 | sean-k-mooney | there is a fix for it im not sure if it has been backported | |
| 21:32:26 | sean-k-mooney | tl;dr qemu change the behavior so that trying to detach a volume that is detaching woudl cancel the detach and reslut in an error | |
| 21:32:34 | sean-k-mooney | that broke nova retry logic | |
| 21:32:34 | JayF | hmm. Maybe I'll dig and see if it backports cleanly. I obviously know a lot more about Ironic than nova, but if I can get it fixed why not :D | |
| 21:32:46 | sean-k-mooney | the fix is to use event instead of a retry loop | |
| 21:33:07 | sean-k-mooney | JayF: gibi and lyarwood have looked at backporting it | |
| 21:33:19 | sean-k-mooney | let me see if i can find it | |
| 21:33:52 | JayF | This patch I'm trying to land fixes a nasty bug in Ironic that we have confirmed multiple operators are experiencing, so I'm trying to get it backported as far as possible :D I appreciate the help | |
| 21:35:11 | sean-k-mooney | did you say this was wallaby or is it an older release | |
| 21:35:37 | sean-k-mooney | it looks like ist on wallaby https://review.opendev.org/c/openstack/nova/+/788720 | |
| 21:35:49 | JayF | Right now wallaby, when that lands I'll go w-1, and so on until it's not clean :) | |
| 21:36:03 | JayF | but I know my deployment will need it for wallaby, and a couple others at a minimum | |
| 21:36:40 | JayF | Uh oh. So it shouldn't be happening :-| | |
| 21:36:53 | sean-k-mooney | so the event change patch has been backported to wallaby already so you might be hitting the other detach bug | |
| 21:37:36 | sean-k-mooney | there is an issue where the guest seam to not respond to the detach properly in all caes | |
| 21:37:54 | sean-k-mooney | as a workaround we were looking ad modifying tempest to not do detaches when not needed | |
| 21:39:24 | JayF | That sounds good, so for now I'm going to keep rechecking and hope to win the gate-lottery. I'd offer to review the tempest workaround but I'm not sure a +1 form me would be that helpful :D | |
| 21:40:14 | sean-k-mooney | JayF: https://review.opendev.org/c/openstack/tempest/+/794757 i think this might be it | |
| 21:40:53 | sean-k-mooney | unfortuetly lee i snot around for the next 2 week due to real world stuff so not sure if that is going to proguress | |
| 21:41:19 | JayF | That's fine :) I'm honestly surprised to have gotten the concierge bug walkthrough with you, and I appreciate it :D. | |
| 21:41:25 | sean-k-mooney | JayF: in general device detach has been flakeing in the jobs for the last why and it does not really seam to be related ot change in nova code | |
| 21:41:44 | JayF | It's extra annoying for VM errors to be cratering an ironic-driver-only change :P | |
| 21:41:50 | sean-k-mooney | the latest issue may be a regression in qemu | |
| 21:41:57 | sean-k-mooney | JayF: yes | |
| 21:42:31 | JayF | I guess targetting a bunch of different qemu versions might have a similar problems to what we see in Ironic with ... unfortunately designed hardware BMCs :P | |
| 21:43:39 | sean-k-mooney | ill ask gibi tomorrow if we should consider making the job non voteing. it generally has not been wedgeing the gate but it has been failing frequetly enough that we might want to consider skiping the test or just manually reviewing the job for now | |
| 21:44:47 | JayF | thanks! I'll keep idling in here, feel free to ping me if there's anything I can do to help :) | |
| 21:45:07 | JayF | [or if you ever need someone with the big review hammer in Ironic :D] | |
| 21:45:48 | sean-k-mooney | i wonder if we could make the live migation jobs skip on change that only affect the ironic driver | |
| 21:46:15 | sean-k-mooney | you know since i dont precive that mattering to your use case unless the laws of physic cahnge | |
| 21:46:51 | JayF | I think that's a potentially wise change to make, but don't underestimate the ability of python to violate the laws of physics :P | |
| 21:47:43 | sean-k-mooney | im pretty sure we could add nova/test/unit/virt/ironic and nova/virt/ironic to the irrelevent files list | |
| 21:48:23 | JayF | I'd be happy to +1 such a change | |
| 21:49:03 | sean-k-mooney | so we would jsut need to add those here https://github.com/openstack/nova/blob/master/.zuul.yaml#L93-L107 | |
| 21:49:44 | JayF | and, for purposes of my patch, it'd also need to be backported | |
| 21:50:20 | sean-k-mooney | well that used selse where so we would have to refactor slightly it should not be terrible. yep we would have to backport it first | |
| 21:50:52 | JayF | hmm where else would it need to go :) | |
| 21:51:03 | JayF | I was already on the way to making the change against main branch | |
| 21:52:20 | sean-k-mooney | currently the deffinition of *nova-base-irrelevant-files is in the live migraiton job we would need to move that to say nova-next which also uses it | |
| 21:52:44 | sean-k-mooney | then defein a new nova-live-migration-irrelavent-files that the ceph job would also use | |
| 21:53:01 | sean-k-mooney | although that may or may not be enough to get your patch to pass | |
| 21:53:19 | sean-k-mooney | e.g. the grenade job might fail in a similar way | |
| 21:53:45 | JayF | Yeah, we've seen it fail on several different jobs for sure. Reducing the number of times we have to roll the dice will only raise the likelihood of that patch getting landed, though | |
| 21:54:25 | sean-k-mooney | am its late for me corrently so im going to call it a night | |
| 21:54:45 | JayF | o/ have a good one | |
| 21:54:52 | sean-k-mooney | for now i would just recheck it again o/ | |
| 21:55:07 | JayF | I'm getting really good at it by now :D Thanks again, have a nice night | |
| 23:07:20 | opendevreview | MarkMielke proposed openstack/nova master: extend_volume of libvirt/volume/iscsi should not use device_path https://review.opendev.org/c/openstack/nova/+/801003 | |
| #openstack-nova - 2021-07-16 | |||
| 07:32:54 | opendevreview | Yongli He proposed openstack/nova master: Smartnic support - cyborg drive https://review.opendev.org/c/openstack/nova/+/771362 | |
| 07:32:56 | opendevreview | Yongli He proposed openstack/nova master: smartnic support - new vnic type https://review.opendev.org/c/openstack/nova/+/771363 | |
| 07:32:59 | opendevreview | Yongli He proposed openstack/nova master: smartnic support - create arqs https://review.opendev.org/c/openstack/nova/+/758944 | |
| 07:33:01 | opendevreview | Yongli He proposed openstack/nova master: smartnic support - build instance with smartnic arqs https://review.opendev.org/c/openstack/nova/+/798249 | |
| 07:33:02 | opendevreview | Yongli He proposed openstack/nova master: smartnic support - cleanup arqs https://review.opendev.org/c/openstack/nova/+/798054 | |
| 07:33:12 | opendevreview | Yongli He proposed openstack/nova master: smartnic support - reject server move and suspend https://review.opendev.org/c/openstack/nova/+/779913 | |
| 07:33:28 | opendevreview | Yongli He proposed openstack/nova master: smartnic support - functional tests https://review.opendev.org/c/openstack/nova/+/780147 | |
| 08:32:30 | opendevreview | Merged openstack/nova stable/wallaby: [ironic] Minimize window for a resource provider to be lost https://review.opendev.org/c/openstack/nova/+/799772 | |
| 10:58:41 | opendevreview | Balazs Gibizer proposed openstack/placement master: Bump os-resource-classes requirements https://review.opendev.org/c/openstack/placement/+/796595 | |
| 11:22:52 | gibi | kashyap: I'm just loking at https://blueprints.launchpad.net/nova/+spec/remove-support-for-floppy-disks which seem to be in a limbo state as it is targeted to xena but it was never set to approved. Do you happen to remember if there was agreemet about that in Xena timeframe? | |
| 11:29:00 | sean-k-mooney | at a minium we proably want to deprecate in xena | |
| 11:29:34 | sean-k-mooney | i dont recalls if we said we would remvoe or not in xena | |
| 12:27:06 | gibi | sean-k-mooney: my problem is that bp state is in limbo so I'm not sure we ever approve this bp | |
| 12:27:16 | gibi | I hope kashyap remembers :D | |
| 13:04:27 | kashyap | gibi: Hi, was away for lunch | |
| 13:06:19 | kashyap | gibi: I only recall a "general agreement" (no active objections on the list - from more than a year ago) | |
| 13:06:40 | kashyap | gibi: Yes, deprecate for sure in Xena | |
| 13:07:10 | gibi | kashyap: do you have intention to working on that bp before Milestone 3 (Sept 3)? | |
| 13:07:16 | kashyap | gibi: Yes | |
| 13:07:47 | kashyap | gibi: Can we remove it in Xena, even? Who uses floppy drives anyway | |
| 13:07:57 | kashyap | I'm expecting a "well, actually" here :D | |
| 13:12:28 | gibi | so there was generic agreement around it so I guees I made a mistake not recording that properly in the bp early in xena | |
| 13:12:43 | gibi | so I will move it to approved now | |
| 13:15:50 | gibi | kashyap: as of the implementation I think we need deprecation first | |
| 13:16:36 | kashyap | gibi: Yeah fair enough | |
| 13:27:18 | opendevreview | Dmitrii Shcherbakov proposed openstack/nova-specs master: Integration With Off-path Network Backends https://review.opendev.org/c/openstack/nova-specs/+/787458 | |
| 14:08:11 | kashyap | gibi: Thanks; I saw the changes | |
| 14:42:07 | dmitriis | gibi, sean-k-mooney: re-proposed https://review.opendev.org/c/openstack/nova-specs/+/787458 for Yoga. There's some progress with Neutron and OVN. | |
| 14:42:07 | dmitriis | Neutron: the RFE https://bugs.launchpad.net/neutron/+bug/1932154 got rfe-approved after the discussion in the drivers meeting https://meetings.opendev.org/irclogs/%23openstack-neutron/%23openstack-neutron.2021-07-09.log.html. We also had a +1 from Brian Haley on the spec https://review.opendev.org/c/openstack/neutron-specs/+/788821 (auto-cleared | |
| 14:42:07 | dmitriis | after a re-upload with nit fixes). | |
| 14:42:07 | dmitriis | OVN: Things moved forward in this thread https://patchwork.ozlabs.org/project/ovn/patch/20210509140305.1910796-1-frode.nordahl@canonical.com/#2699171 and some OVN-side implementation details have changed but nothing Nova needs to care about. We are currently waiting for feedback from Ilya Maximets (hopefully next week) and had some feedback from | |