| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-10-29 | |||
| 14:42:05 | gibi | I'm trying to figure out the issue in the bandwidth tempest tests https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9ab/802060/18/check/nova-next/9ab4682/testr_results.html | |
| 14:42:24 | gibi | it seems that the tempest code now thinks that the new neutron API extension is enabled | |
| 14:42:35 | gibi | but that is not merged yet https://review.opendev.org/c/openstack/neutron/+/805637 | |
| 14:44:05 | gmann | gibi: ah, got it. | |
| 14:44:15 | gibi | api_extensions = all | |
| 14:44:15 | gibi | there is | |
| 14:44:17 | gibi | in the tempest conf | |
| 14:44:22 | gmann | gibi: I think we should make depends-on in otherway around | |
| 14:44:46 | gmann | tempest tests should be depends-on on neutron? or neutron patch fail on existing test? | |
| 14:45:11 | gibi | the neutron patch is OK to land as that adds the extension that makes the test passing | |
| 14:45:22 | gibi | I have another nova patch the fails now until the neutron patch lands | |
| 14:45:43 | gibi | and also the newer tempest patches adding pps coverage are failing unit the neutron patch lands | |
| 14:45:47 | gmann | gibi: you mean tempest existing test (before https://review.opendev.org/c/openstack/tempest/+/809168) fail ? | |
| 14:46:36 | gibi | I mean the test we adapeted to the new neutron API in https://review.opendev.org/c/openstack/tempest/+/809168 now fails for other patches as the neutron patch haven't landed yet | |
| 14:47:10 | gibi | which is due to the fact that tempest is configured that all the API extensions are available even the one that is not merged yet | |
| 14:47:19 | gmann | gibi: yeah. but if we revert 809168 and merge the series after neutron patch then is it fine? | |
| 14:47:32 | gmann | tempest seriesw | |
| 14:48:08 | gibi | I would keep the 809168 and wait for https://review.opendev.org/c/openstack/neutron/+/805637 to land clearing up the situation | |
| 14:48:13 | gmann | or existing assert which was modified in 809168 will fail if neutron patch merged as it change the response? | |
| 14:48:38 | gibi | yepp we need 809168 before we land the neutron API change as that would break existing tempest | |
| 14:48:44 | gibi | so I think we are OK as is now | |
| 14:49:01 | gibi | just cannot land the nova change https://review.opendev.org/c/openstack/nova/+/802060 until the neutron API extension lands | |
| 14:49:02 | gmann | gibi: I am wondering why neutron gate is not blocked as neutron-ovs-tempest-multinode-full fail after 809168 | |
| 14:49:15 | gibi | maybe it is :) | |
| 14:49:24 | gibi | let me check | |
| 14:49:39 | gmann | yeah because my DNM patch was just on neutron master https://review.opendev.org/c/openstack/neutron/+/815898 | |
| 14:50:17 | gmann | https://zuul.openstack.org/builds?job_name=neutron-ovs-tempest-multinode-full | |
| 14:51:05 | gibi | yepp your DNM patch shows that the ovs job is blocked | |
| 14:53:48 | gibi | this is a deadlock, we cannot have 809168 before the neutron API extension as that block everythin except the API extension patch, but we cannot have the neutron API extension before 809168, because that also blocks everything except the tempest patch | |
| 14:53:56 | gibi | /o\ | |
| 14:54:51 | gibi | and this is basically because tempest + devstack declares a neutron API extension exists before it is really exists | |
| 14:55:04 | gmann | gibi: but i am confused as test modified/merged in 809168 is passing in few pass of same job https://zuul.openstack.org/builds?job_name=neutron-ovs-tempest-multinode-full | |
| 14:55:17 | gmann | and failing instances of that job does now run that tests | |
| 14:55:34 | gmann | so I am not clear if master gate is blocked and if not then why | |
| 14:56:36 | gmann | gibi: we have the way to unblock this type of deadlock in tempest which is common way when API change. but test is passing on current master after 809168 is not clear to me | |
| 14:57:02 | gibi | I think the test is not passing on current master, except in https://review.opendev.org/c/openstack/neutron/+/805637 | |
| 14:57:41 | gibi | our patches top that | |
| 14:57:49 | gibi | s/our/or/ | |
| 14:58:57 | gmann | gibi: ah yeah, its in the series patches only. let me check failing one then | |
| 14:59:21 | gibi | your DNM patch is a good example how the neutron master fails today | |
| 14:59:45 | gmann | yeah | |
| 15:01:34 | prometheanfire | gibi: thanks | |
| 15:02:00 | gmann | gibi: for deadlock we can do: step1. revert the 809168 to unblock the gate and for neutron/nova change to land -> step 1. skip the test in tempest which is allow neutron change to merge 2. unskip the tempest test along with revert of revert of809168 | |
| 15:02:57 | gmann | that is how we land bug fixes in projects if tempest test block that | |
| 15:03:29 | gibi | sounds complicated | |
| 15:03:49 | gmann | gibi: ah this is +A so we can just wait for this? https://review.opendev.org/c/openstack/neutron/+/805637/12 | |
| 15:04:24 | gibi | gmann: yeah, if I know that the current recheck of 805637 will be successfully then yes | |
| 15:04:37 | gibi | it is friday, I can recheck it during the weekend | |
| 15:04:52 | gibi | so probably we not loosing much if we keep the gate broken for the weekend | |
| 15:05:24 | gmann | gibi: +1, sure. I will also keep monitoring it. | |
| 15:05:30 | gibi | gmann: OK, thanks | |
| 15:05:40 | gibi | then fingers crossed | |
| 15:05:40 | gmann | gibi: just for future ref if needed. this is process in tempest side https://docs.openstack.org/tempest/latest/HACKING.html#bug-fix-on-core-project-needing-tempest-changes | |
| 15:05:55 | gmann | my bad that i did not realize the full flow | |
| 15:07:32 | gibi | OK, so we all the new neutron API extension that breaks existing tempest test we need to temporary disable the test, merge the API extension, merge the tempest adaptation, and then re-enable the test | |
| 15:07:43 | gibi | s/we/with/ | |
| 15:08:03 | gmann | yeah, and for any type of bug fix in project if that is blocked by tempest tests | |
| 15:08:09 | gibi | ack | |
| 15:19:39 | gibi | melwitt: hi! I've fixed and replied to your comments in https://review.opendev.org/c/openstack/nova/+/802060 please ignore the nova-next failure for now, it is due and ordering issue, we need to land https://review.opendev.org/c/openstack/neutron/+/805637 (which is on the gate) first. See our discussion with gmann above ^^ | |
| 16:00:17 | opendevreview | Merged openstack/nova stable/xena: Reproduce bug 1944759 https://review.opendev.org/c/openstack/nova/+/810910 | |
| 17:03:35 | spatel | sean-k-mooney[m] morning | |
| 18:35:01 | gibi | gmann: so far so good, 805637 cleared the check queue and now running in the gate queue | |
| 18:53:56 | gmann | +1 | |
| 20:04:45 | gmann | seems ovs multinode job again failing | |
| #openstack-nova - 2021-10-30 | |||
| 05:19:24 | gibi | an now neutron functional and tempest slow :/ | |
| #openstack-nova - 2021-10-31 | |||
| 07:46:17 | EugenMayer | Good Morning. Using wallaby, creating a snapshot queues, but after some time, the snapshot task just disapears | |
| 07:48:17 | EugenMayer | does a snapshot need swift as a storage? We are using local disks as storage. So no centralized storage | |
| 07:55:32 | EugenMayer | Same happens if i uses openstack server backup create vpn on the cli - i see the job queued, see it in the ui, but then after some time it is gone and i do not see any snapshots | |
| 13:21:31 | EugenMayer0 | Are there any plans to add firecracker as a virtualization type in nova? | |
| 13:24:42 | sean-k-mooney[m] | not currently but i think is an interesting tech stack to consider | |
| 13:25:11 | sean-k-mooney[m] | i dont think it currently has libvirt integration but that might be the quickest way to integrate it | |
| 14:18:51 | EugenMayer0 | libvirt does include firecracker support, doesnt it? | |
| #openstack-nova - 2021-11-01 | |||
| 07:59:53 | opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: trigger nova-next https://review.opendev.org/c/openstack/nova/+/816127 | |
| 08:22:21 | opendevreview | Balazs Gibizer proposed openstack/nova master: [nova-manage]support extended resource request https://review.opendev.org/c/openstack/nova/+/802060 | |
| 08:23:23 | opendevreview | Balazs Gibizer proposed openstack/nova master: Reno for qos-minimum-guaranteed-packet-rate https://review.opendev.org/c/openstack/nova/+/805046 | |
| 08:24:36 | opendevreview | Balazs Gibizer proposed openstack/nova master: Reno for qos-minimum-guaranteed-packet-rate https://review.opendev.org/c/openstack/nova/+/805046 | |
| 08:56:36 | opendevreview | Danil proposed openstack/nova master: Check if the InstanceNUMACell.pcpuset attribute exist prior to relying on it. This change enables support for the VM instances with pinned CPUs (hw:cpu_policy=dedicated) that were created before upgrading the OpenStack Nova to Victoria release. https://review.opendev.org/c/openstack/nova/+/815923 | |
| 09:33:08 | stephenfin | gibi: Are you planning to backport that nova-manage command for extended resource requests? Maybe that doesn't make sense, since the neutron feature didn't land in time? | |
| 09:33:40 | gibi | stephenfin: hi! no, it does not make sense as you said, it only adds value if there is the new neutron api extension | |
| 09:33:47 | stephenfin | ack | |
| 09:34:03 | gibi | with the old extension the old nova-manage works OK | |
| 09:39:20 | lyarwood | Morning all \o | |
| 09:43:11 | gibi | lyarwood: welcome back! | |
| 09:43:26 | lyarwood | thanks gibi, good to be back | |
| 09:45:25 | gibi | ahh, sh*t. I think nova-next is broken https://review.opendev.org/c/openstack/neutron/+/805637 landed during the weekend. | |
| 09:45:37 | gibi | we had some ordering issue with the pps patches I think | |
| 09:46:25 | gibi | https://zuul.opendev.org/t/openstack/build/f499bc60c7124acea23b05593405f278/log/job-output.txt#62786 | |
| 09:46:54 | gibi | we should have landed https://review.opendev.org/c/openstack/nova/+/802060 before the neutron patch | |
| 09:47:48 | gibi | so now we either turn off the nova-manage placement heal_allocations testing in the post hook in nova-next, or we quickly land https://review.opendev.org/c/openstack/nova/+/802060 | |
| 09:49:05 | lyarwood | I can't guarantee that I'll get to the open review today so it might be easier to turn it off, rebase the change onto that while also turning the testing back on etc | |
| 09:49:34 | gibi | lyarwood: stephenfin and melwitt reviewed https://review.opendev.org/c/openstack/nova/+/802060 already so it is close to land | |
| 09:49:40 | lyarwood | ah | |
| 09:49:49 | gibi | so you are off the hook :) | |
| 09:49:51 | lyarwood | it's your call then if you already have two cores on it | |
| 09:49:53 | stephenfin | lyarwood: welcome back o/ | |
| 09:49:57 | lyarwood | \o | |
| 09:50:38 | gibi | I will propose the turn off of the test quickly that is easy and we have enough cores online now to land that it quickly | |
| 09:50:52 | opendevreview | Merged openstack/nova stable/train: address open redirect with 3 forward slashes https://review.opendev.org/c/openstack/nova/+/806629 | |
| 09:56:53 | opendevreview | Balazs Gibizer proposed openstack/nova master: Temp disable nova-manage placement heal_allocation testing https://review.opendev.org/c/openstack/nova/+/816156 | |
| 09:57:11 | gibi | stephenfin, lyarwood: ^^ this hopefully fixes the gate | |