| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-08-31 | |||
| 12:53:26 | sean-k-mooney | gibi: on the plus side since stephenfin has written the patches those are easy to merge in early wallaby | |
| 12:53:49 | gibi | yeah, but first we have to merge some deprecation patches for these | |
| 12:53:58 | sean-k-mooney | yep | |
| 12:54:18 | sean-k-mooney | while i have your attention there is a revert of one of aarents patches proposed https://review.opendev.org/#/c/749035/1 | |
| 12:54:53 | sean-k-mooney | i dont see how we can be getting None also the ci really does not like the revert for some reason | |
| 12:55:14 | sean-k-mooney | but apprenlty this is failing in rdo | |
| 12:55:26 | sean-k-mooney | im not sure if it was a one off failure or if its blocking there gate | |
| 12:55:38 | openstack | Launchpad bug 1893618 in tripleo "periodic-tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-ussuri tempest test_shelve_unshelve_server failing in component-pipeline " [Critical,Triaged] | |
| 12:55:38 | sean-k-mooney | https://bugs.launchpad.net/tripleo/+bug/1893618 | |
| 12:55:41 | gibi | looking... | |
| 12:57:16 | sean-k-mooney | for some reason instance.system_metadata.get('image_base_image_ref') is retruning none | |
| 12:57:45 | sean-k-mooney | but we set that in exactly one place and i dont really see how it can be none as that implies the instace has no image | |
| 12:57:56 | sean-k-mooney | which makes no sense for a qcow backed vm | |
| 13:00:03 | sean-k-mooney | actully from the pre shelve xml i can see <nova:root type="image" uuid="5cc451d5-7abe-478a-9f4f-1a804f49a3f3"/> | |
| 13:01:26 | sean-k-mooney | so its implying that the system_metadata table is populated incorrectly or instance.system_metadata does not have all the info at this point | |
| 13:01:48 | sean-k-mooney | its really strange because i would have expected this to work | |
| 13:02:46 | stephenfin | gibi, sean-k-mooney: I think those notes are wrong. Have we not already in-effect deprecated them? https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L606-L614 | |
| 13:03:14 | sean-k-mooney | no | |
| 13:03:31 | sean-k-mooney | testing is differnt | |
| 13:03:42 | sean-k-mooney | they are not deprecated yet | |
| 13:03:56 | sean-k-mooney | unless you can point to a release note | |
| 13:05:46 | sean-k-mooney | for what its worth i did not think we planned on removing support for any libvirt backends this cycle just deprecations the only removals were going to be xenapi and vmware | |
| 13:05:54 | sean-k-mooney | vmware have now fixed the ci | |
| 13:06:03 | sean-k-mooney | so they have been undeprecated? | |
| 13:06:10 | sean-k-mooney | i know gibi has a patch for that at least | |
| 13:07:18 | sean-k-mooney | stephenfin: if we just use that waring as a depercation warning then all of libvirt arm and power support would be deprecated and its not | |
| 13:08:36 | gibi | that warning was added 7 years ago https://review.opendev.org/#/c/69919/ | |
| 13:09:17 | gibi | I agree with sean-k-mooney here to state the deprecation explicitly for these first | |
| 13:09:36 | stephenfin | Okay, fair. I'll do that now and -W what's there for another few months | |
| 13:09:39 | stephenfin | Thanks for the reviews :) | |
| 13:09:56 | sean-k-mooney | :) | |
| 13:10:15 | gibi | sean-k-mooney: vmware undeprecation is still open https://review.opendev.org/#/c/742407/ but I think we have a good chance to merge that | |
| 13:12:00 | gibi | dansmith: could you check back to the vmware undeprecation patch ^^ ? | |
| 13:13:01 | openstackgerrit | Brin Zhang proposed openstack/nova master: Refactor check and exception https://review.opendev.org/749052 | |
| 13:16:44 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Add ability to download Glance images into the libvirt image cache via RBD https://review.opendev.org/574301 | |
| 13:19:16 | gibi | sean-k-mooney: I have no idea how image_base_image_ref can be None for a qcow2 instance | |
| 13:19:47 | gibi | sean-k-mooney: we should make a reproduction of the tripleo test case | |
| 13:22:12 | openstackgerrit | Brin Zhang proposed openstack/nova master: Refactor check and exception https://review.opendev.org/749052 | |
| 13:24:49 | sean-k-mooney | gibi: it failed on one of your patches too | |
| 13:24:52 | sean-k-mooney | https://review.opendev.org/#/c/739246/ | |
| 13:24:58 | sean-k-mooney | http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22_finalize_unshelve_qcow2_image%5C%22&from=30d | |
| 13:25:14 | sean-k-mooney | there are only 2 hits in logstach so far in the last month | |
| 13:25:16 | openstackgerrit | Wenping Song proposed openstack/nova master: Reject resize operation for accelerator https://review.opendev.org/748560 | |
| 13:29:06 | sean-k-mooney | gibi: so apparently the filed is not set in the system metadata | |
| 13:32:20 | gibi | so we have a tempes test that can reproduce the problem but it only does it really infrequently | |
| 13:37:48 | gibi | sean-k-mooney: nope, the error that was in my patch has a different stack trace https://zuul.opendev.org/t/openstack/build/de80ec8f00204a92b5e667bdf7a72cee/log/controller/logs/screen-n-cpu.txt#24116 | |
| 13:38:49 | gibi | also the another nova hit in kibana is different too https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_990/739246/3/check/tempest-integrated-compute/990dace/controller/logs/screen-n-cpu.txt | |
| 13:39:28 | sean-k-mooney | gibi: kibana found that error too however | |
| 13:39:43 | sean-k-mooney | on patchset 3 and 4 | |
| 13:40:34 | gibi | the two error I see in kibana on review 739246 have different stack traces | |
| 13:40:51 | sean-k-mooney | oh they do | |
| 13:41:05 | gibi | it does hit _finalize_unshelve_qcow2_image but different way | |
| 13:41:06 | sean-k-mooney | just the same function | |
| 13:41:07 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Drop support for UML https://review.opendev.org/743230 | |
| 13:41:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Remove 'hypervisor_version' from 'libvirt_info' https://review.opendev.org/744199 | |
| 13:41:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Drop support for Xen https://review.opendev.org/743231 | |
| 13:41:09 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Remove '[vnc] keymap', '[spice] keymap' options https://review.opendev.org/749056 | |
| 13:41:09 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Deprecate support for non-QEMU/KVM backends https://review.opendev.org/749055 | |
| 13:41:29 | gibi | one of the errors I even recognize as a rebase issue from my patch | |
| 13:41:38 | stephenfin | sean-k-mooney: gibi: There you go. I'll mark the actual removal patch as -W for six months or so, heh | |
| 13:42:30 | gibi | stephenfin: thanks | |
| 13:43:16 | gibi | stephenfin: if you are itcign for a real removal then the xenserver driver can be removed I tihnk :) | |
| 13:43:47 | stephenfin | Sure, if you're happy to review it, I'm happy to do it :) | |
| 13:44:11 | dansmith | gibi: the ciwatch page shows more red than green for vmware lately, but I'm not sure how to figure out what is failing because the log dump isn't complete | |
| 13:44:26 | gibi | stephenfin: sure, why not | |
| 13:44:54 | dansmith | gibi: oh actually, I guess I'm missing a bunch of green to the right | |
| 13:45:08 | gibi | dansmith: looking.. | |
| 13:45:17 | dansmith | but I'm not sure what to make of the fails that have no logs, maybe those are aborts due to a new patch rev going up or something? | |
| 13:46:44 | dansmith | okay, I found a fail that looks like a real fail, so I dunno what those other ones are, if not aborts | |
| 13:47:16 | dansmith | I just wanted to find something that looks like a real failure to contrast it to a passing run, and also to make sure it's actually reporting failures | |
| 13:47:46 | gibi | yeah, me neither | |
| 13:48:12 | gibi | I'm trying to figure out if Yingji Sun or jhui@vmware.com is on IRC or not | |
| 13:48:44 | gibi | so we can ask them | |
| 13:53:48 | sean-k-mooney | dansmith: they might be hitting the same keystone error that is breaking the first party ci | |
| 13:54:09 | dansmith | sean-k-mooney: and reporting empty logs? seems weird, but okay | |
| 13:55:09 | sean-k-mooney | well no that is likely another issue | |
| 13:55:23 | sean-k-mooney | i just noticed they started failing when the upstream jobs also started failng | |
| 14:00:05 | sean-k-mooney | hum the hyperv ci also seams to be broken | |
| 14:00:20 | sean-k-mooney | its been red for at least the last 7 days | |
| 14:33:24 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Add a routed networks scheduler pre-filter https://review.opendev.org/749068 | |
| 14:33:37 | bauzas | gibi: sean-k-mooney: there it is ^ | |
| 14:36:02 | bauzas | gibi: sean-k-mooney: I now use a pre-filter that looks at the existing physnets that were provided by https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L2082 | |
| 14:39:16 | bauzas | I mean the network metadata that isn't persisted in the spec object but reused by every move operation | |
| 15:08:49 | gibi | bauzas: then I think you have to resolve TODO in the codepath that gather the physnets https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/network/neutron.py#L2020 | |
| 15:09:06 | gibi | as today it only get one physnet per network, even if there are multiple | |
| 15:09:32 | bauzas | holy shit. | |
| 15:14:13 | bauzas | gibi: in a meeting | |
| 15:25:31 | bauzas | gibi: actually, looking at vladik's comment, I'm not expert, but do we really support multiple networks per vlans? | |
| 15:25:49 | bauzas | multiple physnets* | |
| 15:27:34 | gibi | as far as I know in neutron nothing prevents you to create two vlan segments with different physnets in the same network | |
| 15:47:28 | bauzas | gibi: okay, then I don't feel enough expert to fix this TODO | |
| 15:48:05 | bauzas | gibi: the other way would be to find a way to pass requested networks down in the request spedc | |
| 15:48:07 | bauzas | spec* | |
| 15:48:19 | bauzas | which is something we don't do | |
| 15:49:39 | gibi | the way the pervious patches did it was to pass the aggregates via request_spec.request_level_params.member_of | |
| 15:50:04 | gibi | and do the network segment -> host aggregate translation in an upper layer (conductor) | |
| 15:50:20 | bauzas | gibi: I know | |
| 15:50:33 | bauzas | but a pre-filter is better, right? | |
| 15:50:55 | bauzas | and we don't need to pass the aggregates by a new field | |
| 15:51:08 | bauzas | the destination object already contains what we need | |