| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-02-03 | |||
| 16:14:10 | spatel | server load is 0.66 average | |
| 16:15:36 | spatel | I can do nova list etc.. which works fine. only vm creation choking up | |
| 16:15:48 | spatel | let me check nova api logs etc.. | |
| 16:17:18 | spatel | HAproxy showing all green health for nova members | |
| 16:18:25 | spatel | openstack compute service list also showing all service up | |
| 16:22:47 | spatel | This logs has nothing to do with this issue correct - https://paste.opendev.org/show/bhP6raKj9pFgARdlb0ES/ | |
| 16:25:08 | spatel | sean-k-mooney it works after i restarted all nova-* service | |
| 16:25:25 | spatel | do you think this is bug? I am running wallaby release | |
| 16:28:57 | dansmith | bauzas: you still around? | |
| 16:29:09 | bauzas | dansmith: yes | |
| 16:29:40 | dansmith | any chance you could look at the stable compute stuff today? it's +2d up to the top, most of what is left are additional checks and tests | |
| 16:29:51 | dansmith | I'm going to work on docs next week | |
| 16:30:44 | sean-k-mooney | spatel: maybe but without you debuging it and finding a root cause or an indication of the problem we dont dont ahve anything to go on | |
| 16:31:19 | bauzas | dansmith: I'm a bit short in time but I can try | |
| 16:31:23 | sean-k-mooney | spatel: i dont think that log is related | |
| 16:32:07 | spatel | sean-k-mooney i think better i should upgrade to Xena or yoga. | |
| 16:32:15 | spatel | wallaby is little behind now | |
| 16:32:25 | dansmith | bauzas: ack, well, if we have to recheck grind them, I just don't want to be racing next week.. and monday is a holiday for some folks | |
| 16:34:52 | bauzas | dansmith: I'm running up to finish to write functests on https://review.opendev.org/c/openstack/nova/+/868237/ but again, I'll try | |
| 16:35:20 | dansmith | okay, well, it's not that critical I guess | |
| 16:36:12 | dansmith | maybe I could bribe melwitt to have a look-see | |
| 16:40:27 | bauzas | sean-k-mooney: quick q, when reading https://docs.openstack.org/nova/latest/admin/cpu-topologies.html#configure-libvirt-pinning | |
| 16:40:56 | bauzas | sean-k-mooney: on my functest, I only did set cpu_dedicated_set but when trying to boot a regular non-pinned instance, it errors out | |
| 16:41:03 | bauzas | with NoValidHost | |
| 16:41:13 | bauzas | so I guess I also need to set cpu_shared_set ? | |
| 16:42:14 | bauzas | https://docs.openstack.org/nova/latest/configuration/config.html#compute.cpu_dedicated_set doesn't mention the dependency | |
| 16:43:11 | bauzas | oh wait, see my bug | |
| 16:44:58 | bauzas | my self.flags was setting None to cpu_shared_set | |
| 16:45:00 | bauzas | ffff | |
| 16:45:17 | sean-k-mooney | cpu_shared_set is for non pined core yes | |
| 16:45:24 | sean-k-mooney | or floating vms | |
| 16:45:37 | bauzas | I know tyhis | |
| 16:45:39 | sean-k-mooney | you only need to set it if cpu_dedicated_set is defiend | |
| 16:45:52 | bauzas | ok, both are dependent then | |
| 16:46:16 | sean-k-mooney | i dont wnat to go into the complexiyt i can explain it in detail next week if you like | |
| 16:46:16 | bauzas | you can't just set cpu_dedicated_set without opt-in the cpu_shared_set | |
| 16:46:28 | bauzas | no worries, it works | |
| 16:46:40 | bauzas | it was a pebkac | |
| 16:46:40 | sean-k-mooney | you can but if you dont set cpu_shared_set then the host can only be used for pinnded vms | |
| 16:46:59 | bauzas | I see | |
| 16:47:09 | bauzas | with the restricted set of dedicated cpus | |
| 16:47:24 | bauzas | like, I have 10 cpus, I set 5 of them as dedicated and period. | |
| 16:47:24 | sean-k-mooney | yep | |
| 16:47:32 | bauzas | that means that the 5 lefts won't be used | |
| 16:47:36 | bauzas | gotcha | |
| 16:47:43 | sean-k-mooney | ya they are reserved for the host | |
| 16:47:54 | sean-k-mooney | the ones that are not listed | |
| 16:48:30 | sean-k-mooney | so normally i would expect the first core in each socket to not be in either set | |
| 16:48:44 | sean-k-mooney | and then you devied up the rest depending on your workloads | |
| 16:50:36 | bauzas | cool enough | |
| 17:05:26 | opendevreview | Elod Illes proposed openstack/nova stable/yoga: ignore deleted server groups in validation https://review.opendev.org/c/openstack/nova/+/867989 | |
| 17:05:27 | opendevreview | Elod Illes proposed openstack/nova stable/yoga: add repoducer test for bug 1890244 https://review.opendev.org/c/openstack/nova/+/872663 | |
| 18:31:12 | opendevreview | Merged openstack/nova stable/yoga: Improving logging at '_allocate_mdevs'. https://review.opendev.org/c/openstack/nova/+/871414 | |
| 18:46:13 | opendevreview | Sylvain Bauza proposed openstack/nova master: enable cpus when an instance is spawning https://review.opendev.org/c/openstack/nova/+/868237 | |
| 19:50:56 | dvo-plv | Hello | |
| 19:52:10 | dvo-plv | I have a question about trait. I added new trait COMPUTE_NET_VIRTIO_PACKED. WHen I execute next command : openstack allocation candidate list --resource VCPU=1 I can see that one compute node has this trait and another does not | |
| 19:53:13 | sean-k-mooney | you did not request the trait in the query | |
| 19:53:36 | sean-k-mooney | so you will get back a list of all host that have 1 cpu core free | |
| 19:53:48 | dvo-plv | When I do live migration from compute node with new trait to the comptue without this trait, soes scheduler should check if requested compute node has this trait and forbid to execute migration ? | |
| 19:54:17 | sean-k-mooney | if and only if you have requested it in the flavor or implemnted that in your change correctly | |
| 19:54:49 | sean-k-mooney | you would ideally have a prefilter that add the trait request | |
| 19:55:15 | dvo-plv | I set this trait in the flavor with next command openstack flavor set --property trait:COMPUTE_NET_VIRTIO_PACKED=required 3 | |
| 19:55:41 | sean-k-mooney | that will take effect for new vms but not existing ones | |
| 19:55:57 | sean-k-mooney | if you have that set however then it will be included with the placment request and enforced | |
| 19:56:17 | dvo-plv | Yes, I create flavor with this trait and after that create VM and try to do live migration | |
| 19:56:47 | sean-k-mooney | then placement should enable this to work as intended | |
| 19:57:04 | sean-k-mooney | the vm will only shcedule to a host with that trait including on migrations | |
| 19:58:34 | sean-k-mooney | this is part of this proposal correct https://review.opendev.org/c/openstack/nova-specs/+/868377/1/specs/2023.1/approved/virtio_packedring_configuration_support.rst | |
| 20:00:43 | dvo-plv | In the spec file you told that packed ring option should be handled in some specific way to be sure that vm with packed_ring qemu option will not migrate to the node without this feature support and i can not get this feature from the libvirt. I found that this feature is enabled only from libvirt 6.3. So if I migrate to the node with lower libvirt the next method _check_compatible_with_source_hypervisor will forbid to migrate | |
| 20:01:29 | sean-k-mooney | that is not a sufficent check unfortunatly | |
| 20:01:47 | sean-k-mooney | this is not a feature of libvirt its a qmeu one and that check would already be too late | |
| 20:02:28 | sean-k-mooney | what we need to do is have nova report the trait on all hosts that support this | |
| 20:03:09 | sean-k-mooney | and then either have the guest opt into this with a image property or flavor extra specs | |
| 20:03:27 | sean-k-mooney | we can then use that image property/extra spec to request the trait automatically | |
| 20:03:45 | sean-k-mooney | that is the simple approch | |
| 20:04:00 | sean-k-mooney | the harder approch is to figure out a way to auto enable this | |
| 20:04:32 | sean-k-mooney | i dont know a simple way to do that off the top of my head unfortunetly | |
| 20:05:20 | sean-k-mooney | its a bit late here but we should proably disucss this some more next week | |
| 20:05:30 | dvo-plv | another way, we can check qemu version, this feature was implemented from qemu 4.2 | |
| 20:06:24 | dvo-plv | or grep funtionality like in this port http://blog.vmsplice.net/2020/05/how-to-check-virtio-feature-bits-inside.html | |
| 20:07:06 | dvo-plv | Thank you, let's move it to next weekÑ” | |
| 20:07:06 | dvo-plv | Thank you, let's move it to next weekє | |
| 20:07:33 | sean-k-mooney | the issue is we need to make the scudleing work before we have selected any host | |
| 20:08:22 | sean-k-mooney | so in general we cannot check the qemu/libvirt verion at the schduling step | |
| 20:08:39 | dvo-plv | I tried to get this feature at this method static_traits in the nova/virt/libvirt/driver.py | |
| 20:09:18 | sean-k-mooney | the best i can think of would be to stash a flag in the instance_system_metadta to recored it was booted on a host that supported the pact format | |
| 20:09:36 | sean-k-mooney | and then include a request for the trait if that is found in the instance_system_metadata | |
| 20:09:54 | sean-k-mooney | that is proably the best we can do since we dont know if its used by the guest | |
| 20:10:31 | sean-k-mooney | the static_traits function is the correct plase to report the trait for the compute host | |
| 21:06:28 | gmann | bauzas: stephenfin: need one more review in this to unblock stable/wallaby gate (removing broken sdk job) https://review.opendev.org/c/openstack/nova/+/871798 | |
| 22:39:50 | melwitt | dansmith: I have look-see'd | |
| 22:40:31 | opendevreview | Merged openstack/nova stable/wallaby: [stable-only] Remove broken sdk job from wallaby https://review.opendev.org/c/openstack/nova/+/871798 | |
| 22:54:55 | dansmith | melwitt: um | |
| 22:55:04 | dansmith | I'm not sure you can past-tense-verb look-see | |
| 22:55:22 | dansmith | (but thanks :) | |
| 22:55:35 | melwitt | :) | |
| 22:56:49 | sean-k-mooney | its english there is always a way to do things | |
| 23:00:52 | sean-k-mooney | 023-02-03 23:00:25.664 41 ERROR nova.virt.driver [None req-80a00ce3-332e-4a98-be2e-d0a1bd9329d7 - - - - - -] Compute driver option required, but not specified | |
| 23:01:07 | sean-k-mooney | lol ok that explains why its not working | |
| 23:05:37 | opendevreview | Sylvain Bauza proposed openstack/nova master: Enable cpus when an instance is spawning https://review.opendev.org/c/openstack/nova/+/868237 | |