| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-08-04 | |||
| 07:47:41 | gibi | with post-copy the guest will be activated on the dest earlier in the process and libvirt will copy the memory from the inactive guest from the source | |
| 07:48:09 | ignazio | Do you think hugepage can help ? | |
| 07:48:37 | ignazio | I read page size flavor deafaul is 4k. Is true ? | |
| 07:49:05 | gibi | have you checked that the network used by libvirt for transferring the memory data has enough bandwidth? | |
| 07:49:28 | gibi | I'm not sure if hugepage will make a difference. yes the default small page on x86 is 4k | |
| 07:51:04 | ignazio | So, if I migrate an istance it spend a lot of time searching memory to migrate. When I migrate back it takes few seconds . | |
| 07:52:25 | ignazio | Probably after the first migration, the destination host reorganize the memory. I do not know very much the tlb mechanism | |
| 08:10:33 | gibi | moving back is an idependent process, there is no way to remember the past memeory content | |
| 08:23:01 | ignazio | <giby> the instance is using heavily the ram also on the first destination host after the first live migration. What I mean, because if I migrate again it takes few seconds ? | |
| 08:31:51 | opendevreview | Merged openstack/placement master: Clarify trait filtering in the API doc https://review.opendev.org/c/openstack/placement/+/825501 | |
| 08:44:42 | ignazio | <gibi> I have 10Gbs bandwiths available | |
| 08:44:54 | gibi | I'm out of ideas :/ sorry | |
| 08:45:35 | ignazio | <gibi> thanks | |
| 09:22:52 | gibi | bauzas_away: o/ | |
| 09:23:20 | bauzas_away | gibi: apologies about the PCI series, totally got out of my scope btw. | |
| 09:23:38 | gibi | bauzas_away: no worries, you are on vacation :) | |
| 09:23:38 | bauzas_away | yesterday, I thought about it and said "oh shit, forgot to review it" | |
| 09:23:57 | bauzas_away | gibi: I'll do a couple of series this morning | |
| 09:24:38 | bauzas_away | my vacation trip is only next week, so this week this is just staying with the kids at home and preparing for the journey | |
| 09:50:18 | bauzas_away | gibi: thanks for your comment yesterdaty on https://review.opendev.org/c/openstack/nova/+/851924 | |
| 09:50:36 | bauzas_away | replying to it made me realize I was again writing a wrong logic | |
| 09:51:07 | bauzas_away | the FUP can't fix the recreate issue, it should just remove the need for mdev_name2uuid | |
| 10:09:27 | opendevreview | Nobuhiro MIKI proposed openstack/nova master: pci: fix checking for physical function https://review.opendev.org/c/openstack/nova/+/852135 | |
| 10:35:23 | sean-k-mooney[m] | gibi: can i assume you agree we shoudl just proceed with https://review.opendev.org/c/openstack/nova/+/845775/4 instead of ^ | |
| 10:58:25 | sean-k-mooney[m] | gibi can you rereivew https://review.opendev.org/c/openstack/nova/+/848886/18 you were previously +2 but i asked for the release note to be changed slightly | |
| 11:28:04 | gibi | sean-k-mooney[m]: I'm not sure that the two change are equivalent as after my patch we still uses is_physical_function and that queries sriov_totalvfs | |
| 11:29:16 | gibi | sean-k-mooney[m]: I'm +2+A on the evac change | |
| 11:30:23 | sean-k-mooney[m] | ack | |
| 11:30:43 | sean-k-mooney[m] | i dont think we shoudl really be basiing this off sriov_totalvfs | |
| 11:31:28 | sean-k-mooney[m] | that can be 0 if you disable sriov VFs in the bios | |
| 11:31:41 | gibi | I agree that this is shakey | |
| 11:31:47 | gibi | shaky | |
| 11:32:07 | sean-k-mooney[m] | we dont really handel type-pci well with neutron | |
| 11:32:52 | sean-k-mooney[m] | since we dont use the mac update logic for direct phyiscal when you are using a type-pci device via vnic_type=direct | |
| 11:33:52 | sean-k-mooney[m] | so what i dont want to see is use report a nic as type-pci becuase you change the bios setting to disable sriov and then prevent the port form working with direct-physical | |
| 11:34:27 | sean-k-mooney[m] | i guess we can see if there is a better way to do the detection | |
| 11:35:09 | gibi | would be the proposed phys_port_name based detection better? | |
| 11:35:36 | sean-k-mooney[m] | i would prefer if we honetly did not use these fucntions for this if we can avoid it | |
| 11:35:46 | sean-k-mooney[m] | and use the objects form the virt driver instead | |
| 11:36:15 | sean-k-mooney[m] | i have not looked recently but why are we determining if its a physical fucntion in that code path currently | |
| 11:36:52 | sean-k-mooney[m] | this does not influcne if its reported as type-pci ectra today | |
| 11:36:58 | sean-k-mooney[m] | that is done seperatly | |
| 11:39:23 | gibi | I agree that we should not use the sysfs and user the libvirt driver instead | |
| 11:39:47 | gibi | right now I cannot precisely answer why we look up the type during the whitelist parsing | |
| 11:39:54 | sean-k-mooney[m] | well rather the objects returned by the virt driver genericlly | |
| 11:40:06 | gibi | but I have many theorethical issues with that code | |
| 11:40:36 | gibi | +1 on an abstraction over the libvirt virt driver | |
| 11:40:38 | sean-k-mooney[m] | its proably used to supprot the feature where if we whitelist the pf addres with vf product id | |
| 11:40:41 | sean-k-mooney[m] | we allow the VFs | |
| 11:40:51 | gibi | that could be one reason yes | |
| 11:40:56 | sean-k-mooney[m] | well you should not be calling the driver | |
| 11:41:08 | gibi | it is a very convoluted code | |
| 11:41:10 | sean-k-mooney[m] | the whitelist is used to fileter the objects returned by the driver | |
| 11:41:24 | sean-k-mooney[m] | by objects i mean the dicts | |
| 11:41:37 | sean-k-mooney[m] | so the dict already has the type set | |
| 11:41:54 | sean-k-mooney[m] | so we shoudl just be able to look at the type in the dict | |
| 11:43:17 | gibi | yes, that would be nice | |
| 11:43:57 | gibi | it could be a natural continuation of the https://review.opendev.org/q/topic:pci-device-spec-cleanup series | |
| 11:44:24 | sean-k-mooney[m] | yes | |
| 11:44:38 | sean-k-mooney[m] | escpically since the sysfs way only works on linux anyway | |
| 11:44:57 | sean-k-mooney[m] | it does ont work for other virt drivers where as the dict approch would | |
| 11:45:01 | gibi | yes, I also want to remove the sysfs deps from the code | |
| 11:45:21 | sean-k-mooney[m] | we cant remove all of them unforcunetly | |
| 11:45:29 | gibi | true, so limit them :) | |
| 11:45:37 | sean-k-mooney[m] | since we cant trust libvirt because of its caching in some cases | |
| 11:46:05 | gibi | yeah I saw what you and bauzas_away found about the mdev cache in libvirt | |
| 11:46:11 | sean-k-mooney[m] | the sysfs part might be able to move to a libvirt dirver util file or somehting | |
| 11:46:23 | sean-k-mooney[m] | well its not just that | |
| 11:46:39 | gibi | yepp, currently even the nova/network/neutron code is depend on sysfs too | |
| 11:46:40 | sean-k-mooney[m] | i fixed a similar caching issue where libvirt would miss mac adress changes | |
| 11:49:04 | gibi | probably in general if we change something via sysfs and not via libvirt then libvirt will have a stale cache | |
| 11:49:23 | gibi | which is sort of understandable | |
| 11:49:28 | kashyap | Yeah | |
| 11:50:08 | gibi | wondering if neutron also manipulate the host via sysfs or ethtool | |
| 11:50:20 | gibi | as there they have no libvirt interface at all | |
| 11:50:45 | sean-k-mooney[m] | in general if we could disable the caching in libvirt entirly i think we would | |
| 11:51:12 | sean-k-mooney[m] | i have serriously considered if we would be better not using libvirt for device tracking a few times | |
| 11:51:52 | sean-k-mooney[m] | https://review.opendev.org/c/openstack/nova/+/739131 | |
| 11:52:02 | sean-k-mooney[m] | that was the change i was thinkin of | |
| 11:52:05 | gibi | yeah, or creating a way to force libvirt to refresh the cache | |
| 11:52:36 | sean-k-mooney[m] | bind mount it to /dev/null on. disk ? | |
| 11:53:13 | gibi | is it an on disk cache? | |
| 11:53:19 | gibi | I assumed it is just in memory | |
| 11:53:28 | sean-k-mooney[m] | i think both | |
| 11:53:46 | sean-k-mooney[m] | its in memory in the vritnodedevd container | |
| 11:54:02 | sean-k-mooney[m] | but i think it also caches some info on disk | |
| 11:54:16 | sean-k-mooney[m] | i wonder can you just not use that container/deamon | |
| 11:54:19 | sean-k-mooney[m] | and run without it | |
| 11:54:26 | sean-k-mooney[m] | i should ask danpb | |
| 11:55:07 | sean-k-mooney[m] | i assume that wont actully work | |
| 11:55:21 | sean-k-mooney[m] | that deamon is more then just a cache | |
| 11:57:47 | gibi | I assume so | |
| 11:59:52 | sean-k-mooney[m] | what we really. want i think is to add a flag to https://libvirt.org/html/libvirt-libvirt-nodedev.html#virNodeListDevices to force it to probe | |
| 12:00:28 | gibi | yepp, that would help | |
| 12:00:36 | gibi | we would just alway set that flag | |
| 12:06:46 | opendevreview | Rajesh Tailor proposed openstack/nova master: Transport context to all threads https://review.opendev.org/c/openstack/nova/+/827467 | |
| 12:16:41 | gibi | sean-k-mooney[m]: when you have 2 minutes, could you look at this please? https://review.opendev.org/c/openstack/nova/+/845922 I got hit by it recently | |
| 12:17:10 | sean-k-mooney[m] | sure im just working on a doc for a meeting tomrrow | |
| 12:17:40 | sean-k-mooney[m] | hum im not familar with this bug but sure sound strait forward | |
| 12:18:08 | gibi | it happens on slow nodes | |