| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-06-10 | |||
| 08:18:56 | sean-k-mooney | at least it has sicne we got rid fo the local conducor stuff like 2-3 years ago | |
| 08:20:54 | bauzas | sean-k-mooney: operators can pipe their needs for just a FFU upgrade if they want | |
| 08:21:25 | sean-k-mooney | "pipe there needs" ? | |
| 08:22:53 | sean-k-mooney | by the way look like some people from vmware are fixing the vmware nsx ci | |
| 08:24:36 | sean-k-mooney | http://207.189.188.190/logs/58/734658/1/check-vote/ext-nova-zuul/f8d2e76/tempest_results.html.gz | |
| 08:25:57 | sean-k-mooney | i think it has to be manually triggered via vmware-recheck-patch but i guess that is still an improvement | |
| 08:26:16 | bauzas | sean-k-mooney: I mean, for example, to do something like piping netcat | |
| 08:27:49 | sean-k-mooney | bauzas: they could but i dont think we sould require direct db access | |
| 08:28:24 | sean-k-mooney | making them proxy it via netcat or ssh is a hack IMO | |
| 08:28:27 | bauzas | it's a nova-manage command so we need it anyway :) | |
| 08:28:40 | bauzas | they need to provide DB creds | |
| 08:28:57 | sean-k-mooney | bauzas: so that means nova-manage is not suitable solution to this probelem | |
| 08:29:14 | bauzas | but if you need to run this command on a nova-compute (and then you don't have a specific DB cred for it), then you could just use netcat | |
| 08:30:06 | bauzas | sean-k-mooney: it would work for any operator that doesn't want to have compute nodes be in a specific network | |
| 08:30:11 | sean-k-mooney | i think we should be providing a way to pass in some data structro to the nova-manage command which it uses to then do the reshapes | |
| 08:30:33 | bauzas | sean-k-mooney: we could do it, that's one of the things we said | |
| 08:30:41 | bauzas | hence two commands | |
| 08:31:07 | bauzas | and piping between them (or passing a data structure) | |
| 08:31:31 | sean-k-mooney | yep once command that you run on the compute node that gets the info form the virt driver and produces the reshap yaml or whatever and then a second that you run on a node with db access | |
| 08:31:36 | bauzas | sean-k-mooney: but tbc, https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/reshape-provider-tree.html#direct-interface-to-placement already thought about it | |
| 08:32:44 | sean-k-mooney | bauzas: that was not thinking about reshapes | |
| 08:32:49 | sean-k-mooney | it was thinking about db upgrades | |
| 08:32:55 | sean-k-mooney | those are very different things | |
| 08:33:27 | sean-k-mooney | well it mentions reshpaes but we did not have any at that point | |
| 08:34:45 | sean-k-mooney | actully i guess look ing at the spec in general this is the resahpe spec | |
| 08:35:17 | sean-k-mooney | i think it was making the assumtion however that reshaps did not need compute specific infomation | |
| 08:36:43 | sean-k-mooney | looking at the ironic_flavor_migration examle i dont think that is the same as the libvirt case | |
| 08:36:45 | sean-k-mooney | https://review.opendev.org/#/c/501025/12/nova/cmd/manage.py | |
| 08:36:58 | sean-k-mooney | because the ironic virt dirver talks to the ironic api | |
| 08:37:45 | sean-k-mooney | so the compute serivce virt driver does not need to be colocated on the same host as the hypervior | |
| 08:37:59 | gibi | the spec gives solution for one side of the problem. We don't have a running placement API service during FFU but we need to write into the placement db due to reshape | |
| 08:38:14 | gibi | placement direct allows that. | |
| 08:38:27 | sean-k-mooney | yes it will | |
| 08:38:28 | gibi | sean-k-mooney is right that collecting the data for the reshape is not solved in tha spec | |
| 08:39:21 | gibi | except that it mentions to start up a virt driver, but it does not handle the implication of that statemen | |
| 08:39:24 | gibi | t | |
| 08:39:41 | sean-k-mooney | gibi: https://review.opendev.org/#/c/501025/ is the implemeation for ironic | |
| 08:39:52 | sean-k-mooney | but that is very different then libvirt | |
| 08:41:13 | sean-k-mooney | the ironic virt dirver is used here https://review.opendev.org/#/c/501025/12/nova/cmd/manage.py@988 | |
| 08:42:30 | gibi | interesting | |
| 08:42:31 | sean-k-mooney | we cant just call the reshape function in the libvirt dirver however. | |
| 08:42:53 | sean-k-mooney | ironic works because all the info it need is either in the nova db or in the ironic api | |
| 08:43:50 | sean-k-mooney | this is all that was doing https://github.com/openstack/nova/blob/b6f3d393aa5cb755f30fa70d4acc614c32216c99/nova/virt/ironic/driver.py#L570-L582 | |
| 08:44:50 | sean-k-mooney | thats doing an online data migration but its not doing a reshape | |
| 08:46:00 | gibi | the analogous call would be nova.virt.libvirt.driver.LibvirtDriver.update_provider_tree | |
| 08:46:37 | gibi | but that needs a hypervisor connection as far as I see | |
| 08:46:46 | sean-k-mooney | well yes an no that would update the provider i guess but it would do more then just run the reshapes | |
| 08:47:28 | sean-k-mooney | gibi: yes it need the resouce track and a bunch of other things that reuiqre not just a connection to libvirt but file access too | |
| 08:47:49 | sean-k-mooney | im pretty sure we need to check /sys for some things | |
| 08:48:31 | sean-k-mooney | like sev support? we need the compute nodes config for allocation ratio stuff at the very least | |
| 08:48:40 | gibi | sean-k-mooney: agree that update_provider_tree does more than what is needed for reshape | |
| 08:49:22 | gibi | I guess we need to see what are the bare minimum information to run each reshape codepath and see how can we provide that | |
| 08:51:01 | sean-k-mooney | the pcpu reshape would need the compute node cpu_shared_set and cpu_dedicated_set config options techincally vcpu_pin_set too | |
| 08:51:35 | sean-k-mooney | i think the vgpu reshape to have seperate RP per pGPU needs libvirt and the mdev type config options | |
| 08:51:59 | sean-k-mooney | gibi: if we proceeed with numa in placment that is going to need the host numa toplogy info from libfirt | |
| 08:54:22 | sean-k-mooney | gibi: this is the vgpu reshape https://github.com/openstack/nova/blob/b6f3d393aa5cb755f30fa70d4acc614c32216c99/nova/virt/libvirt/driver.py#L7757 | |
| 08:56:09 | sean-k-mooney | well actully _update_provider_tree_for_vgpu is the top level function we would be calling and _update_provider_tree_for_pcpu is the cpu one | |
| 08:58:02 | gibi | I see | |
| 08:58:32 | sean-k-mooney | _get_all_assigned_mediated_devices which is used in the vgpu reshape directly read the guest xml https://github.com/openstack/nova/blob/b6f3d393aa5cb755f30fa70d4acc614c32216c99/nova/virt/libvirt/driver.py#L6940-L6973 | |
| 09:35:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Don't allow "reserving" file-backed memory https://review.opendev.org/734658 | |
| 09:35:51 | openstackgerrit | Merged openstack/nova stable/ussuri: Add functional test for bug 1550919 https://review.opendev.org/731247 | |
| 09:35:51 | openstack | bug 1550919 in OpenStack Compute (nova) ussuri "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Lee Yarwood (lyarwood) | |
| 09:36:15 | stephenfin | bauzas: Can you look at https://review.opendev.org/#/c/733668 ? | |
| 09:36:54 | bauzas | stephenfin: ack, will do | |
| 09:37:23 | bauzas | gibi: sean-k-mooney: sorry was on a meeting, saw your discussion | |
| 09:38:04 | bauzas | yup, from a vgpu reshape perspective, we need to lookup the guest XMLs hence libvirt | |
| 09:38:13 | bauzas | (and just not do a DB check) | |
| 09:39:29 | openstackgerrit | Merged openstack/nova-specs master: Stop to use the __future__ module. https://review.opendev.org/732879 | |
| 09:51:04 | gibi | stephenfin: about https://review.opendev.org/#/c/734658 would it worth to add a reno too to communicate a future breaking change? | |
| 09:51:41 | stephenfin | gibi: Good idea. "upgrade" section? | |
| 09:51:51 | gibi | stephenfin: yeah | |
| 09:52:00 | stephenfin | Cool. Sec | |
| 09:52:17 | gibi | thanks | |
| 09:53:50 | openstackgerrit | Alexandre Arents proposed openstack/nova master: Snapshot: offload glance upload in a native thread https://review.opendev.org/734776 | |
| 09:58:00 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Mark e1000e VIF as supported https://review.opendev.org/734777 | |
| 09:58:01 | stephenfin | sean-k-mooney: Can you sanity check that too, please? ^ | |
| 10:15:24 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Don't allow "reserving" file-backed memory https://review.opendev.org/734658 | |
| 10:15:27 | stephenfin | gibi: ^ | |
| 10:15:30 | stephenfin | as requested | |
| 10:16:30 | stephenfin | bauzas: Another (easier) one for your list https://review.opendev.org/#/c/734777/ | |
| 10:16:50 | bauzas | stephenfin: the exception one is a bit hairy, I need to make sure you don't forget anything | |
| 10:17:33 | sean-k-mooney | stephenfin: yep ill take a look shortly | |
| 10:19:14 | sean-k-mooney | stephenfin: did you check if we report hugepages in the host numa toplogy blob | |
| 10:19:23 | sean-k-mooney | when file backed memory is enabled | |
| 10:19:40 | stephenfin | I did not. What would be the implication? | |
| 10:19:53 | stephenfin | Just that we shouldn't do it? | |
| 10:20:02 | sean-k-mooney | we cant use filebacked memory and hugepages at the same time | |
| 10:20:15 | sean-k-mooney | but the numa toplogy filter has no idea if this config value is set | |
| 10:20:23 | sean-k-mooney | so it wont elimiate those hosts | |
| 10:20:31 | stephenfin | Hmm, good point. That'd be another good fix | |
| 10:20:33 | stephenfin | Lemme go look | |
| 10:20:52 | sean-k-mooney | so unless we dont report hugepages when there vailable if its set then we could get boot fails i think | |
| 10:21:02 | sean-k-mooney | ya seperate patch if its not done | |
| 10:25:41 | stephenfin | Does anyone know how to get a KCS taken down? https://access.redhat.com/solutions/3947641 | |
| 10:26:20 | stephenfin | For some daft reason, someone suggested people modify their DB rather than ask us to resolve a trivial bug :-\ | |
| 10:30:53 | stephenfin | ^ Ignore that. Intended for downstream IRC :) | |
| 10:37:56 | sean-k-mooney | damit is there way way we can just stop support form publishing kcs artical without having them go through tech review | |
| 10:38:09 | sean-k-mooney | ya i dont know but i reviewd https://review.opendev.org/#/c/734658/3 | |
| 10:41:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Don't allow "reserving" file-backed memory https://review.opendev.org/734658 | |
| 10:41:29 | stephenfin | sean-k-mooney: Addressed doc typo. Let's track the hugepage/total GB issue separately? | |