Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-06-10
07:38:11 bauzas gibi: surely, but that would be something we would comment
07:38:16 bauzas like a caveat
07:38:28 bauzas and it's just for FFUs
07:38:50 gibi OK
07:46:53 gibi let's go with the original plan from the spe
07:46:54 gibi c
08:14:12 sean-k-mooney gibi: bauzas well ingeneral we whould assume the db server is not accesable from the compute yes
08:15:41 sean-k-mooney this is why we discussed the need to potentialy seperate the data collection from the updating of placment or the db
08:16:18 sean-k-mooney e.g. we might need to run something on the compute node to generate some output which is the canges to be done via ssh then do the update from the contoler instead
08:16:38 sean-k-mooney i think johnthetubaguy brought that up or dansmith in the ptg sesssion
08:18:16 sean-k-mooney gibi: and ya its not even a case of our deployment model does not require it it activly discurages direct db acess form the compute on security grounds
08:18:56 sean-k-mooney at least it has sicne we got rid fo the local conducor stuff like 2-3 years ago
08:20:54 bauzas sean-k-mooney: operators can pipe their needs for just a FFU upgrade if they want
08:21:25 sean-k-mooney "pipe there needs" ?
08:22:53 sean-k-mooney by the way look like some people from vmware are fixing the vmware nsx ci
08:24:36 sean-k-mooney http://207.189.188.190/logs/58/734658/1/check-vote/ext-nova-zuul/f8d2e76/tempest_results.html.gz
08:25:57 sean-k-mooney i think it has to be manually triggered via vmware-recheck-patch but i guess that is still an improvement
08:26:16 bauzas sean-k-mooney: I mean, for example, to do something like piping netcat
08:27:49 sean-k-mooney bauzas: they could but i dont think we sould require direct db access
08:28:24 sean-k-mooney making them proxy it via netcat or ssh is a hack IMO
08:28:27 bauzas it's a nova-manage command so we need it anyway :)
08:28:40 bauzas they need to provide DB creds
08:28:57 sean-k-mooney bauzas: so that means nova-manage is not suitable solution to this probelem
08:29:14 bauzas but if you need to run this command on a nova-compute (and then you don't have a specific DB cred for it), then you could just use netcat
08:30:06 bauzas sean-k-mooney: it would work for any operator that doesn't want to have compute nodes be in a specific network
08:30:11 sean-k-mooney i think we should be providing a way to pass in some data structro to the nova-manage command which it uses to then do the reshapes
08:30:33 bauzas sean-k-mooney: we could do it, that's one of the things we said
08:30:41 bauzas hence two commands
08:31:07 bauzas and piping between them (or passing a data structure)
08:31:31 sean-k-mooney yep once command that you run on the compute node that gets the info form the virt driver and produces the reshap yaml or whatever and then a second that you run on a node with db access
08:31:36 bauzas sean-k-mooney: but tbc, https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/reshape-provider-tree.html#direct-interface-to-placement already thought about it
08:32:44 sean-k-mooney bauzas: that was not thinking about reshapes
08:32:49 sean-k-mooney it was thinking about db upgrades
08:32:55 sean-k-mooney those are very different things
08:33:27 sean-k-mooney well it mentions reshpaes but we did not have any at that point
08:34:45 sean-k-mooney actully i guess look ing at the spec in general this is the resahpe spec
08:35:17 sean-k-mooney i think it was making the assumtion however that reshaps did not need compute specific infomation
08:36:43 sean-k-mooney looking at the ironic_flavor_migration examle i dont think that is the same as the libvirt case
08:36:45 sean-k-mooney https://review.opendev.org/#/c/501025/12/nova/cmd/manage.py
08:36:58 sean-k-mooney because the ironic virt dirver talks to the ironic api
08:37:45 sean-k-mooney so the compute serivce virt driver does not need to be colocated on the same host as the hypervior
08:37:59 gibi the spec gives solution for one side of the problem. We don't have a running placement API service during FFU but we need to write into the placement db due to reshape
08:38:14 gibi placement direct allows that.
08:38:27 sean-k-mooney yes it will
08:38:28 gibi sean-k-mooney is right that collecting the data for the reshape is not solved in tha spec
08:39:21 gibi except that it mentions to start up a virt driver, but it does not handle the implication of that statemen
08:39:24 gibi t
08:39:41 sean-k-mooney gibi: https://review.opendev.org/#/c/501025/ is the implemeation for ironic
08:39:52 sean-k-mooney but that is very different then libvirt
08:41:13 sean-k-mooney the ironic virt dirver is used here https://review.opendev.org/#/c/501025/12/nova/cmd/manage.py@988
08:42:30 gibi interesting
08:42:31 sean-k-mooney we cant just call the reshape function in the libvirt dirver however.
08:42:53 sean-k-mooney ironic works because all the info it need is either in the nova db or in the ironic api
08:43:50 sean-k-mooney this is all that was doing https://github.com/openstack/nova/blob/b6f3d393aa5cb755f30fa70d4acc614c32216c99/nova/virt/ironic/driver.py#L570-L582
08:44:50 sean-k-mooney thats doing an online data migration but its not doing a reshape
08:46:00 gibi the analogous call would be nova.virt.libvirt.driver.LibvirtDriver.update_provider_tree
08:46:37 gibi but that needs a hypervisor connection as far as I see
08:46:46 sean-k-mooney well yes an no that would update the provider i guess but it would do more then just run the reshapes
08:47:28 sean-k-mooney gibi: yes it need the resouce track and a bunch of other things that reuiqre not just a connection to libvirt but file access too
08:47:49 sean-k-mooney im pretty sure we need to check /sys for some things
08:48:31 sean-k-mooney like sev support? we need the compute nodes config for allocation ratio stuff at the very least
08:48:40 gibi sean-k-mooney: agree that update_provider_tree does more than what is needed for reshape
08:49:22 gibi I guess we need to see what are the bare minimum information to run each reshape codepath and see how can we provide that
08:51:01 sean-k-mooney the pcpu reshape would need the compute node cpu_shared_set and cpu_dedicated_set config options techincally vcpu_pin_set too
08:51:35 sean-k-mooney i think the vgpu reshape to have seperate RP per pGPU needs libvirt and the mdev type config options
08:51:59 sean-k-mooney gibi: if we proceeed with numa in placment that is going to need the host numa toplogy info from libfirt
08:54:22 sean-k-mooney gibi: this is the vgpu reshape https://github.com/openstack/nova/blob/b6f3d393aa5cb755f30fa70d4acc614c32216c99/nova/virt/libvirt/driver.py#L7757
08:56:09 sean-k-mooney well actully _update_provider_tree_for_vgpu is the top level function we would be calling and _update_provider_tree_for_pcpu is the cpu one
08:58:02 gibi I see
08:58:32 sean-k-mooney _get_all_assigned_mediated_devices which is used in the vgpu reshape directly read the guest xml https://github.com/openstack/nova/blob/b6f3d393aa5cb755f30fa70d4acc614c32216c99/nova/virt/libvirt/driver.py#L6940-L6973
09:35:30 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Don't allow "reserving" file-backed memory https://review.opendev.org/734658
09:35:51 openstack bug 1550919 in OpenStack Compute (nova) ussuri "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Lee Yarwood (lyarwood)
09:35:51 openstackgerrit Merged openstack/nova stable/ussuri: Add functional test for bug 1550919 https://review.opendev.org/731247
09:36:15 stephenfin bauzas: Can you look at https://review.opendev.org/#/c/733668 ?
09:36:54 bauzas stephenfin: ack, will do
09:37:23 bauzas gibi: sean-k-mooney: sorry was on a meeting, saw your discussion
09:38:04 bauzas yup, from a vgpu reshape perspective, we need to lookup the guest XMLs hence libvirt
09:38:13 bauzas (and just not do a DB check)
09:39:29 openstackgerrit Merged openstack/nova-specs master: Stop to use the __future__ module. https://review.opendev.org/732879
09:51:04 gibi stephenfin: about https://review.opendev.org/#/c/734658 would it worth to add a reno too to communicate a future breaking change?
09:51:41 stephenfin gibi: Good idea. "upgrade" section?
09:51:51 gibi stephenfin: yeah
09:52:00 stephenfin Cool. Sec
09:52:17 gibi thanks
09:53:50 openstackgerrit Alexandre Arents proposed openstack/nova master: Snapshot: offload glance upload in a native thread https://review.opendev.org/734776
09:58:00 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Mark e1000e VIF as supported https://review.opendev.org/734777
09:58:01 stephenfin sean-k-mooney: Can you sanity check that too, please? ^
10:15:24 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Don't allow "reserving" file-backed memory https://review.opendev.org/734658
10:15:27 stephenfin gibi: ^
10:15:30 stephenfin as requested
10:16:30 stephenfin bauzas: Another (easier) one for your list https://review.opendev.org/#/c/734777/
10:16:50 bauzas stephenfin: the exception one is a bit hairy, I need to make sure you don't forget anything
10:17:33 sean-k-mooney stephenfin: yep ill take a look shortly
10:19:14 sean-k-mooney stephenfin: did you check if we report hugepages in the host numa toplogy blob
10:19:23 sean-k-mooney when file backed memory is enabled
10:19:40 stephenfin I did not. What would be the implication?
10:19:53 stephenfin Just that we shouldn't do it?
10:20:02 sean-k-mooney we cant use filebacked memory and hugepages at the same time
10:20:15 sean-k-mooney but the numa toplogy filter has no idea if this config value is set
10:20:23 sean-k-mooney so it wont elimiate those hosts

Earlier   Later