Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-01-21
16:30:31 mriedem so i guess that won't work
16:30:34 sean-k-mooney its not strictly a move operation but i has understood that effectivly start,stop,pause/unpause,suspend and resume were the only actions that would work
16:30:34 gibi mriedem: so it would be an extra check during rebuild, to see if healing is needed
16:30:55 mriedem rebuild is specifically not a move
16:31:01 mriedem there is no resource claim on rebiuld
16:31:02 mriedem *rebuild
16:31:18 sean-k-mooney well unless the image is chaged
16:31:26 mriedem even though we have an outstanding bug about the pci resource usage changing b/c of changing the image
16:31:28 sean-k-mooney in which case we have to go back to the scheuler
16:31:39 mriedem sean-k-mooney: yes but that's for filtering the image, not for claiming resources
16:31:50 mriedem "does the new image work on this existing compute host"
16:32:07 sean-k-mooney well if the image changes the resouces that are need they would have to be claimed/updated
16:32:11 mriedem https://bugs.launchpad.net/nova/+bug/1780441
16:32:12 openstack Launchpad bug 1780441 in OpenStack Compute (nova) "Rebuild does not respect number of PCIe devices" [Undecided,New]
16:32:23 mriedem sean-k-mooney: normal resources don't change based on the image
16:32:26 mriedem except pci
16:32:28 mriedem hence that bug
16:32:32 gibi mriedem: in theory I can trigger a resource healing check during rebuild, but I guess I should not fail the rebuild if bandwidth resource is not available, should I?
16:32:49 sean-k-mooney well you can enable hugepags and cpu pinning and pinning policies and numa too
16:33:06 sean-k-mooney all of which may change the resoces required
16:33:11 sean-k-mooney but in general yes
16:33:37 mriedem for anything nfv related, the answer is don't ever try to do anythign with the server once you created it
16:33:42 sean-k-mooney gibi: mriedem for rebuild with BW aware schduling you would also need to validate if the port request changed
16:33:45 openstack Launchpad bug 1763766 in OpenStack Compute (nova) "nova needs to disallow resource consumption changes on image rebuild" [Medium,Triaged]
16:33:45 cfriesen mriedem: wouldn't that fall under https://bugs.launchpad.net/nova/+bug/1763766 ?
16:33:46 mriedem because nova doesn't support operations on those servers
16:34:04 mriedem cfriesen: oh i thought that was your bug
16:34:06 sean-k-mooney mriedem: well we have never enforced that
16:34:14 cfriesen yeah, it is. I haven't done anything about it yet.
16:34:26 cfriesen I'm just thinking that PCI would fall under it too
16:34:29 sean-k-mooney mriedem: its good advice but its not explcitly stated as far as i know
16:35:00 mriedem sean-k-mooney: yes i know
16:35:02 sean-k-mooney cfriesen: yes likely.
16:35:15 gibi sean-k-mooney: in general we said that changing the resource request of a port is not supported (yet). I don't see how this is special for rebiuld. The port can be changed any time
16:35:21 spotz Anyone able to ask a quick keypair metadata questioon?
16:35:29 spotz err answer... not enough coffee
16:35:54 sean-k-mooney gibi: for rebuild we are recreating the vm from scratch and should validate its resocues
16:36:17 gibi sean-k-mooney: but current nova code assumes that the resource allocation of such VM is unchanged hence no reallocation happen
16:36:24 mriedem correct
16:36:42 gibi sean-k-mooney: so it is not special for banwidth
16:36:45 mriedem because all of the nfv stuff added in juno didn't take rebuild into account
16:36:47 sean-k-mooney https://bugs.launchpad.net/nova/+bug/1763766
16:36:48 openstack Launchpad bug 1763766 in OpenStack Compute (nova) "nova needs to disallow resource consumption changes on image rebuild" [Medium,Triaged]
16:37:03 mriedem s/rebuild/any other operation besides delete/
16:37:07 sean-k-mooney true but the bug cfriesen pointed to would cover it^
16:37:23 cfriesen mriedem: and resize
16:37:34 mriedem "any other operation besides delete"
16:37:35 gibi sean-k-mooney: OK, so you suggest to fail rebuild if the port's resource request changed
16:37:43 sean-k-mooney gibi: yes
16:37:51 gibi sean-k-mooney: that would need nova to store such request
16:38:00 gibi sean-k-mooney: to have something to compare with
16:38:10 mriedem ummm
16:38:17 sean-k-mooney gibi: it could compare it to the existing placement allocation
16:38:22 gibi sean-k-mooney: I'm more like the idea that neutron rejects the change of a qos rule if the port is bound
16:38:29 mriedem that seems pretty shitty for nova have to do that enforcement,
16:38:33 gibi sean-k-mooney: with multiple port, that comparision is really ahrd
16:38:35 gibi hard
16:38:38 sean-k-mooney gibi: that was brought up before
16:38:51 mriedem neutron should be responsible for not allowing bw policy changes on attached ports
16:39:02 sean-k-mooney gibi: it would be a change in the neutron behaivor but not one that they said the were against
16:39:17 mriedem gibi: yes i agree with what you said,
16:39:23 sean-k-mooney mriedem: well this is special for min mandwith
16:39:25 mriedem "I'm more like the idea that neutron rejects the change of a qos rule if the port is bound"
16:39:37 gibi sean-k-mooney: I reacall that this was stated in the spec
16:39:37 sean-k-mooney there is no reason max bandwith of dscp cant change
16:39:43 openstackgerrit Merged openstack/nova master: Reduce calls to placement from _ensure https://review.openstack.org/615677
16:40:15 sean-k-mooney gibi: i raised this before yes but i dont think we made a decision to actully modify neutron to do it
16:40:39 gibi sean-k-mooney: I will ping rubasov tomorrow about it, he is leading the neutron work
16:41:21 sean-k-mooney gibi: anyway this is why assumed rebuild would be out of scope until the migration stuff was done as i assuem someone nova or neutron would have to do this type of validaion
16:41:28 gibi sean-k-mooney: https://review.openstack.org/#/c/595243/2/specs/stein/approved/bandwidth-resource-provider.rst@110 here I state in the spec that changing rule is out of scope https://review.openstack.org/#/c/595243/2/specs/stein/approved/bandwidth-resource-provider.rst@110
16:42:05 gibi sean-k-mooney: but rebuild is not the good time to check, the port might changed without anybody calling rebuild on the server the port is bound to
16:42:20 sean-k-mooney gibi: yes i am aware its out of scope but its required for rebuild and move i think
16:43:06 sean-k-mooney gibi: anyway i dont think it should be a blocker for this in stien
16:43:27 sean-k-mooney its just another edgecase to be address in train
16:43:45 gibi sean-k-mooney: move will use the actual state of the port not because it wants to use the updated value but because neutron is considered the source of truth in case of port resource request so nova does not store such request
16:44:41 gibi when somebody try to change a QoS rule on a boun port, neutron should check if there are enough resource before it accepts the rule change
16:44:43 sean-k-mooney gibi: sure is it not logical that rebuild should also use the actual state of the port for the same reason
16:44:48 mriedem i assume this is already a problem in the neutron api which is much more loosey goosey about making changes to bound ports, unlike cinder's strong enforcement of not making changes to attached volumes
16:44:58 sean-k-mooney gibi: yep we discussed that in dublin
16:45:03 mriedem e..g in neutron am i able to change port binding details about a bound port?
16:45:28 sean-k-mooney mriedem: the other qos type dont consome resouces
16:46:07 sean-k-mooney e.g. max bandwith and dscp marking can be changed on the fly
16:46:08 gibi sean-k-mooney: in generan nova will not be able to prevent the user to change a port in a way that leads to inconsistencies so neutron should do the check
16:46:25 sean-k-mooney gibi: yep im not arguing it should try
16:48:20 cfriesen sean-k-mooney: is this still correct as far as the formatting of the PCI whitelist or is there a newer doc somewhere? https://specs.openstack.org/openstack/nova-specs/specs/juno/implemented/pci-passthrough-sriov.html
16:48:45 gibi sean-k-mooney: but re-allocation during rebuild only needed if neutron allowed the bound port to change.
16:48:48 sean-k-mooney there is a new dock but let me check
16:49:18 sean-k-mooney gibi which it does hand has always supported
16:49:18 gibi sean-k-mooney: so if neutron rejects that then reubild does not need to be changed due to bandwidth
16:49:52 gibi sean-k-mooney: it should not be supported to min bandwidth any more without making sure that the placement allocation is updated
16:50:10 gibi sean-k-mooney: otherwise we end up in resource inconsistencies
16:50:18 mriedem cfriesen: there are examples in https://docs.openstack.org/nova/latest/admin/pci-passthrough.html
16:50:25 sean-k-mooney gibi: sure but doing so was not covered in the spec because it was declared out of scope
16:50:31 gibi sean-k-mooney: exactly
16:50:37 mriedem cfriesen: and https://docs.openstack.org/nova/latest/configuration/config.html#pci.passthrough_whitelist
16:50:42 sean-k-mooney gibi: so in stien it wont be done
16:50:52 gibi sean-k-mooney: so I will ping rubasov tomorrow to make it clear if they will do the reject or not
16:50:58 sean-k-mooney adn we can get out of sync with neutron
16:51:21 sean-k-mooney gibi: that is a neutron api change and would need a neutron spec right
16:51:36 gibi sean-k-mooney: let me check what the neutron spec states about this

Earlier   Later