Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-04-24
15:43:38 jaypipes TheJulia: I'm confused why vif setup actions are being called prior to the node being reserved.
15:44:12 melwitt same
15:44:27 jaypipes TheJulia: I would think that setup_networking_on_host() would only happen after Ironic has notified the Ironic virt driver that the node is ready for provisioining?
15:47:27 TheJulia jaypipes: melwitt: let me grab the link for the change so we can discuss this with more information
15:47:39 TheJulia between three other conversations :(
15:48:19 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Drop MIN_LIBVIRT_BLOCK_LM_WITH_VOLUMES_VERSION https://review.openstack.org/563984
15:48:20 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Drop MIN_LIBVIRT_NUMA_VERSION_PPC https://review.openstack.org/564010
15:48:21 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: Drop BAD_LIBVIRT_NUMA_VERSIONS https://review.openstack.org/564011
15:48:22 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Drop BAD_LIBVIRT_CPU_POLICY_VERSIONS https://review.openstack.org/564012
15:48:23 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Drop MIN_LIBVIRT_PARALLELS_SET_ADMIN_PASSWD https://review.openstack.org/564013
15:49:09 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec https://review.openstack.org/541290
15:49:26 kashyap (Damn, missed the 'libivrt' prefix for one of the commits; will fix it after I figure out to fix the 2 failing unit tests.)
15:50:56 TheJulia melwitt: jaypipes: This is the change that changed the behavior https://github.com/openstack/nova/commit/23d935b3a60741ddb52f076ffeacde9c37f17c8c which should hopefully shed light as to why
15:52:12 melwitt oh, I remember that now
15:53:58 jaypipes TheJulia: gimme a bit to read the original review.
15:54:19 TheJulia jaypipes: no worries, 2 other conversations and a meeting shortly :(
15:55:00 melwitt IP is needed for the volume backend (or some volume backends require it)
15:55:57 TheJulia Correct, as some do IP level filtering on inbound iscsi connections
15:57:23 melwitt so the vif plug is the problem right? I would think you could get the IP early but wait to plug the vif until the normal time
15:57:58 melwitt (until after it's reserved) or would that not help?
15:58:23 openstackgerrit Merged openstack/nova master: Test case: traits don't sync if first access fails https://review.openstack.org/558066
15:59:50 TheJulia Well, getting the IP earlier would help, I think but there is a caveat there I need to try and remember around vif behavior
16:01:31 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Proxy is_volume through DriverBlockDevice https://review.openstack.org/564017
16:01:42 efried mriedem, esberglu, TheJulia: ^^
16:02:03 melwitt what I mean is decouple the two (creating port vs plugging vif). because that patch moved both of them earlier, so I was thinking maybe decoupling them to leave the port create earlier but do the vif plug later (back to the original place). I need to look at it more to see if what I'm saying makes sense or not
16:03:57 melwitt oh, all prepare_networks_before_block_device_mapping does is plug vifs. it doesn't create the port
16:04:36 melwitt hm, I didn't think vif plugging had anything to do with getting an IP address. I thought that was connected to the port creation
16:16:30 TheJulia efried: I completely forgot about that *blink* *blink*
16:16:38 TheJulia feels like a lifetime ago
16:17:16 melwitt TheJulia: I see that the patch gets the IP address from the attached vif from ironic. is there some reason why we couldn't just use the IP address from the neutron port instead of attaching the vif early? https://developer.openstack.org/api-ref/network/v2/#show-port-details
16:17:46 jaypipes melwitt: it's not really port creation that is needed. it's IP allocation.
16:18:04 jaypipes melwitt: but if that's what you mean by port creation, then yes, decoupling those things would be ++
16:18:24 melwitt okay, so with ironic driver we don't get an IP when we create a port in neutron? or do we not even create a port in neutron when ironic driver?
16:18:39 jaypipes melwitt: as with most thing neutron, it depends :)
16:19:16 TheJulia heh
16:19:35 jaypipes melwitt: if the subnet upon which the port resides uses DHCP, then the IP is doled out to the port after the port is set up on the host.
16:19:47 TheJulia I think if we could get the IP address in advance then ++++++ but I'm having strong deja vu, I just can't place it at hte moment
16:19:53 melwitt okay. yeah, from what I understand, creating the port in neutron will allocate the IP(s). plugging the vif will just attach the instance to the network. so I was thinking can we decouple that and get the IP from neutron instead of getting it from ironic by way of the vif
16:19:56 TheJulia uhg, yeah
16:20:04 TheJulia which means the port has to be plugged to have correct dhcp information
16:20:07 jaypipes melwitt: in TheJulia's case, I think that basically the only way that it will work is if an IP is statically assigned ahead of time.
16:20:42 jaypipes melwitt: but I'm definitely no expert in Neutron-isms. Perhaps sean-k-mooney is a good person to ask about this if not mriedem.
16:20:44 melwitt jaypipes: oh, I see. I wasn't thinking of DHCP :(
16:21:57 jaypipes melwitt: in this specific case (the hitachi/fujitsu NFS/SAN thing) I would think that a statically-assigned IP address is really the only way it would work.
16:23:07 jaypipes melwitt: otherwise, the connector info would need to essentially say "this storage NIC is gonna get some IP address in this CIDR but I don't know what that specific IP address is right now". And I'm pretty sure no such affordance is possible in the volume connector info ;)
16:23:12 TheJulia jaypipes: as in operator pre-creation of the ports, definition of said vifs upon spawning a node?
16:23:17 melwitt yeah. huh. I guess that means that any deployment using DHCP must be creating neutron ports in deferred mode or no IP allocation mode?
16:23:51 jaypipes TheJulia: yes, which is a very common thing already.
16:24:16 jaypipes TheJulia: and passing nova boot --nic port=<UUID> (or whatever the magical incantation is...)
16:24:19 TheJulia jaypipes: that is correct, we can't say "later" for the volume connection info because it gets shipped all the way to the backend storage upfront
16:24:33 TheJulia hshiina|afk: ^^^ thoughts
16:24:44 jaypipes TheJulia: although saying "laterz dude" in the connector info would be, well, amaze-balls.
16:24:57 TheJulia jaypipes: totally
16:25:08 jaypipes TheJulia: I think you meant "totes magotes".
16:25:27 melwitt hah
16:25:33 melwitt jobin
16:25:42 TheJulia lol
16:27:11 melwitt going down a different thought path, can we not wait until after the node is reserved to attach the block device instead?
16:27:43 TheJulia we actually should be waiting
16:28:14 TheJulia but the cinderside of that has to be done first with information about the node too
16:28:20 TheJulia wow, what a fun chicken and the egg
16:28:27 melwitt okay, I think I'm confused because the new problem that's happening is vifs are being plugged before the node is reserved. and we need to plug the vifs to get the IP address so we can attach the volume
16:28:41 jaypipes melwitt: we could definitely do that, yes. Though that won't solve the problem if the port has no IP address already, right?
16:28:41 melwitt so I think I don't get why we can do both after the node is reserved
16:28:52 melwitt *can't
16:29:18 melwitt like, why can't we reserve the node, plug vifs, get IP for volume connector, attach volume
16:29:38 TheJulia Ideally that is what we should be doing in the grand scheme of the universe
16:29:42 TheJulia ideally
16:30:09 melwitt there is something preventing that from being possible?
16:30:34 jaypipes melwitt: nope
16:30:41 jaypipes melwitt: nothing preventing that AFAICT
16:30:58 TheJulia what would be more ideal I guess is the question from a nova point of view, a new pre-spawn virt driver method call or we just quietly check/assert when we get the vif plug call
16:30:59 melwitt let's do that
16:32:50 melwitt oh, so the first step in spawn would not be reserve?
16:33:05 melwitt (I don't know where reserve is currently happening)
16:33:24 TheJulia melwitt: in spawn when we begin to populate information in ironic about the instance
16:33:38 melwitt ohhh yeah. sorry
16:33:57 melwitt this network prep thing is called in compute/manager early
16:34:07 TheJulia no worries
16:34:08 TheJulia yeah
16:37:11 melwitt looking in the code for where reserve is currently happening and can't find it
16:37:31 jaypipes melwitt: yeah, we might want to move the network prep for block devices thing to being a method that is called from the virt driver and not the compute manager
16:38:07 jaypipes melwitt: so that the virt driver can dictate when it is appropriate to do that prep work
16:38:23 TheJulia melwitt: instance_uuid being set
16:38:23 dansmith melwitt: are you suggesting plugging vifs before we call virt spawn across the board?
16:38:37 jroll I did a poc for the pre-spawn method call TheJulia mentioned, fwiw: https://review.openstack.org/#/c/563722/
16:38:38 jaypipes dansmith: nope, not plugging vifs.
16:38:40 melwitt dansmith: no, I mean, where can we do the ironic node reserve call
16:39:04 dansmith jaypipes: okay I see vif plugging discussion above, but wasn't following
16:39:19 melwitt because ultimately we want to reserve the node before plugging vifs or attaching volumes. I just don't know where that's currently being done
16:39:28 TheJulia melwitt: let me get you the line, one moment
16:39:35 jroll melwitt: currently the ironic driver reserves the node within spawn()
16:39:46 jroll by sending instance_uuid in a PUT request
16:39:57 melwitt okay I see
16:39:57 jroll or node.update I guess, in ironicclient terms
16:40:25 melwitt I mean, the cheat would be move that call to "prepare networking etc" method
16:40:27 TheJulia https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L360
16:40:59 jroll melwitt: ya, that's the other version: https://review.openstack.org/#/c/563714/
16:41:05 TheJulia :)
16:41:07 jaypipes dansmith: no worries. we were discussing that ironic (actually, some cinder drivers) need to know the IP address of a port up front. and the prep_network_for-blokc_devices() virt driver API method was added in order to address that need. But that method does vif plugging (because that's unfortunately when a port's IP address is guaranteed to be set). We would like to see the IP address allocation decoupled from the vif plugging action.

Earlier   Later