Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-04-24
17:39:14 gibi jaypipes: thanks, +2
17:39:31 mriedem melwitt: i might have been overthinking it
17:40:18 melwitt k, just trying to make sure there's not something I'm missing
17:40:33 mriedem replied to clarify
17:41:16 melwitt cool thanks
17:43:32 sean-k-mooney melwitt: sorry i was in meeting for the last 4 hours. did ye resolve the issue ye were having?
17:44:11 melwitt sean-k-mooney: yeah, I think so. we were talking about, when is a port guaranteed to have an IP allocated to it? maybe never, but it sounded like after host binding it has to?
17:45:46 openstackgerrit Merged openstack/nova master: Extract generate_hostid method into utils.py https://review.openstack.org/555282
17:45:52 sean-k-mooney melwitt: right so with multi segment, l3 routed networks the allocation of the ip happens only when the port is bound to a specific host. for the general case the port will get an ip when it is created but you can have ports that only have mac adresses
17:46:36 melwitt sean-k-mooney: gotcha. thanks
17:46:37 sean-k-mooney melwitt: so after you do port binding it should have an ip if it is connected to a neutron network that has a subnet
17:47:28 openstackgerrit Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306
17:49:26 melwitt sean-k-mooney: okay, so it's not ever necessarily guaranteed (would a neutron network ever not have a subnet?). we were looking at some code in the ironic driver where they need to get the IP in order to hand it to the volume connector (some volume backends require the IP before attaching). and we were wondering, at what point could one reliably query for the IP
17:50:42 sean-k-mooney melwitt: neutron allows your to have interfaces with no ips. nova used to require one for a neutron port to be used with an vm/instance but that requrement was removed.
17:51:03 sean-k-mooney melwitt: the reason it was removed was to cater for non ip networking e.g. infinaband over ethernet
17:51:07 melwitt I see
17:52:27 sean-k-mooney melwitt: it sound like in the ironic case there is a hard requirement for an ip presuable you are using it to configre iscsi targest for the baremtal not or something like that?
17:53:57 melwitt sean-k-mooney: yeah must be a hard requirement. it's already doing it like this https://review.openstack.org/#/c/468353/19/nova/virt/ironic/driver.py but yeah need the IP to set in the volume connector
17:55:07 sean-k-mooney in general l2 only connectivity is not common in neutron so this is an extream edgecase.
17:55:33 melwitt and the problem was that they have to do their equivalent of "plug_vifs" first in order to get the IP and that's happening outside their node reserve lock and there are races happening. so at first we were talking about "can we get the IP before plugging the vif" but it ended up not mattering and the solution needs to be to reserve the node first before doing anything with the networking
17:58:28 openstackgerrit Merged openstack/nova master: Address nits in I00d29e9fd80e6b8f7ba3bbd8e82dde9d4cb1522f https://review.openstack.org/561493
18:00:36 sean-k-mooney from a ironic perspective i think its prefectly resonable to scope the cinder volume support to jsut instance with ips at a first pass. volumes attached over infinaband is something in the storage space where we might not have ips but that is less common.
18:01:35 melwitt yeah, that's what they've done
18:05:10 openstackgerrit Merged openstack/nova stable/ocata: Add regression test for persisted RequestSpec.retry from failed resize https://review.openstack.org/560162
18:05:17 openstackgerrit Merged openstack/nova stable/ocata: Don't persist RequestSpec.retry https://review.openstack.org/560167
18:05:23 openstackgerrit Merged openstack/nova stable/ocata: only increment disk address unit for scsi devices https://review.openstack.org/561613
18:09:20 efried cdent: yt?
18:09:30 cdent efried: hi, yup
18:09:56 efried cdent: Good evening to you. Do you know of any reason Instance.uuid would or would not respond to isinstance(str) ?
18:11:09 cdent I read you talking about that a bit earlier, but didn't chase it. Is it showing up in a particular piece of code that I can poke at? Have you been able to find out what type it is when that happens? Or is it a racey thing that you've got no easy replication for?
18:13:30 sean-k-mooney efried: cdent jaypipes-lunch i need to run very soon but was a decision reached on the mailing list of how to proceed with resouce provider groups?
18:14:11 efried sean-k-mooney: You mean whether numbered groups are separate by default or unrestricted?
18:14:25 cdent sean-k-mooney: I think you mean this https://review.openstack.org/#/c/562687/
18:14:28 sean-k-mooney efried: yes. last i read was matt suggesting that its explcit
18:14:59 efried sean-k-mooney: Yup, what cdent said - that guy has (all the links to) all the info you need.
18:15:30 efried cdent: This is what I'm chasing down: http://184.172.12.213/58/557958/10/check/nova-out-of-tree-pvm/c1d7e99/logs/n-cpu.txt.gz?#_Apr_20_08_51_16_452651
18:15:38 sean-k-mooney cdent: thanks im out of office tomrow but ill print out the current version and review tomrow.
18:15:41 cdent thanks efried
18:15:55 sean-k-mooney efried: im glad ye found a path forward
18:15:56 cdent woah, colors finally merged
18:16:30 sean-k-mooney cdent: colors?
18:16:30 efried cdent: No, I think that's just because it's in our CI env and I have that patched in there (cause that's where I was testing it)
18:17:07 efried cdent: But you should definitely tell everybody how much you like it in https://review.openstack.org/524744 so they might merge it.
18:17:33 cdent oh, sorry, didn't recognize that was your ci efried
18:17:49 openstack Launchpad bug 1765748 in OpenStack Global Requirements "webob-1.8.1 breaks projects" [High,In progress] - Assigned to Matthew Thode (prometheanfire)
18:17:49 melwitt cdent: I had wanted to bring this bug to your attention, in case you might have some idea why the test_wsgi stuff would fail with webob 1.8.1 https://bugs.launchpad.net/nova/+bug/1765748
18:18:23 efried cdent: Anyway, the deal here is that powervm has a utility method that accepts either an integer ID or a UUID. It detects which one it is and then behaves accordingly. The code that does that hasn't changed in yeeears.
18:18:47 efried The code path that calls it is when we're disconnecting a disk, which (I think) also hasn't changed in yeeeears.
18:19:20 sean-k-mooney melwitt: the stack trace look like one that we get when we need to cap a dependcy in upper constrataint because of an incompatible change
18:19:35 cdent melwitt: hmmm. webob often breaks such stuff, but that looks like a bad mock? is there an in-nova example?
18:19:37 efried cdent: But suddenly on Friday afternoon, our CI started failing with that error. I don't actually know what type it thinks it has in there - just know that isinstance(str) is failing where it didn't before.
18:19:55 mriedem sean-k-mooney: totally unrelated to what you're talking about, but i'm a bit lost in https://review.openstack.org/#/c/551370/16/nova/virt/libvirt/migration.py@229 and looking for some feedback when you get a chance
18:20:03 cdent efried: still parsing
18:21:03 sean-k-mooney mriedem: in the generation of the updated xml for the vif on the destination node
18:21:45 mriedem sean-k-mooney: yeah, i'm looking at how _update_volume_xml works and it's basically an update of any volume disks on the source using the destination block device config
18:21:55 mriedem it does that by keying off the volume 'serial' which is the volume id
18:22:09 mriedem we don't have something like that for the interface config in a domain xml
18:22:25 sean-k-mooney mriedem: we could posibly key of the interface mac adress
18:22:28 mriedem i'm not really even sure if i should be updating the interface xml entries, or replacing them
18:23:01 mriedem i.e. i'm trying to handle the case that you're live migrating from a host using ovs to a host using ovn, or linuxbridge, or vhostuser, et al
18:23:06 sean-k-mooney that said im not sure if we can have a vm with two port with the same mac..
18:23:52 mriedem i've been trying to write a test for this for a couple of hours and it just seems like doing a vif type conversion like this down in the bowels of the xml code is going to be....not reasy
18:23:53 mriedem *easy
18:23:55 sean-k-mooney mriedem: right in that case we are replaceing them but we likely want to keep the guest pci adress the same on both ends
18:24:33 mriedem yeah the only thing that _update_volume_xml doesn't change is the address
18:25:09 cdent efried: so "ValueError: invalid literal for int() with base 10: '4E27E1E6-6A24-4F0A-8E7B-2BBE7B4A28BA'" is caused by a conditional faililng somewhere that wasn't before? I'm unable to grep isinstance anywhere in the log?
18:25:59 efried cdent: It's caused by this: https://github.com/powervm/pypowervm/blob/master/pypowervm/utils/uuid.py#L50
18:26:21 efried That isinstance() fails, so we try to int() the UUID on L55, leading to the ValueError.
18:26:48 mriedem efried: have you looked at https://github.com/openstack/oslo.versionedobjects/compare/1.32.0...1.33.1 ?
18:27:04 mriedem https://github.com/openstack/oslo.versionedobjects/commit/b1d0b5d886afef8c08330bf3c2291e180aa1f534
18:27:09 efried cdent: or, shit, I guess it's possible the regex match could be failing. But when I str(instance.uuid) up the stack, it succeeds.
18:27:11 efried mriedem: looking...
18:27:14 sean-k-mooney mriedem: so if we can assume that the vm cannont have 2 intefaces with the same mac then i think that is likely the best way to approch. if we had a way to store the neutron port uuid in the xml that would help alot but i dont think we can do that with libvirt
18:27:50 efried mriedem: Bingo. When did we subsume that req?
18:27:53 efried in nova
18:27:57 efried in queens
18:28:00 efried cause I looked for that
18:28:06 mriedem efried: upper-constraints on friday for rocky
18:28:14 mriedem https://github.com/openstack/requirements/commit/87540884100650cfd1a67f05163a724906efb46f#diff-0bdd949ed8a7fdd4f95240bd951779c8
18:28:15 efried ahhhh, upper-constraints.
18:28:28 efried Yup, that'd do it. Thanks mriedem
18:28:37 efried I knew you would come through for me.
18:30:03 efried mriedem: That same thing must have gone into queens somehow.
18:30:32 mriedem or your CI is using the wrong upper-constraints?
18:30:41 mriedem your queens CI is likely pointing at master
18:31:25 mriedem although maybe not https://github.com/openstack/oslo.versionedobjects/commit/e918eb976fb5a6f9fa7b7644d5a10d383fcfcf21
18:31:38 mriedem https://review.openstack.org/#/q/Ic6b6308fb1960ec40407e6efde30137b64543e72
18:32:04 mriedem that's not released yet on stable though
18:33:46 cdent efried: even if that change isn't your problem, it may be the clue. are you in py2?
18:33:57 mriedem sean-k-mooney: so do you think this is safe https://review.openstack.org/#/c/551370/16/nova/virt/libvirt/migration.py@229 where it's basically taking the source vif and just overwriting whatever we got from the dest to get the new config xml?
18:34:02 efried cdent: Probably
18:34:21 cdent so if you've got a unicode there (for whatever random reason)
18:35:10 efried cdent: Our "official" fix is going to be using is_uuid_like. But that's going to require changes to pypowervm, which is going to need a requirements bump, which I'm not sure if we can swing in stable. Because we're going to have the same problem in nova (right esberglu?)
18:36:37 cdent I'm totally riffing at this point (because multitasking), but does six.text_type make a difference?
18:36:45 sean-k-mooney mriedem: my concern is how to you merge the two so that if i have a vm with 2 interfaces of the same type they dont swap places e.g. eth0 becomes eth1 and vice versa.
18:36:56 efried and pike
18:37:25 esberglu efried: That wasn't hitting queens
18:37:29 esberglu Only master
18:37:33 efried esberglu: See email - it is now.
18:37:47 efried and based on https://review.openstack.org/#/q/Ic6b6308fb1960ec40407e6efde30137b64543e72 it'll hit pike as soon as that percolates down.

Earlier   Later