| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-31 | |||
| 13:45:12 | openstackgerrit | Matthew Booth proposed openstack/nova master: Ensure attachment cleanup on failure in driver.pre_live_migration https://review.openstack.org/587439 | |
| 13:46:46 | sean-k-mooney | by the way this error message really is not helpful http://paste.openstack.org/show/726939/ | |
| 13:47:51 | sean-k-mooney | it had a route to the host. it could not connect on that port because iptabels was blocking it | |
| 13:59:57 | cdent | in the periodic jobs in nova-compute do we expect the context that it starts with to remain the same for the lifetime of the process? Logs are indicating that it is changing. | |
| 14:00:45 | cdent | http://paste.openstack.org/show/726941/ | |
| 14:02:22 | cdent | rgerganov noticed downstream and we assumed it was some local optimizations, but it appears that maybe the original oslo context is a thread not a green thread | |
| 14:03:14 | cdent | mriedem or dansmith ? ^ | |
| 14:03:37 | dansmith | cdent: that's a known issue | |
| 14:03:42 | dansmith | cdent: it shouldn't change, but it is | |
| 14:03:48 | dansmith | a few people have looked at it | |
| 14:04:03 | cdent | so is it sort of a non-damaging buglet? | |
| 14:04:19 | cdent | is there an existing bug you're aware of? | |
| 14:04:21 | mriedem | it's definitely a serviceability issue | |
| 14:04:22 | mriedem | yes, | |
| 14:04:25 | mriedem | i can find some links | |
| 14:04:27 | dansmith | it's not great because it's a tad confusing, but I'm not sure it's actually a big problem | |
| 14:04:46 | dansmith | if we're ever leaking contexts between request threads then *that* is a huge issue, | |
| 14:04:51 | cdent | I think rado may be sufficiently frustrated by it it want to fix it | |
| 14:04:55 | dansmith | but people would see random failures to look up instances and do things, etc | |
| 14:05:03 | dansmith | and I don't see evidence of that | |
| 14:05:16 | mriedem | gibi has some older changes starting here https://review.openstack.org/#/c/524306/ | |
| 14:05:28 | mriedem | oops that's melwitt's ^ | |
| 14:05:36 | mriedem | https://review.openstack.org/#/c/542891/ is gibi's | |
| 14:06:02 | cdent | thanks mriedem | |
| 14:11:56 | bauzas | mriedem: I don't want to be pedantic about microversions, but look at https://docs.openstack.org/nova/latest/contributor/microversions.html#f1 | |
| 14:12:17 | bauzas | mriedem: I was about +2ing https://review.openstack.org/#/c/583566/ given we merged the master one | |
| 14:12:59 | bauzas | mriedem: but here, a HTTP500 will become a HTTP200 right? | |
| 14:13:16 | bauzas | if so, we should possible change the documentation ^ | |
| 14:13:30 | dansmith | 500->200 is fine right? | |
| 14:13:33 | mriedem | it will change a 500 to a 204 | |
| 14:13:34 | dansmith | that's a bug fix | |
| 14:13:41 | mriedem | @wsgi.response(204) | |
| 14:13:56 | mriedem | 500 is not normal and a client should not expect it | |
| 14:14:26 | bauzas | yup, it's fine | |
| 14:14:33 | bauzas | but I looked at the documentation above | |
| 14:14:40 | bauzas | and the note was saying "no" | |
| 14:14:54 | bauzas | anyway, +2ing it | |
| 14:16:12 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Improve NeutronFixture and remove unncessary stubbing https://review.openstack.org/587412 | |
| 14:17:01 | gibi | mriedem: fixed some of your comments, responded on others in ^^ | |
| 14:21:38 | openstack | Launchpad bug 1528743 in OpenStack Compute (nova) "HostState in Scheduler can be incorrect" [Undecided,Confirmed] | |
| 14:21:38 | mriedem | Kevin_Zheng: is this still an issue worth pursuing? https://bugs.launchpad.net/nova/+bug/1528743 | |
| 14:23:19 | Kevin_Zheng | hi, I have to check since it has been too long | |
| 14:28:38 | mriedem | Kevin_Zheng: our public cloud perf team might like to measure this https://review.openstack.org/#/c/576099/ | |
| 14:29:02 | mriedem | efried: if you agree with what i'm saying in ^ then i could just make the change | |
| 14:29:12 | efried | ... | |
| 14:32:13 | efried | mriedem: I'm not sure I'm going to be fully comfortable with anything along these lines until we've figured out how these code paths impact ironic. | |
| 14:32:24 | efried | mriedem: And I don't even know what that would look like. | |
| 14:32:41 | Kevin_Zheng | Ack | |
| 14:34:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove unnecessary PlacementFixture setups https://review.openstack.org/587490 | |
| 14:34:16 | efried | mriedem: Note that I tried something like what you're suggesting via https://review.openstack.org/#/c/576591/ but couldn't get it to work. | |
| 14:40:14 | tbachman | sahid: I was wondering if you might be able to answer a SRIOV question (and/or sean-k-mooney — was referred to you to by mriedem)? Do you happen to know if there’s a way to influence nova’s SRIOV NIC selection on a given host, if a host has mulitiple SRIOV NICs of the same vendor/device ID(and yes, I realize that this is definitely a “pet” approach <sadface>)? | |
| 14:42:00 | jangutter | tbachman: it's possible to distinguish them via the physical network tag | |
| 14:42:41 | tbachman | jangutter: yeah — there’s one other “special” detail I left out (and by special, I mean not wonderful). In this case, the goal is to support the SRIOV NICs on the same neutron network | |
| 14:43:05 | tbachman | I’d looked into HPB, but SRIOV support for HPB seems to have some limitations. | |
| 14:43:12 | tbachman | (which is what led me down this path) | |
| 14:44:18 | jangutter | tbachman: that is... hairy. https://docs.openstack.org/neutron/latest/admin/config-sriov.html <--- I presume you've been over this a couple of times. | |
| 14:44:28 | tbachman | jangutter: ack | |
| 14:44:40 | tbachman | someone else wondered if I could do something with the vif drivers | |
| 14:44:52 | tbachman | but I think at that point, it may be too late | |
| 14:45:23 | tbachman | (and by ack, I meant to say akc on both the doc and the “hairy” bit ;-) ) | |
| 14:45:24 | jangutter | tbachman: yeah, let me find the code that does the actual allocation. | |
| 14:45:42 | efried | In the glorious future, SRIOV VFs will be resources with traits and stuff. So you'd be able to pick 'em however you want. | |
| 14:45:53 | tbachman | efried: :) | |
| 14:47:40 | cdent | mriedem, dansmith: rgerganov found the root of the problem in the import ordering (the patch that changed things). I reckon he'll have a patch soonish | |
| 14:47:49 | jangutter | tbachman: https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L1888 <--- I _think_ this is the piece of code that actually generates the resource request. | |
| 14:48:25 | tbachman | jangutter: ack. mriedem had pointed me to that earlier | |
| 14:48:57 | jangutter | tbachman: so from that code, there is the physnet tag (physical_network) | |
| 14:49:01 | sahid | tbachman: hum like that i don't know, did you try to match by devname? | |
| 14:49:28 | tbachman | sahid: oh — wasn’t aware devname was available in the whitelist | |
| 14:49:35 | tbachman | I guess I should explain my current approah | |
| 14:49:54 | tbachman | in nova.conf, I use the whitelist to provide the physnet => NIC mappings | |
| 14:50:15 | tbachman | but the problem is that the neutron network has only one static segment in this case | |
| 14:50:23 | tbachman | so, although I could assign each NIC it’s own physnet | |
| 14:50:28 | tbachman | the network still only has one | |
| 14:52:19 | sahid | tbachman: when you whitlist NIC you can use devname, all the NIC can be associated with the same physnet | |
| 14:52:25 | openstackgerrit | sean mooney proposed openstack/nova master: libvirt: fix disk_bus handling for root disk https://review.openstack.org/584999 | |
| 14:52:49 | sahid | i'm wondering whether you can create a port in neutron and use port binding to match the devanme | |
| 14:52:51 | tbachman | sahid: sounds good — but when I create the instance, how do I associate that with the instance | |
| 14:53:00 | tbachman | sahid: that’s what I was looking into | |
| 14:53:24 | jangutter | sahid, tbachman: is the dev_name actually _part_ of that pci request generated? | |
| 14:53:48 | tbachman | jangutter: sahid: this is my understanding so far | |
| 14:53:53 | openstack | Launchpad bug 1784579 in OpenStack Compute (nova) "unable to live migrate instance after update to queens" [Undecided,New] | |
| 14:53:53 | mriedem | sean-k-mooney: interesting bug https://bugs.launchpad.net/nova/+bug/1784579 | |
| 14:54:04 | tbachman | I create the neutron network, which has a physnet | |
| 14:54:16 | tbachman | I create a port in neutron, with the “direct” attribute | |
| 14:54:23 | sean-k-mooney | mriedem: taking a look now | |
| 14:54:40 | openstackgerrit | Merged openstack/nova master: Remove superfluous network stubbing in func test https://review.openstack.org/587134 | |
| 14:54:41 | tbachman | nova-api calls into neutron to discover the physnet for the port | |
| 14:54:54 | tbachman | for the network, I should say | |
| 14:55:11 | tbachman | it provides pci request objects to nova-compute | |
| 14:55:23 | sahid | jangutter: devname is considered as a tag | |
| 14:55:35 | tbachman | nova-compute uses its map and the request objects to determine which SRIOV NIC to select | |
| 14:55:49 | sahid | so basically we should try to match it with the request spec generated using port binding profile | |
| 14:55:57 | tbachman | sahid: exactly | |
| 14:56:18 | sean-k-mooney | mriedem: looks like neutron was not able to bind the vif on the destination so set the vif type to binding failed and we eventual get to the code to build the os-vif object and rais that exception | |
| 14:56:46 | mriedem | yes | |
| 14:56:59 | mriedem | and nova persists the "binding_failed" vif type in the info cache so we blow up on compute restart | |
| 14:57:11 | tbachman | sahid: I was able to create a “hack” in nova, where I put the “physical_network” property in the binding:profile member when I create the port in neutron | |
| 14:57:17 | jangutter | tbachmanm sahid: With vnic_type=DIRECT, you the physical network from Neutron too. There are other places in the VIF Neutron stores things (like binding:profile), but... I'm not quite sure if Nova hooks 'em up in the PCI request. | |
| 14:57:17 | sean-k-mooney | mriedem: still reading the bug content however. your multiple binding changes should help with this | |