Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-07
19:27:36 efried sean-k-mooney: I think it's more likely to be like, "You thought procs 0 and 2 were next to each other, and 1 and 3 were next to each other. Well, now it's 0-1 and 2-3. Deal."
19:28:23 sean-k-mooney haha ya maybe. if you ever test it let me know
19:28:36 efried sean-k-mooney: I feel confident in saying that will never happen.
19:28:58 efried ...for some value of "never".
19:29:11 sean-k-mooney efried: you obvioulsy are not spending enough time talking to telcos
19:30:11 efried If it's anything like talking to the AT&T help desk, I assure you I don't want to spend any more.
19:46:49 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Make ResourceTracker.stats node-specific https://review.openstack.org/588037
19:49:03 mriedem was there a zuul reset or something such that we need to recheck things?
19:51:06 dansmith 513 backlog,
19:51:10 dansmith so I imagine it's catching up
19:51:35 dansmith backlog has been up and down all day so I figured something was going on
19:53:31 sean-k-mooney nothing on the openstack-infra twitter for the last 4 days. then normally post somthing if they need to do a reset
19:54:27 sean-k-mooney well it auto posts when then set a status message on the infra channel but in either case the gate is proably just busy/backed up
19:56:20 mriedem ok
20:00:14 openstackgerrit Matt Riedemann proposed openstack/nova stable/ocata: Make ResourceTracker.stats node-specific https://review.openstack.org/588077
20:20:58 openstackgerrit Matt Riedemann proposed openstack/nova master: doc: mark the max microversion for rocky https://review.openstack.org/589598
20:31:30 maciejjozefczyk mriedem: efried please check my response: https://review.openstack.org/#/c/520024
20:32:11 efried Thanks maciejjozefczyk
20:32:47 maciejjozefczyk efried: u2 for pinging me, I forgot about this topic
20:37:34 mriedem melwitt: comments in the reno prelude https://review.openstack.org/#/c/589303/
20:37:46 melwitt ack
20:43:17 mriedem jroll: you might want to check the rebalance question in here https://review.openstack.org/#/c/520024/
20:49:53 jroll "want" is a weird word to use :P
20:49:59 jroll looking
20:53:03 mriedem i'm just asking for a witness that i asked for an ironic person to look before we break the rebalance stuff
20:53:08 jroll heh
20:54:04 jroll so I'm thinking it's fine to remove that, but... I don't know everything that _update does and I'm not sure things like _update_usages_from_instances() and such will do the right thing, if we haven't updated which compute hosts we moved active ironic instances to
20:55:07 jroll I feel like I'm nervous to remove it
20:56:40 sean-k-mooney mriedem: is there an ironic job we can trigger maybe via the experimental pipelien to test that patch
20:57:32 jroll sean-k-mooney: there isn't one that actively kills compute hosts and such to trigger rebalances
20:58:40 sean-k-mooney jroll: it looks like the only ironic job that did run on this skipp 22 out of the 23 test ... http://logs.openstack.org/24/520024/9/check/ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa/1fe9001/testr_results.html.gz
21:07:34 jroll sean-k-mooney: yes, we run a very small set of tests on nova changes because our tests take a long time
21:08:01 jroll sean-k-mooney: that doesn't change the fact that there aren't any full-stack tests that exercise this code
21:08:37 jroll because it's really hard to do - would involve multiple nova-compute instances and orchestrating bringing them up and down and such
21:08:58 efried just mock everything
21:09:59 jroll anyway, I'm gonna post my comments on the change and then not computer anymore today. see y'all later
21:11:57 sean-k-mooney jroll: im not saying there would be test for this just taking note of the fact that the ironic testing is much lighter then i would have expected
21:12:44 sean-k-mooney jroll: that is not a slight againt ironic i just would have expect more tests even if they only ran in the gate pipeline and not check
21:12:50 jroll sean-k-mooney: figure out how to get nested virt in the gate and we can run them fast enough to test more :)
21:13:31 jroll that one instance boot takes like 20 minutes or something iirc
21:14:00 sean-k-mooney whell i honestly dont know why we dont use nested virt in the gate other then the fact that rackspace provides xen based instance and most of our test would like kvm
21:14:13 jroll there's a long history of why that I don't have time to get into
21:14:46 sean-k-mooney jroll: i have been trying to get nested virt in the gate since 2013 im well awaree of the history but no worries
21:18:22 melwitt only sometimes?
21:18:25 sean-k-mooney i just spent 6 hours debuging network connectivit issue with ovs-dpdk because it has buggy numa detection logic
21:19:30 sean-k-mooney melwitt: normally it works fine but when it doesnt its a pain in the ass to figure out why
21:31:10 mriedem dansmith: do you have any idea why we have a Service.availability_zone field on the versioned object?
21:31:31 mriedem i see that we try to lazy-load it for notifications
21:31:32 mriedem 2018-08-07 17:24:53,165 DEBUG [nova.objects.service] Lazy-loading 'availability_zone' on Service id 2
21:31:32 mriedem 2018-08-07 17:24:53,166 DEBUG [nova.notifications.objects.base] Defaulting the value of the field 'availability_zone' to None in ServiceStatusPayload due to 'Object action obj_load_attr failed because: attribute availability_zone not lazy-loadable'
21:34:13 mriedem i guess for https://github.com/openstack/nova/commit/a2c6838ff5ff095940a76ebd4d578e24575c30d8#diff-b9be5fa188b7efd457da79e9c543344bR110
21:39:28 sean-k-mooney jaypipes: are you around?
21:43:49 sean-k-mooney i just realised that we missed
21:45:06 sean-k-mooney * i just realised that https://review.openstack.org/#/c/587378/3/vif_plug_ovs/ovs.py misses passing the ovsdb_connection on on of the vhost-user code paths. should i just submit a patch for the missing fucntion or revert and submit an updated versions
21:46:14 sean-k-mooney im thinking just add a patch on top the get the missing function call but just said i would ask
22:01:30 sean-k-mooney mriedem: by the way im assuming you did not have time to test livemigrating between different neutron backends as part of the multiple port binding blueprint?
22:02:02 mriedem sean-k-mooney: mlavalle did it between ovs and linuxbridge using neutron directly, not via nova
22:02:11 mriedem i don't have a mixed vif type env setup no
22:02:26 sean-k-mooney mriedem: now that i have figured out why ovs-dpdk was not working from me ill try an test it out later this week. ill also try it via linux bridge if i get a chance
22:02:38 sean-k-mooney cool no worries
22:04:00 sean-k-mooney i normaly have ovs, ovs-dpdk and linux bride deployed concurrnetly or at least i did before i move. ill test the matirx of all 3 setups and let you know how it goes
22:30:40 mriedem Kevin_Zheng: comments all over https://review.openstack.org/#/q/topic:bug/1781880+(status:open+OR+status:merged) so it should be clear to update now
22:30:48 mriedem and abandon the functional test since it will never work
22:39:00 openstackgerrit Matt Riedemann proposed openstack/nova master: Update really old comments about vmware hosts managing multiple nodes https://review.openstack.org/589666
22:47:44 openstackgerrit Matt Riedemann proposed openstack/nova master: Add microversion info in the os-server-groups API samples https://review.openstack.org/589006
22:50:34 melwitt mriedem: I'm not sure what to make of this comment after trying to make the changes to make things fail if a cell raises an exception https://review.openstack.org/#/c/540258/10/nova/scheduler/utils.py@843
22:51:16 melwitt it's not clear to me if a cell raising an exception will be part of our "down cell" handling down the line or if we should expect to potentially fail a server create with a 500 if a cell raises an exception when we query it
22:51:55 melwitt pre-down-cell handling
22:53:11 melwitt initially I was thinking yes, it makes sense to fail the create if a cell raises an exception, but that would propagate up to the user as a 500 because there's something wrong with a cell, and then I became unsure if that falls under the "down cell" stuff or if it's just something we should do now
22:58:09 efried mriedem: I assume that +1 is so we wait until stein to land it
22:58:32 efried (the "Update resources once" patch)
22:58:59 mriedem efried: yeah
22:59:33 efried mriedem: I'm going to -2 it just in case, but I'm with ya.
22:59:45 mriedem efried: don't think you need to -2
22:59:50 openstackgerrit Merged openstack/os-traits master: Update reno for stable/rocky https://review.openstack.org/586103
23:00:07 mriedem efried: i -W'ed it
23:00:11 efried mriedem: You mean because no other cores are going to "accidentally" merge it? :)
23:00:19 mriedem melwitt: how does a cell raise an exception?
23:00:26 mriedem if the called function does?
23:00:35 mriedem like the DB API query explodes or something?
23:00:41 melwitt mriedem: yeah, exactly. if whatever is called under target_cell raises
23:00:54 efried mriedem: ack, +2ed
23:01:42 melwitt it makes sense to fail the boot over that if someone is trying to boot with affinity and a cell somehow raises an exception (gibi pointed it out on the review)
23:01:45 mriedem melwitt: it wouldn't be a 500 to the user for server create
23:01:59 mriedem b/c we've already cast from api to conductor which is what calls setup_instance_group
23:02:04 melwitt I'm just getting mixed up about whether that is in the scope of the bug fix I'm working on right now, or if that's going to be later on with the handling of a down cell work
23:02:14 mriedem cold/live migrate + evacuate + unshelve might return a 500....
23:02:33 mriedem well gibi said it was ok to add a TODO and deal with it later yeah?
23:02:46 melwitt oh, okay. my bad, I was thinking setup_instance_group was called from compute/api but was mistaken
23:03:06 mriedem it's called from conductor, but whether or not we've already returned 202 to the user depends on the operation
23:03:23 melwitt he said the TODO for the "did not respond" but for the raised exception case, suggeste failing the boot
23:03:46 mriedem i would lump that into tssurya's bp in stein
23:03:50 mriedem or as a separate bug fix
23:03:57 mriedem it's not the issue for this patch
23:04:16 melwitt okay. thanks. that's what I was thinking as I went to add it, it's adding a lot to the scope
23:04:28 mriedem commented
23:04:49 mriedem melwitt: i also looked at https://review.openstack.org/#/c/582332/ and i'm not sure what it changes,
23:05:01 mriedem but i did notice we're logging in a greenthread in one place there and i thought that was a real no-no
23:05:05 mriedem dansmith: yeah? ^
23:07:13 melwitt mriedem: I'm not 100% sure where the methods that were changed get called during the scheduler run, so I have to look at that to see if the result is visible during a gate run

Earlier   Later