Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-07
21:08:58 efried just mock everything
21:09:59 jroll anyway, I'm gonna post my comments on the change and then not computer anymore today. see y'all later
21:11:57 sean-k-mooney jroll: im not saying there would be test for this just taking note of the fact that the ironic testing is much lighter then i would have expected
21:12:44 sean-k-mooney jroll: that is not a slight againt ironic i just would have expect more tests even if they only ran in the gate pipeline and not check
21:12:50 jroll sean-k-mooney: figure out how to get nested virt in the gate and we can run them fast enough to test more :)
21:13:31 jroll that one instance boot takes like 20 minutes or something iirc
21:14:00 sean-k-mooney whell i honestly dont know why we dont use nested virt in the gate other then the fact that rackspace provides xen based instance and most of our test would like kvm
21:14:13 jroll there's a long history of why that I don't have time to get into
21:14:46 sean-k-mooney jroll: i have been trying to get nested virt in the gate since 2013 im well awaree of the history but no worries
21:18:22 melwitt only sometimes?
21:18:25 sean-k-mooney i just spent 6 hours debuging network connectivit issue with ovs-dpdk because it has buggy numa detection logic
21:19:30 sean-k-mooney melwitt: normally it works fine but when it doesnt its a pain in the ass to figure out why
21:31:10 mriedem dansmith: do you have any idea why we have a Service.availability_zone field on the versioned object?
21:31:31 mriedem i see that we try to lazy-load it for notifications
21:31:32 mriedem 2018-08-07 17:24:53,165 DEBUG [nova.objects.service] Lazy-loading 'availability_zone' on Service id 2
21:31:32 mriedem 2018-08-07 17:24:53,166 DEBUG [nova.notifications.objects.base] Defaulting the value of the field 'availability_zone' to None in ServiceStatusPayload due to 'Object action obj_load_attr failed because: attribute availability_zone not lazy-loadable'
21:34:13 mriedem i guess for https://github.com/openstack/nova/commit/a2c6838ff5ff095940a76ebd4d578e24575c30d8#diff-b9be5fa188b7efd457da79e9c543344bR110
21:39:28 sean-k-mooney jaypipes: are you around?
21:43:49 sean-k-mooney i just realised that we missed
21:45:06 sean-k-mooney * i just realised that https://review.openstack.org/#/c/587378/3/vif_plug_ovs/ovs.py misses passing the ovsdb_connection on on of the vhost-user code paths. should i just submit a patch for the missing fucntion or revert and submit an updated versions
21:46:14 sean-k-mooney im thinking just add a patch on top the get the missing function call but just said i would ask
22:01:30 sean-k-mooney mriedem: by the way im assuming you did not have time to test livemigrating between different neutron backends as part of the multiple port binding blueprint?
22:02:02 mriedem sean-k-mooney: mlavalle did it between ovs and linuxbridge using neutron directly, not via nova
22:02:11 mriedem i don't have a mixed vif type env setup no
22:02:26 sean-k-mooney mriedem: now that i have figured out why ovs-dpdk was not working from me ill try an test it out later this week. ill also try it via linux bridge if i get a chance
22:02:38 sean-k-mooney cool no worries
22:04:00 sean-k-mooney i normaly have ovs, ovs-dpdk and linux bride deployed concurrnetly or at least i did before i move. ill test the matirx of all 3 setups and let you know how it goes
22:30:40 mriedem Kevin_Zheng: comments all over https://review.openstack.org/#/q/topic:bug/1781880+(status:open+OR+status:merged) so it should be clear to update now
22:30:48 mriedem and abandon the functional test since it will never work
22:39:00 openstackgerrit Matt Riedemann proposed openstack/nova master: Update really old comments about vmware hosts managing multiple nodes https://review.openstack.org/589666
22:47:44 openstackgerrit Matt Riedemann proposed openstack/nova master: Add microversion info in the os-server-groups API samples https://review.openstack.org/589006
22:50:34 melwitt mriedem: I'm not sure what to make of this comment after trying to make the changes to make things fail if a cell raises an exception https://review.openstack.org/#/c/540258/10/nova/scheduler/utils.py@843
22:51:16 melwitt it's not clear to me if a cell raising an exception will be part of our "down cell" handling down the line or if we should expect to potentially fail a server create with a 500 if a cell raises an exception when we query it
22:51:55 melwitt pre-down-cell handling
22:53:11 melwitt initially I was thinking yes, it makes sense to fail the create if a cell raises an exception, but that would propagate up to the user as a 500 because there's something wrong with a cell, and then I became unsure if that falls under the "down cell" stuff or if it's just something we should do now
22:58:09 efried mriedem: I assume that +1 is so we wait until stein to land it
22:58:32 efried (the "Update resources once" patch)
22:58:59 mriedem efried: yeah
22:59:33 efried mriedem: I'm going to -2 it just in case, but I'm with ya.
22:59:45 mriedem efried: don't think you need to -2
22:59:50 openstackgerrit Merged openstack/os-traits master: Update reno for stable/rocky https://review.openstack.org/586103
23:00:07 mriedem efried: i -W'ed it
23:00:11 efried mriedem: You mean because no other cores are going to "accidentally" merge it? :)
23:00:19 mriedem melwitt: how does a cell raise an exception?
23:00:26 mriedem if the called function does?
23:00:35 mriedem like the DB API query explodes or something?
23:00:41 melwitt mriedem: yeah, exactly. if whatever is called under target_cell raises
23:00:54 efried mriedem: ack, +2ed
23:01:42 melwitt it makes sense to fail the boot over that if someone is trying to boot with affinity and a cell somehow raises an exception (gibi pointed it out on the review)
23:01:45 mriedem melwitt: it wouldn't be a 500 to the user for server create
23:01:59 mriedem b/c we've already cast from api to conductor which is what calls setup_instance_group
23:02:04 melwitt I'm just getting mixed up about whether that is in the scope of the bug fix I'm working on right now, or if that's going to be later on with the handling of a down cell work
23:02:14 mriedem cold/live migrate + evacuate + unshelve might return a 500....
23:02:33 mriedem well gibi said it was ok to add a TODO and deal with it later yeah?
23:02:46 melwitt oh, okay. my bad, I was thinking setup_instance_group was called from compute/api but was mistaken
23:03:06 mriedem it's called from conductor, but whether or not we've already returned 202 to the user depends on the operation
23:03:23 melwitt he said the TODO for the "did not respond" but for the raised exception case, suggeste failing the boot
23:03:46 mriedem i would lump that into tssurya's bp in stein
23:03:50 mriedem or as a separate bug fix
23:03:57 mriedem it's not the issue for this patch
23:04:16 melwitt okay. thanks. that's what I was thinking as I went to add it, it's adding a lot to the scope
23:04:28 mriedem commented
23:04:49 mriedem melwitt: i also looked at https://review.openstack.org/#/c/582332/ and i'm not sure what it changes,
23:05:01 mriedem but i did notice we're logging in a greenthread in one place there and i thought that was a real no-no
23:05:05 mriedem dansmith: yeah? ^
23:07:13 melwitt mriedem: I'm not 100% sure where the methods that were changed get called during the scheduler run, so I have to look at that to see if the result is visible during a gate run
23:07:42 melwitt basically, any time those methods run, they will replace the thread local context that oslo.context stores underneath, and oslo.log pulls from that to log request-ids
23:08:17 melwitt so what would happen is a request-id for a thread would change midway if one of the methods that created RequestContext without overwrite=False ran during it
23:08:30 mriedem both run on startup
23:08:42 melwitt okay, I think that's why it wouldn't show up
23:08:46 mriedem well, one does
23:08:49 mriedem the cells refresh one
23:08:55 mriedem the other doesn't, presumably b/c it's in a greenthread
23:09:06 mriedem or maybe b/c we have'nt discovered any hosts yet
23:15:47 melwitt yeah, looks like the greenthread is spawned during startup so it's effectively during startup too
23:15:59 openstackgerrit Merged openstack/nova master: [placement] Add version directives in the history doc https://review.openstack.org/589392
23:16:08 openstackgerrit Merged openstack/nova master: Avoid joins in _server_group_count_members_by_user https://review.openstack.org/580764
23:16:16 openstackgerrit Merged openstack/nova master: Use common functions in granular fixture https://review.openstack.org/588113
23:17:11 melwitt but I see now, it doesn't run at all, don't find it in the log
23:17:51 melwitt oh, because logging in the greenthread is expected not to work?
23:19:00 melwitt oh, it's because [filter_scheduler]/track_instance_changes = False
23:19:04 mriedem melwitt: no i think it's because we disable CONF.filter_scheduler.track_instance_changes in superconductor mode in devstack
23:19:53 mriedem http://logs.openstack.org/32/582332/5/check/neutron-grenade/b319aa4/logs/screen-n-sch.txt.gz#_Jul_31_18_48_24_316837
23:20:06 melwitt ahh
23:20:07 mriedem grenade runs in singleconductor mode so it's logged there
23:20:18 melwitt nice
23:20:50 melwitt so let's see if there's a difference in that job before the patch
23:22:20 melwitt looks like it, new request-id as of async_init_instance_info http://logs.openstack.org/58/540258/10/check/neutron-grenade/54f96c0/logs/screen-n-sch.txt.gz#_Jul_24_06_24_31_353068
23:22:43 melwitt wait, but I still see the request-id prior to that being logged too
23:22:52 melwitt hm
23:28:07 melwitt oh, bc it has its own greenthread. duh. yeah so that one wouldn't show anything anyway
23:29:29 melwitt it can't cause a change in any other greenthread's local context
23:38:51 jaypipes sean-k-mooney: sure, just add a patch on top.
23:47:22 openstackgerrit Matt Riedemann proposed openstack/nova master: api-ref: fix min_version for parent_provider_uuid in responses https://review.openstack.org/579577
23:57:25 openstackgerrit Matt Riedemann proposed openstack/nova master: api-ref: Add descriptions for rebuild https://review.openstack.org/588931
#openstack-nova - 2018-08-08
00:12:37 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix bad links for admin-guide https://review.openstack.org/580259
00:24:32 openstackgerrit Merged openstack/os-vif master: Update reno for stable/rocky https://review.openstack.org/586106
00:26:02 mriedem i have no idea how this can be happening https://review.openstack.org/#/c/588455/
00:26:10 mriedem if only we had some multi-node pci ci
00:32:24 openstackgerrit Merged openstack/nova stable/queens: Add recreate test for RT.stats bug 1784705 https://review.openstack.org/587921

Earlier   Later