Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-02-07
21:24:39 bauzas mriedem: saw https://bugs.launchpad.net/nova/+bug/1748004
21:24:40 openstack Launchpad bug 1748004 in OpenStack Compute (nova) "test_evacuate_late_server_group_policy_check randomly fails with KeyError: 'fake-node'" [Undecided,New]
21:24:40 bauzas ouch
21:25:35 cdent bauzas: me too
21:26:00 cdent (on launchpad in trouble)
21:26:12 bauzas what's depressing is that it seems you can still open bugs, but not close them :p
21:28:28 cdent \o/
21:28:44 bauzas yay it's back
21:38:31 openstackgerrit Matt Riedemann proposed openstack/nova master: Workaround glanceclient bug when CONF.glance.api_servers not set https://review.openstack.org/541008
21:40:30 melwitt mriedem: sorry I didn't mention this earlier but we have another regression but it's been regressed since ocata https://bugs.launchpad.net/nova/+bug/1729584
21:40:32 openstack Launchpad bug 1729584 in OpenStack Compute (nova) "boot from volume + configdrive broken (regression)" [High,In progress] - Assigned to Jay Pipes (jaypipes)
21:41:32 mriedem then le meh
21:41:40 melwitt heh. okay
21:43:27 ameeda_ hello
21:45:11 ameeda_ I need +2w here https://review.openstack.org/#/c/528385/ please :$
21:49:03 mriedem this is a db schema migration we likely want to get in since backporting db schema migrations isn't fun https://review.openstack.org/#/c/530429/
21:49:12 mriedem it's a follow up to the API change to page instance actions and filter on changes-since
21:50:28 ameeda_ Hello mriedem :)
21:50:31 mriedem also need to get the queens GA release note prelude in https://review.openstack.org/#/c/541495/
21:50:44 mriedem ameeda_: i don't have time to look at that right now
21:50:55 mriedem ameeda_: you don't need to ping people daily for reviews on your patch
21:51:00 mriedem someone will get to it
21:51:35 bauzas mriedem: thanks for reminding https://review.openstack.org/#/c/541495/1/releasenotes/notes/queens_prelude-4bdf895167f979b2.yaml
21:51:57 ameeda_ mriedem: I didn't ping you for review , I just like to say Hi for you :)
21:52:07 mriedem ameeda_: oh, hi then o/
21:52:50 ameeda_ mriedem : btw I told you about my case with the team, I am sorry for inconvenience :(
21:52:54 ameeda_ mriedem :)
21:53:51 mriedem ameeda_: feel free to fix https://bugs.launchpad.net/placement-osc-plugin/+bug/1747979
21:53:52 openstack Launchpad bug 1747979 in placement-osc-plugin "docs link in readme is wrong" [Medium,Confirmed]
21:53:55 mriedem i have to go
21:54:37 ameeda_ mriedem : assigned :) Thanks !!
21:54:48 ameeda_ see you ^_^
22:15:12 cdent oops, just accidentally tried to boot 15000 instances from 300 different boot commands
22:18:07 openstackgerrit Takashi NATSUME proposed openstack/nova master: Transform aggregate.update_metadata notification https://review.openstack.org/460625
22:40:06 dansmith melwitt: are you around to update this? https://review.openstack.org/#/c/540258/3
22:40:19 melwitt yes
22:40:23 dansmith if not, I could do it, but it'll be more legit if you fix and I ack :)
22:40:25 dansmith okay cool
22:42:00 dansmith gah mriedem nailed me on my patches too
22:43:38 openstackgerrit Dan Smith proposed openstack/nova master: Make service all-cells min version helper use scatter-gather https://review.openstack.org/539590
22:43:39 openstackgerrit Dan Smith proposed openstack/nova master: Make host_manager use scatter-gather and ignore down cells https://review.openstack.org/539617
22:44:33 melwitt okay, so instead in the scheduler we should target each thing to a cell for getting the hosts? I considered that but wasn't sure how to do it if we're accessing things through a field. would we change it to assign the RequestSpec.instance_group.hosts field to the result of a scatter-gather, instead of letting it lazy-load?
22:45:43 melwitt another thought is, the late affinity check is a compute host checking group members against itself and if it can't check other cells ... is that a problem?
22:49:45 dansmith melwitt: you can't do the late affinity check from the comptue,
22:49:59 dansmith that's been one of the caveats of cellsv2 since pike
22:50:17 dansmith (and ocata in devstack since we run multi-level by defalt)
22:50:47 melwitt guh, right. I swear I don't have enough room in my brain for all of this stuff
22:51:01 dansmith melwitt: this isn't for a lazy-load though right?
22:51:25 dansmith oh, we're actually hitting it via lazy load I see from the comment
22:51:29 dansmith er, commit message
22:51:35 melwitt dansmith: I *think* the way this is happening during scheduling is that this line causes a lazy-load and that's how we get the host list for group members https://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L700
22:52:00 dansmith so yeah, we probably need to not let that happen (i.e. make sure we pull the hosts early, or manually fill the group objects from the list with some cells-aware helper
22:52:05 melwitt so I was thinking we need to replace that with a proper scatter-gather calling get_hosts for all cells except cell0 or something?
22:52:10 melwitt yeah
22:52:15 dansmith melwitt: there should be a debug log about it if there is
22:52:48 dansmith melwitt: yeah, a check before that and a call to a dedicated fill routine would be easiest and cleanest I think
22:52:56 melwitt there actually isn't because InstanceGroup override obj_load_attr and doesn't log a message about it, I think. /re-checks
22:53:00 dansmith alternatively a non-remotable helper, but that really blurs the line more which isn't great
22:53:05 dansmith ah
22:53:08 dansmith well that sucks :)
22:53:15 melwitt yeah, srsly
22:53:16 dansmith we should fix that too
22:53:35 melwitt https://github.com/openstack/nova/blob/master/nova/objects/instance_group.py#L294
22:53:36 melwitt yeah
22:54:01 melwitt it was fun times chasing that down
22:54:26 dansmith we probably have more of those I'd guess, but yeah
22:54:29 dansmith anyway,
22:54:48 dansmith you know the path now yes?
22:54:54 melwitt yeah
22:55:14 dansmith suh-weet
23:02:11 cfriesen when instances query the metadata server, what code adds the webob.headers.EnvironHeaders to the request? (I'm trying to figure out how the mapping to instance_id is done)
23:04:49 openstackgerrit melanie witt proposed openstack/nova master: Refine waiting for vif plug events during _hard_reboot https://review.openstack.org/541442
#openstack-nova - 2018-02-08
00:24:12 jroll melwitt: congrats and condolences
00:24:14 jroll :)
00:24:30 melwitt heh, thanks jroll
00:27:51 mriedem gibi: i likely can't run the nova meeting tomorrow, can you do that? i'll update the agenda tonight.
00:30:26 mnaser melwitt: early unofficial congrats too :)
00:30:40 melwitt thanks mnaser :)
00:43:23 mriedem efried: apparently we lookup the image service endpoint url from the catalog 410 times in n-api alone in a successful tempest run
00:43:28 mriedem which seems....excessive
00:48:33 openstackgerrit Matt Riedemann proposed openstack/nova master: Workaround glanceclient bug when CONF.glance.api_servers not set https://review.openstack.org/541008
00:49:49 openstackgerrit Takashi NATSUME proposed openstack/nova master: [cellv2] Improve getting BDMs in multiple cells https://review.openstack.org/521400
01:15:59 efried mriedem: Accessing an image 410 times in a tempest run doesn't seem excessive to me. And if things are set up correctly, that lookup ought to be really really cheap.
01:16:28 mriedem we just reconstruct the glance client wrapper every time
01:16:36 efried Now if we were doing endpoint discovery every one of those times (or even a tenth of them) that would be excessive.
01:16:37 mriedem i'm pretty sure we do the same for cinder client though too
01:16:45 efried I don't doubt it.
01:17:05 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove unnecessary variables https://review.openstack.org/541964
01:17:07 efried Are you suggesting we could cache the client? Maybe so.
01:17:29 efried FWIW, constructing the client is also pretty cheap.
01:17:45 mriedem i was more worried about the endpoint lookup from ksa every time,
01:17:50 mriedem but is that cached somewhere?
01:18:45 openstackgerrit Matt Riedemann proposed openstack/nova master: Workaround glanceclient bug when CONF.glance.api_servers not set https://review.openstack.org/541008
01:18:52 mriedem btw, this fixes the nova-next job ^, despite using some regex parsing and stripping
01:18:54 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove unnecessary arguments in notification methods https://review.openstack.org/535599
01:20:46 efried mriedem: Yes, the endpoint is cached, if things are working right.
01:21:30 efried oh, you did that, thanks.
01:27:06 openstackgerrit melanie witt proposed openstack/nova master: Make scheduler.utils.setup_instance_group query all cells https://review.openstack.org/540258
01:29:32 openstackgerrit Merged openstack/nova master: Update links in documents https://review.openstack.org/533134
02:04:33 mriedem mordred: would you like it if the GET /flavors/detail response included the extra specs with each flavor also so you don't have to make a 2nd request per flavor to get the specs (as a subresource GET) ?

Earlier   Later