| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-07 | |||
| 21:10:39 | mordred | mriedem: http://paste.openstack.org/show/665249 | |
| 21:11:58 | mriedem | cdent: are you sure? the default is both - did you explicitly configure it? | |
| 21:13:05 | cdent | mriedem: I may be confused. As I understand both is the default only if you have 'notify_on_state_change' set to something and the default for that is to not be set. | |
| 21:13:35 | mriedem | cdent: notify_on_state_change is only for instance updates | |
| 21:13:41 | mriedem | we notify the shit out of you for everything else | |
| 21:13:44 | mriedem | any action taken on an instance | |
| 21:13:53 | mriedem | plus steady state | |
| 21:14:01 | mriedem | "NOTIFY HI I AM STILL WORKING!" | |
| 21:14:11 | cdent | hmmm, the docs are not helping me much here, I will try harder | |
| 21:14:44 | mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#notifications | |
| 21:15:04 | mriedem | set notification_format=unversioned | |
| 21:15:13 | mriedem | i guess we don't have anything for just disabling notifications altogether :( | |
| 21:15:18 | cdent | right | |
| 21:15:21 | cdent | thus my confusion | |
| 21:15:49 | dansmith | mriedem: really? that's.. broken | |
| 21:16:06 | dansmith | maybe rabbit will throw them away if there is no consumer, but I thought not | |
| 21:17:26 | dansmith | anyway, biab | |
| 21:17:28 | openstackgerrit | Christian Berendt proposed openstack/nova stable/ocata: live-mig: keep disk device address same https://review.openstack.org/541904 | |
| 21:17:40 | cdent | so at the moment if I want to at least limit the volume of notifications I should change both to something else? I can at least do that | |
| 21:17:53 | cdent | Being able to turn them off fully seems like it might be a useful thing | |
| 21:18:25 | mriedem | https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html#oslo_messaging_notifications.driver | |
| 21:18:31 | mriedem | you could set the notification driver to 'noop' | |
| 21:18:33 | mriedem | i guess | |
| 21:18:40 | mriedem | however, | |
| 21:18:52 | mriedem | nova is still generating rpc/db load just building the payloads to send them to dev/null | |
| 21:19:07 | mriedem | cdent: for now you'd set it the format to 'unversioned' | |
| 21:19:16 | mriedem | that's as minimal as it gets right now i think | |
| 21:19:33 | cdent | roger that | |
| 21:21:51 | bauzas | is it me or commenting on LP is not working ? | |
| 21:21:58 | bauzas | I got timeouts | |
| 21:23:02 | bauzas | mmmm, Launchpad seems to be in trouble | |
| 21:24:39 | bauzas | mriedem: saw https://bugs.launchpad.net/nova/+bug/1748004 | |
| 21:24:40 | openstack | Launchpad bug 1748004 in OpenStack Compute (nova) "test_evacuate_late_server_group_policy_check randomly fails with KeyError: 'fake-node'" [Undecided,New] | |
| 21:24:40 | bauzas | ouch | |
| 21:25:35 | cdent | bauzas: me too | |
| 21:26:00 | cdent | (on launchpad in trouble) | |
| 21:26:12 | bauzas | what's depressing is that it seems you can still open bugs, but not close them :p | |
| 21:28:28 | cdent | \o/ | |
| 21:28:44 | bauzas | yay it's back | |
| 21:38:31 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Workaround glanceclient bug when CONF.glance.api_servers not set https://review.openstack.org/541008 | |
| 21:40:30 | melwitt | mriedem: sorry I didn't mention this earlier but we have another regression but it's been regressed since ocata https://bugs.launchpad.net/nova/+bug/1729584 | |
| 21:40:32 | openstack | Launchpad bug 1729584 in OpenStack Compute (nova) "boot from volume + configdrive broken (regression)" [High,In progress] - Assigned to Jay Pipes (jaypipes) | |
| 21:41:32 | mriedem | then le meh | |
| 21:41:40 | melwitt | heh. okay | |
| 21:43:27 | ameeda_ | hello | |
| 21:45:11 | ameeda_ | I need +2w here https://review.openstack.org/#/c/528385/ please :$ | |
| 21:49:03 | mriedem | this is a db schema migration we likely want to get in since backporting db schema migrations isn't fun https://review.openstack.org/#/c/530429/ | |
| 21:49:12 | mriedem | it's a follow up to the API change to page instance actions and filter on changes-since | |
| 21:50:28 | ameeda_ | Hello mriedem :) | |
| 21:50:31 | mriedem | also need to get the queens GA release note prelude in https://review.openstack.org/#/c/541495/ | |
| 21:50:44 | mriedem | ameeda_: i don't have time to look at that right now | |
| 21:50:55 | mriedem | ameeda_: you don't need to ping people daily for reviews on your patch | |
| 21:51:00 | mriedem | someone will get to it | |
| 21:51:35 | bauzas | mriedem: thanks for reminding https://review.openstack.org/#/c/541495/1/releasenotes/notes/queens_prelude-4bdf895167f979b2.yaml | |
| 21:51:57 | ameeda_ | mriedem: I didn't ping you for review , I just like to say Hi for you :) | |
| 21:52:07 | mriedem | ameeda_: oh, hi then o/ | |
| 21:52:50 | ameeda_ | mriedem : btw I told you about my case with the team, I am sorry for inconvenience :( | |
| 21:52:54 | ameeda_ | mriedem :) | |
| 21:53:51 | mriedem | ameeda_: feel free to fix https://bugs.launchpad.net/placement-osc-plugin/+bug/1747979 | |
| 21:53:52 | openstack | Launchpad bug 1747979 in placement-osc-plugin "docs link in readme is wrong" [Medium,Confirmed] | |
| 21:53:55 | mriedem | i have to go | |
| 21:54:37 | ameeda_ | mriedem : assigned :) Thanks !! | |
| 21:54:48 | ameeda_ | see you ^_^ | |
| 22:15:12 | cdent | oops, just accidentally tried to boot 15000 instances from 300 different boot commands | |
| 22:18:07 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Transform aggregate.update_metadata notification https://review.openstack.org/460625 | |
| 22:40:06 | dansmith | melwitt: are you around to update this? https://review.openstack.org/#/c/540258/3 | |
| 22:40:19 | melwitt | yes | |
| 22:40:23 | dansmith | if not, I could do it, but it'll be more legit if you fix and I ack :) | |
| 22:40:25 | dansmith | okay cool | |
| 22:42:00 | dansmith | gah mriedem nailed me on my patches too | |
| 22:43:38 | openstackgerrit | Dan Smith proposed openstack/nova master: Make service all-cells min version helper use scatter-gather https://review.openstack.org/539590 | |
| 22:43:39 | openstackgerrit | Dan Smith proposed openstack/nova master: Make host_manager use scatter-gather and ignore down cells https://review.openstack.org/539617 | |
| 22:44:33 | melwitt | okay, so instead in the scheduler we should target each thing to a cell for getting the hosts? I considered that but wasn't sure how to do it if we're accessing things through a field. would we change it to assign the RequestSpec.instance_group.hosts field to the result of a scatter-gather, instead of letting it lazy-load? | |
| 22:45:43 | melwitt | another thought is, the late affinity check is a compute host checking group members against itself and if it can't check other cells ... is that a problem? | |
| 22:49:45 | dansmith | melwitt: you can't do the late affinity check from the comptue, | |
| 22:49:59 | dansmith | that's been one of the caveats of cellsv2 since pike | |
| 22:50:17 | dansmith | (and ocata in devstack since we run multi-level by defalt) | |
| 22:50:47 | melwitt | guh, right. I swear I don't have enough room in my brain for all of this stuff | |
| 22:51:01 | dansmith | melwitt: this isn't for a lazy-load though right? | |
| 22:51:25 | dansmith | oh, we're actually hitting it via lazy load I see from the comment | |
| 22:51:29 | dansmith | er, commit message | |
| 22:51:35 | melwitt | dansmith: I *think* the way this is happening during scheduling is that this line causes a lazy-load and that's how we get the host list for group members https://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L700 | |
| 22:52:00 | dansmith | so yeah, we probably need to not let that happen (i.e. make sure we pull the hosts early, or manually fill the group objects from the list with some cells-aware helper | |
| 22:52:05 | melwitt | so I was thinking we need to replace that with a proper scatter-gather calling get_hosts for all cells except cell0 or something? | |
| 22:52:10 | melwitt | yeah | |
| 22:52:15 | dansmith | melwitt: there should be a debug log about it if there is | |
| 22:52:48 | dansmith | melwitt: yeah, a check before that and a call to a dedicated fill routine would be easiest and cleanest I think | |
| 22:52:56 | melwitt | there actually isn't because InstanceGroup override obj_load_attr and doesn't log a message about it, I think. /re-checks | |
| 22:53:00 | dansmith | alternatively a non-remotable helper, but that really blurs the line more which isn't great | |
| 22:53:05 | dansmith | ah | |
| 22:53:08 | dansmith | well that sucks :) | |
| 22:53:15 | melwitt | yeah, srsly | |
| 22:53:16 | dansmith | we should fix that too | |
| 22:53:35 | melwitt | https://github.com/openstack/nova/blob/master/nova/objects/instance_group.py#L294 | |
| 22:53:36 | melwitt | yeah | |
| 22:54:01 | melwitt | it was fun times chasing that down | |
| 22:54:26 | dansmith | we probably have more of those I'd guess, but yeah | |
| 22:54:29 | dansmith | anyway, | |
| 22:54:48 | dansmith | you know the path now yes? | |
| 22:54:54 | melwitt | yeah | |