| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-04 | |||
| 19:49:25 | dansmith | oh interesting | |
| 19:49:32 | sdague | dansmith: you have the journals all down already? | |
| 19:49:40 | dansmith | sdague: just one journal, right? | |
| 19:49:41 | sdague | search for -k systemd | |
| 19:49:46 | dansmith | devstack.journal.gz? | |
| 19:50:02 | sdague | well, there are usual multiples of them because they roll over | |
| 19:50:17 | sdague | but, yeh, only looks like one in there | |
| 19:50:32 | dansmith | -k is dmesg, which you can't do on a file | |
| 19:50:45 | dansmith | unless you mean something else | |
| 19:50:46 | dansmith | -u? | |
| 19:51:23 | dansmith | journalctl | grep systemd yields not much of relevance | |
| 19:51:39 | sdague | -t | |
| 19:51:40 | sdague | sorry | |
| 19:51:53 | dansmith | same as the grep | |
| 19:52:05 | edleafe | bauzas: should be ready to upload soon. I've already created a unit test. | |
| 19:52:11 | sdague | ok, so default seems to be 1000 messages per 30s | |
| 19:52:17 | sdague | I am sure we are exceeding that | |
| 19:52:23 | dansmith | surely at startup | |
| 19:54:05 | sdague | let me put in a fix for that | |
| 19:54:13 | dansmith | cool thanks | |
| 19:54:26 | dansmith | link me and I'll depend on it and recheck | |
| 19:56:18 | melwitt | dansmith: so the websocketproxy goes direct to the instance? it doesn't need to know the compute host MQ right | |
| 19:56:44 | dansmith | it goes direct to the host | |
| 19:56:59 | dansmith | consoleauth validate_token does talk to the compute mq, but it does the looku | |
| 19:57:17 | melwitt | okay. we'll just have to see the logs then, nothing is jumping out at me | |
| 19:57:20 | dansmith | I think the token in memcache must have the ip/port that the proxy needs to connect to | |
| 19:57:21 | dansmith | yeah | |
| 19:57:28 | dansmith | hopefully something dumb | |
| 19:57:32 | dansmith | simple and dumb | |
| 19:57:34 | dansmith | like me. | |
| 19:57:44 | sdague | dansmith: I think this is it - https://review.openstack.org/462725 | |
| 19:57:46 | melwitt | lol :P | |
| 19:58:42 | openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack https://review.openstack.org/458537 | |
| 19:58:44 | dansmith | sdague: ack thanks | |
| 19:58:56 | dansmith | sdague: if that's really happening we probably want to merge that like.. yesterday :P | |
| 19:59:03 | mriedem | gibi: i rebased and commented in https://review.openstack.org/#/c/419730/ for adding keypairs to InstancePayload - summary is i don't think we should lazy load instance.keypairs for every instance action, for the same reason I didn't want to do that for instance.tags | |
| 20:00:26 | sdague | dansmith: yep agreed, I want to make sure it looks sane | |
| 20:00:37 | sdague | hey, it's all about learning new things :) | |
| 20:02:13 | dansmith | sdague: I'm not complaining :) | |
| 20:03:11 | sdague | also, what's up with the linuxbridge job? | |
| 20:03:17 | sdague | that seem things to be hating me today | |
| 20:04:24 | dansmith | sdague: that was an impressively incomprehensible exclamation | |
| 20:04:45 | openstackgerrit | Ed Leafe proposed openstack/nova master: Don't delete allocation if instance being scheduled https://review.openstack.org/460177 | |
| 20:04:55 | edleafe | bauzas: Here ya go ^^ | |
| 20:05:33 | dansmith | what is this nonsense? http://logs.openstack.org/79/462679/1/check/gate-nova-tox-functional-ubuntu-xenial/c760042/testr_results.html.gz | |
| 20:05:37 | bauzas | edleafe: okay, I'll update the next top changes | |
| 20:07:06 | melwitt | dansmith: socket_options, duh | |
| 20:07:26 | melwitt | :D | |
| 20:10:49 | cdent | dansmith: might be mismatched versions of urllib3 and wsgi-intercept | |
| 20:11:03 | dansmith | cdent: it's broken is what it is | |
| 20:11:03 | cdent | I saw that a while back but thought it was better. | |
| 20:11:20 | dansmith | looks rather fundamental, so wondering if it's worth a recheck | |
| 20:11:25 | cdent | yessir, but I think it got better. are your deps maybe stale? | |
| 20:11:35 | dansmith | that patch is new from today | |
| 20:11:39 | cdent | hmmm | |
| 20:12:00 | dansmith | I guess edleafe already rechecked it | |
| 20:12:15 | cdent | i'll mess with it a bit | |
| 20:12:19 | cdent | (locally) | |
| 20:12:24 | mriedem | can't provide new key_name on rebuild? | |
| 20:17:28 | melwitt | mriedem: I don't think you can specify anything other than the image. there have been requests to allow new userdata and such, but the answer has been "just boot a new instance" if you're looking at re-imaging it anyway | |
| 20:18:04 | mriedem | do we rebuild the config drive during rebulid? | |
| 20:18:32 | melwitt | I don't think so | |
| 20:18:50 | melwitt | but not sure, not a config drive expert | |
| 20:19:24 | mriedem | i'm looking, just this keypair notification patch has me thinking about weird stuf | |
| 20:19:29 | mriedem | https://review.openstack.org/#/c/419730/ | |
| 20:19:40 | melwitt | we definitely don't change anything on it. I don't know if it is "recreated" at all though | |
| 20:19:40 | mriedem | b/c i don't want to lazy-load instance.keypairs for every gd instance action notification | |
| 20:20:00 | mriedem | jaypipes: dansmith: ^ same predicament as with the instance.tags patch | |
| 20:20:08 | mriedem | for notifications that is | |
| 20:21:28 | mriedem | which would include every vm/task state change if you enabled that option... | |
| 20:21:50 | mriedem | CONF.notifications.notify_on_state_change i mean | |
| 20:22:23 | melwitt | preload all the things | |
| 20:22:29 | melwitt | jk | |
| 20:22:42 | mriedem | well someone proposed that for instance.tags, | |
| 20:22:44 | mriedem | preload by default | |
| 20:22:47 | mriedem | we nacked that | |
| 20:22:47 | jroll | just keep the whole db in RAM, wcpgw | |
| 20:23:10 | melwitt | see, jroll knows what I'm talkin about | |
| 20:23:13 | openstackgerrit | Alex Szarka proposed openstack/nova master: Transform instance.exists notification https://review.openstack.org/403660 | |
| 20:23:14 | mriedem | i worry that some of the perf issues people are reporting in newton/ocata are due to all of the versioned notifications doing lazy-loads like crazy | |
| 20:24:02 | jroll | melwitt: :D | |
| 20:24:07 | jroll | feels like state change notifications shouldn't need tags, keypairs, etc | |
| 20:25:15 | mriedem | that's what i've said | |
| 20:25:19 | mriedem | and pushed back on | |
| 20:25:22 | mriedem | nor bdms... | |
| 20:25:35 | mriedem | we're basically throwing the kitchen sink at the notification consumer | |
| 20:26:00 | mriedem | because (1) we can and (2) we don't know what notification consumers even want/need | |
| 20:26:06 | jaypipes | mriedem: currently reviewing dansmith's target_cell patches. will get to those a little later (all the notification ones) | |
| 20:26:22 | mriedem | jaypipes: you don't really need to probably, just thinking out loud | |
| 20:26:25 | mriedem | i've already -1ed | |
| 20:26:39 | jroll | mriedem: and I just realized (3) notifications are considered lossy, so we can't assume consumers have state n-1 to diff properly | |
| 20:26:49 | melwitt | :\ a lot of people use notifications, I thought. so one would think we would have some idea what notification consumers want | |
| 20:26:55 | jroll | is the most sensible reason to me | |
| 20:27:20 | mriedem | melwitt: rebuild calls virt driver destroy and then spawn, so i'd think we're rebuilding the config drive... | |
| 20:27:33 | mriedem | but i guess you just get the same key as before | |
| 20:28:17 | melwitt | mriedem: yeah, I don't know what the term "rebuild" means in the context of config drive. I know we don't change it | |
| 20:28:25 | mriedem | melwitt: the only consumers i know of really are ceilometer, mistral and searchlight | |
| 20:28:47 | mriedem | not sure how much ceilometer is used, and since they didn't get the stuff they needed in the notifications years ago, they just hit the rest api | |
| 20:28:56 | mriedem | not sure about mistral or searchlight | |
| 20:29:44 | melwitt | notifications were supposed to be the answer to instance actions records. I have used those before in a previous life. and notifications, but just to know the history of what happened to an instance when the customer comes calling | |