| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-20 | |||
| 15:38:47 | efried | Thanks for the talk dansmith sean-k-mooney | |
| 15:39:01 | sean-k-mooney | the other usecase for shelve is public cloud to avoid billing but ya unell you have a server affinity group or somethin its very unlikly to land on the same host | |
| 15:39:08 | dansmith | efried: already been asking some questions in -glance | |
| 15:39:18 | sean-k-mooney | no worres i need to review your spec and code too | |
| 15:39:25 | sean-k-mooney | ill try to do that this week | |
| 15:40:06 | efried | cool | |
| 15:46:01 | bauzas | mriedem: want me to care on backporting https://review.opendev.org/#/c/695145/ to stable branches ? | |
| 15:46:13 | bauzas | or you rather and me reviewing ? | |
| 15:46:29 | mriedem | can you do both? | |
| 15:46:51 | bauzas | both backporting and reviewing ? well, is that a thing ? | |
| 15:47:03 | bauzas | we have elod and lyarwood we can bug | |
| 15:47:51 | mriedem | i have things in stable/train you could review | |
| 15:48:12 | bauzas | mriedem: sure thing | |
| 15:48:21 | mriedem | i guess i don't know if you're just talking about that specific change or reviewing code in general | |
| 15:48:29 | mriedem | because both would be good | |
| 15:48:38 | bauzas | mriedem: no, sorry, was talking of proposing backports of this change | |
| 15:49:01 | bauzas | mriedem: cool, I'm committed to reviewing for the last 2 days | |
| 15:49:28 | bauzas | anyway, let's just backport this stuff... | |
| 15:57:09 | dansmith | jroll: you around? | |
| 15:58:54 | efried | I suspect in this case it would be possible to use the keymgr backend to store this blob. But that's really not a good general model. | |
| 15:59:17 | efried | this is really a job for swift. And then the responsibility of the admin to make sure their swift is appropriately hardened. | |
| 16:00:18 | efried | do we have a precedent for nova talking to swift?? | |
| 16:00:24 | efried | I... don't see one. | |
| 16:00:40 | dansmith | so, if you have no swift, we what.. lose your tpm on snapshot/shelve or refuse to do it? | |
| 16:00:50 | efried | yeah, I would think so. | |
| 16:00:55 | dansmith | which? | |
| 16:01:04 | efried | refuse | |
| 16:01:22 | dansmith | how about instead of that, | |
| 16:01:32 | efried | I mean, there seems to have been plenty of talk on recent features about "not supported" (you can try it but it won't work right) versus actually blocked | |
| 16:01:36 | dansmith | we refuse to ever boot an instance with a tpm if there's no swift endpoint in the catalog? | |
| 16:01:46 | dansmith | efried: yeah and they all suck | |
| 16:02:02 | efried | dansmith: I can live with that; same as the requirement for the keymgr to exist. | |
| 16:02:04 | dansmith | efried: because the user doesn't know any of this, other than that "every openstack cloud they use seems to be different and incompatible" | |
| 16:02:20 | fgonzales3246 | hello all, I'm configuring SSL connections into my MySQL database for the components and when I activate this into the dabase and in each connection my controller starts to slow down and the requests fail due to timeout. | |
| 16:02:23 | dansmith | efried: ack, I'd much rather do that, or at least do that in addition to the other | |
| 16:02:52 | efried | dansmith: IMO it's better to block operations you know are broken than to simply state they're not supported and let them sh*t all over themselves if tried. | |
| 16:02:59 | fgonzales3246 | I tried to isolate one component like, cinder, but it occurs the same... seems that the requests doesn't go to the compute node | |
| 16:03:06 | efried | but I think the counterargument has been that we want to be able to enable the operation without needing a new microversion. | |
| 16:03:32 | fgonzales3246 | but the services are all up and running, do you know what could I be missing? Something related to nova-conductor or something else? | |
| 16:03:38 | dansmith | efried: of course, I'm saying having "the plan" be a thousand places where seemingly fundamental operations can fail with the appropriate combination of obscure backend configs sucks from the user's perspective | |
| 16:03:40 | efried | anyway, in this case we're not talking API changes, and we're talking blocking at boot rather than (well, in addition to) shelve, so I'm good. | |
| 16:03:51 | efried | dansmith: totally agree. | |
| 16:03:56 | efried | block it if you know it's broken. | |
| 16:04:09 | dansmith | ...but try not to rely on that in design | |
| 16:04:27 | efried | now I just hope I can detect a swift | |
| 16:04:46 | efried | oh, | |
| 16:05:03 | efried | cause I wanted to add a check for "key manager configured" to the "do we enable vTPM in the first place" logic | |
| 16:05:11 | efried | but couldn't find a reliable way to inspect CONF for that | |
| 16:05:20 | dansmith | you need the catalog, no? | |
| 16:05:24 | efried | but yeah, | |
| 16:05:36 | efried | I can at least look in the catalog to make sure there's a keymgr endpoint. | |
| 16:05:40 | efried | um | |
| 16:05:59 | efried | except I'm doing this on init_host and don't have a context? | |
| 16:06:05 | dansmith | no, | |
| 16:06:10 | dansmith | you do this at boot time for the instance | |
| 16:06:17 | dansmith | non-tpm instances don't care about this | |
| 16:06:37 | dansmith | what you want to avoid is agreeing to create an instance for the user which in six weeks will be unsnapshottable | |
| 16:06:44 | efried | Sorry, I'm talking about something slightly different: the part where the libvirt driver decides whether to set the "can I vTPM?" flag. | |
| 16:06:53 | dansmith | I'm not talking about that | |
| 16:06:55 | efried | ...causing it to expose the traits | |
| 16:06:57 | efried | I know, I am. | |
| 16:07:02 | efried | I understand and agree with what you're talking about. | |
| 16:07:16 | dansmith | that too may be good, but expecting keystone is up and reachable when compute starts is not okay I think | |
| 16:07:33 | efried | orly? ... okay. | |
| 16:07:34 | dansmith | computes on the other side of a network partition shouldn't just fail to start I think | |
| 16:07:42 | efried | fair enough | |
| 16:07:54 | dansmith | they'll currently wait for conductor if it's not up yet, so... | |
| 16:08:25 | efried | yeah, I don't want to be the guy to introduce a network ping into init_host. | |
| 16:08:45 | dansmith | we have that :) | |
| 16:08:49 | dansmith | I added it | |
| 16:09:01 | efried | from compute driver or from compute manager? | |
| 16:09:04 | dansmith | ...but it's just a ping() rpc method compute makes to conductor to see if it's alive before it goes :) | |
| 16:09:08 | dansmith | manager | |
| 16:09:22 | efried | yeah, that's in manager. I'm talking about libvirt compute driver's init_host, which seems... dirtier. | |
| 16:09:37 | dansmith | yes, I'm just pointing out that I called it ping :) | |
| 16:10:42 | efried | okay, so lacking a jroll for live discussion, I'll mod the spec as if this is the plan, and ping for reviews. | |
| 16:10:45 | efried | Thanks dansmith | |
| 16:11:14 | dansmith | sounds good | |
| 16:13:27 | mnaser | sean-k-mooney: so my new map is 3C per NUMA node (6 threads total), ~32GB per NUMA node, with quite interesting latency numbers | |
| 16:13:37 | mnaser | https://www.irccloud.com/pastebin/91D5sGuS/ | |
| 16:13:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: zuul: Remove unnecessary 'tox_install_siblings' https://review.opendev.org/695235 | |
| 16:16:19 | efried | mriedem: the barbican-gating-on-py3 thing from yesterday seems to be busted https://review.opendev.org/#/c/695052/ -- would you mind having a quick look at some point? | |
| 16:17:51 | mnaser | w | |
| 16:22:13 | stephenfin | efried: didn't AJaeger have a fix for that? | |
| 16:22:26 | sean-k-mooney | mnaser: that kindof look like what i would expect | |
| 16:22:30 | stephenfin | efried: https://review.opendev.org/#/c/689461/ | |
| 16:22:32 | efried | stephenfin: yes, gmann said something in -barbican | |
| 16:22:40 | sean-k-mooney | as the numa distance increase teh latnecy does too | |
| 16:22:50 | stephenfin | ah, maybe not the same thing | |
| 16:22:52 | stephenfin | nvm me | |
| 16:23:04 | mnaser | sean-k-mooney: i also bumped the "4-link xGMI max speed" setting to 18Gbps .. but the machine is now crashing (it was set to 10.667 earlier) | |
| 16:23:09 | efried | stephenfin: yeah, he pointed to https://review.opendev.org/#/c/689458/ (same thing but in train) | |
| 16:23:14 | mnaser | so ill reset it back to that, and hopefully the latency stays at these numbers | |
| 16:23:34 | gmann | stephenfin: yeah, stable/train backport will fix the baribican. but still asking the reason to infra on ML for learning. | |
| 16:23:51 | stephenfin | sweet | |
| 16:24:10 | sean-k-mooney | but it lends more evidence to the idea we should look at cache affinity and numa distance when selecting vm numa node mappings | |
| 16:24:39 | mnaser | sean-k-mooney: yep, giving an instance two NUMA nodes that are '32' away vs '10' .. | |
| 16:24:54 | sean-k-mooney | mnaser: can you dump the numa distance info | |
| 16:24:56 | mnaser | or rather 11/12 | |
| 16:25:15 | sean-k-mooney | i would be gould to confimr the latency corralate with the numa distance reported | |