| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-31 | |||
| 19:59:04 | efried | I'll have another look at the documentation, but I thought it was ultimately explained the right way for users. | |
| 19:59:17 | efried | The logic is necessarily hairy and compound-negative-confusing | |
| 19:59:25 | efried | but the ux winds up being really crisp and clean imo | |
| 20:02:58 | efried | ...you're right, the doc https://review.opendev.org/#/c/667952/10/doc/source/reference/forbidden-aggregates.rst still talks about forbidden aggregates. Though really, the explanation is very good, and "forbidden aggregates" is really just used as the name-of-feature. | |
| 20:16:30 | mriedem | yeah i'm not -1 on it, it's just a bit confusing | |
| 20:16:33 | mriedem | some of the wording | |
| 20:18:15 | efried | I think you've got a valid point that we could s/forbidden/isolated/ and it would be more comprehensible. The docs could be fixed later, but if we want to rename the conf opt we should do it now. | |
| 20:21:25 | prometheanfire | sean-k-mooney: ya, looks like it's lxml https://review.opendev.org/673848 | |
| 20:23:06 | mriedem | efried: question in https://review.opendev.org/#/c/670182/1 | |
| 20:24:23 | efried | mriedem: responded | |
| 20:29:24 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix python3 compatibility of rbd get_fsid https://review.opendev.org/664513 | |
| 20:31:07 | mriedem | amorin: are you still working on https://review.opendev.org/#/c/667294/ ? | |
| 20:45:14 | openstackgerrit | Merged openstack/nova master: neutron: log something more useful in _get_instance_nw_info https://review.opendev.org/673694 | |
| 21:43:21 | efried | mriedem: RequestContext question for ya | |
| 21:43:56 | efried | I've been tracking where nova-compute talks to Placement to do things like creating the compute node RP. | |
| 21:44:42 | efried | IIUC those interactions use a RequestContext that gets created when the service starts up, because they're done in a periodic, not based on a request coming from e.g. conductor or whatever. | |
| 21:45:00 | efried | which would be fine, except... | |
| 21:45:13 | efried | it doesn't seem to be the same context that's being used for logging. | |
| 21:45:25 | mriedem | b/c the request id changes etc | |
| 21:45:34 | mriedem | which is because of a thread local store for the context way up in oslo.context | |
| 21:45:36 | mriedem | and it's mega confusing | |
| 21:45:53 | efried | rightright, but it's a *different* context? | |
| 21:46:02 | mriedem | as mentioned https://bugs.launchpad.net/nova/+bug/1835958/comments/7 | |
| 21:46:02 | openstack | Launchpad bug 1835958 in OpenStack Compute (nova) "Nova sync power state on large clusters causes poor performance" [Undecided,New] | |
| 21:46:21 | efried | viz: http://logs.openstack.org/86/672986/2/check/nova-live-migration/0171fa9/logs/screen-n-cpu.txt.gz#_Jul_30_22_09_35_334853 | |
| 21:46:22 | efried | here ^ you can see that the log line is showing one request_id, but the context we're using for the placement request has a different request_id. | |
| 21:47:58 | mriedem | req-428963d8-bc71-4823-8e35-97db13886690 is just some admin context created when the compute service starts up right? | |
| 21:48:20 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1321 | |
| 21:48:33 | mriedem | so you're using a random context there with a random request_id | |
| 21:48:50 | efried | req-42... is the placement-side local request_id, just ignore that one. | |
| 21:48:51 | mriedem | because https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L273 | |
| 21:49:20 | mriedem | i was looking at that one because | |
| 21:49:20 | mriedem | Jul 30 22:09:35.334349 ubuntu-bionic-rax-ord-0009499845 nova-compute[28986]: INFO nova.scheduler.client.report [None req-fb42d5e6-1c0d-41d0-88c5-a4f1ac127a41 None None] [req-428963d8-bc71-4823-8e35-97db13886690] Created resource provider record via placement API for resource provider with UUID 5dcb9522-0513-440a-80db-d985b5d45704 and name ubuntu-bionic-rax-ord-0009499845. | |
| 21:49:35 | efried | req-fb42d5e6-1c0d-41d0-88c5-a4f1ac127a41 is the oslo.context one that oslo.log is paying attention to | |
| 21:49:35 | efried | req-8a7e2cc5-3870-4118-b55f-4533d7a4de80 is the one in the RequestContext being used for the placement communication. | |
| 21:50:07 | efried | yeah, I know, I had to track that one down as well. I started this whole thing because I was wondering why tf we needed to print that reqid at all. | |
| 21:50:37 | efried | it's the one being sent *back* to us in the placement response headers, which turns out to be the (not-global!) request_id on the placement side | |
| 21:50:51 | efried | and because of this whole snafu, it's currently the only way for us to correlate the requests on the nova & placement sides | |
| 21:51:04 | efried | but that flies in the face of the entire global_request_id concept | |
| 21:51:38 | efried | if we get global_request_id working, we don't need that at all because we'll be able to correlate with global_request_id, which is the whole friggin point. | |
| 21:51:48 | efried | so my question is this | |
| 21:52:21 | mriedem | idk what oslo.log is doing really but i know things get wonky with this https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L300 | |
| 21:52:38 | efried | what about if, when we create the RequestContext on compute startup, we a) set its global_request_id, and b) pass overwrite=True so oslo.context replaces it in the threadlocal | |
| 21:53:15 | efried | yeah, b) would call update_store and replace that. | |
| 21:54:08 | efried | I think having talked through this I've got enough to do that ^ in a WIP and see what shakes out. | |
| 21:55:55 | mriedem | ....ok | |
| 21:56:12 | openstackgerrit | Merged openstack/nova master: Move adding vlans to interfaces to privsep. https://review.opendev.org/635436 | |
| 21:56:30 | mriedem | i know melwitt https://review.opendev.org/#/c/524306/ and gibi https://review.opendev.org/#/c/542891/ have muddled in this and it's a mess | |
| 21:56:36 | mriedem | so i guess welcome to the tarpits | |
| 21:57:37 | efried | mriedem: I see that get_context() and get_admin_context() explicitly set overwrite=False. The former even says it's to not mess with logging. Why?? | |
| 21:57:56 | mriedem | oh eric, i could tell you but that would be too easy | |
| 21:58:47 | mriedem | if you can hunt down vishy from 2012 maybe he can answer https://review.opendev.org/#/c/2923/ | |
| 21:59:51 | mriedem | or i guess alaski from 2016 https://review.opendev.org/#/c/378941/ | |
| 22:01:01 | mriedem | the admin context stuff is also a bit legacy from when db apis were explicitly blocked unless you had an admin context, which alex_xu removed a long time ago | |
| 22:01:09 | mriedem | but still affects project_id filtering in the model_query | |
| 22:01:15 | mriedem | as melwitt can probably go into detail about | |
| 22:01:25 | mriedem | as for the logging and thread local stuff, ....idk | |
| 22:05:51 | openstackgerrit | Eric Fried proposed openstack/nova master: Store a RequestContext on service to reuse for periodic tasks https://review.opendev.org/524306 | |
| 22:05:52 | openstackgerrit | Eric Fried proposed openstack/nova master: Poison context usage in periodic tasks https://review.opendev.org/542891 | |
| 22:07:35 | efried | melwitt, gibi_off: FYI, rebased & resolved merge conflicts ^. I'm going to throw one on top that overwrites the threadlocal context and creates a global_request_id so our logging might be sane. We'll see what explodes. | |
| 22:07:39 | efried | mriedem: fyi ^ | |
| 22:10:53 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Generate and log global_request_id properly https://review.opendev.org/673924 | |
| 22:13:57 | openstackgerrit | Eric Fried proposed openstack/nova master: Move iptables rule fetching and setting to privsep. https://review.opendev.org/636508 | |
| 22:13:58 | openstackgerrit | Eric Fried proposed openstack/nova master: Move dnsmasq restarts to privsep. https://review.opendev.org/639280 | |
| 22:13:58 | openstackgerrit | Eric Fried proposed openstack/nova master: Move router advertisement daemon restarts to privsep. https://review.opendev.org/639281 | |
| 22:13:59 | openstackgerrit | Eric Fried proposed openstack/nova master: Move calls to ovs-vsctl to privsep. https://review.opendev.org/639282 | |
| 22:13:59 | openstackgerrit | Eric Fried proposed openstack/nova master: Move setting of device trust to privsep. https://review.opendev.org/639283 | |
| 22:14:00 | openstackgerrit | Eric Fried proposed openstack/nova master: Move final bridge commands to privsep. https://review.opendev.org/639580 | |
| 22:14:00 | openstackgerrit | Eric Fried proposed openstack/nova master: Cleanup the _execute shim in nova/network. https://review.opendev.org/639581 | |
| 22:14:01 | openstackgerrit | Eric Fried proposed openstack/nova master: We no longer need rootwrap. https://review.opendev.org/554438 | |
| 22:14:01 | openstackgerrit | Eric Fried proposed openstack/nova master: Cleanup no longer required filters and add a release note. https://review.opendev.org/639826 | |
| 22:24:29 | openstackgerrit | Merged openstack/nova master: Remove Request Spec Migration upgrade status check https://review.opendev.org/673580 | |
| 22:31:23 | efried | o/ | |
| #openstack-nova - 2019-08-02 | |||
| 02:53:25 | openstackgerrit | Merged openstack/nova master: Follow-up for I2936ce8cb293dc80e1a426094fdae6e675461470 https://review.opendev.org/672669 | |
| 02:53:35 | openstackgerrit | Merged openstack/nova master: nit: fix the test case of migration obj_make_compatible https://review.opendev.org/673961 | |
| 02:58:37 | openstackgerrit | pengyuesheng proposed openstack/os-resource-classes master: Sync Sphinx requirement https://review.opendev.org/674164 | |
| 03:00:25 | openstackgerrit | Eric Fried proposed openstack/nova master: Correct global_request_id sent to Placement https://review.opendev.org/674129 | |
| 05:34:22 | openstackgerrit | Merged openstack/nova stable/rocky: Fix python3 compatibility of rbd get_fsid https://review.opendev.org/664513 | |
| 07:51:46 | aspiers | o] | |
| 07:52:08 | aspiers | wow, that was an unintentional dance move, let's try again | |
| 07:52:10 | aspiers | o/ | |
| 07:52:34 | kashyap | Hehe | |
| 07:52:59 | kashyap | Morning, /me is especially feeling high-spirited today | |
| 07:53:14 | kashyap | Maybe it's the accidental electric shock I got last night while plugging in a bad cable :D | |
| 07:55:26 | cdent | one more shock and you will become the superhero you were born to be | |
| 07:55:58 | kashyap | Hehe. That really shook me | |
| 08:42:13 | aspiers | X-D | |
| 09:13:14 | openstackgerrit | Huachang Wang proposed openstack/nova master: doc: correct the information of 'cpu_map' https://review.opendev.org/673272 | |
| 09:34:01 | openstackgerrit | Huachang Wang proposed openstack/nova master: Add the support of CPU feature 'AVX512-VNNI' https://review.opendev.org/673480 | |
| 09:39:37 | openstackgerrit | Brin Zhang proposed openstack/nova master: WIP: Add user_id and project_id colume to Migration https://review.opendev.org/673990 | |
| 09:39:37 | openstackgerrit | Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243 | |
| 11:58:39 | openstackgerrit | Merged openstack/os-vif master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/672857 | |
| 12:29:35 | openstackgerrit | Huachang Wang proposed openstack/nova master: doc: correct the information of 'cpu_map' https://review.opendev.org/673272 | |
| 13:05:00 | openstackgerrit | Merged openstack/nova master: (Re-)enable vnc console tests in nova-multi-cell job https://review.opendev.org/667388 | |
| 14:12:49 | aspiers | is there an easy way to get encrypted ephemeral disks with devstack? | |
| 14:16:26 | melwitt | that would be a lyarwood question ^ | |
| 14:20:14 | aspiers | melwitt: thanks! | |
| 14:20:22 | aspiers | it seems that it requires lvm | |
| 14:20:48 | aspiers | so just adding to [[post-config|$NOVA_CONF]] isn't enough... I think | |
| 14:21:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Integrate 'pre-commit' https://review.opendev.org/665518 | |