| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-05-18 | |||
| 12:40:39 | sean-k-mooney | actully they might be spaces but either way it would be nice to remove them | |
| 12:40:55 | sean-k-mooney | i think the release note is ok | |
| 12:41:25 | sean-k-mooney | it does not really explain why you want want to enable it | |
| 12:41:32 | sean-k-mooney | or what it is however | |
| 12:42:18 | sean-k-mooney | ideally an opeartor that does not know what this feature is should be able to look at the release note and get a sense for why they might care about learnign more | |
| 12:43:32 | sean-k-mooney | i woudl add a sentence basically saying that the pack virtqueue format can improve the performance espically when combinind with hadware offloads or dpdk | |
| 12:44:00 | sean-k-mooney | ill try an review this properly later or tomorrow | |
| 12:44:54 | dvo-plv | okay, I see. Should I left sentence regarding packed filter and scheduling or this is too dive for the operator? | |
| 12:46:56 | sean-k-mooney | am no i dont think that is needed but it would be nice to add a link to the spec https://specs.openstack.org/openstack/nova-specs/specs/2023.2/approved/virtio_packedring_configuration_support.html | |
| 12:47:21 | sean-k-mooney | just add it in the "Usefull references about virtqueue:" list | |
| 12:47:39 | sean-k-mooney | maybe make that jsut "Usefull references:" | |
| 12:48:17 | sean-k-mooney | with those change the operator that reads it will have an idea of what the feature is and why they might want it and then links to learn more | |
| 12:48:24 | sean-k-mooney | so that shoudl be good. | |
| 12:51:04 | dvo-plv | okay, thanks, I will apply all advice | |
| 13:12:49 | opendevreview | Danylo Vodopianov proposed openstack/nova master: Packed virtqueue support was added. https://review.opendev.org/c/openstack/nova/+/876075 | |
| 13:23:55 | dansmith | we need another +2 on the CVE stack for wallaby please: https://review.opendev.org/c/openstack/nova/+/882869/2 | |
| 13:32:44 | sean-k-mooney | dansmith: done | |
| 13:32:59 | dansmith | sean-k-mooney: thanks | |
| 13:49:54 | dvo-plv | sean-k-mooney: When we've told about queue feature, you mentioned that in could defined via '/sys/bus/pci/devices/...' path. I found this parser in the nova/pci/utils.py | |
| 13:50:58 | dvo-plv | But you also mentined about getting queue from the libvirt xml. Could you please direct me to the module, which parse libvirt output | |
| 13:51:44 | sean-k-mooney | eiehter will work for your case since the PF will be bound to dpdk | |
| 13:51:58 | sean-k-mooney | and the representors are in user space | |
| 13:52:57 | dvo-plv | class LibvirtConfigNodeDevicePciCap ? | |
| 13:54:04 | sean-k-mooney | that is the libvir class i think yes | |
| 13:54:18 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/host.py#L1568 we list the pci devices here | |
| 13:56:10 | sean-k-mooney | this is what parses the nodedevs https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L3245 | |
| 13:56:22 | sean-k-mooney | and yes https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L3293 | |
| 13:56:34 | sean-k-mooney | is what pares node devs that model pci devices | |
| 13:57:43 | sean-k-mooney | the sturcture returend form libvirt looks somethign like this https://libvirt.org/drvnodedev.html#pci-host-devices | |
| 13:58:03 | dvo-plv | thank you a lot | |
| 14:00:14 | sean-k-mooney | idealy the kernel driver woudl be modifed to report theyse capablits such as the queue avaiale and then libvirt would be modifed to present that in the nodedev api | |
| 14:00:18 | sean-k-mooney | then nova woudl just read that | |
| 14:00:22 | opendevreview | Danylo Vodopianov proposed openstack/nova master: Packed virtqueue support was added. https://review.opendev.org/c/openstack/nova/+/876075 | |
| 14:00:27 | sean-k-mooney | we can read form sysfs if we really need too | |
| 14:00:39 | sean-k-mooney | but thats somehtign we try not to do | |
| 14:26:21 | opendevreview | Sofia Enriquez proposed openstack/nova-specs master: Implement support for LUKS inside qcow2 volumes when using Libvirt https://review.opendev.org/c/openstack/nova-specs/+/883516 | |
| 14:49:36 | noonedeadpunk | hey folks! I was kinda wondering about `send_service_user_token` requirement. For some reason, I thought it should have been covered by `service_token_roles_required` but seems it's an independant thing? | |
| 15:08:05 | opendevreview | Merged openstack/nova master: vmwareapi: Mark driver as experimental https://review.opendev.org/c/openstack/nova/+/863911 | |
| 15:09:39 | dansmith | noonedeadpunk: isn't the "required" one set on the middleware and not the client? | |
| 15:13:00 | noonedeadpunk | Ok, this question is cornering me :) I think it is. I guess that auth_section as an option, which I pointed to keystone_authtoken, mislead me | |
| 15:14:37 | noonedeadpunk | but yeah, now I see what you mean I guess | |
| 15:22:52 | opendevreview | Merged openstack/nova stable/wallaby: Use force=True for os-brick disconnect during delete https://review.opendev.org/c/openstack/nova/+/882869 | |
| 18:04:29 | opendevreview | Ghanshyam proposed openstack/nova master: Add integrated job to test the Nova RBAC old defaults https://review.opendev.org/c/openstack/nova/+/883557 | |
| 19:30:38 | opendevreview | Carl Morris proposed openstack/nova master: Improving logging at '_numa_cells_support_network_metadata'. https://review.opendev.org/c/openstack/nova/+/860930 | |
| 19:36:40 | melwitt | noonedeadpunk: send_service_user_token = True needs to go on the client side (nova) and service_token_roles_required = True needs to go on the server (cinder) | |
| 19:37:22 | melwitt | i.e. cinder will require service tokens and so nova will send one | |
| 20:30:33 | opendevreview | melanie witt proposed openstack/nova master: testing: Use the oslo.log ContextFormatter in the log fixture https://review.opendev.org/c/openstack/nova/+/883466 | |
| 22:24:44 | opendevreview | Merged openstack/nova stable/wallaby: Enable use of service user token with admin context https://review.opendev.org/c/openstack/nova/+/882870 | |
| 22:31:27 | dansmith | tres woot | |
| 22:31:35 | dansmith | melwitt: ^ should be the last one | |
| 22:33:00 | melwitt | woohoo | |
| #openstack-nova - 2023-05-19 | |||
| 00:30:53 | gmann | dansmith: this is enabling the nova rbac new defaults in devstack by default and nova change to add a new job to test old default please check https://review.opendev.org/c/openstack/devstack/+/883556 https://review.opendev.org/c/openstack/nova/+/883557 | |
| 00:36:10 | dansmith | gmann: got it | |
| 00:41:32 | gmann | thanks | |
| 12:45:25 | opendevreview | Danylo Vodopianov proposed openstack/nova master: Packed virtqueue support was added. https://review.opendev.org/c/openstack/nova/+/876075 | |
| 13:16:48 | opendevreview | Danylo Vodopianov proposed openstack/nova-specs master: Add support for Napatech LinkVirt SmartNICs https://review.opendev.org/c/openstack/nova-specs/+/859290 | |
| 13:41:12 | opendevreview | Danylo Vodopianov proposed openstack/nova-specs master: Add support for Napatech LinkVirt SmartNICs https://review.opendev.org/c/openstack/nova-specs/+/859290 | |
| 13:43:09 | opendevreview | Danylo Vodopianov proposed openstack/nova-specs master: Add support for Napatech LinkVirt SmartNICs https://review.opendev.org/c/openstack/nova-specs/+/859290 | |
| 13:44:29 | opendevreview | Amit Uniyal proposed openstack/nova master: [WIP] add initial healthcheck support https://review.opendev.org/c/openstack/nova/+/825015 | |
| 13:44:29 | opendevreview | Amit Uniyal proposed openstack/nova master: [WIP] add healthcheck manager to manager base https://review.opendev.org/c/openstack/nova/+/827844 | |
| 13:44:30 | opendevreview | Amit Uniyal proposed openstack/nova master: [WIP] add healthcheck tracker to nova context https://review.opendev.org/c/openstack/nova/+/829468 | |
| 13:44:30 | opendevreview | Amit Uniyal proposed openstack/nova master: [WIP] add healthcheck utils and constants https://review.opendev.org/c/openstack/nova/+/829469 | |
| 13:44:31 | opendevreview | Amit Uniyal proposed openstack/nova master: add healthcheck endpoint to proxy commands https://review.opendev.org/c/openstack/nova/+/830703 | |
| 15:01:01 | dansmith | sean-k-mooney: this is the binding error I've been seeing quite a bit latey: https://a6fc37e91c861c55cf2e-59e8bddca242bc843b9f9be8c2ce73c4.ssl.cf2.rackcdn.com/879905/5/check/nova-live-migration/e3a0c46/testr_results.html | |
| 15:01:21 | dansmith | it says to check neutron logs, so maybe it's just a neutron thing, but does that look familiar at all? | |
| 15:01:48 | dansmith | actually I though it used to say something about os-vif in the message now that I think about it so maybe it's something different | |
| 15:09:00 | opendevreview | yatin proposed openstack/nova master: Add config option to configure TB cache size https://review.opendev.org/c/openstack/nova/+/868419 | |
| 15:10:59 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: Re-order and parallelize calls to Neutron and Cinder in post_live_migration https://review.opendev.org/c/openstack/nova/+/883678 | |
| 15:17:57 | dansmith | artom: the GIL has nothing to do with your threading there | |
| 15:18:18 | dansmith | what you mean, I think, is "as concurrently as the activities are green(ed)" | |
| 15:20:09 | sean-k-mooney | dansmith: only because you linked it to me like two weeks ago | |
| 15:20:24 | sean-k-mooney | ill tak a look at it again i didnt get around to it then | |
| 15:20:33 | dansmith | sean-k-mooney: thanks | |
| 15:32:22 | sean-k-mooney | Refusing to bind port e3308a61-39ff-4064-abb2-76de0d2139dc to dead agent: <neutron.plugins.ml2.drivers.ovn.agent.neutron_agent.ControllerAgent object at 0x7f6a7a6d2950> | |
| 15:32:45 | dansmith | does that mean the agent on the compute died or something? | |
| 15:33:42 | sean-k-mooney | i think that meanst the ovn metadtaa agent is dead since ovn its slef is agent less | |
| 15:34:04 | sean-k-mooney | but yes neutorn think the agent is dead on the destination host | |
| 15:34:13 | sean-k-mooney | it bound fine orgianly on the other host | |
| 15:34:49 | artom | dansmith, right, I meant it in the sense of GIL only allowing one execution thread at a time, so we're counting to eventlet to do its yield thing | |
| 15:35:02 | artom | I'll just remove it from the commit message :P | |
| 15:35:14 | sean-k-mooney | ill see if i can figure out why | |
| 15:48:11 | sean-k-mooney | dansmith: so it soudn like its hitting the code for https://github.com/openstack/neutron/commit/8a55f091925fd5e6742fb92783c524450843f5a0 | |
| 15:50:38 | sean-k-mooney | hum so at the time of the port bidning | |
| 15:51:30 | sean-k-mooney | there are no errro in the metadta aganet log but there are gaps for 3-6 seconds at a tiem and its interacting with both ovs and privsep | |
| 16:12:18 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: Re-order and parallelize calls to Neutron and Cinder in post_live_migration https://review.opendev.org/c/openstack/nova/+/883678 | |
| 16:12:19 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: Call Neutron immediately upon _post_live_migration() start https://review.opendev.org/c/openstack/nova/+/883682 | |
| 16:15:42 | sean-k-mooney | dansmith: so my best guess is its related to thsi change https://github.com/openstack/neutron/commit/628442aed7400251f12809a45605bd717f494c4e | |
| 16:16:16 | sean-k-mooney | 7 mounts ago they started trying to spread the agent heatbeats | |
| 16:16:42 | sean-k-mooney | im seeing logs to the effect fo delaying update to the cachs table for 10 seconds | |
| 16:17:02 | sean-k-mooney | around when the agent prior to the agent being detected as dead | |
| 16:17:24 | sean-k-mooney | my guess is if the agent is doign somthign like writing to the ovs db | |
| 16:17:36 | sean-k-mooney | it can miss the heatbeat | |
| 16:18:04 | sean-k-mooney | Delaying updating chassis table for 23 seconds {{(pid=38857) run /opt/stack/neutron/neutron/agent/ovn/metadata/agent.py:243}} | |
| 16:18:23 | sean-k-mooney | im seeign quite a spread | |
| 16:20:08 | dansmith | artom: ack I figured, probably better to make it accurate though yeah :) | |
| 16:20:23 | dansmith | sean-k-mooney: ah, interesting | |
| 16:20:37 | dansmith | sean-k-mooney: so like under heavy load they're missing some heartbeats maybe | |
| 16:20:53 | sean-k-mooney | ya perhaps | |
| 16:21:21 | sean-k-mooney | im goign to put up a tiny patch to change that form cfg.CONF.agent_down_time // 2 to cfg.CONF.agent_down_time // 3 | |
| 16:21:33 | sean-k-mooney | that will make it heat beat a little more often | |