| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-15 | |||
| 09:03:15 | bauzas | that's working like a charm | |
| 09:03:40 | bauzas | it just binds a new greenlet where you can access the whole process | |
| 09:04:05 | gibi | bauzas: care to document that process somewhere after you surface from the escalations? | |
| 09:04:14 | gibi | I think it would be useful | |
| 09:04:30 | bauzas | gibi: yeah that's worth of interest, I'm also testing tracemalloc module | |
| 09:04:53 | gibi | +1 for the tracemalloc too. | |
| 09:05:28 | bauzas | context (I can tell if I don't tell the whole context) : the nova-compute service is consuming unusual memory space hence a potential memory leak to identify | |
| 09:06:05 | bauzas | so the strategy is to compare memory allocations snapshots over time | |
| 09:08:15 | gibi | does tracemalloc can differentiate between allocated but not used memory and actually used memory? | |
| 09:08:42 | kashyap | Very good question | |
| 09:09:51 | kashyap | tracemalloc.get_tracemalloc_memory() | |
| 09:10:01 | bauzas | gibi: we're litterally doing baby steps here | |
| 09:10:01 | kashyap | gibi: The above _should_ get the memory usag | |
| 09:10:17 | bauzas | gibi: the idea is that you can generate snapshots | |
| 09:10:31 | bauzas | and each snapshot (like a GMR) gives you bits of memory usage | |
| 09:10:49 | bauzas | tracemalloc even provides a compare method between snapshots | |
| 09:13:21 | bauzas | kashyap: tracemalloc.get_traced_memory() only gives you the memory usage of the tracking module itself | |
| 09:27:26 | opendevreview | Lee Yarwood proposed openstack/nova master: docs: Add nova-volume volume_attachment refresh admin workflow https://review.opendev.org/c/openstack/nova/+/809161 | |
| 09:31:41 | opendevreview | Lee Yarwood proposed openstack/nova master: run-evacuate-hook: Switch to osc evacuate command and add --debug https://review.opendev.org/c/openstack/nova/+/809162 | |
| 09:42:40 | opendevreview | Merged openstack/osc-placement stable/xena: Update .gitreview for stable/xena https://review.opendev.org/c/openstack/osc-placement/+/808456 | |
| 09:45:24 | opendevreview | Merged openstack/python-novaclient stable/xena: Update .gitreview for stable/xena https://review.opendev.org/c/openstack/python-novaclient/+/808459 | |
| 09:45:25 | opendevreview | Merged openstack/python-novaclient stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena https://review.opendev.org/c/openstack/python-novaclient/+/808460 | |
| 09:54:13 | opendevreview | Merged openstack/osc-placement stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena https://review.opendev.org/c/openstack/osc-placement/+/808457 | |
| 11:44:35 | viks__ | hi, what is the use of the code: `consumer_type = 'migration'` in https://github.com/openstack/nova/blob/01eaa627b406af9336a231caea6969aae9a62033/nova/cmd/manage.py#L2550 as the next condition after this line always overrides this? am i missing something? | |
| 11:45:58 | sean-k-mooney | it will not override it if there is not instance that matches | |
| 11:46:30 | sean-k-mooney | so we asume any allocation we find that doen have a consumer type are migration and then set them to instnace if we see a corresponding instnace | |
| 11:47:23 | viks__ | sean-k-mooney: but if we check the earlier condition, we see it always seems to be setting `consumer_type = 'instance'` | |
| 11:47:45 | viks__ | https://www.irccloud.com/pastebin/XOzn2XSW/ | |
| 11:48:07 | sean-k-mooney | most allocation will be of type instance | |
| 11:49:06 | sean-k-mooney | the not is likely wrong | |
| 11:49:24 | sean-k-mooney | infact the not i think is wrong | |
| 11:49:45 | viks__ | sean-k-mooney: ok... that's what i was thinking | |
| 11:50:06 | sean-k-mooney | well unless the not is for deleted instnaces | |
| 11:50:46 | viks__ | ok | |
| 11:51:37 | sean-k-mooney | so the first if is if not (consumer_uuid in inst_uuids or | |
| 11:51:39 | sean-k-mooney | consumer_uuid in mig_uuids): | |
| 11:52:35 | sean-k-mooney | so if we enter it we know the consumer uuid is not in inst_uuids and it is not in mig_uuids | |
| 11:53:04 | sean-k-mooney | so the second if will always be true | |
| 11:53:55 | viks__ | sean-k-mooney: correct.. what should be the correct condition here... i'm not having full context of the overall code | |
| 11:55:05 | sean-k-mooney | i think we need a third state of orpanded or deleted | |
| 11:55:47 | viks__ | ok | |
| 11:55:56 | sean-k-mooney | actully | |
| 11:57:08 | sean-k-mooney | we might just be able to do https://paste.opendev.org/show/809339/ | |
| 11:57:35 | sean-k-mooney | so if the consumer uuid is not an instance or migration | |
| 11:57:42 | sean-k-mooney | then we know it was deleted | |
| 11:58:06 | sean-k-mooney | so i think we can just delete https://github.com/openstack/nova/blob/01eaa627b406af9336a231caea6969aae9a62033/nova/cmd/manage.py#L2548-L2555 | |
| 11:58:57 | sean-k-mooney | viks__: does that make sense | |
| 12:00:52 | sean-k-mooney | we might need to modify https://github.com/openstack/nova/blob/01eaa627b406af9336a231caea6969aae9a62033/nova/cmd/manage.py#L2445 | |
| 12:01:03 | sean-k-mooney | to correctly handel reciving None for the consumer type | |
| 12:01:14 | sean-k-mooney | or https://github.com/openstack/nova/blob/01eaa627b406af9336a231caea6969aae9a62033/nova/scheduler/client/report.py#L2053 | |
| 12:01:24 | sean-k-mooney | but i think that woudl be the correct way to fix this | |
| 12:05:05 | viks__ | ok... | |
| 12:11:24 | viks__ | sean-k-mooney: just changing it to below doesnt resolve: | |
| 12:11:30 | viks__ | https://www.irccloud.com/pastebin/e2Zcf6le/ | |
| 12:13:12 | sean-k-mooney | correct that would not work | |
| 12:19:27 | viks__ | as per the comments in the code, it should work right? i.e. first we set it to migration,then if `consumer_uuid`is not found in `inst_uuids`, it should set it to `instance`... i do not have the detailed contex.. so just asking | |
| 14:40:36 | opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [PoC][yoga] Off-path Networking Backends Support https://review.opendev.org/c/openstack/nova/+/808199 | |
| 15:34:13 | MrClayPole | When attempting to live-migrate a VM between 2 compute nodes it fails with "Exception during message handling: TargetPortalNotFound: Unable to find target portal 1.1.1.1:3260". The storage is iSCSI iSER with multipath enabled in nova.conf. I can see cinder mapping the volume to the target but os_brick is scanning with "iscsiadm -m session" before iscsid has brought up the iSCSI session on the target compute node. Is there | |
| 15:34:13 | MrClayPole | a way to make os_brick wait until the iSCSI session is up? | |
| 15:49:58 | lyarwood | MrClayPole: it should retry for a while and iirc that's now configurable, which version of OpenStack is this? | |
| 15:51:58 | MrClayPole | lyarwood: Openstack Rocky. os_bricks only tries once according to the logs | |
| 15:56:28 | lyarwood | MrClayPole: kk I'm really not sure then, you might want o ask the folks in #openstack-cinder who maintain os-brick for more clues | |
| 15:56:32 | lyarwood | to* | |
| 16:01:05 | MrClayPole | lyarwood: Thanks I'll do that. I wasn't sure who looked after os_bricks. | |
| 16:02:40 | lyarwood | np, sorry I couldn't help more | |
| 19:54:12 | opendevreview | Artom Lifshitz proposed openstack/nova stable/victoria: Test SRIOV port move operations with PCI conflicts https://review.opendev.org/c/openstack/nova/+/796908 | |
| 19:54:13 | opendevreview | Artom Lifshitz proposed openstack/nova stable/victoria: Update SRIOV port pci_slot when unshelving https://review.opendev.org/c/openstack/nova/+/796909 | |
| 19:54:14 | opendevreview | Artom Lifshitz proposed openstack/nova stable/victoria: Fix ImageBackendFixture not to support direct_snapshot https://review.opendev.org/c/openstack/nova/+/809282 | |
| 19:54:33 | artom | That was a PITA to hunt down and figure out it was the ImageBackendFixture backport that was needed... | |
| 23:52:33 | opendevreview | Artom Lifshitz proposed openstack/nova stable/victoria: Update SRIOV port pci_slot when unshelving https://review.opendev.org/c/openstack/nova/+/796909 | |
| #openstack-nova - 2021-09-16 | |||
| 00:21:00 | opendevreview | Merged openstack/nova stable/victoria: address open redirect with 3 forward slashes https://review.opendev.org/c/openstack/nova/+/806626 | |
| 03:40:02 | opendevreview | melanie witt proposed openstack/nova master: Add stub unified limits driver https://review.opendev.org/c/openstack/nova/+/712137 | |
| 03:40:02 | opendevreview | melanie witt proposed openstack/nova master: Assert quota related API behavior when noop https://review.opendev.org/c/openstack/nova/+/712140 | |
| 03:40:03 | opendevreview | melanie witt proposed openstack/nova master: Make unified limits APIs return reserved of 0 https://review.opendev.org/c/openstack/nova/+/712141 | |
| 03:40:03 | opendevreview | melanie witt proposed openstack/nova master: Add logic to enforce local api and db limits https://review.opendev.org/c/openstack/nova/+/712139 | |
| 03:40:04 | opendevreview | melanie witt proposed openstack/nova master: Update quota_class APIs for db and api limits https://review.opendev.org/c/openstack/nova/+/712143 | |
| 03:40:04 | opendevreview | melanie witt proposed openstack/nova master: Enforce api and db limits https://review.opendev.org/c/openstack/nova/+/712142 | |
| 03:40:05 | opendevreview | melanie witt proposed openstack/nova master: Update quota sets APIs https://review.opendev.org/c/openstack/nova/+/712749 | |
| 03:40:05 | opendevreview | melanie witt proposed openstack/nova master: Update limit APIs https://review.opendev.org/c/openstack/nova/+/712707 | |
| 03:40:06 | opendevreview | melanie witt proposed openstack/nova master: Tell oslo.limit how to count nova resources https://review.opendev.org/c/openstack/nova/+/713301 | |
| 03:40:07 | opendevreview | melanie witt proposed openstack/nova master: Enforce resource limits using oslo.limit https://review.opendev.org/c/openstack/nova/+/615180 | |
| 03:40:07 | opendevreview | melanie witt proposed openstack/nova master: Add legacy limits and usage to placement unified limits https://review.opendev.org/c/openstack/nova/+/713498 | |
| 03:40:09 | opendevreview | melanie witt proposed openstack/nova master: Update quota apis with keystone limits and usage https://review.opendev.org/c/openstack/nova/+/713499 | |
| 03:40:09 | opendevreview | melanie witt proposed openstack/nova master: Add reno for unified limits https://review.opendev.org/c/openstack/nova/+/715271 | |
| 07:54:11 | lpetrut | Hi, I have a question about keypairs. Nova allows a single keypair to be associated with a vm, yet in some cases we must inject multiple keys. We're using the k8s CAPO provider so we can't really use the userdata directly to inject additional keys. Now, apparently it's possible to bundle multiple ssh keys with the same keypair. Can we rely on this behavior to remain available? Fwiw, when bundling multiple keypairs, apparently each keypair must | |
| 07:54:12 | lpetrut | have a comment, otherwise nova will fail to generate a fingerprint and reject it. | |
| 07:58:16 | bauzas | good morning Nova | |
| 07:58:32 | bauzas | for the first time during this week, I eventually have a bit time for going upstream... | |
| 07:59:19 | gibi | bauzas: o/ could you please check the comments on the prelude | |
| 07:59:48 | bauzas | lpetrut: the API doesn't look it supports more than one public key for a keypair | |
| 07:59:52 | bauzas | lpetrut: https://docs.openstack.org/api-ref/compute/?expanded=create-or-import-keypair-detail | |
| 07:59:55 | bauzas | gibi: sure, will look | |
| 08:00:01 | gibi | thank you | |
| 08:00:09 | bauzas | I also want to work for the vgpu documentation | |
| 08:00:28 | gibi | also would be nice to land this doc https://review.opendev.org/c/openstack/nova/+/809161 and link it to the prelude | |
| 08:00:50 | gibi | sure, if you push vgpu doc ping me and I will prioritize it | |
| 08:01:14 | bauzas | gibi: ack, will look at it today | |
| 08:01:42 | lpetrut | bauzas: we're passing multiple ssh keys separated by newline. apparently other people rely on it as well: https://help.switch.ch/engines/faq/how-to-use-multiple-ssh-keys/ | |
| 08:11:06 | lpetrut | it's an ugly workaround, but it would be nice if we could continue to allow it until nova gets to support associating multiple keypairs | |
| 08:13:02 | bauzas | lpetrut: well, | |