| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-01-28 | |||
| 14:03:54 | lyarwood | gibi: systemctl restart nova-compute? ;) | |
| 14:05:10 | gibi | yepp that what couple of our customers do weekly but they getting impatient | |
| 14:05:24 | gibi | fun stuf | |
| 14:05:26 | lyarwood | ouch | |
| 14:06:29 | gibi | I had the same experience a decade ago with jvm :D the solution was the same, restart it during maintenance window to free up ram | |
| 14:08:00 | sean-k-mooney | gibi: given python is garbage collected any memory leask is likely form interaction with libvirt right | |
| 14:08:17 | sean-k-mooney | e.g. the proxy objects | |
| 14:08:33 | sean-k-mooney | or well interaction with external things via real threads | |
| 14:08:50 | gibi | sean-k-mooney: or thing the code actually stores and accumlates for no reason | |
| 14:09:05 | sean-k-mooney | i know we can have meemory issue if we return excptions instead of raising them too on python 3 | |
| 14:09:06 | sean-k-mooney | *2 | |
| 14:09:22 | gibi | like this https://github.com/openstack/oslo.messaging/blob/00d15eaeaba0ded0330cdcec7b19eee3adbfb1e1/oslo_messaging/_drivers/amqpdriver.py#L426 | |
| 14:09:42 | sean-k-mooney | gibi: ah well that is less a leak but rather using more memory then it need to due to a poor algorithim | |
| 14:10:13 | gibi | yepp | |
| 14:10:30 | gibi | actually I can hit that log in pike devstack by poking rabbitmq | |
| 14:10:37 | sean-k-mooney | oh in this case it also qould be an rabbitissue | |
| 14:11:21 | sean-k-mooney | ya that could grow if the reply are lost right | |
| 14:12:27 | gibi | I see two types of increase one that eventually recovers. after a long timeout the disconnect is logged in nova side and the connection is removed | |
| 14:12:28 | sean-k-mooney | back to debuging why an osp env that is freshly deployed passes tempest but then after 9+ hours apparenlty stops working | |
| 14:12:54 | gibi | but there are connections that does not freed even after an hour | |
| 14:13:29 | sean-k-mooney | that sound like an amqp bug | |
| 14:13:52 | sean-k-mooney | like the clinet has gone away but for some reason it keeps the file desctprot for the connection open | |
| 14:14:00 | sean-k-mooney | and never recognises the disconnect | |
| 14:14:05 | gibi | could be | |
| 14:14:12 | sean-k-mooney | althoug | |
| 14:14:16 | gibi | now I have to proove that this is what happens in the customer env | |
| 14:14:27 | gibi | as it is still in my local devstack | |
| 14:14:44 | sean-k-mooney | ya not sure how to determin that | |
| 14:15:20 | gibi | I bet on the warning log from oslo | |
| 14:15:28 | gibi | If the customer sees that then I have a lead | |
| 14:27:59 | lyarwood | ~. | |
| 15:02:03 | stephenfin | lyarwood: comments left on one, +2 on the other | |
| 15:06:09 | lyarwood | kashyap: libvirt is always starts with a lower case l right? | |
| 15:06:20 | lyarwood | kashyap: I'm sure this came up in the past and has confused me for ages | |
| 15:07:24 | kashyap | lyarwood: On a call; bbiab | |
| 15:07:29 | sean-k-mooney | i have seen both | |
| 15:07:30 | kashyap | But yes, lower case | |
| 15:07:31 | lyarwood | np | |
| 15:07:36 | sean-k-mooney | but its normally lowercase | |
| 15:07:57 | lyarwood | yeah this came up before and I was told always lowercase for $reasons | |
| 15:08:45 | sean-k-mooney | if its in a commit mesage or specs then i always use libvirt instead of Libvirt | |
| 15:08:51 | sean-k-mooney | same for release notes | |
| 15:11:03 | sean-k-mooney | although funally enough it's uppercase here https://github.com/libvirt/libvirt/blob/30703564c2ac8d95279801a821cf5510fa4b8149/docs/ci.rst#libvirt-continuous-integration but i think thats wrong | |
| 15:11:56 | sean-k-mooney | also a few places in the readme too https://github.com/libvirt/libvirt/blob/30703564c2ac8d95279801a821cf5510fa4b8149/README.rst | |
| 15:12:36 | sean-k-mooney | but again that feel more like there editor auto capitalising then intentional as the use lowercase when its not the start of a sentence | |
| 15:16:59 | openstackgerrit | Ghanshyam proposed openstack/nova master: DNM:try l-c with direct deps https://review.opendev.org/c/openstack/nova/+/772780 | |
| 15:24:52 | kashyap | lyarwood: Back; unless it's at the start of a sentence, it's always lowercase. | |
| 15:25:28 | kashyap | sean-k-mooney: Both those occurrences are correct -- because the word came at the start of a sentence | |
| 15:25:39 | kashyap | So it's only natural to capitalize it :-) | |
| 15:26:05 | kashyap | lyarwood: What made you ask? Are you writing some docs / code comments or something? | |
| 15:28:21 | sean-k-mooney | kashyap: ya althoguh some porject for trademark reason say you should not capatilise it or use a spefic capatialistion when you do | |
| 15:28:48 | sean-k-mooney | kashyap: so the english rule dont alway apply if you are follow the project style/trademark guidelines | |
| 15:29:29 | kashyap | sean-k-mooney: Right; but pretty sure I saw what I described "lowercase all the time, except at the start of a sentence" somewhere in libvirt upstream discussions | |
| 15:29:37 | kashyap | But I can't find a link for you at the moment | |
| 15:31:38 | openstackgerrit | Ghanshyam proposed openstack/nova master: DNM:try l-c with direct deps https://review.opendev.org/c/openstack/nova/+/772780 | |
| 15:31:42 | sean-k-mooney | ya makes sense | |
| 15:32:01 | sean-k-mooney | i assume lyarwood was looking at documentaiton or simiarl and just wanted to make sure it was correct | |
| 15:44:36 | lyarwood | sean-k-mooney / kashyap ; https://review.opendev.org/c/openstack/nova/+/771981 was the context, stephenfin had a few nits and I couldn't remember the reasons for libvirt being lowercase all the time etc. | |
| 15:44:47 | lyarwood | thanks kashyap, I'll sort the change out now | |
| 15:45:24 | kashyap | lyarwood: No prob | |
| 15:50:26 | gibi | nova meeting starts in 10 minutes on #openstack-meeting-3 | |
| 15:55:52 | openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Move the LibvirtDistroSupportMatrix wiki page into our docs https://review.opendev.org/c/openstack/nova/+/771981 | |
| 16:08:44 | openstackgerrit | Merged openstack/nova master: tox: Enable parallel docs build https://review.opendev.org/c/openstack/nova/+/751034 | |
| 16:30:18 | sean-k-mooney | lyarwood: ah ok that was in the wiki for easy editing but we very very rarely update it so moving it too docs makes sense i guess | |
| 16:32:15 | lyarwood | yeah it wasn't easy and it's just nice to have it in tree and updated in the same change as the actual bump etc | |
| 16:33:10 | artom | Lol, I just realized running functional tests locally that need an un-mergde os-traits patch is... dumb. | |
| 16:35:24 | sean-k-mooney | artom: you can do that | |
| 16:35:28 | lyarwood | artom: install it into the env | |
| 16:35:34 | sean-k-mooney | you just pip install it in the env | |
| 16:35:50 | artom | Right, I'm sure there are ways | |
| 16:36:25 | sean-k-mooney | basicaly .tox/functional/bin/python3 -m pip install -e ../os-traits | |
| 16:36:44 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: [WIP] libvirt: Remove compareCPU() check on the destination https://review.opendev.org/c/openstack/nova/+/772917 | |
| 16:36:52 | sean-k-mooney | assuming you have os-traits cloneed and checked out beside your nova dir | |
| 17:27:58 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add type annotations to top-level modules https://review.opendev.org/c/openstack/nova/+/705658 | |
| 17:27:58 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add nova.cmd, nova.conf, nova.console https://review.opendev.org/c/openstack/nova/+/705657 | |
| 17:27:59 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Expand type hints for nova.block_device https://review.opendev.org/c/openstack/nova/+/742170 | |
| 17:27:59 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Clean manager.Manager, service.Service signatures https://review.opendev.org/c/openstack/nova/+/764806 | |
| 17:28:00 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Remove dead code https://review.opendev.org/c/openstack/nova/+/772928 | |
| 18:34:15 | openstackgerrit | Lee Yarwood proposed openstack/nova master: Disallow CONF.compute.max_disk_devices_to_attach = 0 https://review.opendev.org/c/openstack/nova/+/755360 | |
| 18:35:24 | artom | Didn't melwitt already do that? | |
| 18:36:34 | lyarwood | artom: that was just a rebase | |
| 18:36:48 | artom | Ag | |
| 18:36:49 | artom | Ah | |
| 18:38:36 | artom | *facepalm* | |
| 18:39:00 | artom | When 0 is an "allowed" value, but None is not, don't just if <value> :( | |
| 18:39:08 | artom | That's a couple of hours wasted | |
| 18:54:13 | sean-k-mooney | its the same as the rpc workers issue we fixed a whiel back | |
| 18:54:19 | sean-k-mooney | jsut missign the min=1 | |
| 18:54:50 | sean-k-mooney | ah we have -1 for unlimited | |
| 18:55:02 | sean-k-mooney | so so we cant do that | |
| 18:55:08 | sean-k-mooney | really we should have used 0 for that | |
| 18:55:21 | sean-k-mooney | in this case since 0 is not valid | |
| 20:12:20 | ganso | Hi Nova folks! I just logged this bug https://bugs.launchpad.net/nova/+bug/1913641 and while thinking about the possible ways to solve it, I started wondering whether it was intentional (and therefore not a bug) to consider shelved_offloaded instances as running, metric-wise. | |
| 20:12:21 | openstack | Launchpad bug 1913641 in OpenStack Compute (nova) "Incorrect Shelved_offloaded instance metrics on openstack usage show output" [Undecided,New] | |
| 21:03:34 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: libvirt: start tracking NUMACell.socket for hosts https://review.opendev.org/c/openstack/nova/+/766816 | |
| 21:03:35 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: Add `socket` PCI NUMA affinity policy request prefilter https://review.opendev.org/c/openstack/nova/+/772749 | |
| 21:03:35 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: extra specs/image pros: add `socket PCI NUMA affinity https://review.opendev.org/c/openstack/nova/+/772748 | |
| 21:03:36 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: pci: implement the SOCKET NUMA affinity policy https://review.opendev.org/c/openstack/nova/+/772779 | |
| 21:04:21 | artom | ganso, I *think* `openstack show usage` is about quotas, and shelved offloaded still count against the quota | |
| 21:04:53 | artom | But you're correct in assuming their resources (disk, VM) should not be counted... | |