| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-04 | |||
| 17:54:41 | claudiub | will try both. :) | |
| 17:56:42 | sean-k-mooney | the balancing without over subctiption or randomisation with it likely could be backproted as a workaround bugfix. with proper feature in the future | |
| 18:24:45 | sean-k-mooney | stephenfin:related to the previous topic it look like libvirt is locking the guest memroy any time a guest has vfio(pci passthough/sriov), mdev or nvme devices assigned nova was previously not aware of that and them means many guests we previously assumed were swapable are not. i still have to file a bug for this but its emrging forlowing the memory locking disucssion we had for vdpa | |
| 18:26:29 | sean-k-mooney | im not sure how to fix that yet without requiring all guest with any kind of passthoug device to be numa guests or otherewise use hw:mem_page_size in some form | |
| 18:27:13 | sean-k-mooney | we can mitagate the problem for q35 guests by enabling the viommu | |
| 18:27:36 | sean-k-mooney | but i suspect this is the cause of many OOM bug that have been filed in the past | |
| 18:29:54 | sean-k-mooney | oh and more fun file backed memroy does not seam to work the way we tought either | |
| 18:30:36 | sean-k-mooney | i should do some more testing but usign it i was not actully able to allocate more vms then i had memory for without OOM issues killing the running vms | |
| 18:31:08 | sean-k-mooney | so it looks like instead of mmaping the guest memory form the files as the qemu/libvirt docs impleis | |
| 18:31:39 | sean-k-mooney | qemu just malloc the memroy normally and then also create a mapping of the memory to a file | |
| 18:32:22 | sean-k-mooney | that might be because we are usign the legacy api for this or it might be for a different reason but either way it makes me sad :( | |
| 18:49:25 | sean-k-mooney | actully i think i know why its broken if we want vms to only have file backed memory which is what we inteded we need to explitly set the normal memory to 0 i belive and then add the file using the memory hotplug feature so ya it likely a qemu bug in the old api. | |
| 19:43:05 | openstackgerrit | Merged openstack/nova master: libvirt: Parse the 'os' element from domainCapabilities https://review.opendev.org/c/openstack/nova/+/673790 | |
| 20:17:26 | openstackgerrit | Merged openstack/nova master: nova-manage: Add libvirt get_machine_type command https://review.opendev.org/c/openstack/nova/+/769548 | |
| 23:03:14 | melwitt | lyarwood: could you pls take a look at these stable/victoria backports? they've disabled tests in tripleo ci to workaround intermittent failures due to the bug https://review.opendev.org/c/openstack/nova/+/777121 and https://review.opendev.org/c/openstack/nova/+/777209 | |
| 23:22:20 | openstackgerrit | Takashi Kajinami proposed openstack/nova master: WIP: Clean up allocations left by evacuation https://review.opendev.org/c/openstack/nova/+/778696 | |
| 23:51:50 | openstackgerrit | melanie witt proposed openstack/nova master: Add functional test for bug 1837995 https://review.opendev.org/c/openstack/nova/+/775449 | |
| 23:51:52 | openstack | bug 1837995 in OpenStack Compute (nova) ""Unexpected API Error" when use "openstack usage show" command" [Undecided,In progress] https://launchpad.net/bugs/1837995 - Assigned to melanie witt (melwitt) | |
| 23:51:52 | openstackgerrit | melanie witt proposed openstack/nova master: Dynamically archive FK related records in archive_deleted_rows https://review.opendev.org/c/openstack/nova/+/773834 | |
| #openstack-nova - 2021-03-05 | |||
| 00:06:21 | openstackgerrit | melanie witt proposed openstack/nova master: Add functional regression test for bug 1853009 https://review.opendev.org/c/openstack/nova/+/695012 | |
| 00:06:23 | openstack | bug 1853009 in OpenStack Compute (nova) ussuri "Ironic node rebalance race can lead to missing compute nodes in DB" [High,In progress] https://launchpad.net/bugs/1853009 - Assigned to Mark Goddard (mgoddard) | |
| 00:06:23 | openstackgerrit | melanie witt proposed openstack/nova master: Clear rebalanced compute nodes from resource tracker https://review.opendev.org/c/openstack/nova/+/695187 | |
| 00:06:23 | openstackgerrit | melanie witt proposed openstack/nova master: Invalidate provider tree when compute node disappears https://review.opendev.org/c/openstack/nova/+/695188 | |
| 00:06:24 | openstackgerrit | melanie witt proposed openstack/nova master: Prevent deletion of a compute node belonging to another host https://review.opendev.org/c/openstack/nova/+/694802 | |
| 00:06:24 | openstackgerrit | melanie witt proposed openstack/nova master: Fix inactive session error in compute node creation https://review.opendev.org/c/openstack/nova/+/695189 | |
| 04:53:48 | openstackgerrit | Yongli He proposed openstack/nova master: Smartnic support - cyborg drive https://review.opendev.org/c/openstack/nova/+/771362 | |
| 04:53:49 | openstackgerrit | Yongli He proposed openstack/nova master: smartnic support - new vnic type https://review.opendev.org/c/openstack/nova/+/771363 | |
| 04:53:49 | openstackgerrit | Yongli He proposed openstack/nova master: smartnic support https://review.opendev.org/c/openstack/nova/+/758944 | |
| 07:04:15 | openstackgerrit | Yongli He proposed openstack/nova master: Smartnic support - cyborg drive https://review.opendev.org/c/openstack/nova/+/771362 | |
| 07:04:16 | openstackgerrit | Yongli He proposed openstack/nova master: smartnic support - new vnic type https://review.opendev.org/c/openstack/nova/+/771363 | |
| 07:04:17 | openstackgerrit | Yongli He proposed openstack/nova master: smartnic support https://review.opendev.org/c/openstack/nova/+/758944 | |
| 08:24:05 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Replace blind retry with libvirt event waiting in detach https://review.opendev.org/c/openstack/nova/+/770246 | |
| 08:27:25 | lyarwood | elod: https://review.opendev.org/c/openstack/nova/+/777209/1 - would you mind hitting this and the regression test below it for melwitt please. | |
| 08:47:29 | elod | lyarwood: sure, looking | |
| 08:48:30 | yonglihe | alex_xu: gibi: test cases fixed, resolve concerns. | |
| 08:57:49 | lyarwood | elod: thanks | |
| 09:23:16 | bauzas | gibi: stephenfin: the whole api db series is now +Wd | |
| 09:23:26 | bauzas | but, | |
| 09:23:56 | bauzas | have we forgotten to add wallaby db placeholders like we did for Rocky ? :scared_emoji_you'd_like_have: | |
| 09:24:19 | stephenfin | Do we need to? | |
| 09:24:27 | stephenfin | We can just use the Victoria placeholders | |
| 09:24:39 | stephenfin | We don't have any DB migrations afaik | |
| 09:25:47 | bauzas | stephenfin: well, in general, we create DB placeholders even if we don't have existing DB things | |
| 09:25:52 | bauzas | https://github.com/openstack/nova/tree/master/nova/db/sqlalchemy/api_migrations/migrate_repo/versions | |
| 09:26:03 | stephenfin | I know. I'm questioning that :) | |
| 09:26:54 | bauzas | stephenfin: the point is, if someone writes a DB migration in Xena and then we need to backport it in Wallaby but not in Victoria, how to do it ? | |
| 09:27:17 | bauzas | surely, it's a corner case, but that's why we have placeholders for every release | |
| 09:27:30 | stephenfin | Hmm, that's a good point | |
| 09:28:03 | stephenfin | I really doubt we'd want/need them but you can add them if you think it's a good idea? | |
| 09:28:09 | bauzas | I can try | |
| 09:28:17 | stephenfin | They'll all be deleted when we switch to alembic in Xena anyway | |
| 09:28:19 | bauzas | it's maybe just a bikeshed | |
| 09:28:35 | bauzas | but in case we have issues, then... | |
| 09:29:05 | bauzas | stephenfin: have we already agreed on accepting to use alembic for nova db migrations in Xena ? :D | |
| 09:29:22 | stephenfin | I'm going on the assumption that we have, yes :D | |
| 09:29:35 | openstackgerrit | Merged openstack/nova stable/victoria: Add regression test for bug 1914777 https://review.opendev.org/c/openstack/nova/+/777121 | |
| 09:29:37 | openstack | bug 1914777 in OpenStack Compute (nova) victoria "Possible race condition between n-cpu and n-api when deleting a building instance" [High,In progress] https://launchpad.net/bugs/1914777 - Assigned to melanie witt (melwitt) | |
| 09:29:43 | stephenfin | If not, I'm core on sqlalchemy-migrate. It'd be a shame if something were to...happen to it | |
| 09:29:57 | stephenfin | "Nice DB migration library you got there..." | |
| 09:30:10 | stephenfin | 😈 | |
| 09:30:21 | bauzas | :) | |
| 09:30:38 | bauzas | at least we should remember why we had concerns before | |
| 09:30:45 | stephenfin | http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020638.html | |
| 09:31:04 | bauzas | stephenfin: add this to the xena ptg etherpad if you haven't done it already :) | |
| 09:31:14 | stephenfin | link? | |
| 09:31:17 | bauzas | sec | |
| 09:31:28 | bauzas | https://etherpad.opendev.org/p/nova-xena-ptg | |
| 09:31:42 | bauzas | haven't you went to the nova meeting yesterday ? maaaaaan | |
| 09:31:52 | stephenfin | I have a conflict :-( | |
| 09:31:56 | bauzas | ;) | |
| 09:31:59 | stephenfin | NDSU mentoring meeting | |
| 09:32:25 | bauzas | what's NSDU ? | |
| 09:33:10 | stephenfin | North Dakota State University | |
| 09:33:12 | stephenfin | Students | |
| 09:34:51 | bauzas | hah | |
| 09:35:06 | bauzas | I'd love to help some students here | |
| 09:35:20 | bauzas | but... | |
| 09:35:49 | bauzas | french universities don't like to have remote mentors | |
| 09:36:53 | stephenfin | this year could be your year | |
| 09:36:58 | stephenfin | it's all remote now :-\ | |
| 09:37:30 | bauzas | meh | |
| 09:37:38 | bauzas | it's... difficult, trust me | |
| 09:37:43 | bauzas | even for our company :) | |
| 10:09:43 | stephenfin | gibi: URI qemu:///system does not support events: module 'nova.tests.unit.virt.libvirt.fakelibvirt' has no attribute 'VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED' | |
| 10:09:48 | stephenfin | I'm seeing that in unit tests | |
| 10:10:00 | stephenfin | Looks like you might want to add stuff to fakelibvirt, else mute the warning? | |
| 10:44:17 | openstackgerrit | Lee Yarwood proposed openstack/nova master: nova-next: Start testing the q35 machine type https://review.opendev.org/c/openstack/nova/+/708701 | |
| 10:44:43 | lyarwood | ^ gibi / kashyap / stephenfin ; this should be ready for reviews now, just had a quick nit to resolve. | |
| 10:45:35 | kashyap | lyarwood: Thanks for the sustained effort on it; will look today | |
| 10:51:02 | gibi | bauzas, stephenfin: I think we only need to add the wallaby db placeholders at the start of X as in X we need to differentiate between db changes that are X only from db changes that are backported to W | |
| 10:52:02 | gibi | stephenfin: regarding the unit test logs I will look into it | |
| 10:53:57 | stephenfin | lyarwood: cool | |
| 10:58:09 | kashyap | lyarwood: If you're respinning, can you please add the comment I noted in the review? It'll be useful for those who don't dwell on machine types | |
| 10:59:49 | kashyap | I don't insist; just that it avoids some head-scratching; I added my +1 regardless, FWIW. | |
| 11:01:33 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Add support for virtio-based input devices https://review.opendev.org/c/openstack/nova/+/756552 | |
| 11:01:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Stop mocking 'nova.virt.libvirt.utils.get_arch' https://review.opendev.org/c/openstack/nova/+/778879 | |
| 11:02:22 | openstackgerrit | Merged openstack/nova stable/victoria: Handle instance = None in _local_delete_cleanup https://review.opendev.org/c/openstack/nova/+/777209 | |
| 11:05:32 | stephenfin | lyarwood, gibi: Mind taking at look at ^ again? Took a while but I got the mocking nailed down eventually | |
| 11:09:34 | lyarwood | kashyap: ack I did note something about that in the commit but if I respin I'll add a note | |
| 11:09:37 | lyarwood | stephenfin: ack looking | |