| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-11 | |||
| 16:40:10 | bauzas | see you folks tomorrow morning | |
| 16:41:11 | openstackgerrit | Alex Xu proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup https://review.opendev.org/678455 | |
| 16:41:12 | openstackgerrit | Alex Xu proposed openstack/nova master: Parse vpmem related flavor extra spec https://review.opendev.org/678456 | |
| 16:41:12 | openstackgerrit | Alex Xu proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | |
| 16:41:13 | openstackgerrit | Alex Xu proposed openstack/nova master: Add functional tests for virtual persistent memory https://review.opendev.org/678470 | |
| 16:41:13 | openstackgerrit | Alex Xu proposed openstack/nova master: objects: use all_things_equal from objects.base https://review.opendev.org/681397 | |
| 16:42:01 | alex_xu | sean-k-mooney: actually, it is all about this little bit https://review.opendev.org/#/c/678456/32/nova/virt/hardware.py@1688 | |
| 16:43:35 | sean-k-mooney | alex_xu: ya that shoudl cause nodes to be 1 | |
| 16:44:17 | sean-k-mooney | and i think we will take this brance when creating the numa toplgoy right https://review.opendev.org/#/c/678456/32/nova/virt/hardware.py@1742 | |
| 16:45:34 | sean-k-mooney | well we will also call numa_topology = _get_numa_topology_auto(...) | |
| 16:47:15 | sean-k-mooney | ya | |
| 16:47:19 | alex_xu | sean-k-mooney: yes, it is go to the _get_nuam_toplogy_auto | |
| 16:47:23 | sean-k-mooney | the singel numa toplogy is created by https://review.opendev.org/#/c/678456/32/nova/virt/hardware.py@1514 | |
| 16:47:42 | sean-k-mooney | which is exactly what we want | |
| 16:47:53 | alex_xu | yea | |
| 16:53:15 | stephenfin | efried: What do you think about https://review.opendev.org/#/c/678455/25/nova/virt/libvirt/driver.py@1299 ? | |
| 16:54:12 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: NUMA live migration support https://review.opendev.org/634606 | |
| 16:54:12 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration https://review.opendev.org/640021 | |
| 16:54:13 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595 | |
| 16:54:18 | artom | dansmith, mriedem ^^ | |
| 16:54:19 | efried | stephenfin: It is critical that a dirty pmem not be allowed to be assigned to another guest | |
| 16:55:16 | efried | sorry, you're asking whether we should continue cleaning up wait until the end to fail out? | |
| 16:55:32 | stephenfin | yup | |
| 16:55:40 | efried | this is by no means the only way we can fail out of a destroy() early | |
| 16:55:43 | efried | is it? | |
| 16:55:50 | stephenfin | That's what I'm trying to figure out | |
| 16:56:05 | stephenfin | _unplug_vifs is called with ignore_errors=True | |
| 16:56:19 | efried | looking at L1207-30 | |
| 16:56:34 | efried | anything that's not LibvirtException will cause immediate failure. | |
| 16:56:47 | efried | and even some LibvirtErrorZ | |
| 16:56:57 | efried | so this is far from unprecedented it would seem. | |
| 16:57:11 | stephenfin | good point. If it's broken, it was already broken | |
| 16:57:30 | efried | stephenfin: likewise below when cleaning up bdms | |
| 16:57:31 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix race in _test_live_migration_force_complete https://review.opendev.org/681540 | |
| 16:57:33 | mriedem | efried: dansmith: ^ this fixes a gate bug | |
| 16:57:37 | mriedem | which reset us earlier | |
| 16:57:51 | sean-k-mooney | stephenfin: its called where | |
| 16:58:21 | sean-k-mooney | oh in distroy | |
| 16:58:26 | stephenfin | https://review.opendev.org/#/c/678455/25/nova/virt/libvirt/driver.py@1196 | |
| 16:58:28 | stephenfin | sean-k-mooney: ^ | |
| 16:58:44 | stephenfin | heh | |
| 16:59:55 | gibi_bus | stephenfin: yepp, try to get to Athlone | |
| 17:00:13 | sean-k-mooney | so there is a diffreenc between pmens and vifs | |
| 17:00:22 | efried | mriedem: qualified +2 | |
| 17:00:42 | sean-k-mooney | pmem namespace contain user data which we want to be very careful with ensuring it is earased | |
| 17:01:03 | sean-k-mooney | we shoudl clean up vifs but if noting is plug into it it has little effect | |
| 17:01:12 | sean-k-mooney | ideal neighter would fail | |
| 17:01:28 | sean-k-mooney | but we shoudl make sure we dont skip the rest of the cleanup if they do | |
| 17:01:30 | dansmith | artom: -1 | |
| 17:01:45 | artom | For the sheep joke or the code? | |
| 17:02:18 | dansmith | artom: for the plethora of typos | |
| 17:02:33 | sean-k-mooney | stephenfin: efried so i think we shoudl catch the VPMEMCleanupFailed excetion or put the rest of the cleanup into a finally block | |
| 17:02:37 | dansmith | I assumed you were spending lots of time wordsmithing and checking all that stuff | |
| 17:02:49 | sean-k-mooney | stephenfin: efried or do the pmem cleanup last | |
| 17:03:00 | sean-k-mooney | actullly no we want to always try it | |
| 17:03:05 | stephenfin | sean-k-mooney: You see what efried said about us already failing hard on other things though? | |
| 17:03:28 | sean-k-mooney | yes and that has bit us over and over again | |
| 17:03:49 | sean-k-mooney | this is one of the places we leak resouces on failed migrations | |
| 17:04:22 | efried | seems like yet another thing we shouldn't start fixing as part of this series, but should do later when we can be more complete | |
| 17:04:40 | sean-k-mooney | i guess | |
| 17:04:51 | artom | dansmith, clearly not enough :( | |
| 17:05:00 | efried | f man, if we try to fix everything ugly we notice or touch or get near, we'll never get anything done anywhere ever. | |
| 17:05:01 | sean-k-mooney | what is the behavior when we raise we go to error then we clean it up later? | |
| 17:05:11 | sean-k-mooney | i assume that woudl jsut fail again | |
| 17:05:20 | dansmith | artom: fix those quick so I can apply my +2 and we can move on | |
| 17:05:33 | mriedem | artom: https://review.opendev.org/#/c/634606/85 | |
| 17:05:38 | mriedem | FUP needed but won't block | |
| 17:06:09 | artom | Ack, thanks | |
| 17:07:48 | sean-k-mooney | efried: ok so we will just leave it raise. | |
| 17:08:17 | efried | ++ | |
| 17:08:25 | sean-k-mooney | efried: its not that i want to fix every ugly thing we see. i just dont want use adding more | |
| 17:08:46 | efried | Yeah, I understand, but in this case it would entail reengineering the whole exception flow. | |
| 17:08:51 | sean-k-mooney | as long as we dont return the namesapce to the pool of alocatable ones then its fine | |
| 17:08:57 | sean-k-mooney | if we did it woudl be a CVE | |
| 17:09:01 | efried | right, IIUC we're careful enough about that. | |
| 17:09:16 | sean-k-mooney | even if you delete the vm | |
| 17:09:18 | efried | I think in all cases where we could have leaked a pmem, we scrub it. | |
| 17:09:52 | mriedem | now that sean jinxed it | |
| 17:10:25 | sean-k-mooney | if only the hardware did secure erase.... | |
| 17:10:37 | efried | what was that unix util called? | |
| 17:10:41 | efried | shred | |
| 17:10:47 | sean-k-mooney | dban? | |
| 17:10:59 | sean-k-mooney | shred is proably a thing too | |
| 17:11:24 | efried | shred exists on my ubuntu. dban doesn't. | |
| 17:11:47 | mriedem | shred is used in the lvm image backend code | |
| 17:11:52 | artom | dansmith, you made me enable spell check for vim | |
| 17:11:55 | sean-k-mooney | dban is not a utility its a thing you boot https://dban.org/ | |
| 17:12:32 | efried | anyway, I would think a VM wouldn't be able to go looking at a low enough level to recover regular-deleted data. | |
| 17:12:47 | sean-k-mooney | efried: of couse it can | |
| 17:12:49 | efried | thought you needed specialized hardware for that | |
| 17:12:59 | sean-k-mooney | its directly mapped into the guests adress speace | |
| 17:13:04 | sean-k-mooney | it is byte adressable | |
| 17:13:14 | efried | right, but if it's deleted | |
| 17:13:16 | sean-k-mooney | that is why we need to 0 it out | |
| 17:13:46 | efried | isn't that what daxio -z does?? | |
| 17:13:58 | sean-k-mooney | efried: there is no filesystem so to delete it you have to write over it | |
| 17:14:00 | sean-k-mooney | and yes | |
| 17:14:06 | dansmith | artom: no, you did. | |
| 17:14:15 | artom | dansmith, that was a good thing :) | |
| 17:14:41 | dansmith | efried: uh what? | |