| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-27 | |||
| 19:04:33 | donnyd | If I could speed up the downloading part, i wouldn't even notice | |
| 19:04:53 | artom | Which is why I wanted to strongly to avoid touching them altogether | |
| 19:04:54 | donnyd | because a new image would be downloaded in 30-40 seconds | |
| 19:05:04 | dansmith | artom: we stash a copy of the requests to be applied int he migration context with other things.. if the sriov migration code is not playing nice with that, then it's wrong | |
| 19:05:08 | sean-k-mooney | artom: we wont clobber anything | |
| 19:05:18 | sean-k-mooney | but we will end up trying to claim pci device twice | |
| 19:05:42 | artom | sean-k-mooney, well no, dansmith's point was that we still skip the actual claiming (right?) | |
| 19:05:49 | artom | Just don't mess with any DB stuff | |
| 19:05:54 | sean-k-mooney | and then whe might not move them form claimed to allocated correctly and leak pci deivces | |
| 19:06:03 | dansmith | artom: I don't think I made that claim | |
| 19:06:05 | sean-k-mooney | e.g. once that are claimed for the instance but not allocated to it | |
| 19:06:07 | dansmith | artom: I might have asked that | |
| 19:06:19 | mriedem | "I don't think I made that claim" | |
| 19:06:25 | artom | We need to stop overloading "claim" >_< | |
| 19:06:39 | dansmith | I still don't understand why this needs to be different for cold and live migration, with respect to the accounting | |
| 19:06:57 | melwitt | donnyd: right... that does sound strange, but I'm definitely not that knowledgable about what is reasonable to expect there. I'll run those details by penick and find out if it rings a bell | |
| 19:06:58 | sean-k-mooney | claimed and allocated are states in the pci resouce tracker. | |
| 19:07:08 | sean-k-mooney | and claimes generally refers to the RT | |
| 19:07:14 | artom | dansmith, it doesn't :) But SRIOV live migration was implemented without it, so now we're in this mess | |
| 19:07:18 | sean-k-mooney | we have moved to using allocaiton to refer to placmeent | |
| 19:07:30 | dansmith | artom: yeah, sounds like that is the real problem here | |
| 19:08:04 | donnyd | I think the last time we dug into it, it had something to do with the requests library only being able to use one core, which in turn make sense because my controllers cores aren't super fast | |
| 19:08:24 | sean-k-mooney | i can proably make the sriov stuff work with move claims. but i would prefer to keep move claimes for cold migration | |
| 19:08:52 | sean-k-mooney | sriov migration did not use claims because we did not need to and live migration never used them before | |
| 19:09:32 | mriedem | so something was bolted on instead and now we have a mess | |
| 19:09:33 | donnyd | I thought the http.store option in glance would allow compute to grab from a http server (like apache or something), but i have no idea how to configure it | |
| 19:09:36 | mriedem | is the summary yeah? | |
| 19:09:51 | dansmith | rightm | |
| 19:09:52 | dansmith | that's the problem | |
| 19:09:53 | dansmith | "live migration is different so I can be more different" | |
| 19:10:05 | artom | mriedem, I think sean-k-mooney would object to the "bolted on" wording, but yeah | |
| 19:10:13 | sean-k-mooney | when we were first proposing sirov migration we were not planning to use move claime for numa migration | |
| 19:10:39 | artom | It's also on me for not having reviewed that spec/code | |
| 19:11:01 | sean-k-mooney | i am testing your code right now by the way | |
| 19:11:03 | artom | Could have said "hey let's use claims since we'll need them for NUMA LM anyways" | |
| 19:11:58 | sean-k-mooney | well we dont you could alos do the calimes the way we do but anyway should i start looking at how to convert the sriov code | |
| 19:12:08 | dansmith | which cores were reviewing that? | |
| 19:12:11 | dansmith | I don't think it was me, | |
| 19:12:17 | mriedem | jay and stephen | |
| 19:12:21 | dansmith | okay | |
| 19:12:46 | melwitt | donnyd: the last comment confuses me because compute downloads images over glance API (http), so what is the difference between that and the http.store option you mention, I wonder? | |
| 19:15:49 | dansmith | Seems like jay was actually in favor of managing the pci stuff like the rest of it: https://review.opendev.org/#/c/620115/ | |
| 19:16:00 | dansmith | "If we managed PCI devices using the same system we do for CPU, memory and local disk, then we could do the resource *claim* in the scheduler's claim_resources() " | |
| 19:16:29 | dansmith | actually, | |
| 19:16:37 | dansmith | maybe he means more placement-like by that | |
| 19:16:41 | sean-k-mooney | yes | |
| 19:17:24 | sean-k-mooney | he wanted to avoid the move cliams becuae he wante use to use placment as much as possible | |
| 19:17:59 | dansmith | I don't really see anywhere that he said that, but I don't doubt it | |
| 19:19:47 | artom | I really didn't want to start a witch hunt :( | |
| 19:19:52 | sean-k-mooney | that was my preference too by the way so im biased in recolection but i think if we need too i can adapt the sriov code to get the device form the move claim if needed | |
| 19:20:00 | dansmith | artom: no, just looking for context | |
| 19:20:10 | artom | Choices were made, I'm sure at the time with the available info they were optimal | |
| 19:20:14 | artom | dansmith, ack | |
| 19:20:43 | dansmith | artom: this is the problem with bolting on incremental "okay we'll justsupport live migration with pci if they're like this" kind of features | |
| 19:20:48 | dansmith | after you do that a couple times, you end up here | |
| 19:20:51 | donnyd | melwitt: well I am really just guessing because I don't understand how the http.store option works... I was hopeful that I could configure the filestore to save images in a particular location, and then have that exposed directly by apache (or something of the like) https://opendev.org/openstack/glance_store/src/branch/master/glance_store/_drivers/http.py | |
| 19:21:58 | sean-k-mooney | well we evenually wanted to use the multiple port binding workflow for could migration and then move the sriov port claiming to always use the live migration flow when we did that | |
| 19:22:05 | artom | dansmith, isn't there a "law" about that? | |
| 19:22:15 | mriedem | ooo i'm scheduled for that tomorrow | |
| 19:22:19 | mriedem | we're sympatico | |
| 19:22:37 | artom | Products reproduce the communication structures of their organisations, or something like that? | |
| 19:22:53 | donnyd | melwitt: but i honestly don't know because I have never had to get that far into glance... in my normal uses cases... it just works | |
| 19:22:55 | artom | So us, being distributed, loose and non-cohesive, will produce features that are the same? | |
| 19:23:48 | artom | Aha, https://en.wikipedia.org/wiki/Conway%27s_law | |
| 19:24:10 | mriedem | "organizations that are a clusterfuck will produce software that is a clusterfuck" | |
| 19:24:11 | mriedem | got it | |
| 19:24:32 | artom | I mean, yes. :P | |
| 19:25:28 | sean-k-mooney | fyi i just migrated an instace with a numa toployg and an sriov macvtap port | |
| 19:25:33 | melwitt | donnyd: understood. I'm grepping around and also can't find the config option in glance about how to disable checksum verification for the image_cache. I'll need to chat with penick and get back to you. I'll ask him if he knows anything about http.store while I'm at it | |
| 19:25:38 | sean-k-mooney | i need to now look at the xml and see what happend | |
| 19:25:44 | sean-k-mooney | but it succeded | |
| 19:26:08 | donnyd | melwitt: much appreciate... or just any possible way to make it faster would be a massive help | |
| 19:26:14 | mriedem | melwitt: that stuff should be disabled by default in nova | |
| 19:26:15 | mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#glance.verify_glance_signatures | |
| 19:26:32 | melwitt | mriedem: yeah, he said it's different than the signature verify | |
| 19:26:48 | melwitt | and I just can't find it anywhere | |
| 19:26:57 | mriedem | is this based on his ocata cloud? | |
| 19:27:10 | melwitt | it might be | |
| 19:27:14 | mriedem | https://docs.openstack.org/nova/pike/configuration/config.html#libvirt.checksum_base_images | |
| 19:27:21 | mriedem | that stuff has been removed | |
| 19:27:30 | melwitt | oh, dangit | |
| 19:27:38 | mriedem | dagnabbit even | |
| 19:28:03 | melwitt | yeah, for sure | |
| 19:28:40 | melwitt | there goes that lead :( | |
| 19:28:41 | sean-k-mooney | artom: the xml looks correct and it was updated | |
| 19:29:16 | donnyd | what would be real slick is to just mount the same share for glance filestore backend and nova image cache | |
| 19:29:23 | mriedem | make way | |
| 19:29:35 | donnyd | that way once the image is uploaded the hypervisor would have immediate access | |
| 19:30:37 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image https://review.opendev.org/637605 | |
| 19:30:38 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add power_on kwarg to ComputeDriver.spawn() method https://review.opendev.org/642590 | |
| 19:30:38 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor ComputeManager.remove_volume_connection https://review.opendev.org/642183 | |
| 19:30:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.opendev.org/614037 | |
| 19:30:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.opendev.org/614035 | |
| 19:30:40 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method https://review.opendev.org/633293 | |
| 19:30:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.opendev.org/634832 | |
| 19:30:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask https://review.opendev.org/627890 | |
| 19:30:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method https://review.opendev.org/635080 | |
| 19:30:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.opendev.org/627891 | |
| 19:30:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.opendev.org/635668 | |
| 19:30:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask https://review.opendev.org/635646 | |