| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-28 | |||
| 13:36:32 | sean-k-mooney | artom: well you could start by adding it to the experimental pipeline | |
| 13:36:52 | sean-k-mooney | that requires a comment ot trigger it | |
| 13:36:58 | artom | sean-k-mooney, wouldn't I have to start by moving whitebox to the openstack namespace? | |
| 13:37:08 | sean-k-mooney | yes | |
| 13:37:15 | sean-k-mooney | i ment after that | |
| 13:37:27 | sean-k-mooney | and you know after we write all the tests | |
| 13:37:41 | artom | Hey, I have some written already :) | |
| 13:38:04 | sean-k-mooney | did you look at the list i gave in the email | |
| 13:38:17 | artom | Yeah | |
| 13:38:19 | sean-k-mooney | feel free to automate those while i test them | |
| 13:38:52 | artom | sean-k-mooney, that'll kinda depend on whether the absence of func tests is a blocker for mriedem | |
| 13:39:04 | artom | If it is, those take priority before FF | |
| 13:39:15 | sean-k-mooney | yep | |
| 13:39:16 | artom | Other than that timing issue, I'll happily hax those up | |
| 13:43:03 | sean-k-mooney | artom: once you get the first functional test working i suspect the rest will be fairly staight forward | |
| 13:44:05 | sean-k-mooney | if you can add the functional test as a follow up that will reduce churn and make my testing eaisier | |
| 13:44:42 | artom | sean-k-mooney, yep, good idea. I can start with the tests in a new patch on top, and we can squash later | |
| 13:45:04 | sean-k-mooney | yep or even just keep it on top | |
| 13:45:46 | sean-k-mooney | squashing them to keep them with the feature code is nice but i dont think its required | |
| 14:00:24 | sean-k-mooney | mnaser: by they way do you use kvm on ubuntu 18.04 for vexhost? | |
| 14:00:33 | mnaser | sean-k-mooney: centos 7 | |
| 14:00:38 | sean-k-mooney | ah ok | |
| 14:00:46 | mnaser | but we've managed to get the perfect combo for functional nested virt | |
| 14:00:51 | mnaser | thanks a lot to the kata containers folks | |
| 14:00:57 | sean-k-mooney | there is a bug in the ubunut 18.04 kernel that breaks nested virt | |
| 14:01:18 | sean-k-mooney | i had to replace mine with the mainline 4.20 kernel to fix issues | |
| 14:01:19 | mnaser | yeah our ci is worked out pretty often both by magnum and kata and their nested virt has been super stable | |
| 14:02:46 | sean-k-mooney | ya i went with ubuntu 18.04 because kolla-ansibel source deploys work better on ubuntu then centos in my previos experience | |
| 14:03:02 | sean-k-mooney | mnaser: do ye use openstack ansible for your production installation | |
| 14:03:07 | mnaser | sean-k-mooney: yeah | |
| 14:03:56 | sean-k-mooney | i must get around to checking that out at some point but since its working im not going to touch it untill i need to upgrade :) | |
| 14:10:04 | mdbooth | bauzas: Do you recall why rebuild is forced to be on the same host? Is it just because the way rebuild is written doesn't allow for a migration, or is it fundamental to the intent of rebuild? Or some other reason? | |
| 14:12:10 | sean-k-mooney | mdbooth: why would a rebuild ever need to migrate if the image does not change | |
| 14:12:21 | sean-k-mooney | if the image does change we hit the schulder | |
| 14:12:25 | mdbooth | bauzas: I ask because I was looking at the attach/detach root volume patches. They look fine, but shelve/unshelve is just distasteful, so having it as the only available option seems undesirable. Reading the original spec, it sounds to me like they want to rebuild on a different host... | |
| 14:12:30 | mdbooth | Which is an evacuate# | |
| 14:12:42 | mdbooth | Evacuate with a different root volume. | |
| 14:12:43 | mdbooth | Hmm | |
| 14:13:06 | mnaser | sean-k-mooney: be happy to help you get setup on OSA | |
| 14:13:19 | mdbooth | sean-k-mooney: Yes, we hit the scheduler but constrain it to return the original host or return NoValidHost | |
| 14:14:01 | mdbooth | Rebuild on a different host with a different root volume... | |
| 14:14:02 | sean-k-mooney | mnaser: its more just i have not taken the time to try it out. i learn how kolla ansible worked a few years ago and it served well enough for my personal needs | |
| 14:14:20 | sean-k-mooney | mdbooth: ah ok | |
| 14:14:30 | bauzas | mdbooth: because it doesn't ask for other resources, just for an image | |
| 14:14:43 | bauzas | unless you evacuate :) | |
| 14:15:05 | mdbooth | bauzas: Given that evacuate exists, I'm guessing most of this is already wired up. | |
| 14:15:55 | bauzas | yeah, that's actually the change in between evacuate and rebuild | |
| 14:16:09 | mdbooth | In fact: cold migrate with a different root volume. | |
| 14:16:10 | bauzas | evacuate is just the same rebuild path but with a different API action :) | |
| 14:16:18 | bauzas | hum no | |
| 14:16:23 | bauzas | cold migrate != evacuate | |
| 14:16:32 | bauzas | evacuate == rebuild(different_host) | |
| 14:16:47 | mdbooth | bauzas: Hmm, no. It doesn't consider a change of image. | |
| 14:17:14 | mdbooth | But I was thinking that you might have a root volume and ephemeral disks, in which case you might want to migrate them. | |
| 14:19:38 | mdbooth | bauzas: So, going back to the original question, why do we go to lengths to ensure that rebuild doesn't change host? | |
| 14:20:04 | bauzas | mdbooth: I'm not the rebuild specialist | |
| 14:20:30 | bauzas | some others hitted so many times issues with rebuild that they're more knowledgable than me, like... mriedem ? :) | |
| 14:20:38 | mdbooth | bauzas: Ah, sorry. I pinged you because you were co-author in a patch which added this constraint. | |
| 14:20:47 | mdbooth | mriedem was the other author. | |
| 14:20:52 | bauzas | knowledgable * (not sure it's a word) | |
| 14:21:44 | mdbooth | bauzas: Yeah, it's a word. Pretty sure that's not how you spell it, but I wouldn't have called you on it as I'm not 100% sure myself ;) | |
| 14:23:03 | mriedem | what is the question? | |
| 14:23:12 | mriedem | why doesn't rebuild migrate? | |
| 14:23:21 | mriedem | why do we validate the image against the scheduler when it changes? | |
| 14:23:28 | mriedem | because build is on the same host | |
| 14:23:37 | mriedem | why? because that's the way it originated | |
| 14:23:41 | mdbooth | mriedem: Yeah. Specifically why does it go to length to prevent a change of host even when calling the scheduler? | |
| 14:23:52 | mdbooth | mriedem: Right, that was my first guess. | |
| 14:24:04 | mriedem | then over the years it turned out it was buggy and to maintain the api contract but resolve those bugs we had to build stuff into it | |
| 14:24:11 | mdbooth | So there's no fundamental reason that it shouldn't, it's just that the control flow doesn't support it. | |
| 14:24:25 | mriedem | i wasn't around when rebuild was added | |
| 14:24:26 | sean-k-mooney | mdbooth: it calls the schudler to validate the current host is still valide as changeing the image means the image metadata could invaldate the current host | |
| 14:24:42 | mriedem | fwiw, | |
| 14:25:01 | mriedem | it's also got a latent bug where the image can change numa topology which does not send new numa topology limits to the compute service to do a resource claim | |
| 14:25:03 | sean-k-mooney | mdbooth: you can rebuild instance with sriov | |
| 14:25:18 | mriedem | because rebuild doesn't do any resource claim, because it was assumed that the flavor never changed (because it doesn't in the api) and the scheduler wouldn't be used | |
| 14:25:23 | sean-k-mooney | actully it cold migration ignore that | |
| 14:25:30 | mriedem | so over time we added more features to nova (numa support) which broke existing flows | |
| 14:25:54 | mdbooth | mriedem: Going up a level, I was just considering the shelve/unshelve approach for attach/detach root volume. While that's obviously great if it allows the user to do something which is currently impossible, it still requires shelve/unshelve. I was considering if it could also be added to some other less nasty operation. | |
| 14:26:16 | mriedem | what is nasty about shelve/unshelve? | |
| 14:26:31 | mriedem | i don't see how that's any nastier than trying to make rebuild trigger a migration | |
| 14:26:54 | mriedem | lots of people actually want the stop operation to be a shelve | |
| 14:26:55 | mdbooth | I specifically don't like the way shelve/unshelve abuses the image cache. | |
| 14:27:48 | mdbooth | Probably fixing that would be simpler, I guess. | |
| 14:28:00 | sean-k-mooney | thats specifcally for non bvf instances right. bfv instnace should not mess with the image cache | |
| 14:28:40 | mdbooth | However, if all the user is trying to do is DR an instance (from the spec), going via glance isn't the most efficient solution. | |
| 14:29:22 | openstackgerrit | Jim Rollenhagen proposed openstack/nova master: Remove TypeError handling for get_info https://review.openstack.org/640043 | |
| 14:29:22 | openstackgerrit | Jim Rollenhagen proposed openstack/nova master: Fix typo in vmware get_info docstring https://review.openstack.org/640042 | |
| 14:29:52 | mdbooth | That's a point... in the DR case you'd presumably be evacuating anyway, so you still couldn't evacuate with a different root volume. | |
| 14:30:05 | sean-k-mooney | mdbooth: that depend some of the usescase in the spec impliend at least to me that the remote fail over sigt may not have direct conectivity at the hyperviosr level. e.g. a migrtion to the second sight may now actully work | |
| 14:31:15 | sean-k-mooney | *sigt -> site sight->site | |
| 14:32:03 | mdbooth | Shelve/unshelve is *extremely* space inefficient. You're basically storing a flattened snapshot at the point of shelve in the image cache forever. | |
| 14:32:29 | mdbooth | But not for BFV, I guess. | |
| 14:32:40 | sean-k-mooney | mdbooth: doesnt ath depend on your glance backend | |
| 14:33:35 | mdbooth | sean-k-mooney: No. It might depend on Nova's imagebackend, though. | |
| 14:34:23 | sean-k-mooney | ah right. i was thinking it might be effienet on ceph backed glance but may ceph imagebackend would help | |
| 14:34:40 | sean-k-mooney | e.g. for non bfv instnaces | |
| 14:35:07 | mdbooth | I don't think we're at all clever about shelve of ceph instance to ceph glance. | |
| 14:35:30 | mdbooth | And ceph glance to ceph instance is efficient but problematic. | |
| 14:38:04 | efried | gibi, bauzas: Sorry, I meant to mention it in the change set - feel free to squash. | |