| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-10 | |||
| 13:21:53 | aspiers | kashyap: oh, I thought you were talking about cpuid inside the guest | |
| 13:22:09 | mriedem | gibi: yeah i saw you updated but i haven't looked at your replies or changes, | |
| 13:22:13 | aspiers | kashyap: I think there are multiple binaries with SB | |
| 13:22:17 | mriedem | it kind of sucks that we don't store the information needed for revert in the migration context | |
| 13:22:41 | gibi | mriedem: I have better plans, described in my reply | |
| 13:22:56 | gibi | mriedem: we can let neutron remember the mapping | |
| 13:23:09 | kashyap | aspiers: Sorry, I was not clear. I don't _know_ if it is also reported in the guest `cpuid` -- maybe you can tell from your hardware? | |
| 13:23:10 | gibi | mriedem: by using multiple portbinding | |
| 13:23:27 | aspiers | kashyap: I don't have a guest with cpuid installed currently | |
| 13:23:46 | aspiers | kashyap: I think I misremembered - before I probably ran it on the host | |
| 13:24:19 | kashyap | aspiers: For SB -- IMHO, you don't _need_ all those 4M, 2M variants -- it's beyond overkill. Your life, and admins life will be far more simpler, if you used the "two pairs" approach I noted in the comment | |
| 13:24:43 | aspiers | kashyap: You're preaching to the choir. You really need to file a bug report on bugzilla.opensuse.org | |
| 13:24:47 | mriedem | gibi: ah yeah | |
| 13:24:53 | sean-k-mooney | artom: i have confirmed that the migration context correctly contains the new numa toplogy blob but it does not get saved to the db | |
| 13:25:02 | aspiers | kashyap: or reach the right team in some other way | |
| 13:25:07 | kashyap | aspiers: Aaah, sorry; didn't realize we're on the same line, same word | |
| 13:25:23 | openstackgerrit | Adam Spiers proposed openstack/nova master: Improve SEV documentation and other minor tweaks https://review.opendev.org/681254 | |
| 13:25:50 | aspiers | efried, gibi, stephenfin: there's the follow-up ^^^ | |
| 13:25:58 | sean-k-mooney | artom: so do ing apply migration context followed by drop to delete it form the db does not work | |
| 13:26:23 | sean-k-mooney | artom: i would assume that for some reason the filed is not flaged as dirty and is not being saved | |
| 13:26:37 | gibi | aspiers: ack! | |
| 13:27:59 | Sundar | Hi sean-k-mooney, how are you doing? | |
| 13:28:07 | sean-k-mooney | Sundar: hi | |
| 13:28:11 | efried | luyao: I'm trying to make sense of those scenarios right now. | |
| 13:28:34 | Sundar | sean-k-mooney: The Cyborg patches for nova-integ have merged; https://review.opendev.org/#/q/project:openstack/cyborg+branch:master+topic:nova-integ+owner:Sundar | |
| 13:28:58 | sean-k-mooney | Sundar: ok so its just the nova code that is pending | |
| 13:29:07 | Sundar | There is one related patch for Nova notification https://review.opendev.org/674520 that is close | |
| 13:29:24 | aspiers | kashyap: Where did you hear that an SEV-enabled kernel would fail to boot if SEV is not provided? I have empirically proven that false by booting the same image both with and without SEV | |
| 13:29:58 | sean-k-mooney | Sundar: ok yes without that nova will timeout waiting and rollback the spawn | |
| 13:30:16 | mriedem | Sundar: no testing at all for those cyborg patches? | |
| 13:30:18 | kashyap | aspiers: DanPB; but to be fair to him, he used a qualifier "IIUC". If you've tested it, I'll go with your data | |
| 13:30:18 | Sundar | Yes. BTW, I updated the Nova patches too. Notification works with 674520. If you look at the seeming UT failures, they are mostly unrelated to Cyborg | |
| 13:30:28 | aspiers | kashyap: gotcha | |
| 13:30:44 | Sundar | mriedem: What do you mean by 'no testing'? | |
| 13:30:53 | sean-k-mooney | Sundar: have we had a end to end run of the tempest job | |
| 13:31:09 | mriedem | Sundar: there are no tests associated with that patch | |
| 13:31:12 | Sundar | The tempest CI is working with the patches, and we are hoping to merge it by this week. | |
| 13:31:17 | efried | mriedem: do we use "migration instance" for evac? | |
| 13:31:23 | mriedem | efried: no | |
| 13:31:29 | mriedem | efried: you mean migratoin-based allocations? | |
| 13:31:29 | efried | so what happens to allocations? | |
| 13:31:34 | mriedem | efried: they sit on the source | |
| 13:31:34 | efried | yeah | |
| 13:31:56 | efried | thanks | |
| 13:31:58 | mriedem | which is why we can't delete resource providers when deleting a compute service that has evacuated instance allocations against it | |
| 13:32:03 | mriedem | that whole thread i had in the ML | |
| 13:32:31 | mriedem | efried: related to this https://review.opendev.org/#/c/678100/ | |
| 13:32:41 | sean-k-mooney | mriedem: that has been changed recently right. we delete the allcoation if they exisits if and only if you bring back up the compute agent on the failed host | |
| 13:32:55 | sean-k-mooney | so that only help if you repair whatever the issue was | |
| 13:33:06 | mriedem | sean-k-mooney: yes the source allocations are deleted if you bring up the evacuated-from compute service | |
| 13:33:10 | sean-k-mooney | if you dont you get into the stiutation in that ml thread | |
| 13:33:42 | mriedem | i don't think it's probably uncommon to have a host failure, down the compute service, evacuate it, and then try to delete the comptue service before redeploying on that host | |
| 13:34:40 | mriedem | Sundar: tempest only covers happy path testing for the most part; unit tests are good for testing error conditions and such - exceptional cases | |
| 13:34:53 | mriedem | anyway, that's up to the cyborg core team for how they want to enforce testing standards | |
| 13:35:04 | efried | mriedem: also rebuild? | |
| 13:35:10 | mriedem | efried: a rebuild isn't a migration | |
| 13:35:17 | mriedem | and you can't rebuild on a down host | |
| 13:35:53 | mriedem | so the host stays the same and the flavor stays the same, but the image might change on a rebuild | |
| 13:36:00 | sean-k-mooney | Sundar: have the depencies been fixed in https://review.opendev.org/#/c/670999/ | |
| 13:36:12 | efried | mriedem: right, separate topic, I'm saying: does rebuild also end up with multiple sets of allocations for the same instance uuid? | |
| 13:36:28 | sean-k-mooney | Sundar: you rebased it but did not run the new job via "check experimental" | |
| 13:36:40 | sean-k-mooney | so we still dont have a run that worked | |
| 13:37:03 | sean-k-mooney | the previous run failded https://c3308e17743765936b80-6c7fec3fffbf24afb7394804bcdecfae.ssl.cf2.rackcdn.com/670999/7/experimental/cyborg-tempest/2fe52ec/testr_results.html.gz | |
| 13:37:12 | mriedem | efried: no | |
| 13:37:17 | sean-k-mooney | that said we knew the depencies were wrong so we expected that | |
| 13:38:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fixing broken links https://review.opendev.org/681206 | |
| 13:39:31 | luyao | mriedem: what about in the process of rebuild | |
| 13:40:08 | luyao | mriedem: rebuild is not done, how many allocations will an instance have? | |
| 13:40:10 | efried | we destroy the instance but keep the allocations, then respawn it with the existing allocations? | |
| 13:40:23 | efried | ...and maybe a different image? | |
| 13:40:30 | sean-k-mooney | Sundar: it looks like there are 3 patches still remaining on the cyborg side. 2 for nova integration and 1 for python3 support | |
| 13:41:28 | mriedem | luyao: i don't understand your question | |
| 13:41:36 | mriedem | efried: correct | |
| 13:41:49 | efried | mriedem: luyao and I are asking about the same thing. Thanks for the help. | |
| 13:42:14 | mriedem | rebuild is basically re-spawn in place on the same host with maybe a different image but the same ports/volumes/flavor - if you're on shared storage you keep your root disk, if not your root disk is rebuilt from the specified image | |
| 13:42:28 | Sundar | sean-k-mooney: I'll rerun with 'check experimental'. Re. dependencies for https://review.opendev.org/#/c/670999/, one has already merged, and the other is the tempest code itself, which is working with the patches and should merge soon. There should be only 1 for nova-integ (i.e. Nova notification) -- I fixed the topic now. | |
| 13:42:44 | sean-k-mooney | Sundar: wait a minute | |
| 13:42:51 | mriedem | though i might be thinking of evacuate for that root disk comment | |
| 13:42:56 | sean-k-mooney | im going to fix it to not waste gate resouces and fix the depencies | |
| 13:43:00 | mriedem | anyway, root disk doesn't matter for what you're asking | |
| 13:43:22 | mriedem | sean-k-mooney: cyborg integration isn't happening in nova in train - where are we with test runs on the numa live migration series? | |
| 13:43:44 | efried | stephenfin, dansmith: btw, vpmem now has a CI passing. I haven't opened one up yet, but it's taking half an hour, so it's doing... *something* :P | |
| 13:43:53 | luyao | mriedem: I saw an instance have new and old allocations in an functional test when it's in evacuating. | |
| 13:43:55 | sean-k-mooney | mriedem: i confrimed that it is not saving the updated numa toplogy so it looks like we need instance.refresh | |
| 13:44:05 | sean-k-mooney | mriedem: the migration context has the correct numa toplogy | |
| 13:44:07 | luyao | mriedem: I thought rebuild is similar to evacuate | |
| 13:44:22 | sean-k-mooney | mriedem: so it looks liek its not marking the field as changed for some reason | |
| 13:44:36 | sean-k-mooney | mriedem: perhapes because apply migration context uses setattr | |
| 13:44:45 | mriedem | luyao: evacuate is rebuild to another host when the source host is down | |
| 13:44:49 | sean-k-mooney | mriedem: i -1'd the patch | |
| 13:45:06 | mriedem | evacuate and rebuild use the same code flows in conductor and compute services with conditionals for any differences | |
| 13:45:08 | efried | luyao, mriedem: I find all of it so confusing that I don't even try to remember, because that just makes things worse. It goes against the very fiber of my being, but this is one where I'll ask. Every. Time. | |
| 13:45:15 | mriedem | e.g. evacuate does a claim on the dest host, rebuild does not | |
| 13:45:58 | mriedem | sean-k-mooney: ok i didn't realize artom removed the instance.refresh in post live migration | |
| 13:46:03 | mriedem | but haven't looked at changes since yesterday | |
| 13:46:29 | mriedem | efried: rebuild+evacuate and resize+cold-migrate are confusing in that they share code but don't do the exact same things | |
| 13:46:40 | mriedem | but are like 90% the same | |
| 13:47:37 | mriedem | efried: do we need a "what's the difference between evacuate and rebuild" thing in the contributor docs? | |
| 13:47:41 | mriedem | b/c that's probably pretty easy to write up | |
| 13:47:58 | efried | that must be why luyao was conflating rebuild+evacuate wrt allocations. That must fall in the 10% that's different. | |