| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-05 | |||
| 16:22:46 | mriedem | ack | |
| 16:22:49 | artom | mriedem, ah, yeah, it we never update the port bindings for a failed LM, we don't need to roll them back | |
| 16:23:04 | artom | So, would there be value in a DNM to repro that in the gate? | |
| 16:23:07 | artom | *DNM patch | |
| 16:26:41 | watersj | I'm getting, There are allocations remaining against the source host that might need to be removed | |
| 16:27:39 | sean-k-mooney | artom: if you can repro it in the gate we can use that to test both of mriedem patches so yes i think there is still value | |
| 16:28:01 | sean-k-mooney | although long term a functional regression test would be optimal | |
| 16:28:13 | artom | sean-k-mooney, mriedem was saying we need to fix the Neutron fixture for that | |
| 16:28:24 | sean-k-mooney | proably | |
| 16:28:59 | artom | sean-k-mooney, so what was that thing about the MTU? Removing it should trigger thing? Seems weird | |
| 16:29:05 | sean-k-mooney | it does not really do the multiple port bindings workflow right | |
| 16:29:35 | sean-k-mooney | artom: ya remove it form the port forces libvirt to change form teh 64bit encodeing to the 32bit encoding | |
| 16:29:59 | sean-k-mooney | well adding it forces it | |
| 16:31:04 | sean-k-mooney | anyway the root of that secific libvirt/qemu issue is they ran out of feature bits and adding and removing hte mtu is one way to chagne it | |
| 16:31:28 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Create BDM in API during attach volume https://review.opendev.org/692940 | |
| 16:31:28 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Provide a better error when _verify_response hits a TypeError https://review.opendev.org/693042 | |
| 16:33:30 | mriedem | watersj: ah yeah...so in placement the migration record (id=76) holds the allocations for the source host and the instance holds allocations for the dest host, and you need to move the instance allocations back to the source host and drop the allocations from the dest host, which doesn't sound fun right | |
| 16:33:49 | mriedem | luckily i just wrote a guide on how to do this https://review.opendev.org/#/c/691427/ | |
| 16:34:03 | watersj | lol | |
| 16:34:27 | mriedem | you might want to just skip straight to the heal_allocations part | |
| 16:34:44 | mriedem | i.e. delete the allocations for both the migration and instance consumer and then heal allocations for the instance | |
| 16:34:59 | mriedem | https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-allocation-delete | |
| 16:35:11 | mriedem | you'd pass the migration uuid and instance uuid to ^ to remove the allocations in placement | |
| 16:35:23 | mriedem | then run: nova-manage placemet heal_allocations | |
| 16:35:45 | mriedem | https://docs.openstack.org/nova/stein/cli/nova-manage.html#placement | |
| 16:36:28 | mriedem | if you want to target the specific instance, or do a dry-run heal_allocations, you'd need to setup a train nova instance in a vm/container/virtualenv https://docs.openstack.org/nova/train/cli/nova-manage.html#placement since --dry-run and --instance options aren't in stein | |
| 16:37:02 | watersj | have backup of volume | |
| 16:37:08 | watersj | ready really break it | |
| 16:37:16 | mriedem | i don't know what that means | |
| 16:37:27 | mriedem | oh you mean you do'nt need a dry run | |
| 16:37:52 | mriedem | heal allocations just recreates the missing allocation records in placement for the instance using the flavor data associated with the instance | |
| 16:37:58 | watersj | mean I ok to try without dryrun capabilities | |
| 16:37:59 | mriedem | doesn't touch the guest or anything like that | |
| 16:38:04 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: DNM: break live migration to trigger bug 1788014 https://review.opendev.org/693045 | |
| 16:38:04 | openstack | bug 1788014 in OpenStack Compute (nova) "when live migration fails due to a internal error rollback is not handeled correctly." [Medium,In progress] https://launchpad.net/bugs/1788014 - Assigned to Matt Riedemann (mriedem) | |
| 16:38:32 | watersj | k, ty for the help | |
| 16:39:15 | openstackgerrit | Merged openstack/nova stable/stein: Add integration testing for heal_allocations https://review.opendev.org/692923 | |
| 16:57:10 | mriedem | dansmith: replied in https://review.opendev.org/#/c/635668/ - i'm not sure exactly what you're looking for so i gave some options | |
| 16:57:56 | mriedem | if you want to see a paste of what i'm talking about i can do that quick | |
| 16:59:07 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: DNM: break live migration to trigger bug 1788014 https://review.opendev.org/693045 | |
| 16:59:07 | openstack | bug 1788014 in OpenStack Compute (nova) "when live migration fails due to a internal error rollback is not handeled correctly." [Medium,In progress] https://launchpad.net/bugs/1788014 - Assigned to Matt Riedemann (mriedem) | |
| 17:01:58 | dansmith | mriedem: seems like you get it to me :) | |
| 17:02:14 | mriedem | ok | |
| 17:02:19 | mriedem | updating thusly | |
| 17:23:51 | melwitt | whoa, one of my things merged the first time | |
| 17:26:08 | dansmith | you know | |
| 17:26:22 | dansmith | it has been said that on several providers we're our own noisy neighbor | |
| 17:26:32 | dansmith | if we see a massive drop in the IO-based failures this week, | |
| 17:26:45 | dansmith | that would probably just go to confirm that as a likely problem | |
| 17:26:51 | dansmith | less volume for samples for sure, but.. | |
| 17:29:01 | melwitt | yeah, I was wondering that too. summit/ptg and if suddenly less gate failures | |
| 17:30:56 | mriedem | gate has been smooth has rich creamery butter this week | |
| 17:32:00 | mriedem | https://www.youtube.com/watch?v=3ds0vWfoTwU | |
| 17:33:51 | melwitt | lol | |
| 17:35:10 | mriedem | i didn't actually think of the simpsons when i said that, but now i realize it's just baked into my brain from years of watching those recorded episodes on vhs as a kid | |
| 17:35:35 | mriedem | "we didn't have streaming in my day..." | |
| 17:35:46 | melwitt | that's impressive | |
| 17:36:30 | sean-k-mooney | even with out streaming like friends the simsons is one of those shows that is on a loop on cable/satialite tv constantly | |
| 17:36:52 | mriedem | we didn't have cable until i was 17 | |
| 17:36:54 | mriedem | hence the vhs | |
| 17:37:18 | mriedem | and then it was all WWE and Love Line years after my friends were sick of those shows | |
| 17:37:33 | sean-k-mooney | some what similar i was maybe 12/14 but that just becasue ireland and it not being a thing until the 90s | |
| 17:38:15 | dansmith | "This channel is for Nova development and mriedem's childhood nostalgia" | |
| 17:38:40 | sean-k-mooney | dansmith: and music references form the 80s | |
| 17:38:51 | dansmith | sean-k-mooney: same category | |
| 17:39:52 | mriedem | let it be so | |
| 17:40:01 | sean-k-mooney | well now that im donw backporting a patch we backported 6 months ago i think im going to get food then do something useful... | |
| 17:50:44 | openstackgerrit | melanie witt proposed openstack/nova stable/stein: Add regression test for bug 1824435 https://review.opendev.org/692906 | |
| 17:50:44 | openstack | bug 1824435 in OpenStack Compute (nova) stein "fill_virtual_interface_list migration fails on second attempt" [Medium,In progress] https://launchpad.net/bugs/1824435 - Assigned to melanie witt (melwitt) | |
| 17:50:44 | openstackgerrit | melanie witt proposed openstack/nova stable/stein: Remove redundant call to get/create default security group https://review.opendev.org/692907 | |
| 18:06:37 | watersj | is openstack resource provider a command available to stein? or am I supposed to replace "resource provider" with something else ? | |
| 18:07:37 | melwitt | watersj: you need to install osc-placement in order to get the placement cli commands | |
| 18:10:08 | watersj | melwitt, ty | |
| 18:17:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.opendev.org/635668 | |
| 18:17:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refresh instance in MigrationTask.execute Exception handler https://review.opendev.org/669012 | |
| 18:17:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize() https://review.opendev.org/635684 | |
| 18:17:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted() https://review.opendev.org/636224 | |
| 18:17:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize https://review.opendev.org/636253 | |
| 18:17:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle target host cross-cell cold migration in conductor https://review.opendev.org/642591 | |
| 18:17:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Validate image/create during cross-cell resize functional testing https://review.opendev.org/642592 | |
| 18:17:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add zones wrinkle to TestMultiCellMigrate https://review.opendev.org/643450 | |
| 18:17:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add negative test for cross-cell finish_resize failing https://review.opendev.org/643451 | |
| 18:17:13 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add negative test for prep_snapshot_based_resize_at_source failing https://review.opendev.org/669013 | |
| 18:17:13 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add confirm_snapshot_based_resize_at_source compute method https://review.opendev.org/637058 | |
| 18:17:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add ConfirmResizeTask https://review.opendev.org/637070 | |
| 18:17:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add confirm_snapshot_based_resize conductor RPC method https://review.opendev.org/637075 | |
| 18:17:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize from the API https://review.opendev.org/637316 | |
| 18:17:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize_at_dest compute method https://review.opendev.org/637630 | |
| 18:17:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Deal with cross-cell resize in _remove_deleted_instances_allocations https://review.opendev.org/639453 | |
| 18:17:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add finish_revert_snapshot_based_resize_at_source compute method https://review.opendev.org/637647 | |
| 18:17:17 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add RevertResizeTask https://review.opendev.org/638046 | |
| 18:25:21 | dansmith | mriedem: I'm missing something | |
| 18:27:48 | dansmith | okay I think I see, but I feel like now the flow is more confusing than it was before | |
| 18:28:12 | dansmith | all the nested conditions make it hard for me to tell what states you can end up in after your fall out of all of it | |
| 18:28:35 | dansmith | and having the two "restricting" cases is not helping | |
| 18:29:08 | dansmith | I feel like that is a lot more complicated just because we're trying to log a message there about something that will happen later based on a flag we're not even concerned with | |
| 18:30:15 | dansmith | maybe it's more complicated because of the later change you commented about but... it's pretty confusing | |
| 18:30:42 | dansmith | ....which adds another level of nested conditional | |
| 18:30:50 | dansmith | and more logs | |
| 18:31:27 | dansmith | maybe it's just me, and sorry if so, but I think that end state tripley-nested conditional thing is just a total nightmare | |