| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-06-16 | |||
| 14:54:29 | lyarwood | stephenfin: awesome thanks | |
| 14:56:17 | sean-k-mooney | looks good to me | |
| 14:56:41 | sean-k-mooney | we had the env varable beforefore for downstream? | |
| 14:56:56 | sean-k-mooney | or was it there for another reason | |
| 14:57:13 | sean-k-mooney | having it be a different tox enve removes the need for it but just wondering | |
| 14:57:40 | lyarwood | downstream | |
| 14:58:14 | sean-k-mooney | ack, thats what i tought but was not sure | |
| 14:58:19 | lyarwood | but I never got around to replacing the original hackaround with it | |
| 14:58:30 | lyarwood | the original being to truncate the file | |
| 14:58:38 | lyarwood | \o/ | |
| 15:07:25 | opendevreview | Merged openstack/nova master: tests: Merge flavor tests https://review.opendev.org/c/openstack/nova/+/783662 | |
| 15:08:19 | opendevreview | Merged openstack/nova master: tests: Remove useless mocks https://review.opendev.org/c/openstack/nova/+/778730 | |
| 15:10:20 | admin1 | if vms ( backed up by ceph) fail to live migrate, is a pause + migrate enough .. or they need to be shutdown and then migrated ? | |
| 15:11:14 | admin1 | i used this setting: live_migration_downtime: 10000 ; live_migration_downtime_steps: 20 ; live_migration_downtime_delay: 100 ; live_migration_completion_timeout: 15000 | |
| 15:11:27 | admin1 | which helped 8 stuck instances to migrate .. 2 fail to migrate no matter what | |
| 15:41:20 | opendevreview | Lee Yarwood proposed openstack/nova master: tests: Allow bindep and test-setup.sh to run on EL distros https://review.opendev.org/c/openstack/nova/+/796428 | |
| 15:41:21 | opendevreview | Lee Yarwood proposed openstack/nova master: DNM - Test bindep and test-setup.sh changes against centos8 https://review.opendev.org/c/openstack/nova/+/796684 | |
| 15:49:01 | opendevreview | Merged openstack/nova master: Allow X-OpenStack-Nova-API-Version header in CORS https://review.opendev.org/c/openstack/nova/+/796319 | |
| 15:49:51 | opendevreview | Merged openstack/nova master: tests: Remove duplicate policy tests https://review.opendev.org/c/openstack/nova/+/778731 | |
| 15:50:11 | melwitt | ganso: ack will do | |
| 15:51:50 | sean-k-mooney | admin1: you can jsut do a cloud migrate | |
| 15:51:56 | sean-k-mooney | no need to pause | |
| 15:52:02 | melwitt | sean-k-mooney: ack thanks for the hint | |
| 15:52:08 | sean-k-mooney | or you can do a live migrate with a force complete call | |
| 15:52:19 | sean-k-mooney | force-complete will pause the vm | |
| 15:52:40 | admin1 | oh .. ok | |
| 15:53:34 | admin1 | openstack server migrate --help .. there is no such parameter ( force-complete) | |
| 15:54:09 | sean-k-mooney | https://docs.openstack.org/api-ref/compute/?expanded=force-migration-complete-action-force-complete-action-detail#force-migration-complete-action-force-complete-action | |
| 15:55:16 | sean-k-mooney | "openstack server migration force complete" is the comand | |
| 15:55:54 | sean-k-mooney | sean@p50:~$ openstack help server migration force complete | |
| 15:55:56 | sean-k-mooney | usage: openstack server migration force complete [-h] <server> <migration> | |
| 15:55:58 | sean-k-mooney | Force an ongoing live migration to complete. This command requires ``--os-compute-api-version`` 2.22 or greater. | |
| 15:56:00 | sean-k-mooney | positional arguments: | |
| 15:56:02 | sean-k-mooney | <server> Server (name or ID) | |
| 15:56:04 | sean-k-mooney | <migration> Migration (ID) | |
| 15:56:06 | sean-k-mooney | so basically you have to start the live migration | |
| 15:56:22 | sean-k-mooney | then you can get the migration id and then force the completeion of the live migation | |
| 15:57:41 | sean-k-mooney | melwitt: sorry dont rememebr the context of that but if the hint? helpped im glad | |
| 15:57:57 | sean-k-mooney | oh | |
| 15:58:07 | sean-k-mooney | melwitt: the socket activation? | |
| 15:58:23 | melwitt | sean-k-mooney: haha yeah the fixing for bionic thing | |
| 16:00:09 | melwitt | stephenfin, lyarwood, bauzas: I'm meh about approving older branches before the newer one merges, I don't think it solves the ping problem. in my case it's just I don't always remember to review all the older versions and I don't often like to do them all at the same time anyway | |
| 16:01:11 | stephenfin | I figured it would solve the ping problem, or at least significantly reduce it | |
| 16:01:35 | stephenfin | in theory, if all branches were approved, one could get them merged by a simple recheck | |
| 16:05:00 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Speed up 'servers' API tests https://review.opendev.org/c/openstack/nova/+/778732 | |
| 16:05:56 | melwitt | yeah. I guess one potential thing that could happen is if the author/rechecker doesn't realize they can't merge out of order and they go on a recheck loop in the wrong order. I have seen that before anyway but I'd imagine it's more tempting when you see the shiny +W on there | |
| 16:10:23 | admin1 | sean-k-mooneythanks .. it worked | |
| 16:22:00 | opendevreview | Merged openstack/nova master: Retry lvm volume and volume group query https://review.opendev.org/c/openstack/nova/+/796269 | |
| 16:31:40 | opendevreview | Stephen Finucane proposed openstack/nova stable/wallaby: Retry lvm volume and volume group query https://review.opendev.org/c/openstack/nova/+/796707 | |
| 16:59:57 | stephenfin | le sigh https://storyboard.openstack.org/#!/story/2008549 | |
| 17:00:27 | stephenfin | "I set the state of my shelved offloaded instance to active and it broke things" | |
| 17:01:55 | lyarwood | lol | |
| 17:01:56 | melwitt | lol | |
| 17:02:06 | melwitt | we need a required option on set state like --I-acknowledge-this-might-break-something | |
| 17:03:16 | lyarwood | -i-really-know-what-im-doing-and-if-this-breaks-anything-i-am-to-blame | |
| 17:03:59 | melwitt | 😂 yeah the longer the option, the better | |
| 17:05:14 | melwitt | realistically, we should put a big fat warning in the set state help that says something similar. if there already is one, then ... :| | |
| 17:06:00 | lyarwood | I think there might be in the api-ref but yeah osc could say more | |
| 17:12:09 | stephenfin | * derekh has quit (Quit: Leaving) | |
| 17:12:09 | stephenfin | <stephenfin> "I set the state of my shelved offloaded instance to active and it broke things" | |
| 17:12:09 | stephenfin | ||
| 17:12:09 | stephenfin | melwitt: lyarwood: 96707 | |
| 17:12:10 | stephenfin | whoops | |
| 17:12:21 | stephenfin | lyarwood: melwitt: https://review.opendev.org/c/openstack/python-openstackclient/+/796713 | |
| 17:12:35 | stephenfin | (HexChat copies whatever you highlight to the clipboard) | |
| 17:15:07 | melwitt | sweet | |
| 17:20:04 | lyarwood | stephenfin: that could've been so much worse ^_^ | |
| 17:30:32 | opendevreview | Merged openstack/nova stable/victoria: Error anti-affinity violation on migrations https://review.opendev.org/c/openstack/nova/+/795542 | |
| 17:31:19 | opendevreview | Merged openstack/placement master: Bump os-resource-classes deps to 1.0.0 https://review.opendev.org/c/openstack/placement/+/796593 | |
| 17:39:55 | sean-k-mooney | admin1: useing force complete cool | |
| 17:40:04 | sean-k-mooney | admin1: that makes it a semi live migration | |
| 17:40:25 | sean-k-mooney | the vm does not restart or anythng but it proably will notice its clock is wrong and or drop packets | |
| 17:41:02 | sean-k-mooney | so normally we avoid that but its good to know its an option when you really need to move soemthing and is still friendlier then a cold migrate | |
| 17:41:50 | sean-k-mooney | why is that in story board | |
| 17:42:04 | sean-k-mooney | and ya we had a downstream issue to the same effect recently | |
| 17:42:45 | sean-k-mooney | melwitt: i wonder if we should actully consier blocking that in the api | |
| 17:43:11 | sean-k-mooney | its never valide to reset state form shelve offloaded to any other state then error | |
| 17:45:16 | sean-k-mooney | lyarwood: stephenfin melwitt by the way we still have a todo item somehwer to extend the reset state api to allow more states and to allow settign the task_state too | |
| 17:46:13 | sean-k-mooney | if we do that we should harden the allowed state transations IMO | |
| 17:46:49 | melwitt | sean-k-mooney: yeah... conflicted about that because reset state is kind of a wild west api call to make but on the other hand if we can block something that's guaranteed to be invalid, that makes sense too | |
| 17:46:51 | sean-k-mooney | e.g. ERROR we woudl allow to go to any sate as an escape hatch. but any none ERROR state should not be changable arbitrailly | |
| 17:47:57 | sean-k-mooney | melwitt: im pretty sure it can result in data lose | |
| 17:48:24 | sean-k-mooney | if you do resetstat form shelve offloaded and then do either hard reboot or a migration i cant rememebr which | |
| 17:48:29 | melwitt | ? | |
| 17:48:36 | sean-k-mooney | we had a customer do that with ceph recently | |
| 17:48:42 | sean-k-mooney | and they lost the snapshot | |
| 17:49:15 | melwitt | I guess I'm not realizing how that happens. if it's shelved offloaded there's no guest to reboot | |
| 17:49:59 | sean-k-mooney | yes so if i rememebr correctly unshleve failed so the reset state then hard rebooted | |
| 17:50:03 | sean-k-mooney | that failed because no host | |
| 17:50:42 | sean-k-mooney | the then reset state again and tired to shleve i think? this is where my memory is blank | |
| 17:51:02 | sean-k-mooney | when we got the bz they had tried some cobination of that several times | |
| 17:51:14 | sean-k-mooney | and the rbd volume was nolonger preset for the vm root disk | |
| 17:52:05 | melwitt | ah ok, so you're saying if we prevent that path from even starting, it can avoid whatever future thing they managed to do to lose the data | |
| 17:52:14 | sean-k-mooney | yes | |
| 17:52:58 | melwitt | yeah ok | |
| 17:58:56 | opendevreview | Rodrigo Barbieri proposed openstack/nova stable/ussuri: Error anti-affinity violation on migrations https://review.opendev.org/c/openstack/nova/+/796719 | |
| #openstack-nova - 2021-06-17 | |||
| 01:46:55 | opendevreview | melanie witt proposed openstack/nova master: Add func test for nova-manage db archive_deleted_rows --before https://review.opendev.org/c/openstack/nova/+/796744 | |
| 01:57:32 | opendevreview | melanie witt proposed openstack/nova master: Add --task-log option to nova-manage db archive_deleted_rows https://review.opendev.org/c/openstack/nova/+/780395 | |
| 02:00:14 | melwitt | lyarwood, elodilles: it's funny passing CI \o/ (but recheck-a-thon) https://review.opendev.org/c/openstack/nova/+/795432 | |
| 02:00:21 | melwitt | s/funny/finally/ | |