| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-29 | |||
| 16:49:52 | melwitt | their argument has been that stop + insta delete should not break the volume, IIUC | |
| 16:50:00 | lyarwood | waiting assumes we don't kill it before it's finished shutting down | |
| 16:50:14 | dansmith | lyarwood: I meant to make sure journal buffers are written before we nuke the guest | |
| 16:50:36 | dansmith | melwitt: that argument is fine as long as you wait between stop and delete :) | |
| 16:51:01 | dansmith | melwitt: because again, start->shutdown and then pulling the plug before it finishes yields a corrupted disk :) | |
| 16:51:21 | melwitt | dansmith: that was the first thing I said on the bug report but it's gone on for a long time now and gotten into the weeds | |
| 16:52:04 | bauzas | melwitt: I haven't seen any bug report against artom's patch | |
| 16:52:12 | bauzas | I guess you're talking internally | |
| 16:52:19 | melwitt | bauzas: yes internally | |
| 16:52:42 | bauzas | yeah, because that's what worried me originally | |
| 16:52:51 | bauzas | technically, destroy works like expected | |
| 16:52:52 | dansmith | we also really need to do a better job of making this a sanitized bug externally if we're going to claim this is a bug in nova | |
| 16:53:25 | bauzas | I'm not happy with claiming this as an upstream bug | |
| 16:53:37 | dansmith | me either, fwiw :) | |
| 16:53:40 | bauzas | a blueprint or a wishlist bug | |
| 16:54:08 | dansmith | we could add a feature as gibi said, but destroy is doing the right thing here | |
| 16:54:21 | bauzas | (18:52:51) bauzas: technically, destroy works like expected | |
| 16:54:28 | melwitt | +1 to all of that | |
| 16:54:30 | bauzas | strong agreement here | |
| 16:54:30 | dansmith | an alternative to gibi's idea, would be a delete flag that says "assuming task_state=None" meaning "delete this if nothing else is going on" | |
| 16:54:47 | dansmith | but it would require the client to retry, which they could currently do by just waiting for the stop to finish | |
| 16:54:50 | dansmith | so, meh | |
| 16:55:22 | bauzas | dansmith: yeah, that's why I was considering the vm state or the task state | |
| 16:55:38 | bauzas | if we really want to do *something* | |
| 16:55:39 | bauzas | but, | |
| 16:55:49 | bauzas | this can't be done with the current destroy API | |
| 16:55:52 | dansmith | it doesn't make it do what they want, and honestly it's kinda weird since they could just wait for the stop just as well, but it's less new orchestration stuff | |
| 16:56:09 | dansmith | [09:47:37] <dansmith> novaclient has a --poll option for some things. "nova stop server --poll; nova delete server" would solve this pretty easy :) | |
| 16:56:11 | dansmith | ^ :) | |
| 16:56:12 | bauzas | it would be a 'destroy++" API | |
| 16:56:34 | bauzas | heh | |
| 16:56:43 | bauzas | (18:44:41) bauzas: "ask your orchestration to be smarter ?" | |
| 16:56:43 | dansmith | also, if melwitt is right and they're using heat, then FFS, get heat to wait or something | |
| 16:56:47 | dansmith | lol | |
| 16:56:53 | lyarwood | we keep saying they could wait for the stop to finish but wasn't that part of the issue here? Even if they did wait the libvirt driver would kill the instance prematurely before it had finished shutting down? | |
| 16:57:01 | dansmith | if we're just re-quoting ourselves, are we done here? :P | |
| 16:57:19 | gibi | if currently stop + wait for STOPPED + delete works, then a new delete-with-gracefull-shutdown could also be orchestrated from the conductor but it is obviously an orchestration and adds complexity to the already complext delete codepaths | |
| 16:57:42 | bauzas | gibi: and again, this can't be the straight delete | |
| 16:57:53 | dansmith | gibi: we don't go through conductor directly for stop or delete right now, AFAIK, so it would really get confusing to add another whole path like that, IMHO | |
| 16:57:54 | gibi | bauzas: yepp this is delete++ :D | |
| 16:58:25 | gibi | dansmith: then we either need the make the stop RCP sync or do a polling from the nova-api | |
| 16:58:33 | gibi | RPC | |
| 16:58:35 | melwitt | lyarwood: I didn't think so? | |
| 16:58:53 | bauzas | can't we just have as dansmith suggested a "destroy++" API return some 40x when the task state is not None ? | |
| 16:59:15 | lyarwood | that's fine if they aren't waiting | |
| 16:59:18 | melwitt | if that's the case I missed it | |
| 16:59:21 | dansmith | gibi: we'd need a cast to conductor, a new method there, and a sync stop operation to compute from conductor followed by delete.. waiting from the api is not reasonable because stop can take a *long* time | |
| 16:59:35 | gibi | yeah | |
| 16:59:41 | lyarwood | melwitt: I was sure we had talked about them waiting and libvirt still killing the domain before things had shutdown sorry | |
| 16:59:45 | lyarwood | I'll look at the bug again | |
| 16:59:47 | dansmith | in addition to the other four ways we can delete things :) | |
| 16:59:48 | melwitt | lyarwood: my understanding (and I could be wrong) is that they have never tried waiting to stopped | |
| 17:00:00 | lyarwood | I was sure we suggested that early on | |
| 17:00:01 | dansmith | melwitt: right that seems like it to me | |
| 17:00:03 | melwitt | lyarwood: ok same, maybe I'm way off | |
| 17:00:07 | bauzas | I'd rather prefer this destroy++ API to stop synchronously at the API level if some conditions aren't met | |
| 17:00:36 | bauzas | rather than us pursuing the idea we could achieve some distributed locking mechanism | |
| 17:00:56 | dansmith | gibi: also delete returns "I will eventually do this" to the client, which it would no longer be able to guarantee | |
| 17:01:32 | dansmith | bauzas: fwiw, I don't think that is really going to do what this person wants, so we should make sure they think it would help before we do that work | |
| 17:01:37 | gibi | dansmith: ohh, that is correct and bad :/ | |
| 17:02:05 | gibi | delete++ is just too complex | |
| 17:02:08 | bauzas | dansmith: welp, good point | |
| 17:07:07 | bauzas | either way, I think we need a proper tracking | |
| 17:07:41 | bauzas | artom: I guess you need to fill a blueprint and honestly given the brainstorm efforts we made over last hour, you need to write a bit of a spec | |
| 17:08:16 | bauzas | mostly because the current destroy action can't be used for this and we need to consider a new action parameter (or any change in our API) | |
| 17:08:41 | bauzas | artom: feel also free to write a PTG proposal for this one, we could continue the talk there | |
| 17:57:36 | spatel | Any idea how to do VM nic bonding using two VF with SRIOV ? | |
| 17:58:39 | spatel | i am looking for redundancy with sriov implementation and only solution is to do bonding inside vm | |
| 18:04:32 | opendevreview | Ivan Kolodyazhny proposed openstack/nova master: Add release note which descrube NVMe attach issue https://review.opendev.org/c/openstack/nova/+/811447 | |
| 19:15:02 | opendevreview | melanie witt proposed openstack/nova stable/train: Reject open redirection in the console proxy https://review.opendev.org/c/openstack/nova/+/791807 | |
| 19:15:03 | opendevreview | melanie witt proposed openstack/nova stable/train: address open redirect with 3 forward slashes https://review.opendev.org/c/openstack/nova/+/806629 | |
| 19:15:04 | opendevreview | melanie witt proposed openstack/nova stable/train: [stable-only] Set lower-constraints job as non-voting https://review.opendev.org/c/openstack/nova/+/811762 | |
| 21:59:20 | opendevreview | David Vallee Delisle proposed openstack/nova master: rephrasing config description for num_pcie_ports in libvirt https://review.opendev.org/c/openstack/nova/+/811173 | |
| 22:19:17 | opendevreview | Goutham Pacha Ravi proposed openstack/nova stable/wallaby: DNM: Test wallaby with ceph pacific https://review.opendev.org/c/openstack/nova/+/811802 | |
| 22:33:16 | melwitt | gouthamr: thanks for that! ^ I was going to do it and forgot | |
| 22:34:56 | gouthamr | o/ melwitt - i may have hit a circular dependency of sorts | |
| 22:35:45 | melwitt | oh hm.. (looking at it now) | |
| 22:36:54 | melwitt | gouthamr: I wonder if you need to use the url instead of the change-id? not sure how it would know which of the two (master or stable/wallaby) to use | |
| 22:37:02 | melwitt | for the depends-on | |
| 22:37:05 | gouthamr | +1 can try that | |
| 22:38:04 | opendevreview | Goutham Pacha Ravi proposed openstack/nova stable/wallaby: DNM: Test wallaby with ceph pacific https://review.opendev.org/c/openstack/nova/+/811802 | |
| 22:38:22 | clarkb | the url is prefered now because you can depends on things in other code review systems | |
| 22:38:26 | clarkb | change id is gerrit specific | |
| 22:39:11 | gmann | gouthamr: i do not see circular deps, it should work fine. nova->plugin->devstack | |
| 22:39:19 | melwitt | good to know | |
| 22:40:32 | gouthamr | thanks, but zuul stull tells me "Unable to freeze job graph: 0" | |
| 22:40:54 | clarkb | gouthamr: that implies you have a bug in your zuul config I think | |
| 22:41:15 | clarkb | gouthamr: can you give me links to all the changes involved? | |
| 22:41:24 | gouthamr | clarkb: yep: https://review.opendev.org/c/openstack/nova/+/811802/ | |
| 22:42:16 | gouthamr | https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/810059 and https://review.opendev.org/c/openstack/devstack/+/810202 | |
| 22:42:44 | opendevreview | Ghanshyam proposed openstack/nova stable/wallaby: DNM: Test wallaby with ceph pacific https://review.opendev.org/c/openstack/nova/+/811802 | |
| 22:43:31 | gmann | gouthamr: I think running now ^^ ? | |
| 22:43:37 | gouthamr | oh! | |
| 22:43:48 | gmann | yeah | |
| 22:43:54 | gouthamr | gmann: thanks! i am used to pushing empty changes to trigger the CI elsewhere :) | |
| 22:44:24 | gmann | with right deps too https://zuul.openstack.org/status#nova | |
| 22:44:36 | melwitt | gmann++ | |
| 22:47:34 | clarkb | the issue is you have no files in the commit | |
| 22:47:42 | clarkb | seems like you figured that out | |
| 22:48:07 | clarkb | I don't know that that is a use case we should support. If you aren't changing anything then why bother | |