| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-06 | |||
| 14:13:01 | jaypipes | mdbooth: then I don't understand what is different between the pause VM, volume migrate, start VM. | |
| 14:13:10 | mriedem | i wonder when k8s is going to implement swap volume for cinder to use | |
| 14:13:14 | mriedem | that will be awesome | |
| 14:13:23 | kashyap | jaypipes: And then, map it to libvirt's blockRebase() API, which is called by _swap_volume(). | |
| 14:13:25 | mdbooth | jaypipes: How would you prevent unpause during the volume migrate? | |
| 14:13:29 | jaypipes | kashyap: no... unless you look at the LIBVIRT driver _swap_volume() method. | |
| 14:13:33 | jaypipes | kashyap: which is my point. | |
| 14:13:40 | kashyap | jaypipes: That's what I said :-) | |
| 14:13:44 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transfer port.resource_request to the scheduler https://review.openstack.org/567268 | |
| 14:14:00 | jaypipes | kashyap: no, you said "the Nova _swap_volume() method". | |
| 14:14:21 | kashyap | jaypipes: Yes, you're right - Nova's libvirt driver. | |
| 14:14:43 | jaypipes | mdbooth: it's an administrative action that involves multiple orchestrated pieces. | |
| 14:15:40 | mdbooth | jaypipes: Note that we're redefining swap_volume here, which is a bit out of scope. My concern relates to multiattach. | |
| 14:15:45 | jaypipes | mdbooth: and just because libvirt/QEMU has some magic foo to do all that orchestration in a (sometimes) live fashion, we've made the Compute API look like the libvirt/QEMU API and embedded expectations of full coordination/orchestration into our API. and that's wrong IMHO. | |
| 14:16:17 | mdbooth | However, there are 2 parts to it. | |
| 14:16:41 | MultipleCrashes | Please have a look at this review and leave a feedback : https://review.openstack.org/#/c/570370/ | |
| 14:16:45 | jaypipes | mdbooth: yes, I understand your concern about multiattach and artom's suggestion of just barfing back a 400 Bad Request when any of the source or dest volumes in the swap operation are multiattach is a perfectly reasonable "solution" to that. | |
| 14:16:49 | mdbooth | If we removed swap volume and implemented this with pause/resume, we would still need an additional api in nova to 'lock' the pause. | |
| 14:17:08 | jaypipes | mdbooth: I'm just annoyed at everything, sorry. | |
| 14:17:09 | mdbooth | Otherwise, orchestration or not, a concurrent operation could unpause it. | |
| 14:17:26 | artom | jaypipes, would you like a lawn and some kids to yell at? ;) | |
| 14:17:35 | jaypipes | mdbooth: precisely. which is why it belongs in an orchestration layer, not the virt driver layer :) | |
| 14:17:38 | mdbooth | Also, customers who don't like 6 hours of downtime would be grumpy. | |
| 14:17:48 | jaypipes | artom: :) | |
| 14:18:08 | mdbooth | jaypipes: But you *can't* orchestrate an *inability* for somebody else to do something without a new api. | |
| 14:18:28 | mdbooth | Your orchestrator can pause, migrate, unpause | |
| 14:18:29 | mriedem | mdbooth: so your use case in the ML reply to jay was that people are doing retype to get from old shitty storage to new shiny storage, | |
| 14:18:36 | mriedem | but that's still initiated through cinder's retype API | |
| 14:18:38 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send resource allocations in the port binding https://review.openstack.org/569459 | |
| 14:18:44 | mriedem | what's the use case for using the swap volume API directly w/o cinder? | |
| 14:18:47 | mdbooth | But it can't prevent my user, frustrated that his data has been unavailable for 6 hours, from unpausing | |
| 14:19:19 | mdbooth | mriedem: No idea. I don't think it matters here, though. | |
| 14:20:16 | mriedem | i think it matters to understand how people are using or abusing the APIs | |
| 14:20:31 | mriedem | os-server-external-events is a public REST API which i could call directly with admin creds, i don't know why i would, but i can | |
| 14:20:33 | mriedem | doesn't mean i should | |
| 14:20:38 | jaypipes | mriedem: ++ | |
| 14:20:41 | mdbooth | mriedem: Agreed. I don't think it's specific to this case, though. | |
| 14:21:01 | mdbooth | mriedem: It doesn't impact the multiattach thing. | |
| 14:21:03 | artom | mriedem, I think the difference here is that, calling os-server-external-events doesn't potentially corrupt data | |
| 14:21:25 | mriedem | you guys are missing the point | |
| 14:21:27 | jaypipes | mdbooth: BTW, I'm not insinuating that you personally have anything to do with this swap_volume() code. Just pointing out I don't think this particular API call belongs in the compute API. | |
| 14:21:28 | artom | For instance, if we discovered that calling it with a certain event caused a VM to go down (suspend your disbelief, work with me) | |
| 14:21:43 | artom | We'd fix that, regardless of how silly the "use case" (or lack thereof) was | |
| 14:21:55 | bhagyashri_s | efried, jaypipes, bauzas: Hi, Addressed review comments on https://review.openstack.org/#/c/560459 and replied to few comments. Request to review the same. Thank you in advanced :) | |
| 14:22:17 | mdbooth | jaypipes: Don't get me wrong, I've wasted many, many curse words on swap_volume. However, I don't see how you can implement this without something like it. | |
| 14:22:54 | mriedem | mdbooth: have you opened a nova bug for this yet? | |
| 14:24:12 | mdbooth | mriedem: No, but I will. I was kinda hoping somebody, either Nova or Cinder, would pipe up with a simple fix. | |
| 14:24:28 | mriedem | mdbooth: i can but wanted a but quick | |
| 14:25:08 | artom | mdbooth, I mean, I did? | |
| 14:25:11 | artom | Return a 400? | |
| 14:25:19 | mdbooth | Oh, I think you're saying that the direct call use case is relevant here because it's all that's stopping us from implementing this in cinder? | |
| 14:25:24 | artom | jaypipes seems on board with it | |
| 14:25:43 | mdbooth | In which case, yeah. | |
| 14:26:03 | mriedem | mdbooth: yes | |
| 14:26:10 | mriedem | i would prefer to just put the blocker in cinder | |
| 14:26:21 | mriedem | reasons are in my ML reply | |
| 14:26:44 | mriedem | 1. source of truth so less racy and 2. no rollbacks for retype or live migration for volumes | |
| 14:26:47 | artom | mriedem, so Cinder would barf if we "fix" it *just* in Nova, right? | |
| 14:26:51 | openstackgerrit | Jan Gutter proposed openstack/nova master: Add support for vrouter HW offloads https://review.openstack.org/572082 | |
| 14:26:55 | mriedem | artom: yes, but way late | |
| 14:27:00 | smcginnis | I haven't been following the discussion at all, but if we want to block something in cinder, make sure it makes sense for use cases outside of nova. | |
| 14:27:05 | mriedem | and then you better pray that cinder rolls back properly | |
| 14:27:26 | mriedem | smcginnis: you probably missed my k8s snark earlier | |
| 14:27:37 | artom | mriedem, so we'd have to 1. block it in cinder early on 2. block it in nova for the ludicrous direct-call case | |
| 14:27:46 | mriedem | how would a volume live migratoin / retype work with k8s when k8s doesn't implement a swap volume callback API? | |
| 14:27:53 | mriedem | artom: i guess | |
| 14:27:59 | mriedem | i can hack something up for the nova side blocker | |
| 14:28:01 | artom | And I then I don't think we'd need a "3. make cinder handle 2" because we'd never get there | |
| 14:28:41 | smcginnis | mriedem + k8s == snark :) | |
| 14:33:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Block swap volume with multiattach volumes https://review.openstack.org/572790 | |
| 14:33:18 | mriedem | there ^ - afk for a couple of hours for a kid's school thing | |
| 14:34:57 | dansmith | mriedem: back for cells meeting? | |
| 14:35:40 | artom | mriedem, https://bugzilla.redhat.com/show_bug.cgi?id=1446446 <--- downstream bug with direct call to volume update | |
| 14:36:05 | openstack | artom: Error: Could not parse XML returned by bugzilla.redhat.com: Unknown host. (https://bugzilla.redhat.com/show_bug.cgi?id=1446446&ctype=xml) | |
| 14:36:20 | artom | openstack, ok :) | |
| 14:36:36 | artom | (It just took me forever to find it) | |
| 14:41:42 | mnaser | dansmith: regarding the build failure limit change, nova does include all weighers by default, right? | |
| 14:41:54 | dansmith | mnaser: yup | |
| 14:42:20 | dansmith | mnaser: you adjust the weight to make a weigher a no-op | |
| 14:42:46 | mnaser | dansmith: cool, with that, its lgtm, there are a few copy-paste thing but it's not a big deal, only if you have to do another revision you can fix it i guess | |
| 14:43:21 | dansmith | mnaser: ah, not copy-paste, but "mriedem decided he wanted to s/boot/build/" and I missed one :) | |
| 14:43:36 | dansmith | oh, | |
| 14:43:42 | dansmith | the ram one is copy pasta yeah | |
| 14:43:58 | mnaser | :P | |
| 14:44:07 | dansmith | I'll update so it's clean since this is backportable | |
| 14:44:08 | dansmith | plus nobody has really looked yet :/ | |
| 14:45:10 | mnaser | your call :) | |
| 14:45:18 | dansmith | mnaser: | |
| 14:45:19 | openstackgerrit | Dan Smith proposed openstack/nova master: Change consecutive build failure limit to a weigher https://review.openstack.org/572195 | |
| 14:46:21 | mnaser | dansmith: awesome, thank you very much for your work on that | |
| 14:46:27 | dansmith | np | |
| 14:54:14 | efried | bauzas, jaypipes: I think https://review.openstack.org/#/c/560459/ is ready when you get a chance. I'm not +2ing because I contributed. | |
| 14:54:24 | bauzas | ok | |
| 15:01:52 | sahid | stephenfin: thanks for the reviex on https://review.openstack.org/#/c/553072/ | |
| 15:04:38 | bauzas | dansmith: just saw https://review.openstack.org/#/c/572195/ | |
| 15:05:09 | bauzas | dansmith: fortunately to https://review.openstack.org/#/c/572195/4/nova/scheduler/host_manager.py, it will work with queens computes | |
| 15:05:35 | stephenfin | sahid: no problem | |
| 15:05:37 | dansmith | bauzas: right that's the point | |
| 15:06:11 | bauzas | dansmith: but https://review.openstack.org/#/c/572195/4/nova/conf/scheduler.py is asking for a default number that's huge | |
| 15:06:33 | dansmith | bauzas: did you read the justification for that number in the commit message? | |