| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-14 | |||
| 12:57:06 | efried | lifeless_ where's the "python/cpython issue tracker" | |
| 13:07:54 | mriedem | cdent: ha, no | |
| 13:08:02 | mriedem | i think i saw mention of that in the cinder channel last night though | |
| 13:08:20 | cdent | yeah, there's a few related reports | |
| 13:08:44 | cdent | at first I was "oh noes, lots of new bugs, something is very broken" | |
| 13:10:02 | bauwser | hey folks, just a question, is it an US holiday today ? | |
| 13:10:16 | mriedem | bauwser: not for me | |
| 13:10:31 | bauwser | depending on your company, then ? | |
| 13:10:35 | mriedem | yes | |
| 13:10:42 | mriedem | easter is not a US holiday | |
| 13:11:31 | mriedem | cdent: closing all of these bugs is going to do wonders for my LP karma | |
| 13:15:18 | cdent | mriedem: my treatment of and respect for you is entirely based on your lp karma | |
| 13:15:52 | mriedem | i knew it | |
| 13:21:53 | smcginnis | mriedem: I reported it and someone with launchpad has been doing a decent job watching for them. | |
| 13:22:15 | smcginnis | It was mostly one account for awhile, but they've been switching over to a couple others. Probably as the one gets disabled. | |
| 13:32:56 | cdent | mriedem: I can't remember: Is it safe to assume that placement api will be upgraded before compute nodes? More specifically: can I make a change to the resource-tracker that requires a newer placement microversion? | |
| 13:37:21 | mriedem | cdent: i thought we doc'ed something about that here? https://docs.openstack.org/developer/nova/placement.html#upgrade-notes | |
| 13:37:23 | mriedem | since we'd forget | |
| 13:37:35 | mriedem | cdent: either way you need to use microversions and be graceful about when it's not available | |
| 13:37:42 | mriedem | curtsy and whatnot | |
| 13:38:09 | cdent | meh. I was hoping to be able to delete 30 lines or so lines of code and replace it with < 10 | |
| 13:41:38 | efried | This soft delete thing: can a guy "undelete" a soft-deleted instance? | |
| 13:43:18 | mriedem | efried: yes | |
| 13:43:19 | mriedem | restore | |
| 13:43:34 | mriedem | efried: https://developer.openstack.org/api-ref/compute/#restore-soft-deleted-instance-restore-action | |
| 13:43:43 | efried | mriedem (melwitt) And by current design, does the quota get re-hit at that time? | |
| 13:43:57 | mriedem | efried: no | |
| 13:44:05 | mriedem | quota is decremented when we actually delete | |
| 13:44:16 | mriedem | it's one of the reasons the local delete code in the api is terrible | |
| 13:44:34 | mriedem | because we have to create a quota reservation to decrement in the api and then either roll that back or commit it in the compute | |
| 13:44:37 | efried | Ohh, but according to comments on https://review.openstack.org/#/c/386756/ it looks like we *also* decrement the quota when we do the soft delete. | |
| 13:44:39 | mriedem | if we lose the compute, well, oops! | |
| 13:46:17 | efried | melwitt dansmith Am I misunderstanding ^^ ? | |
| 13:46:44 | mriedem | dan is out today | |
| 13:46:55 | mriedem | it's 6:46am for mel | |
| 13:46:59 | mriedem | and she's not a morning person i gather | |
| 13:47:02 | efried | :) | |
| 13:49:09 | mriedem | i left a comment in the patch | |
| 13:51:32 | mriedem | that change needs to be rebased anyway | |
| 13:51:36 | mriedem | the compute api code in there is wayyyyy old | |
| 13:51:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change soft delete resources to prevent DOS https://review.openstack.org/386756 | |
| 13:53:21 | efried | mriedem Cool, thanks. | |
| 13:53:40 | efried | I keep hoping that reviewing stuff I don't understand will eventually lead to some kind of enlightenment. | |
| 13:54:44 | efried | So far, it's like that flashlight in a horror movie. Dim... dimmer... gone. | |
| 13:55:45 | mriedem | well, that's true if you're reviewing anything related to local delete in the api or quotas | |
| 13:56:03 | mriedem | efried: you could triage some new bugs!? | |
| 13:56:06 | efried | mriedem BTW, I'm picking things to review based on the "query_high_prio_bug_fixes.py" result in the Pike Review Focus etherpad. | |
| 13:56:18 | efried | mriedem Sure, was gonna ask earlier if I could help with that. | |
| 13:56:27 | efried | But wasn't sure I'd be qualified - see flashlight analogy above. | |
| 13:56:30 | mriedem | like i came across this today https://bugs.launchpad.net/nova/+bug/1682060 | |
| 13:56:30 | openstack | Launchpad bug 1682060 in kolla-ansible "empty nova service and hypervisor list" [Undecided,New] | |
| 13:56:42 | mriedem | ^ is a thing that just landed which broke how kolla does cell setup | |
| 13:56:53 | mriedem | yay | |
| 13:56:57 | efried | mriedem Okay, looking. (And will have to find out wtf kolla is...) | |
| 13:57:14 | mriedem | kolla is openstack on top of k8s | |
| 13:57:20 | mriedem | deploy openstack with k8s | |
| 13:57:34 | efried | And k8s is...? | |
| 13:57:38 | mriedem | kubernetes | |
| 13:58:15 | mriedem | dude, i'm still reeling from the shelve/unshelve stuff i was digging into for a few hours yesterday | |
| 13:58:26 | mriedem | i find new and terrible things every day it seems | |
| 13:58:54 | mriedem | well, "find" as in new to me, and then i find out someone else found the same thing out over a year ago | |
| 13:58:57 | mriedem | and it's not fixed yet | |
| 14:01:12 | mriedem | cdent: back to your point about dropping code, | |
| 14:01:15 | mriedem | i think we can do that after a release | |
| 14:01:28 | mriedem | cdent: so you request the new microversion and if it's there, great, else fallback, | |
| 14:01:47 | mriedem | cdent: and then in queens, we require the higher microversion - we've been signalling that in nova-status | |
| 14:01:57 | cdent | mriedem: okay, no problem, that's what I'm doing now: trying the new way. kablooie? okay do the old way | |
| 14:02:08 | mriedem | e.g. filter scheduler requires placement>=1.4 in pike, and nova-status checks that too | |
| 14:02:31 | mriedem | yeah https://github.com/openstack/nova/blob/ae637e70a72166067e09f3bbc3acb54e5d92771e/nova/cmd/status.py#L208 | |
| 14:02:33 | mriedem | cool | |
| 14:02:53 | mriedem | huh, never realized we were using pkg_resources there | |
| 14:03:44 | cdent | mriedem: that's a relatively recent bugfix | |
| 14:03:48 | cdent | it was doing floats initially | |
| 14:03:53 | cdent | which would have broken relatively soon | |
| 14:04:31 | mriedem | i remember seeing something about that | |
| 14:06:49 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Clarify wording in compute API _delete method https://review.openstack.org/456910 | |
| 14:13:23 | sdague | mriedem: you know, shelve/unshelve being terrible might be a feature and not a bug, to convince no one to ever use it | |
| 14:15:07 | mriedem | sdague: well, it's also fodder for why nova/openstack is no good | |
| 14:15:17 | mriedem | i'd rather drop things that don't work | |
| 14:16:01 | sdague | mriedem: I agree on the drop front | |
| 14:16:33 | mriedem | i think fixing the volume terminate_connection thing is simple, but that only fixes some backends, because of how cinder works | |
| 14:16:42 | mriedem | which is unfortunate, and what stalled fixing it over a year ago | |
| 14:17:05 | mriedem | because what really needs to be called is this internal remove_export function which only os-detach calls, but that's exactly what we don't want to use for shelve becaues we want the volume to stay attached | |
| 14:17:17 | sdague | mriedem: it would be nice if there was a way we could actually get some real data on whether anyone actually uses this bit | |
| 14:17:44 | smcginnis | +1 | |
| 14:17:47 | mriedem | so cinder exposes a new api, or changes terminate_connection to also call remove_export, of nova calls detach but then immediately also calls reserve_volume so the instance doesn't lose the volume, all of which aren't great | |
| 14:18:00 | mriedem | option 2 there is probably my favorite | |
| 14:18:04 | mriedem | since it abstracts everything within cinder | |
| 14:18:14 | mriedem | sdague: yeah but i don't really know how to get that | |
| 14:18:31 | sdague | mriedem: yeh, me either | |
| 14:18:32 | mriedem | polling the operators, i could ask if they disable it via policy | |
| 14:18:48 | mriedem | can't really ask users though | |
| 14:19:20 | mriedem | i also don't know if operators track incoming request metrics and could figure out how many calls are made to that api in a week/month | |
| 14:19:22 | sdague | mriedem: so... an idea I was thinking about the other day was to actually build something into nova that builds API stats in a format that's easy to annonymously dump back out | |
| 14:19:22 | cdent | turn if off and wait for someone to complain :D | |
| 14:19:40 | sdague | cdent: if the feedback cycle wasn't 1.5 years, that would work :) | |
| 14:20:07 | mriedem | we send an instance.shelve notification, which i'd think telemetry is consuming, but i'm not sure, | |
| 14:20:09 | cdent | we've all become quite patient for other reasons...this one more thing, no problem | |
| 14:20:19 | mriedem | but that also assumes people are using telemetry, which is a bad assumption | |
| 14:20:28 | cdent | yes it is a bad assumption | |