| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-10-22 | |||
| 16:46:52 | mriedem | not sure it needs to be a minor | |
| 16:46:58 | dansmith | that's what I mean yeah | |
| 16:47:13 | mriedem | u-c would get 16.0.1 | |
| 16:47:17 | mriedem | you'd get your thing for osc, | |
| 16:47:22 | mriedem | and horizon would have a release to deprecate it's option | |
| 16:47:29 | mriedem | and we revert the revert in X | |
| 16:47:53 | mriedem | which is what should have happened anyway according to our novaclient deprecation removal guidelines https://docs.openstack.org/python-novaclient/latest/contributor/deprecation-policy.html | |
| 16:48:02 | dansmith | idk, I agree that plan would be the by-the-book way, | |
| 16:48:21 | dansmith | but being a client it seems like it shouldn't be that big of a deal, but I'm clearly not the expert on the politics here | |
| 16:49:14 | mriedem | it's not really politics, it's just that we screwed horizon without time to react, | |
| 16:49:39 | mriedem | so they can find an alternative solution without novaclient (i don't know how to write that patch), they can drop their thing without a deprecation period, or we can try to make it right | |
| 16:50:05 | mriedem | and option 5 is i just stop caring and let someone else sort out this mess :) | |
| 16:50:15 | mriedem | like stephenfin! | |
| 16:50:43 | dansmith | I mean the "getting everyone to agree on the solution" where some want the by-the-book and some might go a more pragmatic route | |
| 16:50:45 | dansmith | anyhoo | |
| 16:56:41 | dansmith | if we leave it to stephenfin then we're not going to get anything resolved until well after ptg | |
| 16:56:54 | dansmith | so hopefully that isn't the strategy | |
| 16:56:54 | mriedem | i know, i was joking | |
| 16:57:10 | mriedem | crap, forgot i was preheating an oven, oops | |
| 17:18:10 | mriedem | will see if this does the trick https://review.opendev.org/690134 | |
| 18:00:44 | dansmith | mriedem: were you going to send an email about this or just hope horizon peeps are okay with that? | |
| 18:02:46 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Adds view builders for keypairs controller https://review.opendev.org/347289 | |
| 18:03:02 | mriedem | dansmith: i started working up the email but then if that change is OK i don't think it's worth the email | |
| 18:03:06 | mriedem | or revert discussion | |
| 18:42:18 | openstackgerrit | Matt Riedemann proposed openstack/nova master: FakeDriver: adding and removing instances on live migration. https://review.opendev.org/243613 | |
| 19:24:52 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: mark device response param as optional for list/show vol attachments https://review.opendev.org/690383 | |
| 19:36:45 | mriedem | this is pretty funny https://review.opendev.org/#/c/323250/ | |
| 19:36:53 | mriedem | "Happy 3rd birthday patch!! Soon you will go to school!" | |
| 19:37:57 | dansmith | lol | |
| 19:50:31 | mriedem | unfortunately i still have to -1 them | |
| 19:54:07 | mriedem | is there some way we could have an instance that is both deleted (instance.deleted != 0) and SOFT_DELETED? (instance.vm_state == 'soft-delete')? | |
| 20:18:17 | openstackgerrit | Matthew Booth proposed openstack/nova master: Functional test for UnexpectedDeletingTaskStateError https://review.opendev.org/689278 | |
| 20:18:17 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add new simple test base for functional tests https://review.opendev.org/689186 | |
| 20:18:18 | openstackgerrit | Matthew Booth proposed openstack/nova master: Fix incorrect vm and task state after build failure race https://review.opendev.org/689388 | |
| 20:18:18 | openstackgerrit | Matthew Booth proposed openstack/nova master: Unplug VIFs as part of cleanup of networks https://review.opendev.org/663382 | |
| 20:18:19 | openstackgerrit | Matthew Booth proposed openstack/nova master: Make image fixtures simpler to reference https://review.opendev.org/690403 | |
| 20:19:55 | mdbooth | mriedem: https://review.opendev.org/#/c/689186/3/nova/tests/functional/integrated_helpers.py is what I did to ProviderUsageBaseTestCase | |
| 20:20:42 | mdbooth | tl;dr renamed it to SimpleTestBase and put all the provider test specific support code in a ProviderUsageBaseTestCase subclass | |
| 20:21:14 | mdbooth | Then I subclassed SimpleTestBase, which is clean without the provider stuff | |
| 20:22:58 | mriedem | i don't want to backport any of those refactors | |
| 20:23:03 | mriedem | i thought i was pretty clear about that | |
| 20:23:13 | mdbooth | I can't explain the locking thing I hit, btw, but it made no sense, went away when I looked harder, and never came back. I assume it was PEBKAC and I probably had an unsaved editor window open or something. | |
| 20:23:19 | mdbooth | mriedem: Sure, if you like. | |
| 20:23:48 | mdbooth | mriedem: For the backport we could just use ProviderUsageBaseTestCase | |
| 20:23:54 | mriedem | yes, please | |
| 20:23:58 | mdbooth | And for the image fixture thing we could just use a hardcoded image | |
| 20:24:04 | mriedem | right now you have 2 refactor changes before the new regression test | |
| 20:24:05 | mdbooth | But lets do that in the backport | |
| 20:26:46 | melwitt | mriedem: without looking at the code first, I'd have thought "yes" because the instance would begin as SOFT_DELETED and then when the reclaim interval comes around and it's time to permadelete it, it would then become deleted != 0 as well. it would surprise me if the vm_state is cleared when that happens, but who knows | |
| 20:27:58 | mriedem | melwitt: the periodic queries the db for instances which are SOFT_DELETED and not undergoing some task_state transition on the given host, | |
| 20:28:10 | mriedem | then deletes them which changes the vm_state to DELETED and destroys the instance | |
| 20:28:26 | mriedem | so if we updated the vm_state to DELETED it shouldn't be in the db query results in the periodic | |
| 20:28:47 | mriedem | and if we didn't update that, like the update failed, then the instance won't be deleted either (deleted == 0) | |
| 20:29:17 | mriedem | the db api query will return deleted instances by default since the filter being used doesn't include deleted=False | |
| 20:29:31 | mriedem | but still, not sure how to get into deleted=id and vm_state=soft-delete at the same time | |
| 20:29:55 | mriedem | i was thinking maybe some local delete in the api wonkiness but didn't see anything obvoius | |
| 20:30:31 | mriedem | melwitt: also while i've got you, it looks like NOVA_CONSOLE_PROXY_COMPUTE_TLS in our nova-next job isn't getting honored anymore, i.e. we don't have the vencrypt config in that job | |
| 20:30:40 | melwitt | ok, it sounds like by "could have" you mean can the periodic return an instance like that. I answered it as though it were, "can we have any records in that state?" and I thought yes. so... nevermind me | |
| 20:30:49 | mriedem | because NOVA_CONSOLE_PROXY_COMPUTE_TLS=true in localrc but https://github.com/openstack/devstack/blob/1a46c898db9c16173013d95e2bc954992121077c/lib/nova#L175 | |
| 20:31:04 | melwitt | mriedem: argh again huh... ok, I'll look. I've had to fix that before in the past | |
| 20:31:26 | mriedem | i'm trying to figure out why https://review.opendev.org/#/c/683988/ is intermittenly failing novnc tests, | |
| 20:31:41 | melwitt | I'll look up what I did last time and compare to see what has changed | |
| 20:31:42 | mriedem | which you recently fixed for multinode jobs here https://review.opendev.org/#/c/675721/ | |
| 20:32:00 | mriedem | i think it's just the conversion of the nova-next job from d-g to zuulv3 | |
| 20:32:55 | mriedem | https://review.opendev.org/#/c/670196/ | |
| 20:33:12 | mriedem | d-g would pass the value through in the localrc but zuulv3 is yaml'ing it and converting True to true | |
| 20:33:37 | mriedem | yaml-ama-ding-dong | |
| 20:33:41 | melwitt | this is what I had to do last time to fix enablement of tls between nova-compute and the proxy https://review.opendev.org/643045 | |
| 20:34:05 | mriedem | we aren't even getting that far because https://review.opendev.org/#/c/643045/9/lib/nova@659 | |
| 20:34:15 | mriedem | [[ ${NOVA_CONSOLE_PROXY_COMPUTE_TLS} = "True" ]] | |
| 20:34:33 | mriedem | NOVA_CONSOLE_PROXY_COMPUTE_TLS: true | |
| 20:34:45 | melwitt | oh, you're saying that flag is not getting reflected as true? O.o | |
| 20:34:51 | KeithMnemonic | is this a valid command option, using --deleted and --limit -1. it seems to cause an infinite hang on Rocky. i did not check newwer releases yet | |
| 20:35:11 | mriedem | melwitt: hard to say because local.conf here looks OK https://zuul.opendev.org/t/openstack/build/9726aebf48804b83bbebe290bd8187ac/log/controller/logs/local_conf.txt.gz#54 | |
| 20:35:30 | melwitt | hmm | |
| 20:36:14 | melwitt | KeithMnemonic: I didn't think that limit -1 was a thing. unlimited happens by omitting --limit, I thought. but regardless of that, there shouldn't be an indefinite hang, that I would call a bug | |
| 20:36:37 | mriedem | limit=-1 is a novaclient thing to page until there are no more results | |
| 20:37:02 | melwitt | oh :\ I thought it did that on its own in the absence of --limit | |
| 20:37:04 | KeithMnemonic | 'osapi_max_limit' will be used instead. | |
| 20:37:04 | KeithMnemonic | than 'osapi_max_limit' option of Nova API, | |
| 20:37:04 | KeithMnemonic | -1, all servers will be displayed. If limit is greater | |
| 20:37:04 | KeithMnemonic | Maximum number of servers to display. If limit equals | |
| 20:37:04 | KeithMnemonic | " --limit <num-servers> | |
| 20:37:05 | KeithMnemonic | " | |
| 20:37:11 | melwitt | nevermind, sorry | |
| 20:37:13 | KeithMnemonic | from the cli help | |
| 20:37:27 | mriedem | melwitt: nope, by default limit is 1000 results in the api | |
| 20:37:27 | KeithMnemonic | ok i can log a bug. i will see if this happens on something newer | |
| 20:37:36 | KeithMnemonic | i did not see anything open around this currently | |
| 20:37:52 | melwitt | mriedem: that I know, but I thought the client would auto-page by itself without need to tell it | |
| 20:37:56 | melwitt | and I was wrong | |
| 20:38:12 | KeithMnemonic | in the env i tested, i only have maybe 20 deleted at most | |
| 20:38:34 | melwitt | yeah, something is whack there, I think | |
| 20:39:35 | KeithMnemonic | let me see what debug i can get to add to the bug to help isolate it | |
| 20:39:49 | mdbooth | mriedem: Responded. I still think that's good to go. I'd prefer to make the requested changes in the backports. They're trivial enough. | |
| 20:40:47 | melwitt | KeithMnemonic: yeah run with --debug to see where it gets stuck | |
| 20:50:16 | mriedem | melwitt: oh here is the vencrypt stuff, i was looking in the wrong config file (there are only 20 nova configs in a devstack job) | |
| 20:50:17 | mriedem | https://zuul.opendev.org/t/openstack/build/9726aebf48804b83bbebe290bd8187ac/log/controller/logs/etc/nova/nova_cell1_conf.txt.gz#32 | |
| 20:51:46 | mriedem | so i wonder if NOVA_CONSOLE_PROXY_COMPUTE_TLS=True just doesn't work for the subnode since we've never had a multinode job use that yet | |
| 20:52:33 | melwitt | mriedem: ah yeah. awhile back I split things at stephenfin request (and I agree was a good idea) to only config which settings were required on the proxy and compute host, respectively. so they don't just all appear in all configs like they used to back in the day | |