Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-10-23
16:53:08 KeithMnemonic let me try it with a market in cell0
16:53:13 KeithMnemonic i have one instance there
16:53:16 melwitt mriedem: uh oh https://github.com/openstack/nova/blob/master/nova/compute/instance_list.py#L59
16:53:34 melwitt oh nevermind
16:53:36 mriedem was just looking at that code,
16:53:41 mriedem but yeah a deleted instance will still have a mapping
16:53:43 melwitt there should still be an instance mapping? or no
16:53:43 mriedem until archived
16:53:45 melwitt yeah
16:53:52 mriedem and https://github.com/openstack/nova/blob/master/nova/compute/instance_list.py#L63
16:53:59 mriedem matches the db api to read deleted markers
16:54:05 melwitt yeah hm
16:54:58 KeithMnemonic nope same error with a deleted instance in cell0,
16:56:15 mriedem ok when i've had to debug these types of failures before i've needed to add a LOG.exception here in the API https://github.com/openstack/nova/blob/2718de6ed7c21f8ff8cf74164ae5054531fdbc30/nova/api/openstack/compute/servers.py#L331 to get a traceback
16:56:23 KeithMnemonic but just --deleted --limit -1 shows deleted from cell0 and cell1 so it is definitely something with the marker
16:56:25 mriedem KeithMnemonic: is that something you can do and then recreate to get the trace?
16:56:47 KeithMnemonic so this is interesting
16:57:01 KeithMnemonic "https://192.168.23.35:8774/v2.1/428982d4248a419a933668b6a4dd14a0/servers?name=c244e139-98fe-44eb-a971-c067954da026" will this work without deleted being passed?
16:57:18 melwitt I'm wondering what this get_marker_by_values is
16:57:28 mriedem KeithMnemonic: that's not a server name
16:57:43 KeithMnemonic that is what --debug shows
16:57:45 mriedem i'd expect you to get no results in that case
16:58:37 mriedem KeithMnemonic: hmm, ok so it's probably the cli trying to determine if the marker provided is a server name or id
16:58:46 KeithMnemonic i pass in --marker c244e139-98fe-44eb-a971-c067954da026
16:58:57 KeithMnemonic no option to specify name vs ID
16:58:59 mriedem --marker <server>¶ The last server of the previous page. Display list of servers after marker. Display all servers if not specified. (name or ID)
16:59:56 mriedem that might be a red herring if osc is just trying to determine if you gave a name or id
17:00:28 mriedem try to recreate with the LOG.exception injection i mentioned above to see where the MarkerNotFound is coming from
17:00:43 mriedem you know, we might not even be getting to the api
17:00:57 mriedem "No server with a name or ID of 'c244e139-98fe-44eb-a971-c067954da026' exists." sounds like an osc bug
17:01:06 mriedem ok i bet i know,
17:01:13 mriedem the osc code is trying to find the marker server by name or id,
17:01:19 mriedem but failing to include the deleted=True filter,
17:01:32 mriedem so the api says "nope a server that's not deleted does not exist for that"
17:01:38 KeithMnemonic but should it worked without specifying deleted in the api call?
17:02:10 mriedem meaning can you specify a marker that is a deleted server? yes
17:02:22 mriedem it seems the bug is in osc though
17:03:03 mriedem https://github.com/openstack/python-openstackclient/blob/4.0.0/openstackclient/compute/v2/server.py#L1415
17:03:09 melwitt wait, how do we know it's not including the deleted=True filter? it should show that in the --debug right
17:03:11 mriedem osc isn't looking for a deleted marker
17:03:44 KeithMnemonic it is osc, you can not do a show on a deleted sever, you get the same curl and same error
17:03:45 mriedem https://192.168.23.35:8774/v2.1/428982d4248a419a933668b6a4dd14a0/servers?name=c244e139-98fe-44eb-a971-c067954da026 means osc is first looking to see if the marker server is given on the command line by name
17:03:45 melwitt I mean, when you pass --deleted doesn't it just send deleted=True regardless? or no
17:03:50 melwitt oh
17:03:52 mriedem it's not, it gets a 404 and then tries by id,
17:04:08 mriedem which returns a 404 because the server is deleted and you're not including the ?deleted=true filter param
17:04:13 KeithMnemonic correct
17:04:23 KeithMnemonic but osc is called with --deleted
17:04:24 mriedem so it's a shell bug
17:04:30 KeithMnemonic it is just not passed to the api call
17:04:35 mriedem right, it's a bug
17:04:42 KeithMnemonic ok against osc
17:04:44 mriedem yes
17:05:17 KeithMnemonic i can file another today. i will reference this discussion and some debug output.
17:05:20 KeithMnemonic is that ok
17:05:31 mriedem sure
17:05:36 mriedem that goes against osc in storyboard though
17:05:52 mriedem where it will languish for 1000 years
17:06:26 KeithMnemonic ok let me test on something newer to see if it still exists is train for example
17:06:34 mriedem that code i linked above is train
17:06:38 mriedem so it's there
17:07:35 mriedem https://github.com/openstack/osc-lib/blob/1.14.1/osc_lib/utils/__init__.py#L169
17:07:49 mriedem looks like the fix is relatively simple:
17:08:12 mriedem utils.find_resource(compute_client.servers, parsed_args.marker, deleted=parsed_args.deleted)
17:10:48 mriedem KeithMnemonic: feel free to have suse send me a check for the openstack support anytime :)
17:12:31 KeithMnemonic you know that is a sore point these days
17:13:07 dansmith speaking of that,
17:13:22 dansmith did aspiers ever show up and/or has anyone else been able to resolve the SEV issue we released with train?
17:13:45 KeithMnemonic AFAIK, aspiers is till with SUSE but not sure in what capacity
17:14:10 KeithMnemonic mriedem since that fix is simple, can i take a stab at it?
17:14:16 mriedem KeithMnemonic: go nuts
17:14:20 KeithMnemonic ok thanks
17:17:06 dansmith supposedly the SEV thing was resolvable by someone else, but has that happened?
17:18:24 KeithMnemonic someone at SUSE? dansmith
17:18:29 mriedem you mean boris? https://review.opendev.org/#/c/684825/
17:18:32 mriedem he's also suse
17:18:46 KeithMnemonic he is still on openstack as well
17:29:05 openstackgerrit Merged openstack/nova master: Adds view builders for keypairs controller https://review.opendev.org/347289
17:34:19 dansmith KeithMnemonic: no, I meant assuming nobody from SUSE is going to fix it, there was an assertion from the libvirty people that they could resolve it too, I just haven't heard that it happened
17:34:20 dansmith efried: ?
17:35:38 KeithMnemonic ok
18:00:45 mriedem dansmith: so i'd be +2 on https://review.opendev.org/#/c/687348/ except,
18:01:05 mriedem i think we should probably drop the dependency on the osc command for now because the horizon thing holding up novaclient 16.0.0 in upper-constraints is not going well
18:01:27 mriedem could use the nova cli in the docs for now and leave a todo in there to replace with osc later
18:01:52 openstackgerrit Dustin Cowles proposed openstack/nova master: Provider Config File: YAML file loading and schema validation https://review.opendev.org/673341
18:01:53 openstackgerrit Dustin Cowles proposed openstack/nova master: Provider Config File: Merge provider configs to provider tree https://review.opendev.org/676522
18:01:53 openstackgerrit Dustin Cowles proposed openstack/nova master: Provider Config File: Function to further validate and retrieve configs https://review.opendev.org/676029
18:02:08 dansmith mriedem: there's no real rush to merge that is there?
18:02:16 mriedem the docs?
18:02:28 mriedem i'd like to merge it sooner than later so i don't lose the context on it
18:02:35 mriedem especially since we have a nova cli for the thing
18:04:13 dansmith okay
18:10:19 efried dansmith: I talked to aspiers a couple weeks ago and he said he was going to try to fix the thing regardless of what SUSE does.
18:10:42 efried but like as a swan song, not any kind of ongoing support.
18:11:11 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove fixed sqlalchemy-migrate deprecation warning filters https://review.opendev.org/690704
18:11:15 mriedem the song....remains the same?
18:11:19 efried but it's also still not out of the question that amd will pony up (or partner with) someone to maintain moving forward.
18:11:35 openstackgerrit Dan Smith proposed openstack/nova master: Add image precaching docs for aggregates https://review.opendev.org/687348
18:11:48 dansmith mriedem: ^
18:11:55 mriedem you forgot to remove the depends-on

Earlier   Later