Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-12-07
21:53:19 melwitt after fighting with the test coverage I wasn't thinking too clearly. that's my lame excuse
21:53:39 mriedem ok. i don't know what stephenfin's excuse was then. :)
21:54:11 mriedem thinking about bangers and mash during the football game at the pub
21:55:09 melwitt how is is_noop not knowing about the driver?
21:55:40 melwitt honestly the way it was before (the values coming from inside the driver) is the way the caller knows the least about it :P
21:55:45 mriedem i'm saying hide that check within QUOTAS
21:55:56 mriedem true
21:56:15 mriedem so, QUOTAS.get_reserved()?
21:56:16 melwitt so I'm like ¯\_(ツ)_/¯
21:56:21 mriedem umm
21:56:29 mriedem ^ is reserved for our api response when a cell is down and you list instances
21:56:51 melwitt wwhhaaaaat
21:57:05 mriedem all we have is the uuid in the instance mapping,
21:57:11 mriedem so we're going to use "¯\_(ツ)_/¯" for all other fields
21:57:15 melwitt oh right
21:57:21 melwitt that would be fun
21:57:28 mriedem reserve that patch for april 1
22:11:47 mriedem melwitt: dansmith: remind me, in the cellsv1 job, is n-api the top-level API or is that the cell API?
22:12:15 melwitt top-level API
22:12:32 melwitt I don't think we run n-api per cell though it is possible to do that (if you want to test a cell in isolation)
22:12:45 mriedem and cell-region is the nova-cells service at the top and cell-child is nova-cells in the cell?
22:13:03 melwitt oh boy, memory cobwebs
22:13:53 melwitt that sounds right though
22:14:08 openstackgerrit melanie witt proposed openstack/nova master: Follow up on removing old-style quotas code https://review.openstack.org/524234
22:18:53 mriedem hmm, ok, i'm trying to figure out http://logs.openstack.org/76/523676/11/check/legacy-tempest-dsvm-cells/1d155c6/logs/screen-n-cpu.txt.gz#_Dec_07_09_02_09_579991
22:19:01 mriedem the patch creates an instance action in the api
22:19:13 mriedem then when the compute is trying to look it up, it can't find it and blows up
22:19:26 mriedem so i'm guessing,
22:19:42 mriedem the top level api database has the action record, but not the cell
22:19:45 mriedem cell db i mean
22:20:12 mriedem but then i don't know how we have action records working for anything in cellsv1
22:20:18 melwitt yeah, so you'll want to check if the instance.save() at the top does a sync down to the cell. it probably does, but it can race
22:20:27 melwitt like compute might try to read before the sync makes it to the cell
22:20:38 mriedem this isn't instance.save()
22:20:46 melwitt okay, then it won't sync it
22:20:54 mriedem objects.InstanceAction.action_start
22:21:28 melwitt hm, so yeah does compute rely on reading instance actions in any other case?
22:21:48 melwitt because if so, like you said those shouldn't work either
22:21:51 mriedem oh i think i see something
22:22:11 mriedem so stop_instance is in this list https://github.com/openstack/nova/blob/master/nova/compute/cells_api.py#L51
22:22:15 mriedem and so is backup_instance
22:22:35 mriedem https://github.com/openstack/nova/blob/master/nova/cells/messaging.py#L1714
22:22:38 mriedem stop_instance has that ^
22:22:41 mriedem backup_instance doesn't
22:23:10 melwitt and you're seeing a fail with backup?
22:24:45 mriedem yeah https://review.openstack.org/#/c/523676/11
22:24:46 melwitt so returning the response from the instance action vs not
22:25:04 melwitt I didn't realize instance action had a rpc response in any case
22:25:22 openstackgerrit Takashi NATSUME proposed openstack/nova master: Deduplicate functional test code https://review.openstack.org/526227
22:27:53 melwitt yeah it looks like for some reason stop_instance can return something but backup_instance can't in nova/compute/rpcapi.py
22:28:07 melwitt I don't see stop_instance ever returning anything in compute/manager though
22:29:25 openstackgerrit Takashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in delete https://review.openstack.org/525231
22:29:34 mriedem melwitt: i think the do_cast thing is something else
22:29:38 openstackgerrit Takashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in delete (2) https://review.openstack.org/526263
22:29:40 mriedem red herring
22:29:43 melwitt yeah, it's just saying wait for response vs not
22:29:53 mriedem so i'm not sure what ever creates instance action records in the top level vs cell db
22:30:05 melwitt and looks correct based on backup vs stop
22:30:28 melwitt well, it would be whichever service does the write. if n-api writes it, it will be in the top level db, if n-cpu writes it, it will be in the cell db
22:30:40 melwitt I think only instance records are synced back and forth in cells v1
22:31:08 melwitt I'm checking to see if anything in backup_instance in compute does a write of instance action records
22:31:14 mriedem but if that's the case, then how does our @wrap_instance_event in the compute not blow up on everything?
22:31:20 melwitt I guess it can't be, else they'd be found
22:31:32 mriedem the action record stuff for backup_instance is new to this patch
22:31:40 mriedem following the same pattern as everywhere else
22:31:48 melwitt okay, I'll look at that then
22:32:44 melwitt yeah, that's just writing it in n-api so it'll go in the top level db only
22:32:52 melwitt I'm not sure how the other actions are currently working
22:44:44 mriedem alaski: you remember anything special about how the hell instance action records get synced between the api cell and the child cells in v1?
22:47:56 openstackgerrit Matt Riedemann proposed openstack/nova master: Add instance action record for backup instances https://review.openstack.org/523676
22:53:07 openstackgerrit Matt Riedemann proposed openstack/nova master: [WIP] POC to use neutron port_list when filtering instance by ip https://review.openstack.org/525505
23:05:23 tasker I'm playing around with the nova-metadata service and every PUT or POST request I send to "openstack/latest/password" is seen by the service as a GET. is this a security feature of the metadata service or is there some configuration issue that I haven't uncovered yet?
23:06:34 tasker I tracked it through the code from api/metadata/password.py:handle_password up through MetadataRequestHandler and it's a GET the whole time.
23:06:51 tasker is something translating it to GET?
23:08:04 mriedem tempest does a POST successfully from the guest to change the password
23:08:08 mriedem i remember because of this bug https://bugs.launchpad.net/nova/+bug/1696848
23:08:10 openstack Launchpad bug 1696848 in OpenStack Compute (nova) pike "POST /openstack/2013-10-17/password returns 500 if instance is not found" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)
23:08:46 mriedem http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/test_server_basic_ops.py#n90
23:09:02 mriedem tasker: are you passing a body?
23:09:24 tasker yeah. though now that I think about it, it's probably malformed
23:10:32 tasker curl -D - -X POST -d 'cloudnull' http://169.254.169.254/openstack/latest/password
23:12:21 tasker before we go much farther .. this Mitaka at best. I haven't been able to find anything that says it won't work there though.
23:12:39 mriedem metadata api doesn't really ever change
23:12:43 mriedem mikal might have some ideas
23:12:54 mriedem he did give the talk about it in sydney
23:13:27 tasker I can't seem to find anything else on the internet about it not working. as if it not working wasn't even a thing.
23:13:42 tasker I'll keep hacking and tracking seeing if I can figure out what's going on.
23:13:52 mriedem are you using service_metadata_proxy in nova.conf?
23:14:13 tasker yes. set to True
23:17:01 tasker or
23:17:02 tasker oh
23:17:10 tasker and .. that explains a lot.
23:17:39 mriedem not set?
23:17:47 tasker it's set to True.
23:18:09 tasker and I don't use the `neutron_agents` nor its metadata service
23:18:18 mriedem ah
23:19:04 tasker thanks, Matt. that shed a lot of light on what's going on.
23:19:07 tasker back to work!
23:19:09 mriedem o/
23:19:12 tasker to me. not you.

Earlier   Later